2024-03-26T13:34:02.047 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-26T13:34:02.047 DEBUG:teuthology.run:Teuthology command: teuthology --name mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi --archive /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940 --owner mchangir --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} -- /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/orig.config.yaml 2024-03-26T13:34:02.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T13:34:02.176 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940 branch: wip-mchangir-testing-main-20240318.032620 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} email: mchangir@redhat.com first_in_suite: false job_id: '7622940' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-testing-main-20240318.032620 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - slow metadata IO - SLOW_OPS - slow request sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-main-20240318.032620 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 owner: mchangir priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3185 sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 sleep_before_teardown: 0 subset: 155/256 suite: fs suite_branch: wip-mchangir-testing-main-20240318.032620 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxphBMmXNF5997hhTfJvfFX87xNP8nNzUJue/G8xxLyEA5aHmZvyq4GdEN1upO8Ac7fqmYN6PW7ZYSGnxHpQc0= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFaz4TwhsPhkEN5rr9iXf6Tvy5HEfimoIVWxx+bM9NLts2Ga8D3Nqf2LP8UVejyAo9PK4I4d/Bv5+mHUl2TFmEc= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMXfKgTlsjA+aIMd4i8oHJ7WfQSVYhz5rZzq7S7A2jCksrbOmDQa07XUBN0GQea2sd8zC5OC7KYWPMsxDGzdo5k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-26_07:59:35 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-26T13:34:02.176 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa; will attempt to use it 2024-03-26T13:34:02.176 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3be9a5c9bc793e11e2800a8c0c696e8b46742033/qa/tasks 2024-03-26T13:34:02.176 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-26T13:34:02.178 INFO:teuthology.task.internal:Checking packages... 2024-03-26T13:34:02.199 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3be9a5c9bc793e11e2800a8c0c696e8b46742033' 2024-03-26T13:34:02.199 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-26T13:34:02.199 INFO:teuthology.packaging:ref: None 2024-03-26T13:34:02.199 INFO:teuthology.packaging:tag: None 2024-03-26T13:34:02.200 INFO:teuthology.packaging:branch: wip-mchangir-testing-main-20240318.032620 2024-03-26T13:34:02.200 INFO:teuthology.packaging:sha1: 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:34:02.200 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-testing-main-20240318.032620 2024-03-26T13:34:02.339 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2145.g3be9a5c9 2024-03-26T13:34:02.341 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-26T13:34:02.348 INFO:teuthology.task.internal:no buildpackages task found 2024-03-26T13:34:02.348 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-26T13:34:02.447 INFO:teuthology.task.internal:Saving configuration 2024-03-26T13:34:02.465 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-26T13:34:02.471 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T13:34:02.493 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 13:28:50.995636', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxphBMmXNF5997hhTfJvfFX87xNP8nNzUJue/G8xxLyEA5aHmZvyq4GdEN1upO8Ac7fqmYN6PW7ZYSGnxHpQc0='} 2024-03-26T13:34:02.516 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 13:28:50.998389', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFaz4TwhsPhkEN5rr9iXf6Tvy5HEfimoIVWxx+bM9NLts2Ga8D3Nqf2LP8UVejyAo9PK4I4d/Bv5+mHUl2TFmEc='} 2024-03-26T13:34:02.537 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 13:28:51.000042', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMXfKgTlsjA+aIMd4i8oHJ7WfQSVYhz5rZzq7S7A2jCksrbOmDQa07XUBN0GQea2sd8zC5OC7KYWPMsxDGzdo5k='} 2024-03-26T13:34:02.537 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-26T13:34:02.544 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-26T13:34:02.544 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-26T13:34:02.544 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-26T13:34:02.544 INFO:teuthology.run_tasks:Running task console_log... 2024-03-26T13:34:02.652 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb9df246f70>, signals=[15]) 2024-03-26T13:34:02.652 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-26T13:34:02.658 INFO:teuthology.task.internal:Opening connections... 2024-03-26T13:34:02.659 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T13:34:02.660 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:34:02.752 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-03-26T13:34:02.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:34:02.838 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-03-26T13:34:02.839 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:34:02.925 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-26T13:34:02.932 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-03-26T13:34:02.950 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-03-26T13:34:02.950 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-03-26T13:34:03.007 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-03-26T13:34:03.007 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-03-26T13:34:03.007 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-03-26T13:34:03.007 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-03-26T13:34:03.007 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-03-26T13:34:03.007 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T13:34:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T13:34:03.008 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-03-26T13:34:03.031 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-03-26T13:34:03.048 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-03-26T13:34:03.049 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-03-26T13:34:03.109 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-03-26T13:34:03.109 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-03-26T13:34:03.109 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-03-26T13:34:03.109 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T13:34:03.110 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T13:34:03.110 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-03-26T13:34:03.130 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-03-26T13:34:03.149 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-03-26T13:34:03.149 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-03-26T13:34:03.205 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-03-26T13:34:03.205 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-03-26T13:34:03.205 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-03-26T13:34:03.205 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-26T13:34:03.206 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-26T13:34:03.206 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-03-26T13:34:03.230 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-26T13:34:03.243 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-26T13:34:03.249 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-26T13:34:03.249 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-03-26T13:34:03.252 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-03-26T13:34:03.254 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-03-26T13:34:03.270 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-26T13:34:03.276 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-26T13:34:03.276 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-03-26T13:34:03.308 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-03-26T13:34:03.314 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-03-26T13:34:03.414 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-26T13:34:03.482 INFO:teuthology.run_tasks:Running task kernel... 2024-03-26T13:34:03.496 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-26T13:34:03.496 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-26T13:34:03.496 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T13:34:03.497 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T13:34:03.497 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-26T13:34:03.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T13:34:03.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T13:34:03.514 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.514 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T13:34:03.514 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-03-26T13:34:03.515 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.515 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T13:34:03.515 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-03-26T13:34:03.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T13:34:03.516 DEBUG:teuthology.task.kernel:sha1 for mon.c is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.516 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-26T13:34:03.516 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-03-26T13:34:03.571 INFO:teuthology.orchestra.run.smithi137.stdout:5.14.0-361.el9.x86_64 2024-03-26T13:34:03.572 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.572 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T13:34:03.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.576 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-361.el9.x86_64 2024-03-26T13:34:03.577 INFO:teuthology.orchestra.run.smithi179.stdout:5.14.0-361.el9.x86_64 2024-03-26T13:34:03.577 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.577 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T13:34:03.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.578 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.578 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-26T13:34:03.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:34:03.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum upgrade -y linux-firmware 2024-03-26T13:34:03.795 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum upgrade -y linux-firmware 2024-03-26T13:34:03.797 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum upgrade -y linux-firmware 2024-03-26T13:34:06.043 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-26T13:34:06.086 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-26T13:34:06.116 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-26T13:34:07.394 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-03-26T13:34:07.499 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.0 MB 00:00 2024-03-26T13:34:07.703 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-03-26T13:34:09.424 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-03-26T13:34:11.175 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-03-26T13:34:13.427 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.0 MB 00:05 2024-03-26T13:34:16.211 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-26T13:34:18.433 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 769 kB/s | 8.0 MB 00:10 2024-03-26T13:34:20.284 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-03-26T13:34:23.400 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 725 kB/s | 6.1 MB 00:08 2024-03-26T13:34:24.177 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.1 MB 00:00 2024-03-26T13:34:24.750 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 16 kB 00:00 2024-03-26T13:34:25.628 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-03-26T13:34:26.310 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-26T13:34:27.769 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 782 kB/s | 6.1 MB 00:08 2024-03-26T13:34:29.211 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 16 kB 00:00 2024-03-26T13:34:30.171 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-03-26T13:34:31.518 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 5.0 kB/s | 1.7 kB 00:00 2024-03-26T13:34:33.738 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-03-26T13:34:33.751 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-03-26T13:34:33.751 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-03-26T13:34:33.751 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-03-26T13:34:33.751 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 1 Package 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 375 M 2024-03-26T13:34:33.752 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-03-26T13:34:35.129 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-26T13:34:36.829 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 21 MB 00:11 2024-03-26T13:34:37.412 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:================================================================================ 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 1 Package 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 375 M 2024-03-26T13:34:37.423 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-03-26T13:34:42.012 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-26T13:34:44.262 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-03-26T13:34:44.275 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-03-26T13:34:44.275 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-03-26T13:34:44.275 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout:================================================================================ 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 1 Package 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 375 M 2024-03-26T13:34:44.276 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-03-26T13:34:49.096 INFO:teuthology.orchestra.run.smithi137.stdout:linux-firmware-20240219-143.el9.noarch.rpm 32 MB/s | 375 MB 00:11 2024-03-26T13:34:49.096 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:34:49.097 INFO:teuthology.orchestra.run.smithi137.stdout:Total 32 MB/s | 375 MB 00:11 2024-03-26T13:34:49.097 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-03-26T13:34:49.191 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-03-26T13:34:49.192 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-03-26T13:34:49.276 INFO:teuthology.orchestra.run.smithi159.stdout:linux-firmware-20240219-143.el9.noarch.rpm 24 MB/s | 375 MB 00:15 2024-03-26T13:34:49.277 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:34:49.278 INFO:teuthology.orchestra.run.smithi159.stdout:Total 24 MB/s | 375 MB 00:15 2024-03-26T13:34:49.278 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-03-26T13:34:49.373 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-03-26T13:34:49.374 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-03-26T13:34:51.626 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-03-26T13:34:51.626 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-03-26T13:34:51.855 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-03-26T13:34:51.855 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-03-26T13:34:52.680 INFO:teuthology.orchestra.run.smithi179.stdout:linux-firmware-20240219-143.el9.noarch.rpm 45 MB/s | 375 MB 00:08 2024-03-26T13:34:52.681 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:34:52.681 INFO:teuthology.orchestra.run.smithi179.stdout:Total 45 MB/s | 375 MB 00:08 2024-03-26T13:34:52.682 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-03-26T13:34:52.778 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-03-26T13:34:52.778 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-03-26T13:34:55.161 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-03-26T13:34:55.161 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-03-26T13:34:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-03-26T13:34:58.127 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T13:34:58.229 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:34:58.238 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-03-26T13:34:58.494 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T13:34:58.577 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:34:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:34:59.477 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T13:34:59.778 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:34:59.778 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T13:34:59.856 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:34:59.856 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:34:59.856 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-03-26T13:34:59.856 INFO:teuthology.orchestra.run.smithi137.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T13:34:59.856 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:34:59.857 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-03-26T13:35:00.124 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:35:00.124 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:35:00.124 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-03-26T13:35:00.124 INFO:teuthology.orchestra.run.smithi159.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T13:35:00.125 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:35:00.125 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-03-26T13:35:00.179 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-26T13:35:00.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:35:00.388 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-26T13:35:00.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:35:00.396 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T13:35:00.396 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T13:35:00.399 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi137.front.sepia.ceph.com to complete... 2024-03-26T13:35:00.605 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T13:35:00.605 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T13:35:00.607 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi159.front.sepia.ceph.com to complete... 2024-03-26T13:35:01.703 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-03-26T13:35:01.958 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T13:35:02.041 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:35:03.260 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:35:03.260 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-26T13:35:03.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-26T13:35:03.701 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:35:03.701 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-03-26T13:35:03.701 INFO:teuthology.orchestra.run.smithi179.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-26T13:35:03.701 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:35:03.701 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-03-26T13:35:03.992 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-26T13:35:03.993 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-26T13:35:04.142 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-26T13:35:04.142 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-26T13:35:04.145 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi179.front.sepia.ceph.com to complete... 2024-03-26T13:35:13.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26 13:35:13 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T13:35:13.706 INFO:teuthology.orchestra.run.smithi137.stderr:FINISHED --2024-03-26 13:35:13-- 2024-03-26T13:35:13.706 INFO:teuthology.orchestra.run.smithi137.stderr:Total wall clock time: 13s 2024-03-26T13:35:13.707 INFO:teuthology.orchestra.run.smithi137.stderr:Downloaded: 1 files, 321M in 13s (24.6 MB/s) 2024-03-26T13:35:13.709 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-26T13:35:13.709 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T13:35:15.639 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying... ######################################## 2024-03-26T13:35:15.768 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing... ######################################## 2024-03-26T13:35:15.847 INFO:teuthology.orchestra.run.smithi137.stdout:Updating / installing... 2024-03-26T13:35:23.308 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T13:35:29.313 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26 13:35:29 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T13:35:29.313 INFO:teuthology.orchestra.run.smithi159.stderr:FINISHED --2024-03-26 13:35:29-- 2024-03-26T13:35:29.313 INFO:teuthology.orchestra.run.smithi159.stderr:Total wall clock time: 29s 2024-03-26T13:35:29.313 INFO:teuthology.orchestra.run.smithi159.stderr:Downloaded: 1 files, 321M in 28s (11.4 MB/s) 2024-03-26T13:35:29.316 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-26T13:35:29.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T13:35:31.252 INFO:teuthology.orchestra.run.smithi159.stdout:Verifying... ######################################## 2024-03-26T13:35:31.315 INFO:teuthology.orchestra.run.smithi159.stdout:Preparing... ######################################## 2024-03-26T13:35:31.395 INFO:teuthology.orchestra.run.smithi159.stdout:Updating / installing... 2024-03-26T13:35:39.015 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T13:35:39.145 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26 13:35:39 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-26T13:35:39.145 INFO:teuthology.orchestra.run.smithi179.stderr:FINISHED --2024-03-26 13:35:39-- 2024-03-26T13:35:39.145 INFO:teuthology.orchestra.run.smithi179.stderr:Total wall clock time: 35s 2024-03-26T13:35:39.145 INFO:teuthology.orchestra.run.smithi179.stderr:Downloaded: 1 files, 321M in 34s (9.33 MB/s) 2024-03-26T13:35:39.149 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.c... 2024-03-26T13:35:39.149 DEBUG:teuthology.orchestra.run.smithi179:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-26T13:35:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T13:35:41.073 INFO:teuthology.orchestra.run.smithi179.stdout:Verifying... ######################################## 2024-03-26T13:35:41.179 INFO:teuthology.orchestra.run.smithi179.stdout:Preparing... ######################################## 2024-03-26T13:35:41.257 INFO:teuthology.orchestra.run.smithi179.stdout:Updating / installing... 2024-03-26T13:35:48.808 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-26T13:35:53.216 INFO:teuthology.orchestra.run.smithi159.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T13:36:01.720 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T13:36:01.721 DEBUG:teuthology.orchestra.run.smithi137:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T13:36:01.791 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T13:36:01.791 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T13:36:01.791 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T13:36:01.792 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T13:36:01.793 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T13:36:01.794 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T13:36:01.795 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T13:36:01.796 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T13:36:01.797 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T13:36:01.798 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T13:36:01.799 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T13:36:01.800 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T13:36:01.801 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T13:36:01.802 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T13:36:01.803 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T13:36:01.804 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T13:36:01.805 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T13:36:01.806 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T13:36:01.807 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T13:36:01.808 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T13:36:01.809 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T13:36:01.810 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T13:36:01.811 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T13:36:01.812 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T13:36:01.813 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T13:36:01.814 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T13:36:01.815 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T13:36:01.816 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T13:36:01.817 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T13:36:01.818 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T13:36:01.819 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T13:36:01.820 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T13:36:01.821 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T13:36:01.822 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T13:36:01.823 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T13:36:01.824 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T13:36:01.825 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T13:36:01.826 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T13:36:01.827 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T13:36:01.828 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T13:36:01.829 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T13:36:01.830 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T13:36:01.831 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T13:36:01.832 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T13:36:01.833 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T13:36:01.834 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T13:36:01.835 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T13:36:01.836 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T13:36:01.837 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T13:36:01.838 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T13:36:01.839 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T13:36:01.840 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T13:36:01.841 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T13:36:01.842 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T13:36:01.843 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T13:36:01.844 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T13:36:01.845 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T13:36:01.846 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T13:36:01.847 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T13:36:01.848 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T13:36:01.849 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T13:36:01.850 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T13:36:01.851 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T13:36:01.852 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T13:36:01.853 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T13:36:01.854 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T13:36:01.855 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T13:36:01.856 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T13:36:01.857 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T13:36:01.858 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T13:36:01.859 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T13:36:01.860 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T13:36:01.861 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T13:36:01.862 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T13:36:01.863 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T13:36:01.864 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T13:36:01.865 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T13:36:01.866 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T13:36:01.867 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T13:36:01.868 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T13:36:01.869 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T13:36:01.870 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T13:36:01.871 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T13:36:01.872 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T13:36:01.873 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T13:36:01.874 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T13:36:01.875 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T13:36:01.876 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T13:36:01.877 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T13:36:01.878 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T13:36:01.879 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T13:36:01.880 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T13:36:01.881 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T13:36:01.882 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T13:36:01.883 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T13:36:01.884 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T13:36:01.885 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T13:36:01.886 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T13:36:01.887 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T13:36:01.888 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T13:36:01.889 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T13:36:01.890 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T13:36:01.891 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T13:36:01.892 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T13:36:01.893 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T13:36:01.894 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T13:36:01.895 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T13:36:01.896 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T13:36:01.897 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T13:36:01.898 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T13:36:01.899 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T13:36:01.900 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T13:36:01.901 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T13:36:01.902 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T13:36:01.903 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T13:36:01.904 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T13:36:01.905 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T13:36:01.906 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T13:36:01.907 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T13:36:01.908 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T13:36:01.909 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T13:36:01.910 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T13:36:01.911 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T13:36:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T13:36:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T13:36:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T13:36:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T13:36:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T13:36:01.917 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T13:36:01.918 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T13:36:01.919 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T13:36:01.920 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T13:36:01.921 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T13:36:01.922 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T13:36:01.923 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T13:36:01.924 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T13:36:01.925 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T13:36:01.926 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T13:36:01.927 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T13:36:01.928 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T13:36:01.929 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T13:36:01.930 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T13:36:01.931 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T13:36:01.932 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T13:36:01.933 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T13:36:01.934 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T13:36:01.935 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T13:36:01.936 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T13:36:01.937 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T13:36:01.938 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T13:36:01.939 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T13:36:01.940 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T13:36:01.941 INFO:teuthology.orchestra.run.smithi137.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T13:36:01.945 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T13:36:01.945 DEBUG:teuthology.orchestra.run.smithi137:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T13:36:01.967 INFO:teuthology.orchestra.run.smithi137.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T13:36:01.967 INFO:teuthology.orchestra.run.smithi137.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T13:36:01.967 INFO:teuthology.orchestra.run.smithi137.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T13:36:01.967 INFO:teuthology.orchestra.run.smithi137.stdout:fi 2024-03-26T13:36:01.967 INFO:teuthology.orchestra.run.smithi137.stdout:for file in vmlinuz System.map config; do 2024-03-26T13:36:01.967 INFO:teuthology.orchestra.run.smithi137.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout: fi 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:done 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:fi 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T13:36:01.968 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T13:36:01.969 INFO:teuthology.orchestra.run.smithi137.stdout:fi 2024-03-26T13:36:01.969 DEBUG:teuthology.orchestra.run.smithi137:> sudo rpm -qi grub2-tools 2024-03-26T13:36:02.046 INFO:teuthology.orchestra.run.smithi137.stdout:Name : grub2-tools 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Epoch : 1 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Version : 2.06 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Release : 68.el9 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: x86_64 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Group : Unspecified 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Size : 8264254 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:License : GPLv3+ 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T13:36:02.047 INFO:teuthology.orchestra.run.smithi137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:Packager : builder@centos.org 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:Vendor : CentOS 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:Summary : Support tools for GRUB. 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:Description : 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:hardware devices. 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:36:02.048 INFO:teuthology.orchestra.run.smithi137.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T13:36:02.049 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T13:36:02.049 INFO:teuthology.task.kernel:Updating grub on smithi137 to boot 6.8.0-g1df0c7e8a235 2024-03-26T13:36:02.049 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T13:36:02.579 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-03-26T13:36:03.473 INFO:teuthology.orchestra.run.smithi179.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-26T13:36:04.094 INFO:teuthology.orchestra.run.smithi137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T13:36:04.134 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-03-26T13:36:04.136 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-03-26T13:36:04.151 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.XbsMA0eFkN 2024-03-26T13:36:04.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XbsMA0eFkN 2024-03-26T13:36:04.220 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.XbsMA0eFkN 2024-03-26T13:36:04.364 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.XbsMA0eFkN is 6KB 2024-03-26T13:36:04.412 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.XbsMA0eFkN 2024-03-26T13:36:04.426 DEBUG:teuthology.orchestra.run.smithi137:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T13:36:04.492 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T13:36:04.492 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T13:36:04.492 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T13:36:04.492 INFO:teuthology.orchestra.run.smithi137.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T13:36:04.494 DEBUG:teuthology.orchestra.run.smithi137:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T13:36:04.645 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-03-26T13:36:16.106 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi159.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T13:36:16.107 DEBUG:teuthology.orchestra.run.smithi159:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T13:36:16.186 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T13:36:16.187 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T13:36:16.188 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T13:36:16.189 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T13:36:16.190 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T13:36:16.191 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T13:36:16.192 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T13:36:16.193 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T13:36:16.194 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T13:36:16.195 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T13:36:16.196 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T13:36:16.197 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T13:36:16.198 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T13:36:16.199 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T13:36:16.200 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T13:36:16.201 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T13:36:16.202 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T13:36:16.203 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T13:36:16.204 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T13:36:16.205 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T13:36:16.206 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T13:36:16.207 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T13:36:16.208 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T13:36:16.209 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T13:36:16.210 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T13:36:16.211 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T13:36:16.212 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T13:36:16.213 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T13:36:16.214 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T13:36:16.215 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T13:36:16.216 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T13:36:16.217 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T13:36:16.218 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T13:36:16.219 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T13:36:16.220 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T13:36:16.221 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T13:36:16.222 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T13:36:16.223 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T13:36:16.224 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T13:36:16.225 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T13:36:16.226 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T13:36:16.227 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T13:36:16.228 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T13:36:16.229 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T13:36:16.230 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T13:36:16.231 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T13:36:16.232 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T13:36:16.233 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T13:36:16.234 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T13:36:16.235 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T13:36:16.236 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T13:36:16.237 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T13:36:16.238 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T13:36:16.239 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T13:36:16.240 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T13:36:16.241 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T13:36:16.242 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T13:36:16.243 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T13:36:16.244 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T13:36:16.245 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T13:36:16.246 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T13:36:16.247 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T13:36:16.248 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T13:36:16.249 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T13:36:16.250 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T13:36:16.251 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T13:36:16.252 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T13:36:16.253 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T13:36:16.254 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T13:36:16.255 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T13:36:16.256 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T13:36:16.257 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T13:36:16.258 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T13:36:16.259 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T13:36:16.260 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T13:36:16.261 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T13:36:16.262 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T13:36:16.263 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T13:36:16.264 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T13:36:16.265 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T13:36:16.266 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T13:36:16.267 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T13:36:16.268 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T13:36:16.269 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T13:36:16.270 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T13:36:16.271 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T13:36:16.272 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T13:36:16.273 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T13:36:16.274 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T13:36:16.275 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T13:36:16.276 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T13:36:16.277 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T13:36:16.278 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T13:36:16.279 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T13:36:16.280 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T13:36:16.281 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T13:36:16.282 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T13:36:16.283 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T13:36:16.284 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T13:36:16.285 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T13:36:16.286 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T13:36:16.287 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T13:36:16.288 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T13:36:16.289 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T13:36:16.290 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T13:36:16.291 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T13:36:16.292 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T13:36:16.293 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T13:36:16.294 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T13:36:16.295 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T13:36:16.296 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T13:36:16.297 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T13:36:16.298 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T13:36:16.299 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T13:36:16.300 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T13:36:16.301 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T13:36:16.302 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T13:36:16.303 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T13:36:16.304 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T13:36:16.305 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T13:36:16.306 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T13:36:16.307 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T13:36:16.308 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T13:36:16.309 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T13:36:16.310 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T13:36:16.311 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T13:36:16.312 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T13:36:16.313 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T13:36:16.314 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T13:36:16.315 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T13:36:16.316 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T13:36:16.317 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T13:36:16.318 INFO:teuthology.orchestra.run.smithi159.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T13:36:16.321 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T13:36:16.322 DEBUG:teuthology.orchestra.run.smithi159:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T13:36:16.345 INFO:teuthology.orchestra.run.smithi159.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T13:36:16.345 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:for file in vmlinuz System.map config; do 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout: fi 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:done 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-03-26T13:36:16.346 INFO:teuthology.orchestra.run.smithi159.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T13:36:16.347 INFO:teuthology.orchestra.run.smithi159.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T13:36:16.347 INFO:teuthology.orchestra.run.smithi159.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T13:36:16.347 INFO:teuthology.orchestra.run.smithi159.stdout:fi 2024-03-26T13:36:16.347 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -qi grub2-tools 2024-03-26T13:36:16.425 INFO:teuthology.orchestra.run.smithi159.stdout:Name : grub2-tools 2024-03-26T13:36:16.425 INFO:teuthology.orchestra.run.smithi159.stdout:Epoch : 1 2024-03-26T13:36:16.425 INFO:teuthology.orchestra.run.smithi159.stdout:Version : 2.06 2024-03-26T13:36:16.425 INFO:teuthology.orchestra.run.smithi159.stdout:Release : 68.el9 2024-03-26T13:36:16.425 INFO:teuthology.orchestra.run.smithi159.stdout:Architecture: x86_64 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Group : Unspecified 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Size : 8264254 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:License : GPLv3+ 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Packager : builder@centos.org 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Vendor : CentOS 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Summary : Support tools for GRUB. 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout:Description : 2024-03-26T13:36:16.426 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:36:16.427 INFO:teuthology.orchestra.run.smithi159.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T13:36:16.427 INFO:teuthology.orchestra.run.smithi159.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T13:36:16.427 INFO:teuthology.orchestra.run.smithi159.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T13:36:16.427 INFO:teuthology.orchestra.run.smithi159.stdout:hardware devices. 2024-03-26T13:36:16.427 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:36:16.427 INFO:teuthology.orchestra.run.smithi159.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T13:36:16.428 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T13:36:16.428 INFO:teuthology.task.kernel:Updating grub on smithi159 to boot 6.8.0-g1df0c7e8a235 2024-03-26T13:36:16.428 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T13:36:17.003 INFO:teuthology.orchestra.run.smithi159.stderr:Generating grub configuration file ... 2024-03-26T13:36:18.490 INFO:teuthology.orchestra.run.smithi159.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T13:36:18.510 INFO:teuthology.orchestra.run.smithi159.stderr:done 2024-03-26T13:36:18.513 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-03-26T13:36:18.527 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.gYVzkQc3A2 2024-03-26T13:36:18.527 DEBUG:teuthology.orchestra.run.smithi159:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gYVzkQc3A2 2024-03-26T13:36:18.596 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.gYVzkQc3A2 2024-03-26T13:36:18.740 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.gYVzkQc3A2 is 6KB 2024-03-26T13:36:18.789 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.gYVzkQc3A2 2024-03-26T13:36:18.803 DEBUG:teuthology.orchestra.run.smithi159:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T13:36:18.871 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T13:36:18.871 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T13:36:18.871 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T13:36:18.872 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T13:36:18.873 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T13:36:19.026 DEBUG:teuthology.orchestra.run.smithi159:> sudo shutdown -r now 2024-03-26T13:36:26.016 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi179.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-26T13:36:26.017 DEBUG:teuthology.orchestra.run.smithi179:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-26T13:36:26.085 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-26T13:36:26.086 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-26T13:36:26.087 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-26T13:36:26.088 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-26T13:36:26.089 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-26T13:36:26.090 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-26T13:36:26.091 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-26T13:36:26.092 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-26T13:36:26.093 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-26T13:36:26.094 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-26T13:36:26.095 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-26T13:36:26.096 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-26T13:36:26.097 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-26T13:36:26.098 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-26T13:36:26.099 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-26T13:36:26.100 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-26T13:36:26.101 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-26T13:36:26.102 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-26T13:36:26.103 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-26T13:36:26.104 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-26T13:36:26.105 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-26T13:36:26.106 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-26T13:36:26.107 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-26T13:36:26.108 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-26T13:36:26.109 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-26T13:36:26.110 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-26T13:36:26.111 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-26T13:36:26.112 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-26T13:36:26.113 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-26T13:36:26.114 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-26T13:36:26.115 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-26T13:36:26.116 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-26T13:36:26.117 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-26T13:36:26.118 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-26T13:36:26.119 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-26T13:36:26.120 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-26T13:36:26.121 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-26T13:36:26.122 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-26T13:36:26.123 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-26T13:36:26.124 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-26T13:36:26.125 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-26T13:36:26.126 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-26T13:36:26.127 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-26T13:36:26.128 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-26T13:36:26.129 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-26T13:36:26.130 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-26T13:36:26.131 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-26T13:36:26.132 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-26T13:36:26.133 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-26T13:36:26.134 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-26T13:36:26.135 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-26T13:36:26.136 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-26T13:36:26.137 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-26T13:36:26.138 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-26T13:36:26.139 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-26T13:36:26.140 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-26T13:36:26.141 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-26T13:36:26.142 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-26T13:36:26.143 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-26T13:36:26.144 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-26T13:36:26.145 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-26T13:36:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-26T13:36:26.147 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-26T13:36:26.148 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-26T13:36:26.149 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-26T13:36:26.150 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-26T13:36:26.151 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-26T13:36:26.152 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-26T13:36:26.153 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-26T13:36:26.154 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-26T13:36:26.155 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-26T13:36:26.156 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-26T13:36:26.157 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-26T13:36:26.158 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-26T13:36:26.159 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-26T13:36:26.160 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-26T13:36:26.161 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-26T13:36:26.162 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-26T13:36:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-26T13:36:26.164 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-26T13:36:26.165 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-26T13:36:26.166 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-26T13:36:26.167 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-26T13:36:26.168 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-26T13:36:26.169 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-26T13:36:26.170 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-26T13:36:26.171 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-26T13:36:26.172 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-26T13:36:26.173 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-26T13:36:26.174 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-26T13:36:26.175 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-26T13:36:26.176 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-26T13:36:26.177 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-26T13:36:26.178 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-26T13:36:26.179 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-26T13:36:26.180 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-26T13:36:26.181 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-26T13:36:26.182 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-26T13:36:26.183 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-26T13:36:26.184 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-26T13:36:26.185 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-26T13:36:26.186 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-26T13:36:26.187 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-26T13:36:26.188 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-26T13:36:26.189 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-26T13:36:26.190 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-26T13:36:26.191 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-26T13:36:26.192 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-26T13:36:26.193 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-26T13:36:26.194 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-26T13:36:26.195 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-26T13:36:26.196 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-26T13:36:26.197 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-26T13:36:26.198 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-26T13:36:26.199 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-26T13:36:26.200 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-26T13:36:26.201 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-26T13:36:26.202 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-26T13:36:26.203 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-26T13:36:26.204 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-26T13:36:26.205 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-26T13:36:26.206 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-26T13:36:26.207 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-26T13:36:26.208 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-26T13:36:26.209 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-26T13:36:26.210 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-26T13:36:26.211 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-26T13:36:26.212 INFO:teuthology.orchestra.run.smithi179.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T13:36:26.216 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-26T13:36:26.216 DEBUG:teuthology.orchestra.run.smithi179:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout:postinstall scriptlet (using /bin/sh): 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout:fi 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout:for file in vmlinuz System.map config; do 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout: fi 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout:done 2024-03-26T13:36:26.238 INFO:teuthology.orchestra.run.smithi179.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:fi 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-26T13:36:26.239 INFO:teuthology.orchestra.run.smithi179.stdout:fi 2024-03-26T13:36:26.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo rpm -qi grub2-tools 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Name : grub2-tools 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Epoch : 1 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Version : 2.06 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Release : 68.el9 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Architecture: x86_64 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Group : Unspecified 2024-03-26T13:36:26.315 INFO:teuthology.orchestra.run.smithi179.stdout:Size : 8264254 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:License : GPLv3+ 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Packager : builder@centos.org 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Vendor : CentOS 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Summary : Support tools for GRUB. 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:Description : 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:hardware devices. 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:36:26.316 INFO:teuthology.orchestra.run.smithi179.stdout:This subpackage provides tools for support of all platforms. 2024-03-26T13:36:26.317 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-26T13:36:26.317 INFO:teuthology.task.kernel:Updating grub on smithi179 to boot 6.8.0-g1df0c7e8a235 2024-03-26T13:36:26.317 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-26T13:36:26.985 INFO:teuthology.orchestra.run.smithi179.stderr:Generating grub configuration file ... 2024-03-26T13:36:28.385 INFO:teuthology.orchestra.run.smithi179.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-26T13:36:28.428 INFO:teuthology.orchestra.run.smithi179.stderr:done 2024-03-26T13:36:28.438 DEBUG:teuthology.orchestra.run.smithi179:> mktemp 2024-03-26T13:36:28.453 INFO:teuthology.orchestra.run.smithi179.stdout:/tmp/tmp.wYNqTuq9jq 2024-03-26T13:36:28.453 DEBUG:teuthology.orchestra.run.smithi179:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wYNqTuq9jq 2024-03-26T13:36:28.521 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 0666 /tmp/tmp.wYNqTuq9jq 2024-03-26T13:36:28.665 DEBUG:teuthology.orchestra.remote:smithi179:/tmp/tmp.wYNqTuq9jq is 6KB 2024-03-26T13:36:28.713 DEBUG:teuthology.orchestra.run.smithi179:> rm -fr /tmp/tmp.wYNqTuq9jq 2024-03-26T13:36:28.727 DEBUG:teuthology.orchestra.run.smithi179:> sudo /bin/ls /boot/loader/entries || true 2024-03-26T13:36:28.792 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-26T13:36:28.792 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-26T13:36:28.792 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-26T13:36:28.792 INFO:teuthology.orchestra.run.smithi179.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-26T13:36:28.793 DEBUG:teuthology.orchestra.run.smithi179:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-26T13:36:28.974 DEBUG:teuthology.orchestra.run.smithi179:> sudo shutdown -r now 2024-03-26T13:36:34.652 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-26T13:36:34.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-26T13:36:34.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:36:49.042 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-26T13:36:49.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:36:49.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:36:53.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-26T13:36:58.978 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-26T13:36:58.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-03-26T13:36:58.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:02.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-26T13:37:02.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:05.086 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-03-26T13:37:07.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-26T13:37:16.367 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:37:16.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:17.088 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-26T13:37:17.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:17.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-03-26T13:37:19.423 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-03-26T13:37:26.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-03-26T13:37:26.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:29.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.179 2024-03-26T13:37:31.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:37:31.426 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:41.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-03-26T13:37:41.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:37:49.064 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-26T13:37:49.679 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-26T13:37:49.679 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-26T13:37:49.679 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-03-26T13:37:49.720 INFO:teuthology.orchestra.run.smithi137.stdout:6.8.0-g1df0c7e8a235 2024-03-26T13:37:49.720 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T13:37:49.721 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T13:37:49.721 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T13:37:50.722 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-26T13:37:50.722 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T13:37:50.844 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-03-26T13:37:50.888 DEBUG:teuthology.parallel:result is None 2024-03-26T13:38:13.505 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-03-26T13:38:14.060 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-03-26T13:38:14.060 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-g1df0c7e8a235"... 2024-03-26T13:38:14.060 DEBUG:teuthology.orchestra.run.smithi179:> uname -r 2024-03-26T13:38:14.098 INFO:teuthology.orchestra.run.smithi179.stdout:6.8.0-g1df0c7e8a235 2024-03-26T13:38:14.098 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T13:38:14.098 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T13:38:14.098 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T13:38:15.099 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-26T13:38:15.099 DEBUG:teuthology.orchestra.run.smithi179:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T13:38:15.227 INFO:teuthology.orchestra.run.smithi179.stdout:ttyS1 2024-03-26T13:38:15.266 DEBUG:teuthology.parallel:result is None 2024-03-26T13:38:31.429 DEBUG:teuthology.orchestra.remote:timed out 2024-03-26T13:38:46.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:38:46.431 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:38:46.796 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-03-26T13:38:47.431 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:38:47.431 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-26T13:38:47.432 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-03-26T13:38:47.483 INFO:teuthology.orchestra.run.smithi159.stdout:6.8.0-g1df0c7e8a235 2024-03-26T13:38:47.483 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-26T13:38:47.483 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-26T13:38:47.483 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-26T13:38:48.485 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-26T13:38:48.485 DEBUG:teuthology.orchestra.run.smithi159:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-26T13:38:48.628 INFO:teuthology.orchestra.run.smithi159.stdout:ttyS1 2024-03-26T13:38:48.697 DEBUG:teuthology.parallel:result is None 2024-03-26T13:38:48.697 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-26T13:38:48.704 INFO:teuthology.task.internal:Creating test directory... 2024-03-26T13:38:48.704 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T13:38:48.712 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T13:38:48.720 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-26T13:38:48.779 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-26T13:38:48.786 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-26T13:38:48.794 INFO:teuthology.task.internal:Creating archive directory... 2024-03-26T13:38:48.794 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T13:38:48.801 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T13:38:48.808 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-26T13:38:49.160 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-26T13:38:49.167 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-26T13:38:49.167 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T13:38:49.175 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T13:38:49.182 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-26T13:38:49.283 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T13:38:49.289 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T13:38:49.299 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T13:38:49.325 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T13:38:49.332 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T13:38:49.344 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-26T13:38:49.349 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-26T13:38:49.356 INFO:teuthology.task.internal:Configuring sudo... 2024-03-26T13:38:49.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T13:38:49.363 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T13:38:49.370 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-26T13:38:49.467 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-26T13:38:49.475 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-26T13:38:49.476 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T13:38:49.483 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T13:38:49.490 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-26T13:38:49.543 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T13:38:49.623 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T13:38:49.751 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T13:38:49.832 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T13:38:49.926 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:38:49.926 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T13:38:50.021 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T13:38:50.107 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T13:38:50.240 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T13:38:50.323 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T13:38:50.420 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:38:50.420 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T13:38:50.511 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T13:38:50.600 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-26T13:38:50.725 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T13:38:50.808 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-26T13:38:50.906 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:38:50.906 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-26T13:38:50.999 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-03-26T13:38:51.006 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-03-26T13:38:51.014 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-03-26T13:38:51.127 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T13:38:51.134 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T13:38:51.181 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T13:38:51.599 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-26T13:38:51.607 INFO:teuthology.task.internal:Starting timer... 2024-03-26T13:38:51.607 INFO:teuthology.run_tasks:Running task pcp... 2024-03-26T13:38:51.620 INFO:teuthology.run_tasks:Running task selinux... 2024-03-26T13:38:51.628 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-26T13:38:51.629 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-03-26T13:38:51.753 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-03-26T13:38:51.757 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-03-26T13:38:51.907 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-03-26T13:38:51.912 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-03-26T13:38:52.053 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-03-26T13:38:52.058 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-26T13:38:52.058 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-03-26T13:38:52.105 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-03-26T13:38:52.106 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-03-26T13:38:52.159 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-03-26T13:38:52.159 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-03-26T13:38:52.211 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-03-26T13:38:52.211 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-03-26T13:38:52.211 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T13:38:52.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T13:38:52.297 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T13:38:52.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T13:38:52.378 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T13:38:52.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T13:38:52.462 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-26T13:38:52.462 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-03-26T13:38:52.561 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-03-26T13:38:52.646 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-03-26T13:38:52.732 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-26T13:38:52.742 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-26T13:38:52.757 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-26T13:38:52.758 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi137.front.sepia.ceph.com,smithi159.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-26T13:45:10.063 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-03-26T13:45:10.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-26T13:45:10.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:45:10.206 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-03-26T13:45:10.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-03-26T13:45:10.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:45:10.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:45:10.491 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-03-26T13:45:10.636 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-03-26T13:45:10.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-03-26T13:45:10.637 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-26T13:45:10.778 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-03-26T13:45:10.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-03-26T13:45:10.920 INFO:teuthology.run_tasks:Running task clock... 2024-03-26T13:45:10.930 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-26T13:45:10.930 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T13:45:10.930 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T13:45:10.938 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T13:45:10.938 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T13:45:10.945 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-26T13:45:10.945 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T13:45:11.034 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T13:45:11.042 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T13:45:11.051 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-26T13:45:11.086 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T13:45:11.091 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T13:45:11.102 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-26T13:45:11.178 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-03-26T13:45:11.183 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-03-26T13:45:11.193 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-03-26T13:45:11.219 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-03-26T13:45:11.224 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-03-26T13:45:11.235 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-03-26T13:45:11.269 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T13:45:11.276 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T13:45:11.287 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-26T13:45:11.323 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T13:45:11.329 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T13:45:11.339 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-26T13:45:11.448 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-03-26T13:45:11.456 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T13:45:11.457 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-03-26T13:45:11.457 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.457 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.457 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.457 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.466 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-03-26T13:45:11.471 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-03-26T13:45:11.473 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T13:45:11.473 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-03-26T13:45:11.473 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.473 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.473 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.473 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.479 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T13:45:11.479 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-03-26T13:45:11.479 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.479 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.479 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.479 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T13:45:11.480 INFO:teuthology.run_tasks:Running task install... 2024-03-26T13:45:11.489 DEBUG:teuthology.task.install:project ceph 2024-03-26T13:45:11.489 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'}} 2024-03-26T13:45:11.489 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033'} 2024-03-26T13:45:11.489 INFO:teuthology.task.install:Using flavor: default 2024-03-26T13:45:11.494 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-26T13:45:11.494 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-26T13:45:11.495 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T13:45:11.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:45:11.497 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T13:45:11.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:45:11.498 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'tag': None, 'wait_for_package': False} 2024-03-26T13:45:11.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:45:11.637 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T13:45:11.637 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T13:45:11.718 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T13:45:11.718 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T13:45:11.724 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/ 2024-03-26T13:45:11.724 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2145.g3be9a5c9 2024-03-26T13:45:11.825 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T13:45:11.825 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:45:11.825 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T13:45:11.886 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T13:45:11.887 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:45:11.887 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T13:45:11.899 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-26T13:45:11.899 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:45:11.899 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-26T13:45:11.922 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T13:45:11.923 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T13:45:11.988 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T13:45:11.988 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T13:45:12.000 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-26T13:45:12.001 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3be9a5c9bc793e11e2800a8c0c696e8b46742033/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-26T13:45:12.061 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T13:45:12.124 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T13:45:12.138 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-26T13:45:12.190 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T13:45:12.248 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T13:45:12.268 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-26T13:45:12.279 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-03-26T13:45:12.283 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-03-26T13:45:12.335 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-03-26T13:45:12.339 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-03-26T13:45:12.360 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-03-26T13:45:12.364 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-03-26T13:45:12.636 INFO:teuthology.orchestra.run.smithi137.stdout:74 files removed 2024-03-26T13:45:12.678 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T13:45:12.691 INFO:teuthology.orchestra.run.smithi159.stdout:74 files removed 2024-03-26T13:45:12.729 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T13:45:12.737 INFO:teuthology.orchestra.run.smithi179.stdout:74 files removed 2024-03-26T13:45:12.775 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-26T13:45:13.661 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-03-26T13:45:13.694 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-03-26T13:45:13.763 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 145 kB/s | 83 kB 00:00 2024-03-26T13:45:14.322 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-03-26T13:45:14.395 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 17 kB/s | 10 kB 00:00 2024-03-26T13:45:14.412 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 15 kB/s | 10 kB 00:00 2024-03-26T13:45:14.872 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-03-26T13:45:14.896 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-03-26T13:45:14.954 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-26T13:45:15.490 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-26T13:45:15.574 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-26T13:45:15.668 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - AppStream 9.3 MB/s | 6.7 MB 00:00 2024-03-26T13:45:17.267 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-26T13:45:17.400 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-03-26T13:45:17.735 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS-9 - BaseOS 2.5 MB/s | 2.0 MB 00:00 2024-03-26T13:45:18.387 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-26T13:45:20.864 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.0 MB 00:02 2024-03-26T13:45:22.798 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-03-26T13:45:27.174 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.1 MB 00:00 2024-03-26T13:45:27.877 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 818 kB/s | 8.0 MB 00:10 2024-03-26T13:45:28.775 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-03-26T13:45:30.420 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-03-26T13:45:30.498 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-03-26T13:45:34.829 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-26T13:45:35.687 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-03-26T13:45:36.276 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-26T13:45:36.430 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-03-26T13:45:37.482 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-26T13:45:38.910 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T13:45:38.911 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T13:45:38.935 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T13:45:38.936 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T13:45:38.938 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T13:45:38.939 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T13:45:38.942 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T13:45:38.945 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T13:45:38.947 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T13:45:39.034 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T13:45:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T13:45:39.060 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T13:45:39.061 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T13:45:39.062 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T13:45:39.063 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T13:45:39.064 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T13:45:39.065 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T13:45:39.066 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T13:45:39.067 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T13:45:39.068 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T13:45:39.069 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T13:45:39.070 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================================ 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout:Install 150 Packages 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 24 Packages 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 225 M 2024-03-26T13:45:39.071 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-03-26T13:45:41.410 INFO:teuthology.orchestra.run.smithi137.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 4.1 kB/s | 6.6 kB 00:01 2024-03-26T13:45:42.685 INFO:teuthology.orchestra.run.smithi137.stdout:(2/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-03-26T13:45:42.727 INFO:teuthology.orchestra.run.smithi137.stdout:(3/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 677 kB/s | 890 kB 00:01 2024-03-26T13:45:42.810 INFO:teuthology.orchestra.run.smithi137.stdout:(4/174): ceph-immutable-object-cache-19.0.0-214 1.1 MB/s | 145 kB 00:00 2024-03-26T13:45:42.837 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.1 MB 00:03 2024-03-26T13:45:43.248 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-03-26T13:45:43.319 INFO:teuthology.orchestra.run.smithi137.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-03-26T13:45:44.111 INFO:teuthology.orchestra.run.smithi137.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 5.9 MB/s | 4.7 MB 00:00 2024-03-26T13:45:44.486 INFO:teuthology.orchestra.run.smithi137.stdout:(7/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 1.3 MB/s | 2.3 MB 00:01 2024-03-26T13:45:44.546 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-03-26T13:45:44.995 INFO:teuthology.orchestra.run.smithi137.stdout:(8/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 4.0 MB/s | 20 MB 00:05 2024-03-26T13:45:45.095 INFO:teuthology.orchestra.run.smithi137.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 254 kB/s | 25 kB 00:00 2024-03-26T13:45:45.877 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T13:45:45.879 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T13:45:45.898 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T13:45:45.900 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T13:45:45.901 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T13:45:45.902 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T13:45:45.904 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T13:45:45.908 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T13:45:45.909 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T13:45:45.998 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-03-26T13:45:46.026 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-03-26T13:45:46.026 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-03-26T13:45:46.026 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-03-26T13:45:46.026 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T13:45:46.027 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T13:45:46.028 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T13:45:46.029 INFO:teuthology.orchestra.run.smithi179.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T13:45:46.030 INFO:teuthology.orchestra.run.smithi179.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T13:45:46.031 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T13:45:46.032 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T13:45:46.033 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T13:45:46.034 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T13:45:46.035 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T13:45:46.036 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T13:45:46.037 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T13:45:46.038 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout:============================================================================================ 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout:Install 150 Packages 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 24 Packages 2024-03-26T13:45:46.039 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:45:46.040 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 225 M 2024-03-26T13:45:46.040 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-03-26T13:45:47.112 INFO:teuthology.orchestra.run.smithi137.stdout:(10/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x8 5.6 MB/s | 17 MB 00:02 2024-03-26T13:45:47.187 INFO:teuthology.orchestra.run.smithi137.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 2.9 MB/s | 220 kB 00:00 2024-03-26T13:45:47.191 INFO:teuthology.orchestra.run.smithi179.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-03-26T13:45:47.237 INFO:teuthology.orchestra.run.smithi137.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 548 kB/s | 27 kB 00:00 2024-03-26T13:45:47.288 INFO:teuthology.orchestra.run.smithi137.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 628 kB/s | 31 kB 00:00 2024-03-26T13:45:47.680 INFO:teuthology.orchestra.run.smithi137.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 1.9 MB/s | 746 kB 00:00 2024-03-26T13:45:47.830 INFO:teuthology.orchestra.run.smithi137.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 1.1 MB/s | 163 kB 00:00 2024-03-26T13:45:47.930 INFO:teuthology.orchestra.run.smithi137.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.2 MB/s | 127 kB 00:00 2024-03-26T13:45:48.214 INFO:teuthology.orchestra.run.smithi137.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 1.7 MB/s | 501 kB 00:00 2024-03-26T13:45:48.348 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 21 MB 00:03 2024-03-26T13:45:48.600 INFO:teuthology.orchestra.run.smithi179.stdout:(2/174): ceph-common-19.0.0-2145.g3be9a5c9.el9. 12 MB/s | 20 MB 00:01 2024-03-26T13:45:48.650 INFO:teuthology.orchestra.run.smithi179.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 2.9 MB/s | 145 kB 00:00 2024-03-26T13:45:48.692 INFO:teuthology.orchestra.run.smithi179.stdout:(4/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 594 kB/s | 890 kB 00:01 2024-03-26T13:45:48.867 INFO:teuthology.orchestra.run.smithi179.stdout:(5/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 2.3 MB 00:00 2024-03-26T13:45:49.243 INFO:teuthology.orchestra.run.smithi179.stdout:(6/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 13 MB/s | 4.7 MB 00:00 2024-03-26T13:45:49.280 INFO:teuthology.orchestra.run.smithi137.stdout:(18/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 4.8 MB/s | 5.1 MB 00:01 2024-03-26T13:45:49.372 INFO:teuthology.orchestra.run.smithi137.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 498 kB/s | 45 kB 00:00 2024-03-26T13:45:49.472 INFO:teuthology.orchestra.run.smithi137.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.3 MB/s | 131 kB 00:00 2024-03-26T13:45:49.573 INFO:teuthology.orchestra.run.smithi137.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.6 MB/s | 162 kB 00:00 2024-03-26T13:45:49.748 INFO:teuthology.orchestra.run.smithi137.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 1.8 MB/s | 321 kB 00:00 2024-03-26T13:45:49.925 INFO:teuthology.orchestra.run.smithi137.stdout:(23/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.7 MB/s | 298 kB 00:00 2024-03-26T13:45:50.024 INFO:teuthology.orchestra.run.smithi137.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.0 MB/s | 99 kB 00:00 2024-03-26T13:45:50.124 INFO:teuthology.orchestra.run.smithi137.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 862 kB/s | 85 kB 00:00 2024-03-26T13:45:50.376 INFO:teuthology.orchestra.run.smithi179.stdout:(7/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 15 MB/s | 17 MB 00:01 2024-03-26T13:45:50.652 INFO:teuthology.orchestra.run.smithi179.stdout:(8/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 782 kB/s | 1.5 MB 00:01 2024-03-26T13:45:50.774 INFO:teuthology.orchestra.run.smithi137.stdout:(26/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 4.8 MB/s | 3.1 MB 00:00 2024-03-26T13:45:50.785 INFO:teuthology.orchestra.run.smithi179.stdout:(9/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el9 190 kB/s | 25 kB 00:00 2024-03-26T13:45:50.866 INFO:teuthology.orchestra.run.smithi137.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-26T13:45:50.958 INFO:teuthology.orchestra.run.smithi137.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 373 kB/s | 34 kB 00:00 2024-03-26T13:45:51.000 INFO:teuthology.orchestra.run.smithi137.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 3.3 MB/s | 139 kB 00:00 2024-03-26T13:45:51.094 INFO:teuthology.orchestra.run.smithi179.stdout:(10/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 14 MB/s | 9.9 MB 00:00 2024-03-26T13:45:51.144 INFO:teuthology.orchestra.run.smithi179.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 4.4 MB/s | 220 kB 00:00 2024-03-26T13:45:51.194 INFO:teuthology.orchestra.run.smithi179.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 547 kB/s | 27 kB 00:00 2024-03-26T13:45:51.244 INFO:teuthology.orchestra.run.smithi179.stdout:(13/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 636 kB/s | 31 kB 00:00 2024-03-26T13:45:51.250 INFO:teuthology.orchestra.run.smithi137.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 7.2 MB/s | 1.8 MB 00:00 2024-03-26T13:45:51.320 INFO:teuthology.orchestra.run.smithi179.stdout:(14/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 9.8 MB/s | 746 kB 00:00 2024-03-26T13:45:51.378 INFO:teuthology.orchestra.run.smithi179.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 2.8 MB/s | 163 kB 00:00 2024-03-26T13:45:51.429 INFO:teuthology.orchestra.run.smithi179.stdout:(16/174): librados-devel-19.0.0-2145.g3be9a5c9. 2.5 MB/s | 127 kB 00:00 2024-03-26T13:45:51.527 INFO:teuthology.orchestra.run.smithi179.stdout:(17/174): libradosstriper1-19.0.0-2145.g3be9a5c 5.1 MB/s | 501 kB 00:00 2024-03-26T13:45:51.771 INFO:teuthology.orchestra.run.smithi179.stdout:(18/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x 1.1 MB/s | 5.4 MB 00:04 2024-03-26T13:45:51.880 INFO:teuthology.orchestra.run.smithi179.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 420 kB/s | 45 kB 00:00 2024-03-26T13:45:51.988 INFO:teuthology.orchestra.run.smithi179.stdout:(20/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-26T13:45:52.039 INFO:teuthology.orchestra.run.smithi179.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 3.2 MB/s | 162 kB 00:00 2024-03-26T13:45:52.064 INFO:teuthology.orchestra.run.smithi179.stdout:(22/174): python3-ceph-common-19.0.0-2145.g3be9 716 kB/s | 131 kB 00:00 2024-03-26T13:45:52.097 INFO:teuthology.orchestra.run.smithi179.stdout:(23/174): python3-rados-19.0.0-2145.g3be9a5c9.e 5.4 MB/s | 321 kB 00:00 2024-03-26T13:45:52.148 INFO:teuthology.orchestra.run.smithi179.stdout:(24/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 2.0 MB/s | 99 kB 00:00 2024-03-26T13:45:52.198 INFO:teuthology.orchestra.run.smithi179.stdout:(25/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-26T13:45:52.248 INFO:teuthology.orchestra.run.smithi179.stdout:(26/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.6 MB/s | 298 kB 00:00 2024-03-26T13:45:52.490 INFO:teuthology.orchestra.run.smithi179.stdout:(27/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 11 MB/s | 3.1 MB 00:00 2024-03-26T13:45:52.515 INFO:teuthology.orchestra.run.smithi179.stdout:(28/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 640 kB/s | 170 kB 00:00 2024-03-26T13:45:52.541 INFO:teuthology.orchestra.run.smithi179.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2145.g 687 kB/s | 34 kB 00:00 2024-03-26T13:45:52.567 INFO:teuthology.orchestra.run.smithi137.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 5.6 MB/s | 7.4 MB 00:01 2024-03-26T13:45:52.641 INFO:teuthology.orchestra.run.smithi179.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 1.1 MB/s | 139 kB 00:00 2024-03-26T13:45:52.708 INFO:teuthology.orchestra.run.smithi179.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 11 MB/s | 1.8 MB 00:00 2024-03-26T13:45:52.717 INFO:teuthology.orchestra.run.smithi137.stdout:(32/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el 1.2 MB/s | 9.9 MB 00:08 2024-03-26T13:45:52.751 INFO:teuthology.orchestra.run.smithi137.stdout:(33/174): ceph-mgr-modules-core-19.0.0-2145.g3b 1.3 MB/s | 248 kB 00:00 2024-03-26T13:45:52.766 INFO:teuthology.orchestra.run.smithi179.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 4.2 MB/s | 248 kB 00:00 2024-03-26T13:45:52.801 INFO:teuthology.orchestra.run.smithi137.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 334 kB/s | 16 kB 00:00 2024-03-26T13:45:52.817 INFO:teuthology.orchestra.run.smithi179.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 1.0 MB/s | 49 kB 00:00 2024-03-26T13:45:52.826 INFO:teuthology.orchestra.run.smithi137.stdout:(35/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 458 kB/s | 49 kB 00:00 2024-03-26T13:45:52.875 INFO:teuthology.orchestra.run.smithi179.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 285 kB/s | 16 kB 00:00 2024-03-26T13:45:52.885 INFO:teuthology.orchestra.run.smithi137.stdout:(36/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 3.1 MB/s | 264 kB 00:00 2024-03-26T13:45:52.934 INFO:teuthology.orchestra.run.smithi179.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 4.5 MB/s | 264 kB 00:00 2024-03-26T13:45:52.935 INFO:teuthology.orchestra.run.smithi137.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 372 kB/s | 18 kB 00:00 2024-03-26T13:45:53.026 INFO:teuthology.orchestra.run.smithi179.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 8.0 MB/s | 748 kB 00:00 2024-03-26T13:45:53.077 INFO:teuthology.orchestra.run.smithi179.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 370 kB/s | 18 kB 00:00 2024-03-26T13:45:53.219 INFO:teuthology.orchestra.run.smithi137.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 662 kB 00:00 2024-03-26T13:45:53.327 INFO:teuthology.orchestra.run.smithi137.stdout:(39/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 1.5 MB/s | 748 kB 00:00 2024-03-26T13:45:53.369 INFO:teuthology.orchestra.run.smithi137.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 6.0 MB/s | 920 kB 00:00 2024-03-26T13:45:53.394 INFO:teuthology.orchestra.run.smithi137.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-26T13:45:53.428 INFO:teuthology.orchestra.run.smithi137.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-03-26T13:45:53.454 INFO:teuthology.orchestra.run.smithi137.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 571 kB/s | 32 kB 00:00 2024-03-26T13:45:53.460 INFO:teuthology.orchestra.run.smithi179.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.7 MB/s | 662 kB 00:00 2024-03-26T13:45:53.585 INFO:teuthology.orchestra.run.smithi179.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 920 kB 00:00 2024-03-26T13:45:53.621 INFO:teuthology.orchestra.run.smithi137.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-26T13:45:53.636 INFO:teuthology.orchestra.run.smithi179.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 106 kB 00:00 2024-03-26T13:45:53.646 INFO:teuthology.orchestra.run.smithi137.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 17 kB 00:00 2024-03-26T13:45:53.671 INFO:teuthology.orchestra.run.smithi137.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 204 kB/s | 10 kB 00:00 2024-03-26T13:45:53.696 INFO:teuthology.orchestra.run.smithi137.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-03-26T13:45:53.719 INFO:teuthology.orchestra.run.smithi179.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 3.6 MB/s | 307 kB 00:00 2024-03-26T13:45:53.722 INFO:teuthology.orchestra.run.smithi137.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 918 kB/s | 44 kB 00:00 2024-03-26T13:45:53.763 INFO:teuthology.orchestra.run.smithi137.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 661 kB 00:00 2024-03-26T13:45:53.770 INFO:teuthology.orchestra.run.smithi179.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 642 kB/s | 32 kB 00:00 2024-03-26T13:45:53.797 INFO:teuthology.orchestra.run.smithi137.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 8.0 MB/s | 578 kB 00:00 2024-03-26T13:45:53.823 INFO:teuthology.orchestra.run.smithi137.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 867 kB/s | 46 kB 00:00 2024-03-26T13:45:53.864 INFO:teuthology.orchestra.run.smithi137.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-26T13:45:53.898 INFO:teuthology.orchestra.run.smithi137.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-26T13:45:53.932 INFO:teuthology.orchestra.run.smithi137.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-26T13:45:53.957 INFO:teuthology.orchestra.run.smithi137.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 587 kB/s | 34 kB 00:00 2024-03-26T13:45:53.961 INFO:teuthology.orchestra.run.smithi179.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-03-26T13:45:54.029 INFO:teuthology.orchestra.run.smithi179.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 259 kB/s | 17 kB 00:00 2024-03-26T13:45:54.096 INFO:teuthology.orchestra.run.smithi179.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 153 kB/s | 10 kB 00:00 2024-03-26T13:45:54.146 INFO:teuthology.orchestra.run.smithi179.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-03-26T13:45:54.157 INFO:teuthology.orchestra.run.smithi137.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-03-26T13:45:54.189 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-26T13:45:54.196 INFO:teuthology.orchestra.run.smithi179.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 895 kB/s | 44 kB 00:00 2024-03-26T13:45:54.208 INFO:teuthology.orchestra.run.smithi137.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.1 MB/s | 1.0 MB 00:00 2024-03-26T13:45:54.233 INFO:teuthology.orchestra.run.smithi137.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-03-26T13:45:54.272 INFO:teuthology.orchestra.run.smithi179.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-26T13:45:54.414 INFO:teuthology.orchestra.run.smithi179.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 578 kB 00:00 2024-03-26T13:45:54.456 INFO:teuthology.orchestra.run.smithi179.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-03-26T13:45:54.466 INFO:teuthology.orchestra.run.smithi137.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-26T13:45:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 95 kB/s | 24 kB 00:00 2024-03-26T13:45:54.522 INFO:teuthology.orchestra.run.smithi179.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-26T13:45:54.533 INFO:teuthology.orchestra.run.smithi137.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 447 kB/s | 18 kB 00:00 2024-03-26T13:45:54.550 INFO:teuthology.orchestra.run.smithi137.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-26T13:45:54.573 INFO:teuthology.orchestra.run.smithi179.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-26T13:45:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 788 kB/s | 32 kB 00:00 2024-03-26T13:45:54.648 INFO:teuthology.orchestra.run.smithi179.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-03-26T13:45:54.690 INFO:teuthology.orchestra.run.smithi179.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 831 kB/s | 34 kB 00:00 2024-03-26T13:45:54.734 INFO:teuthology.orchestra.run.smithi137.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-26T13:45:54.768 INFO:teuthology.orchestra.run.smithi137.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-26T13:45:54.802 INFO:teuthology.orchestra.run.smithi137.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-03-26T13:45:54.827 INFO:teuthology.orchestra.run.smithi137.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-26T13:45:54.852 INFO:teuthology.orchestra.run.smithi137.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 41 kB 00:00 2024-03-26T13:45:54.882 INFO:teuthology.orchestra.run.smithi179.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-26T13:45:54.886 INFO:teuthology.orchestra.run.smithi137.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 4.1 MB/s | 238 kB 00:00 2024-03-26T13:45:54.911 INFO:teuthology.orchestra.run.smithi137.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-26T13:45:54.945 INFO:teuthology.orchestra.run.smithi137.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-26T13:45:54.957 INFO:teuthology.orchestra.run.smithi179.stdout:(56/174): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-03-26T13:45:54.987 INFO:teuthology.orchestra.run.smithi137.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 650 kB/s | 47 kB 00:00 2024-03-26T13:45:55.007 INFO:teuthology.orchestra.run.smithi179.stdout:(57/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 78 kB 00:00 2024-03-26T13:45:55.012 INFO:teuthology.orchestra.run.smithi137.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 650 kB/s | 43 kB 00:00 2024-03-26T13:45:55.350 INFO:teuthology.orchestra.run.smithi179.stdout:(58/174): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-03-26T13:45:55.392 INFO:teuthology.orchestra.run.smithi179.stdout:(59/174): python3-jsonpatch-1.21-16.el9.noarch. 602 kB/s | 24 kB 00:00 2024-03-26T13:45:55.433 INFO:teuthology.orchestra.run.smithi179.stdout:(60/174): python3-jsonpointer-2.0-4.el9.noarch. 443 kB/s | 18 kB 00:00 2024-03-26T13:45:55.476 INFO:teuthology.orchestra.run.smithi179.stdout:(61/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-26T13:45:55.487 INFO:teuthology.orchestra.run.smithi137.stdout:(74/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-26T13:45:55.509 INFO:teuthology.orchestra.run.smithi179.stdout:(62/174): python3-markupsafe-1.1.1-12.el9.x86_6 971 kB/s | 32 kB 00:00 2024-03-26T13:45:55.512 INFO:teuthology.orchestra.run.smithi137.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 80 kB/s | 39 kB 00:00 2024-03-26T13:45:55.554 INFO:teuthology.orchestra.run.smithi137.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 41 kB 00:00 2024-03-26T13:45:55.588 INFO:teuthology.orchestra.run.smithi137.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 300 kB 00:00 2024-03-26T13:45:55.613 INFO:teuthology.orchestra.run.smithi137.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-26T13:45:55.639 INFO:teuthology.orchestra.run.smithi137.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-03-26T13:45:55.664 INFO:teuthology.orchestra.run.smithi137.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-26T13:45:55.689 INFO:teuthology.orchestra.run.smithi137.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-26T13:45:55.714 INFO:teuthology.orchestra.run.smithi137.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.8 MB/s | 241 kB 00:00 2024-03-26T13:45:55.717 INFO:teuthology.orchestra.run.smithi179.stdout:(63/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-03-26T13:45:55.768 INFO:teuthology.orchestra.run.smithi179.stdout:(64/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2024-03-26T13:45:55.773 INFO:teuthology.orchestra.run.smithi137.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-03-26T13:45:55.798 INFO:teuthology.orchestra.run.smithi137.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-26T13:45:55.810 INFO:teuthology.orchestra.run.smithi179.stdout:(65/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-03-26T13:45:55.824 INFO:teuthology.orchestra.run.smithi137.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-26T13:45:55.857 INFO:teuthology.orchestra.run.smithi137.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-26T13:45:55.870 INFO:teuthology.orchestra.run.smithi179.stdout:(66/174): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 69 kB 00:00 2024-03-26T13:45:55.910 INFO:teuthology.orchestra.run.smithi179.stdout:(67/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-03-26T13:45:55.933 INFO:teuthology.orchestra.run.smithi137.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-03-26T13:45:55.961 INFO:teuthology.orchestra.run.smithi179.stdout:(68/174): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-03-26T13:45:55.999 INFO:teuthology.orchestra.run.smithi137.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 5.6 MB/s | 803 kB 00:00 2024-03-26T13:45:56.002 INFO:teuthology.orchestra.run.smithi179.stdout:(69/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2024-03-26T13:45:56.041 INFO:teuthology.orchestra.run.smithi137.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-03-26T13:45:56.053 INFO:teuthology.orchestra.run.smithi179.stdout:(70/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 211 kB 00:00 2024-03-26T13:45:56.100 INFO:teuthology.orchestra.run.smithi137.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-26T13:45:56.104 INFO:teuthology.orchestra.run.smithi179.stdout:(71/174): python3-pytz-2021.1-5.el9.noarch.rpm 956 kB/s | 47 kB 00:00 2024-03-26T13:45:56.125 INFO:teuthology.orchestra.run.smithi137.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 260 kB/s | 21 kB 00:00 2024-03-26T13:45:56.145 INFO:teuthology.orchestra.run.smithi179.stdout:(72/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-26T13:45:56.201 INFO:teuthology.orchestra.run.smithi137.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 256 kB/s | 25 kB 00:00 2024-03-26T13:45:56.454 INFO:teuthology.orchestra.run.smithi179.stdout:(73/174): ceph-mgr-diskprediction-local-19.0.0- 1.9 MB/s | 7.4 MB 00:03 2024-03-26T13:45:56.659 INFO:teuthology.orchestra.run.smithi137.stdout:(93/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 7.8 MB/s | 4.1 MB 00:00 2024-03-26T13:45:56.751 INFO:teuthology.orchestra.run.smithi137.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 9.8 MB/s | 911 kB 00:00 2024-03-26T13:45:56.771 INFO:teuthology.orchestra.run.smithi179.stdout:(74/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-03-26T13:45:56.797 INFO:teuthology.orchestra.run.smithi179.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 118 kB/s | 39 kB 00:00 2024-03-26T13:45:56.801 INFO:teuthology.orchestra.run.smithi137.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 2.7 MB/s | 133 kB 00:00 2024-03-26T13:45:56.822 INFO:teuthology.orchestra.run.smithi179.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 833 kB/s | 41 kB 00:00 2024-03-26T13:45:56.842 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T13:45:56.844 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-26T13:45:56.855 INFO:teuthology.orchestra.run.smithi179.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-03-26T13:45:56.866 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-26T13:45:56.868 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-26T13:45:56.869 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-26T13:45:56.870 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-26T13:45:56.873 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-26T13:45:56.876 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-26T13:45:56.878 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-26T13:45:56.881 INFO:teuthology.orchestra.run.smithi179.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-26T13:45:56.906 INFO:teuthology.orchestra.run.smithi179.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-26T13:45:56.927 INFO:teuthology.orchestra.run.smithi137.stdout:(96/174): postgresql-13.14-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-03-26T13:45:56.931 INFO:teuthology.orchestra.run.smithi179.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-26T13:45:56.967 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-03-26T13:45:56.982 INFO:teuthology.orchestra.run.smithi179.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-26T13:45:56.984 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-26T13:45:56.984 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-03-26T13:45:56.984 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-26T13:45:56.984 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 6.6 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.4 M 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 890 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 145 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 1.5 M 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 139 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 1.8 M 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 7.4 M 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 49 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 9.9 M 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 48 M 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 264 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 748 k 2024-03-26T13:45:56.985 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 220 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 27 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 18 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 31 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 746 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 127 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-26T13:45:56.986 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 162 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 321 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 298 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 99 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 85 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 170 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-26T13:45:56.987 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.4 M 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 3.1 M 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-26T13:45:56.988 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 20 M 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 34 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 2.3 M 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 248 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 4.7 M 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 17 M 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2145.g3be9a5c9.el9 ceph-noarch 16 k 2024-03-26T13:45:56.989 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 25 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 163 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-26T13:45:56.990 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 501 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 5.1 M 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-26T13:45:56.991 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 45 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.0.0-2145.g3be9a5c9.el9 ceph 131 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-26T13:45:56.992 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-26T13:45:56.993 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-26T13:45:56.994 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-26T13:45:56.995 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-26T13:45:56.996 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:============================================================================================ 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:Install 150 Packages 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 24 Packages 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 225 M 2024-03-26T13:45:56.997 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-03-26T13:45:57.015 INFO:teuthology.orchestra.run.smithi179.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-03-26T13:45:57.019 INFO:teuthology.orchestra.run.smithi137.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-03-26T13:45:57.074 INFO:teuthology.orchestra.run.smithi179.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2024-03-26T13:45:57.107 INFO:teuthology.orchestra.run.smithi179.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-26T13:45:57.119 INFO:teuthology.orchestra.run.smithi137.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-03-26T13:45:57.132 INFO:teuthology.orchestra.run.smithi179.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.1 MB/s | 124 kB 00:00 2024-03-26T13:45:57.166 INFO:teuthology.orchestra.run.smithi179.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-26T13:45:57.361 INFO:teuthology.orchestra.run.smithi137.stdout:(99/174): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-03-26T13:45:57.394 INFO:teuthology.orchestra.run.smithi137.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 804 kB/s | 221 kB 00:00 2024-03-26T13:45:57.420 INFO:teuthology.orchestra.run.smithi137.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 621 kB/s | 36 kB 00:00 2024-03-26T13:45:57.476 INFO:teuthology.orchestra.run.smithi137.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 593 kB/s | 47 kB 00:00 2024-03-26T13:45:57.512 INFO:teuthology.orchestra.run.smithi137.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.0 MB/s | 94 kB 00:00 2024-03-26T13:45:57.571 INFO:teuthology.orchestra.run.smithi137.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 237 kB/s | 22 kB 00:00 2024-03-26T13:45:57.696 INFO:teuthology.orchestra.run.smithi137.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-03-26T13:45:57.974 INFO:teuthology.orchestra.run.smithi159.stdout:(1/174): ceph-19.0.0-2145.g3be9a5c9.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-03-26T13:45:58.021 INFO:teuthology.orchestra.run.smithi137.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-03-26T13:45:58.091 INFO:teuthology.orchestra.run.smithi179.stdout:(87/174): jq-1.6-16.el9.x86_64.rpm 196 kB/s | 187 kB 00:00 2024-03-26T13:45:58.097 INFO:teuthology.orchestra.run.smithi137.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 773 kB/s | 309 kB 00:00 2024-03-26T13:45:58.122 INFO:teuthology.orchestra.run.smithi137.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 195 kB/s | 19 kB 00:00 2024-03-26T13:45:58.222 INFO:teuthology.orchestra.run.smithi137.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-03-26T13:45:58.314 INFO:teuthology.orchestra.run.smithi137.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 534 kB/s | 49 kB 00:00 2024-03-26T13:45:58.316 INFO:teuthology.orchestra.run.smithi159.stdout:(2/174): ceph-fuse-19.0.0-2145.g3be9a5c9.el9.x8 2.6 MB/s | 890 kB 00:00 2024-03-26T13:45:58.416 INFO:teuthology.orchestra.run.smithi159.stdout:(3/174): ceph-immutable-object-cache-19.0.0-214 1.4 MB/s | 145 kB 00:00 2024-03-26T13:45:58.465 INFO:teuthology.orchestra.run.smithi137.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-26T13:45:58.490 INFO:teuthology.orchestra.run.smithi137.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 386 kB/s | 67 kB 00:00 2024-03-26T13:45:58.611 INFO:teuthology.orchestra.run.smithi137.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-26T13:45:58.700 INFO:teuthology.orchestra.run.smithi159.stdout:(4/174): ceph-base-19.0.0-2145.g3be9a5c9.el9.x8 5.4 MB/s | 5.4 MB 00:00 2024-03-26T13:45:58.740 INFO:teuthology.orchestra.run.smithi137.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 4.2 MB/s | 548 kB 00:00 2024-03-26T13:45:58.840 INFO:teuthology.orchestra.run.smithi137.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-03-26T13:45:58.866 INFO:teuthology.orchestra.run.smithi137.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-03-26T13:45:58.900 INFO:teuthology.orchestra.run.smithi179.stdout:(88/174): libgfortran-11.4.1-3.el9.x86_64.rpm 463 kB/s | 803 kB 00:01 2024-03-26T13:45:58.916 INFO:teuthology.orchestra.run.smithi137.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2024-03-26T13:45:58.950 INFO:teuthology.orchestra.run.smithi159.stdout:(5/174): ceph-mgr-19.0.0-2145.g3be9a5c9.el9.x86 6.0 MB/s | 1.5 MB 00:00 2024-03-26T13:45:58.974 INFO:teuthology.orchestra.run.smithi137.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-03-26T13:45:59.016 INFO:teuthology.orchestra.run.smithi137.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-26T13:45:59.042 INFO:teuthology.orchestra.run.smithi159.stdout:(6/174): ceph-mds-19.0.0-2145.g3be9a5c9.el9.x86 3.7 MB/s | 2.3 MB 00:00 2024-03-26T13:45:59.042 INFO:teuthology.orchestra.run.smithi179.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 206 kB/s | 195 kB 00:00 2024-03-26T13:45:59.092 INFO:teuthology.orchestra.run.smithi137.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-03-26T13:45:59.175 INFO:teuthology.orchestra.run.smithi137.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-03-26T13:45:59.192 INFO:teuthology.orchestra.run.smithi137.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 355 kB/s | 35 kB 00:00 2024-03-26T13:45:59.261 INFO:teuthology.orchestra.run.smithi137.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-03-26T13:45:59.342 INFO:teuthology.orchestra.run.smithi179.stdout:(90/174): elfutils-libelf-devel-0.190-2.el9.x86 72 kB/s | 21 kB 00:00 2024-03-26T13:45:59.368 INFO:teuthology.orchestra.run.smithi137.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-26T13:45:59.401 INFO:teuthology.orchestra.run.smithi137.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-03-26T13:45:59.451 INFO:teuthology.orchestra.run.smithi179.stdout:(91/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 236 kB/s | 25 kB 00:00 2024-03-26T13:45:59.452 INFO:teuthology.orchestra.run.smithi137.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2024-03-26T13:45:59.485 INFO:teuthology.orchestra.run.smithi137.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 213 kB/s | 18 kB 00:00 2024-03-26T13:45:59.536 INFO:teuthology.orchestra.run.smithi137.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 279 kB/s | 23 kB 00:00 2024-03-26T13:45:59.569 INFO:teuthology.orchestra.run.smithi137.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2024-03-26T13:45:59.620 INFO:teuthology.orchestra.run.smithi137.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-03-26T13:45:59.651 INFO:teuthology.orchestra.run.smithi159.stdout:(7/174): ceph-mon-19.0.0-2145.g3be9a5c9.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-03-26T13:45:59.728 INFO:teuthology.orchestra.run.smithi137.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 430 kB/s | 46 kB 00:00 2024-03-26T13:45:59.759 INFO:teuthology.orchestra.run.smithi179.stdout:(92/174): python3-requests-2.25.1-8.el9.noarch. 146 kB/s | 125 kB 00:00 2024-03-26T13:45:59.820 INFO:teuthology.orchestra.run.smithi137.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 868 kB/s | 79 kB 00:00 2024-03-26T13:45:59.912 INFO:teuthology.orchestra.run.smithi137.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-03-26T13:45:59.938 INFO:teuthology.orchestra.run.smithi137.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 497 kB/s | 58 kB 00:00 2024-03-26T13:46:00.021 INFO:teuthology.orchestra.run.smithi137.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 199 kB/s | 16 kB 00:00 2024-03-26T13:46:00.055 INFO:teuthology.orchestra.run.smithi137.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-03-26T13:46:00.105 INFO:teuthology.orchestra.run.smithi137.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-26T13:46:00.138 INFO:teuthology.orchestra.run.smithi137.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-03-26T13:46:00.205 INFO:teuthology.orchestra.run.smithi137.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-03-26T13:46:00.247 INFO:teuthology.orchestra.run.smithi137.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-26T13:46:00.298 INFO:teuthology.orchestra.run.smithi137.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 647 kB/s | 59 kB 00:00 2024-03-26T13:46:00.326 INFO:teuthology.orchestra.run.smithi137.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 458 kB/s | 36 kB 00:00 2024-03-26T13:46:00.360 INFO:teuthology.orchestra.run.smithi179.stdout:(93/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 4.6 MB/s | 4.1 MB 00:00 2024-03-26T13:46:00.376 INFO:teuthology.orchestra.run.smithi137.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 653 kB/s | 51 kB 00:00 2024-03-26T13:46:00.427 INFO:teuthology.orchestra.run.smithi137.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-03-26T13:46:00.477 INFO:teuthology.orchestra.run.smithi137.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 897 kB/s | 90 kB 00:00 2024-03-26T13:46:00.553 INFO:teuthology.orchestra.run.smithi137.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-03-26T13:46:00.595 INFO:teuthology.orchestra.run.smithi137.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-03-26T13:46:00.637 INFO:teuthology.orchestra.run.smithi137.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-03-26T13:46:00.760 INFO:teuthology.orchestra.run.smithi179.stdout:(94/174): postgresql-13.14-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-03-26T13:46:00.868 INFO:teuthology.orchestra.run.smithi179.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-03-26T13:46:00.945 INFO:teuthology.orchestra.run.smithi137.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-03-26T13:46:01.167 INFO:teuthology.orchestra.run.smithi159.stdout:(8/174): ceph-osd-19.0.0-2145.g3be9a5c9.el9.x86 7.9 MB/s | 17 MB 00:02 2024-03-26T13:46:01.368 INFO:teuthology.orchestra.run.smithi159.stdout:(9/174): ceph-radosgw-19.0.0-2145.g3be9a5c9.el9 5.7 MB/s | 9.9 MB 00:01 2024-03-26T13:46:01.384 INFO:teuthology.orchestra.run.smithi159.stdout:(10/174): ceph-selinux-19.0.0-2145.g3be9a5c9.el 116 kB/s | 25 kB 00:00 2024-03-26T13:46:01.443 INFO:teuthology.orchestra.run.smithi159.stdout:(11/174): cephfs-mirror-19.0.0-2145.g3be9a5c9.e 3.7 MB/s | 220 kB 00:00 2024-03-26T13:46:01.462 INFO:teuthology.orchestra.run.smithi137.stdout:(150/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 6.1 MB/s | 3.1 MB 00:00 2024-03-26T13:46:01.494 INFO:teuthology.orchestra.run.smithi159.stdout:(12/174): cephfs-shell-19.0.0-2145.g3be9a5c9.el 551 kB/s | 27 kB 00:00 2024-03-26T13:46:01.512 INFO:teuthology.orchestra.run.smithi137.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.4 MB/s | 321 kB 00:00 2024-03-26T13:46:01.577 INFO:teuthology.orchestra.run.smithi179.stdout:(96/174): postgresql-contrib-13.14-1.el9.x86_64 749 kB/s | 911 kB 00:01 2024-03-26T13:46:01.811 INFO:teuthology.orchestra.run.smithi159.stdout:(13/174): ceph-common-19.0.0-2145.g3be9a5c9.el9 5.0 MB/s | 20 MB 00:04 2024-03-26T13:46:01.827 INFO:teuthology.orchestra.run.smithi159.stdout:(14/174): libcephfs-devel-19.0.0-2145.g3be9a5c9 94 kB/s | 31 kB 00:00 2024-03-26T13:46:01.869 INFO:teuthology.orchestra.run.smithi179.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 844 kB/s | 245 kB 00:00 2024-03-26T13:46:01.877 INFO:teuthology.orchestra.run.smithi159.stdout:(15/174): libcephsqlite-19.0.0-2145.g3be9a5c9.e 3.2 MB/s | 163 kB 00:00 2024-03-26T13:46:01.953 INFO:teuthology.orchestra.run.smithi159.stdout:(16/174): libcephfs2-19.0.0-2145.g3be9a5c9.el9. 5.2 MB/s | 746 kB 00:00 2024-03-26T13:46:01.963 INFO:teuthology.orchestra.run.smithi137.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 5.1 MB/s | 2.3 MB 00:00 2024-03-26T13:46:01.978 INFO:teuthology.orchestra.run.smithi159.stdout:(17/174): librados-devel-19.0.0-2145.g3be9a5c9. 1.2 MB/s | 127 kB 00:00 2024-03-26T13:46:02.062 INFO:teuthology.orchestra.run.smithi159.stdout:(18/174): libradosstriper1-19.0.0-2145.g3be9a5c 4.5 MB/s | 501 kB 00:00 2024-03-26T13:46:02.102 INFO:teuthology.orchestra.run.smithi179.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-03-26T13:46:02.145 INFO:teuthology.orchestra.run.smithi159.stdout:(19/174): python3-ceph-argparse-19.0.0-2145.g3b 547 kB/s | 45 kB 00:00 2024-03-26T13:46:02.196 INFO:teuthology.orchestra.run.smithi137.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 7.9 MB/s | 1.8 MB 00:00 2024-03-26T13:46:02.246 INFO:teuthology.orchestra.run.smithi159.stdout:(20/174): python3-ceph-common-19.0.0-2145.g3be9 1.3 MB/s | 131 kB 00:00 2024-03-26T13:46:02.263 INFO:teuthology.orchestra.run.smithi137.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 641 kB/s | 42 kB 00:00 2024-03-26T13:46:02.338 INFO:teuthology.orchestra.run.smithi159.stdout:(21/174): python3-cephfs-19.0.0-2145.g3be9a5c9. 1.7 MB/s | 162 kB 00:00 2024-03-26T13:46:02.361 INFO:teuthology.orchestra.run.smithi179.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 856 kB/s | 221 kB 00:00 2024-03-26T13:46:02.446 INFO:teuthology.orchestra.run.smithi159.stdout:(22/174): python3-rados-19.0.0-2145.g3be9a5c9.e 2.9 MB/s | 321 kB 00:00 2024-03-26T13:46:02.503 INFO:teuthology.orchestra.run.smithi179.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 254 kB/s | 36 kB 00:00 2024-03-26T13:46:02.572 INFO:teuthology.orchestra.run.smithi137.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 807 kB/s | 248 kB 00:00 2024-03-26T13:46:02.605 INFO:teuthology.orchestra.run.smithi159.stdout:(23/174): librgw2-19.0.0-2145.g3be9a5c9.el9.x86 8.2 MB/s | 5.1 MB 00:00 2024-03-26T13:46:02.647 INFO:teuthology.orchestra.run.smithi159.stdout:(24/174): python3-rbd-19.0.0-2145.g3be9a5c9.el9 1.5 MB/s | 298 kB 00:00 2024-03-26T13:46:02.653 INFO:teuthology.orchestra.run.smithi179.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 317 kB/s | 47 kB 00:00 2024-03-26T13:46:02.655 INFO:teuthology.orchestra.run.smithi137.stdout:(156/174): librados2-19.0.0-2145.g3be9a5c9.el9. 1.7 MB/s | 3.4 MB 00:02 2024-03-26T13:46:02.680 INFO:teuthology.orchestra.run.smithi159.stdout:(25/174): python3-rgw-19.0.0-2145.g3be9a5c9.el9 1.3 MB/s | 99 kB 00:00 2024-03-26T13:46:02.706 INFO:teuthology.orchestra.run.smithi137.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 743 kB/s | 36 kB 00:00 2024-03-26T13:46:02.706 INFO:teuthology.orchestra.run.smithi159.stdout:(26/174): rbd-fuse-19.0.0-2145.g3be9a5c9.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-26T13:46:02.731 INFO:teuthology.orchestra.run.smithi137.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 451 kB/s | 71 kB 00:00 2024-03-26T13:46:02.795 INFO:teuthology.orchestra.run.smithi179.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 666 kB/s | 94 kB 00:00 2024-03-26T13:46:02.798 INFO:teuthology.orchestra.run.smithi159.stdout:(27/174): rbd-nbd-19.0.0-2145.g3be9a5c9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-03-26T13:46:02.831 INFO:teuthology.orchestra.run.smithi137.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.5 MB/s | 192 kB 00:00 2024-03-26T13:46:02.873 INFO:teuthology.orchestra.run.smithi137.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.8 MB/s | 256 kB 00:00 2024-03-26T13:46:02.906 INFO:teuthology.orchestra.run.smithi159.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2145.g 316 kB/s | 34 kB 00:00 2024-03-26T13:46:02.921 INFO:teuthology.orchestra.run.smithi179.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-03-26T13:46:02.932 INFO:teuthology.orchestra.run.smithi137.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 232 kB/s | 23 kB 00:00 2024-03-26T13:46:02.974 INFO:teuthology.orchestra.run.smithi137.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-26T13:46:03.008 INFO:teuthology.orchestra.run.smithi137.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-26T13:46:03.023 INFO:teuthology.orchestra.run.smithi159.stdout:(29/174): rbd-mirror-19.0.0-2145.g3be9a5c9.el9. 9.0 MB/s | 3.1 MB 00:00 2024-03-26T13:46:03.057 INFO:teuthology.orchestra.run.smithi159.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2145.g3be9a5c 929 kB/s | 139 kB 00:00 2024-03-26T13:46:03.066 INFO:teuthology.orchestra.run.smithi137.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.1 MB/s | 64 kB 00:00 2024-03-26T13:46:03.104 INFO:teuthology.orchestra.run.smithi179.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-03-26T13:46:03.108 INFO:teuthology.orchestra.run.smithi137.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 136 kB 00:00 2024-03-26T13:46:03.150 INFO:teuthology.orchestra.run.smithi137.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 347 kB/s | 29 kB 00:00 2024-03-26T13:46:03.249 INFO:teuthology.orchestra.run.smithi159.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2145.g3be9a 8.0 MB/s | 1.8 MB 00:00 2024-03-26T13:46:03.292 INFO:teuthology.orchestra.run.smithi137.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 6.6 MB/s | 1.2 MB 00:00 2024-03-26T13:46:03.324 INFO:teuthology.orchestra.run.smithi159.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2145.g3b 3.3 MB/s | 248 kB 00:00 2024-03-26T13:46:03.399 INFO:teuthology.orchestra.run.smithi159.stdout:(33/174): ceph-mgr-rook-19.0.0-2145.g3be9a5c9.e 667 kB/s | 49 kB 00:00 2024-03-26T13:46:03.434 INFO:teuthology.orchestra.run.smithi137.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 7.6 MB/s | 2.2 MB 00:00 2024-03-26T13:46:03.438 INFO:teuthology.orchestra.run.smithi179.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-03-26T13:46:03.466 INFO:teuthology.orchestra.run.smithi159.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2145.g3 250 kB/s | 16 kB 00:00 2024-03-26T13:46:03.530 INFO:teuthology.orchestra.run.smithi179.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-03-26T13:46:03.534 INFO:teuthology.orchestra.run.smithi137.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.6 MB/s | 465 kB 00:00 2024-03-26T13:46:03.550 INFO:teuthology.orchestra.run.smithi159.stdout:(35/174): ceph-volume-19.0.0-2145.g3be9a5c9.el9 3.1 MB/s | 264 kB 00:00 2024-03-26T13:46:03.601 INFO:teuthology.orchestra.run.smithi137.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 7.4 MB/s | 2.3 MB 00:00 2024-03-26T13:46:03.626 INFO:teuthology.orchestra.run.smithi137.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 341 kB/s | 31 kB 00:00 2024-03-26T13:46:03.680 INFO:teuthology.orchestra.run.smithi179.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-03-26T13:46:03.684 INFO:teuthology.orchestra.run.smithi159.stdout:(36/174): cephadm-19.0.0-2145.g3be9a5c9.el9.noa 5.5 MB/s | 748 kB 00:00 2024-03-26T13:46:03.751 INFO:teuthology.orchestra.run.smithi159.stdout:(37/174): cephfs-top-19.0.0-2145.g3be9a5c9.el9. 277 kB/s | 18 kB 00:00 2024-03-26T13:46:03.802 INFO:teuthology.orchestra.run.smithi137.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.4 MB/s | 1.1 MB 00:00 2024-03-26T13:46:03.827 INFO:teuthology.orchestra.run.smithi137.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 373 kB/s | 74 kB 00:00 2024-03-26T13:46:03.956 INFO:teuthology.orchestra.run.smithi179.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-26T13:46:04.014 INFO:teuthology.orchestra.run.smithi179.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 453 kB/s | 26 kB 00:00 2024-03-26T13:46:04.018 INFO:teuthology.orchestra.run.smithi159.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-26T13:46:04.081 INFO:teuthology.orchestra.run.smithi179.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 738 kB/s | 49 kB 00:00 2024-03-26T13:46:04.093 INFO:teuthology.orchestra.run.smithi159.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2024-03-26T13:46:04.215 INFO:teuthology.orchestra.run.smithi179.stdout:(111/174): postgresql-server-13.14-1.el9.x86_64 1.7 MB/s | 5.8 MB 00:03 2024-03-26T13:46:04.240 INFO:teuthology.orchestra.run.smithi179.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 426 kB/s | 67 kB 00:00 2024-03-26T13:46:04.243 INFO:teuthology.orchestra.run.smithi159.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-03-26T13:46:04.261 INFO:teuthology.orchestra.run.smithi159.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 636 kB/s | 106 kB 00:00 2024-03-26T13:46:04.294 INFO:teuthology.orchestra.run.smithi159.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 307 kB 00:00 2024-03-26T13:46:04.311 INFO:teuthology.orchestra.run.smithi159.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 682 kB/s | 32 kB 00:00 2024-03-26T13:46:04.315 INFO:teuthology.orchestra.run.smithi179.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-03-26T13:46:04.391 INFO:teuthology.orchestra.run.smithi179.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 838 kB 00:00 2024-03-26T13:46:04.437 INFO:teuthology.orchestra.run.smithi159.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-03-26T13:46:04.441 INFO:teuthology.orchestra.run.smithi179.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-03-26T13:46:04.461 INFO:teuthology.orchestra.run.smithi159.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-03-26T13:46:04.466 INFO:teuthology.orchestra.run.smithi179.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 626 kB/s | 43 kB 00:00 2024-03-26T13:46:04.495 INFO:teuthology.orchestra.run.smithi159.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2024-03-26T13:46:04.496 INFO:teuthology.orchestra.run.smithi179.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 601 kB/s | 32 kB 00:00 2024-03-26T13:46:04.502 INFO:teuthology.orchestra.run.smithi137.stdout:(174/174): ceph-test-19.0.0-2145.g3be9a5c9.el9. 2.5 MB/s | 48 MB 00:19 2024-03-26T13:46:04.512 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:46:04.512 INFO:teuthology.orchestra.run.smithi137.stdout:Total 8.8 MB/s | 225 MB 00:25 2024-03-26T13:46:04.520 INFO:teuthology.orchestra.run.smithi159.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-26T13:46:04.546 INFO:teuthology.orchestra.run.smithi159.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 915 kB/s | 44 kB 00:00 2024-03-26T13:46:04.580 INFO:teuthology.orchestra.run.smithi179.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 133 kB/s | 15 kB 00:00 2024-03-26T13:46:04.596 INFO:teuthology.orchestra.run.smithi159.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-26T13:46:04.605 INFO:teuthology.orchestra.run.smithi179.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-03-26T13:46:04.638 INFO:teuthology.orchestra.run.smithi159.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-03-26T13:46:04.655 INFO:teuthology.orchestra.run.smithi179.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 4.8 MB/s | 358 kB 00:00 2024-03-26T13:46:04.663 INFO:teuthology.orchestra.run.smithi159.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 756 kB/s | 46 kB 00:00 2024-03-26T13:46:04.689 INFO:teuthology.orchestra.run.smithi179.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-03-26T13:46:04.713 INFO:teuthology.orchestra.run.smithi159.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-26T13:46:04.715 INFO:teuthology.orchestra.run.smithi179.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 631 kB/s | 35 kB 00:00 2024-03-26T13:46:04.747 INFO:teuthology.orchestra.run.smithi159.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2024-03-26T13:46:04.748 INFO:teuthology.orchestra.run.smithi179.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-26T13:46:04.797 INFO:teuthology.orchestra.run.smithi159.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-03-26T13:46:04.857 INFO:teuthology.orchestra.run.smithi159.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 314 kB/s | 34 kB 00:00 2024-03-26T13:46:04.882 INFO:teuthology.orchestra.run.smithi179.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-26T13:46:04.957 INFO:teuthology.orchestra.run.smithi179.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 695 kB/s | 144 kB 00:00 2024-03-26T13:46:05.016 INFO:teuthology.orchestra.run.smithi179.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-03-26T13:46:05.031 INFO:teuthology.orchestra.run.smithi159.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-03-26T13:46:05.041 INFO:teuthology.orchestra.run.smithi179.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-03-26T13:46:05.073 INFO:teuthology.orchestra.run.smithi159.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-03-26T13:46:05.075 INFO:teuthology.orchestra.run.smithi179.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 402 kB/s | 23 kB 00:00 2024-03-26T13:46:05.098 INFO:teuthology.orchestra.run.smithi159.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-26T13:46:05.100 INFO:teuthology.orchestra.run.smithi179.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 335 kB/s | 19 kB 00:00 2024-03-26T13:46:05.134 INFO:teuthology.orchestra.run.smithi179.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 339 kB/s | 20 kB 00:00 2024-03-26T13:46:05.376 INFO:teuthology.orchestra.run.smithi179.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-03-26T13:46:05.391 INFO:teuthology.orchestra.run.smithi159.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-03-26T13:46:05.401 INFO:teuthology.orchestra.run.smithi179.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 174 kB/s | 46 kB 00:00 2024-03-26T13:46:05.416 INFO:teuthology.orchestra.run.smithi159.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 77 kB/s | 24 kB 00:00 2024-03-26T13:46:05.433 INFO:teuthology.orchestra.run.smithi159.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 443 kB/s | 18 kB 00:00 2024-03-26T13:46:05.451 INFO:teuthology.orchestra.run.smithi179.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-03-26T13:46:05.476 INFO:teuthology.orchestra.run.smithi179.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 774 kB/s | 58 kB 00:00 2024-03-26T13:46:05.483 INFO:teuthology.orchestra.run.smithi159.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-03-26T13:46:05.500 INFO:teuthology.orchestra.run.smithi159.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 476 kB/s | 32 kB 00:00 2024-03-26T13:46:05.535 INFO:teuthology.orchestra.run.smithi179.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-26T13:46:05.560 INFO:teuthology.orchestra.run.smithi179.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-03-26T13:46:05.602 INFO:teuthology.orchestra.run.smithi179.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-03-26T13:46:05.674 INFO:teuthology.orchestra.run.smithi179.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 251 kB/s | 28 kB 00:00 2024-03-26T13:46:05.694 INFO:teuthology.orchestra.run.smithi179.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 337 kB/s | 31 kB 00:00 2024-03-26T13:46:05.836 INFO:teuthology.orchestra.run.smithi179.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-03-26T13:46:05.853 INFO:teuthology.orchestra.run.smithi179.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 374 kB/s | 59 kB 00:00 2024-03-26T13:46:05.895 INFO:teuthology.orchestra.run.smithi179.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 626 kB/s | 36 kB 00:00 2024-03-26T13:46:05.921 INFO:teuthology.orchestra.run.smithi179.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 770 kB/s | 51 kB 00:00 2024-03-26T13:46:06.012 INFO:teuthology.orchestra.run.smithi179.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-03-26T13:46:06.038 INFO:teuthology.orchestra.run.smithi179.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 771 kB/s | 90 kB 00:00 2024-03-26T13:46:06.088 INFO:teuthology.orchestra.run.smithi179.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.7 MB/s | 427 kB 00:00 2024-03-26T13:46:06.113 INFO:teuthology.orchestra.run.smithi179.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 269 kB/s | 20 kB 00:00 2024-03-26T13:46:06.123 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-03-26T13:46:06.201 INFO:teuthology.orchestra.run.smithi159.stdout:(64/174): ceph-test-19.0.0-2145.g3be9a5c9.el9.x 9.9 MB/s | 48 MB 00:04 2024-03-26T13:46:06.214 INFO:teuthology.orchestra.run.smithi179.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-03-26T13:46:06.309 INFO:teuthology.orchestra.run.smithi159.stdout:(65/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.8 MB/s | 4.8 MB 00:00 2024-03-26T13:46:06.326 INFO:teuthology.orchestra.run.smithi159.stdout:(66/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 264 kB/s | 217 kB 00:00 2024-03-26T13:46:06.359 INFO:teuthology.orchestra.run.smithi159.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2024-03-26T13:46:06.418 INFO:teuthology.orchestra.run.smithi159.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 462 kB/s | 41 kB 00:00 2024-03-26T13:46:06.443 INFO:teuthology.orchestra.run.smithi159.stdout:(69/174): python3-packaging-20.9-5.el9.noarch.r 599 kB/s | 69 kB 00:00 2024-03-26T13:46:06.464 INFO:teuthology.orchestra.run.smithi179.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-03-26T13:46:06.468 INFO:teuthology.orchestra.run.smithi159.stdout:(70/174): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 238 kB 00:00 2024-03-26T13:46:06.485 INFO:teuthology.orchestra.run.smithi159.stdout:(71/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-26T13:46:06.504 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-03-26T13:46:06.504 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-03-26T13:46:06.527 INFO:teuthology.orchestra.run.smithi159.stdout:(72/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 211 kB 00:00 2024-03-26T13:46:06.553 INFO:teuthology.orchestra.run.smithi159.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 581 kB/s | 47 kB 00:00 2024-03-26T13:46:06.578 INFO:teuthology.orchestra.run.smithi159.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 468 kB/s | 43 kB 00:00 2024-03-26T13:46:06.789 INFO:teuthology.orchestra.run.smithi179.stdout:(150/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-03-26T13:46:06.881 INFO:teuthology.orchestra.run.smithi179.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2024-03-26T13:46:06.953 INFO:teuthology.orchestra.run.smithi159.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 107 kB/s | 39 kB 00:00 2024-03-26T13:46:06.978 INFO:teuthology.orchestra.run.smithi159.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 106 kB/s | 41 kB 00:00 2024-03-26T13:46:07.279 INFO:teuthology.orchestra.run.smithi159.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-03-26T13:46:07.312 INFO:teuthology.orchestra.run.smithi159.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 176 kB/s | 56 kB 00:00 2024-03-26T13:46:07.346 INFO:teuthology.orchestra.run.smithi159.stdout:(79/174): socat-1.7.4.1-5.el9.x86_64.rpm 849 kB/s | 300 kB 00:00 2024-03-26T13:46:07.371 INFO:teuthology.orchestra.run.smithi159.stdout:(80/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 688 kB/s | 63 kB 00:00 2024-03-26T13:46:07.396 INFO:teuthology.orchestra.run.smithi159.stdout:(81/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 551 kB/s | 44 kB 00:00 2024-03-26T13:46:07.438 INFO:teuthology.orchestra.run.smithi159.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2024-03-26T13:46:07.472 INFO:teuthology.orchestra.run.smithi159.stdout:(83/174): fuse-2.9.9-15.el9.x86_64.rpm 794 kB/s | 79 kB 00:00 2024-03-26T13:46:07.522 INFO:teuthology.orchestra.run.smithi159.stdout:(84/174): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.1 MB 00:00 2024-03-26T13:46:07.547 INFO:teuthology.orchestra.run.smithi159.stdout:(85/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-26T13:46:07.573 INFO:teuthology.orchestra.run.smithi159.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-26T13:46:07.606 INFO:teuthology.orchestra.run.smithi159.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 187 kB 00:00 2024-03-26T13:46:07.890 INFO:teuthology.orchestra.run.smithi159.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 596 kB/s | 187 kB 00:00 2024-03-26T13:46:07.948 INFO:teuthology.orchestra.run.smithi159.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 573 kB/s | 195 kB 00:00 2024-03-26T13:46:07.982 INFO:teuthology.orchestra.run.smithi159.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-03-26T13:46:07.998 INFO:teuthology.orchestra.run.smithi179.stdout:(152/174): librados2-19.0.0-2145.g3be9a5c9.el9. 1.9 MB/s | 3.4 MB 00:01 2024-03-26T13:46:08.107 INFO:teuthology.orchestra.run.smithi159.stdout:(91/174): elfutils-libelf-devel-0.190-2.el9.x86 136 kB/s | 21 kB 00:00 2024-03-26T13:46:08.141 INFO:teuthology.orchestra.run.smithi159.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 803 kB 00:00 2024-03-26T13:46:08.166 INFO:teuthology.orchestra.run.smithi159.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 139 kB/s | 25 kB 00:00 2024-03-26T13:46:08.850 INFO:teuthology.orchestra.run.smithi159.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 1.3 MB/s | 911 kB 00:00 2024-03-26T13:46:08.877 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-03-26T13:46:08.877 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-03-26T13:46:08.958 INFO:teuthology.orchestra.run.smithi159.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-03-26T13:46:08.984 INFO:teuthology.orchestra.run.smithi159.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-03-26T13:46:09.201 INFO:teuthology.orchestra.run.smithi159.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-26T13:46:09.368 INFO:teuthology.orchestra.run.smithi159.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-03-26T13:46:09.509 INFO:teuthology.orchestra.run.smithi159.stdout:(99/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-03-26T13:46:09.587 INFO:teuthology.orchestra.run.smithi159.stdout:(100/174): qatlib-service-24.02.0-1.el9.x86_64. 471 kB/s | 36 kB 00:00 2024-03-26T13:46:09.649 INFO:teuthology.orchestra.run.smithi179.stdout:(153/174): ceph-test-19.0.0-2145.g3be9a5c9.el9. 2.5 MB/s | 48 MB 00:18 2024-03-26T13:46:09.652 INFO:teuthology.orchestra.run.smithi159.stdout:(101/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 742 kB/s | 47 kB 00:00 2024-03-26T13:46:09.752 INFO:teuthology.orchestra.run.smithi159.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 946 kB/s | 94 kB 00:00 2024-03-26T13:46:10.027 INFO:teuthology.orchestra.run.smithi159.stdout:(103/174): postgresql-server-13.14-1.el9.x86_64 5.4 MB/s | 5.8 MB 00:01 2024-03-26T13:46:10.052 INFO:teuthology.orchestra.run.smithi159.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 75 kB/s | 22 kB 00:00 2024-03-26T13:46:10.328 INFO:teuthology.orchestra.run.smithi159.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-03-26T13:46:10.412 INFO:teuthology.orchestra.run.smithi159.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-03-26T13:46:10.424 INFO:teuthology.orchestra.run.smithi179.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 54 kB/s | 42 kB 00:00 2024-03-26T13:46:10.445 INFO:teuthology.orchestra.run.smithi159.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 599 kB/s | 19 kB 00:00 2024-03-26T13:46:10.654 INFO:teuthology.orchestra.run.smithi159.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-03-26T13:46:10.771 INFO:teuthology.orchestra.run.smithi159.stdout:(109/174): openssl-devel-3.0.7-27.el9.x86_64.rp 1.6 MB/s | 4.1 MB 00:02 2024-03-26T13:46:10.796 INFO:teuthology.orchestra.run.smithi159.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-03-26T13:46:10.821 INFO:teuthology.orchestra.run.smithi159.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 980 kB/s | 49 kB 00:00 2024-03-26T13:46:10.847 INFO:teuthology.orchestra.run.smithi159.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-26T13:46:10.880 INFO:teuthology.orchestra.run.smithi159.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-03-26T13:46:10.922 INFO:teuthology.orchestra.run.smithi159.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-03-26T13:46:10.956 INFO:teuthology.orchestra.run.smithi159.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-03-26T13:46:10.981 INFO:teuthology.orchestra.run.smithi159.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 810 kB/s | 43 kB 00:00 2024-03-26T13:46:10.998 INFO:teuthology.orchestra.run.smithi159.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 790 kB/s | 32 kB 00:00 2024-03-26T13:46:11.049 INFO:teuthology.orchestra.run.smithi159.stdout:(118/174): protobuf-compiler-3.14.0-13.el9.x86_ 846 kB/s | 863 kB 00:01 2024-03-26T13:46:11.074 INFO:teuthology.orchestra.run.smithi159.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-03-26T13:46:11.099 INFO:teuthology.orchestra.run.smithi159.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-26T13:46:11.157 INFO:teuthology.orchestra.run.smithi159.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.3 MB/s | 358 kB 00:00 2024-03-26T13:46:11.191 INFO:teuthology.orchestra.run.smithi159.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.4 MB/s | 303 kB 00:00 2024-03-26T13:46:11.216 INFO:teuthology.orchestra.run.smithi159.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 313 kB/s | 35 kB 00:00 2024-03-26T13:46:11.250 INFO:teuthology.orchestra.run.smithi159.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-26T13:46:11.334 INFO:teuthology.orchestra.run.smithi159.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 18 MB/s | 2.0 MB 00:00 2024-03-26T13:46:11.367 INFO:teuthology.orchestra.run.smithi159.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 978 kB/s | 144 kB 00:00 2024-03-26T13:46:11.392 INFO:teuthology.orchestra.run.smithi159.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-03-26T13:46:11.418 INFO:teuthology.orchestra.run.smithi159.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 362 kB/s | 18 kB 00:00 2024-03-26T13:46:11.443 INFO:teuthology.orchestra.run.smithi159.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 314 kB/s | 23 kB 00:00 2024-03-26T13:46:11.468 INFO:teuthology.orchestra.run.smithi159.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-03-26T13:46:11.510 INFO:teuthology.orchestra.run.smithi159.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 299 kB/s | 20 kB 00:00 2024-03-26T13:46:11.549 INFO:teuthology.orchestra.run.smithi179.stdout:(155/174): NetworkManager-1.47.2-1.el9.x86_64.r 506 kB/s | 2.3 MB 00:04 2024-03-26T13:46:11.560 INFO:teuthology.orchestra.run.smithi159.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-03-26T13:46:11.586 INFO:teuthology.orchestra.run.smithi159.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 432 kB/s | 46 kB 00:00 2024-03-26T13:46:11.611 INFO:teuthology.orchestra.run.smithi159.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-03-26T13:46:11.616 INFO:teuthology.orchestra.run.smithi179.stdout:(156/174): NetworkManager-tui-1.47.2-1.el9.x86_ 209 kB/s | 248 kB 00:01 2024-03-26T13:46:11.636 INFO:teuthology.orchestra.run.smithi159.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 776 kB/s | 58 kB 00:00 2024-03-26T13:46:11.670 INFO:teuthology.orchestra.run.smithi159.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.3 MB/s | 272 kB 00:00 2024-03-26T13:46:11.695 INFO:teuthology.orchestra.run.smithi159.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 297 kB/s | 16 kB 00:00 2024-03-26T13:46:11.721 INFO:teuthology.orchestra.run.smithi159.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-26T13:46:11.746 INFO:teuthology.orchestra.run.smithi159.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 382 kB/s | 28 kB 00:00 2024-03-26T13:46:11.771 INFO:teuthology.orchestra.run.smithi159.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 629 kB/s | 31 kB 00:00 2024-03-26T13:46:11.788 INFO:teuthology.orchestra.run.smithi159.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-03-26T13:46:11.822 INFO:teuthology.orchestra.run.smithi159.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 804 kB/s | 59 kB 00:00 2024-03-26T13:46:11.847 INFO:teuthology.orchestra.run.smithi159.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 628 kB/s | 36 kB 00:00 2024-03-26T13:46:11.872 INFO:teuthology.orchestra.run.smithi159.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 623 kB/s | 51 kB 00:00 2024-03-26T13:46:11.897 INFO:teuthology.orchestra.run.smithi159.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.1 MB/s | 230 kB 00:00 2024-03-26T13:46:11.956 INFO:teuthology.orchestra.run.smithi159.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-26T13:46:11.989 INFO:teuthology.orchestra.run.smithi159.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.7 MB/s | 427 kB 00:00 2024-03-26T13:46:12.015 INFO:teuthology.orchestra.run.smithi159.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 179 kB/s | 20 kB 00:00 2024-03-26T13:46:12.048 INFO:teuthology.orchestra.run.smithi159.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-03-26T13:46:12.050 INFO:teuthology.orchestra.run.smithi179.stdout:(157/174): NetworkManager-libnm-1.47.2-1.el9.x8 463 kB/s | 1.8 MB 00:04 2024-03-26T13:46:12.107 INFO:teuthology.orchestra.run.smithi159.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-03-26T13:46:12.174 INFO:teuthology.orchestra.run.smithi159.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2024-03-26T13:46:12.441 INFO:teuthology.orchestra.run.smithi159.stdout:(152/174): librbd1-19.0.0-2145.g3be9a5c9.el9.x8 8.1 MB/s | 3.1 MB 00:00 2024-03-26T13:46:12.458 INFO:teuthology.orchestra.run.smithi179.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 43 kB/s | 36 kB 00:00 2024-03-26T13:46:12.524 INFO:teuthology.orchestra.run.smithi159.stdout:(153/174): librados2-19.0.0-2145.g3be9a5c9.el9. 6.8 MB/s | 3.4 MB 00:00 2024-03-26T13:46:12.525 INFO:teuthology.orchestra.run.smithi179.stdout:(159/174): acl-2.3.1-4.el9.x86_64.rpm 73 kB/s | 71 kB 00:00 2024-03-26T13:46:12.606 INFO:teuthology.orchestra.run.smithi159.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 522 kB/s | 42 kB 00:00 2024-03-26T13:46:12.731 INFO:teuthology.orchestra.run.smithi159.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 2.0 MB/s | 248 kB 00:00 2024-03-26T13:46:12.815 INFO:teuthology.orchestra.run.smithi159.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 864 kB/s | 71 kB 00:00 2024-03-26T13:46:12.882 INFO:teuthology.orchestra.run.smithi159.stdout:(157/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.3 MB/s | 2.3 MB 00:00 2024-03-26T13:46:12.908 INFO:teuthology.orchestra.run.smithi159.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 396 kB/s | 36 kB 00:00 2024-03-26T13:46:12.958 INFO:teuthology.orchestra.run.smithi159.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-26T13:46:13.017 INFO:teuthology.orchestra.run.smithi159.stdout:(160/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.2 MB/s | 1.8 MB 00:00 2024-03-26T13:46:13.042 INFO:teuthology.orchestra.run.smithi159.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 278 kB/s | 23 kB 00:00 2024-03-26T13:46:13.099 INFO:teuthology.orchestra.run.smithi159.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.3 MB/s | 256 kB 00:00 2024-03-26T13:46:13.134 INFO:teuthology.orchestra.run.smithi159.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 931 kB/s | 108 kB 00:00 2024-03-26T13:46:13.159 INFO:teuthology.orchestra.run.smithi159.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-03-26T13:46:13.185 INFO:teuthology.orchestra.run.smithi159.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-26T13:46:13.201 INFO:teuthology.orchestra.run.smithi159.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 961 kB/s | 64 kB 00:00 2024-03-26T13:46:13.209 INFO:teuthology.orchestra.run.smithi179.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 166 kB/s | 192 kB 00:01 2024-03-26T13:46:13.218 INFO:teuthology.orchestra.run.smithi159.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 494 kB/s | 29 kB 00:00 2024-03-26T13:46:13.234 INFO:teuthology.orchestra.run.smithi179.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 32 kB/s | 23 kB 00:00 2024-03-26T13:46:13.536 INFO:teuthology.orchestra.run.smithi159.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.5 MB/s | 1.2 MB 00:00 2024-03-26T13:46:13.619 INFO:teuthology.orchestra.run.smithi159.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.7 MB/s | 2.3 MB 00:00 2024-03-26T13:46:13.668 INFO:teuthology.orchestra.run.smithi179.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 212 kB/s | 256 kB 00:01 2024-03-26T13:46:13.678 INFO:teuthology.orchestra.run.smithi159.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.5 MB/s | 2.2 MB 00:00 2024-03-26T13:46:13.703 INFO:teuthology.orchestra.run.smithi159.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 376 kB/s | 31 kB 00:00 2024-03-26T13:46:13.736 INFO:teuthology.orchestra.run.smithi159.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.3 MB/s | 465 kB 00:00 2024-03-26T13:46:13.796 INFO:teuthology.orchestra.run.smithi159.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 815 kB/s | 74 kB 00:00 2024-03-26T13:46:13.904 INFO:teuthology.orchestra.run.smithi159.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.8 MB/s | 1.1 MB 00:00 2024-03-26T13:46:13.913 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:46:13.914 INFO:teuthology.orchestra.run.smithi159.stdout:Total 13 MB/s | 225 MB 00:16 2024-03-26T13:46:14.193 INFO:teuthology.orchestra.run.smithi179.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 110 kB/s | 108 kB 00:00 2024-03-26T13:46:14.318 INFO:teuthology.orchestra.run.smithi179.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 143 kB/s | 155 kB 00:01 2024-03-26T13:46:14.618 INFO:teuthology.orchestra.run.smithi179.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 143 kB/s | 136 kB 00:00 2024-03-26T13:46:15.063 INFO:teuthology.orchestra.run.smithi179.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 73 kB/s | 64 kB 00:00 2024-03-26T13:46:15.089 INFO:teuthology.orchestra.run.smithi179.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 37 kB/s | 29 kB 00:00 2024-03-26T13:46:15.562 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-03-26T13:46:15.946 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-03-26T13:46:15.947 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-03-26T13:46:16.066 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-03-26T13:46:16.150 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T13:46:16.190 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T13:46:16.229 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T13:46:16.273 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T13:46:16.368 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T13:46:16.473 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T13:46:16.538 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T13:46:16.570 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T13:46:16.640 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T13:46:17.052 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T13:46:17.064 INFO:teuthology.orchestra.run.smithi179.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 522 kB/s | 1.2 MB 00:02 2024-03-26T13:46:17.101 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:46:17.233 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:46:17.261 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:46:17.364 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:46:17.401 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T13:46:17.464 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T13:46:17.761 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T13:46:17.816 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T13:46:17.840 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:46:17.987 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:46:18.025 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T13:46:18.114 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T13:46:18.168 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T13:46:18.218 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T13:46:18.269 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-03-26T13:46:18.270 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-03-26T13:46:18.282 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T13:46:18.322 INFO:teuthology.orchestra.run.smithi179.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 370 kB/s | 465 kB 00:01 2024-03-26T13:46:18.464 INFO:teuthology.orchestra.run.smithi179.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 650 kB/s | 2.2 MB 00:03 2024-03-26T13:46:18.572 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T13:46:18.618 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T13:46:18.668 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T13:46:18.890 INFO:teuthology.orchestra.run.smithi179.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 615 kB/s | 2.3 MB 00:03 2024-03-26T13:46:18.984 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T13:46:19.123 INFO:teuthology.orchestra.run.smithi179.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 39 kB/s | 31 kB 00:00 2024-03-26T13:46:19.150 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T13:46:19.231 INFO:teuthology.orchestra.run.smithi179.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 218 kB/s | 74 kB 00:00 2024-03-26T13:46:19.247 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T13:46:19.444 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:19.495 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:19.806 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:19.992 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T13:46:20.111 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T13:46:20.150 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T13:46:20.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T13:46:20.269 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T13:46:20.387 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T13:46:20.453 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T13:46:20.506 INFO:teuthology.orchestra.run.smithi179.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 538 kB/s | 1.1 MB 00:02 2024-03-26T13:46:20.519 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:46:20.520 INFO:teuthology.orchestra.run.smithi179.stdout:Total 6.5 MB/s | 225 MB 00:34 2024-03-26T13:46:20.532 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T13:46:20.638 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T13:46:20.681 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T13:46:20.743 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T13:46:20.786 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T13:46:20.817 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T13:46:20.930 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T13:46:20.971 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T13:46:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T13:46:21.078 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T13:46:21.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T13:46:21.255 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T13:46:21.363 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T13:46:21.538 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T13:46:21.599 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T13:46:22.128 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-03-26T13:46:22.339 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T13:46:22.441 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T13:46:22.507 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-03-26T13:46:22.507 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-03-26T13:46:22.546 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T13:46:22.592 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T13:46:22.616 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T13:46:22.653 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T13:46:22.709 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T13:46:22.762 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T13:46:22.827 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T13:46:23.251 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T13:46:23.670 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T13:46:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T13:46:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T13:46:24.172 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T13:46:24.242 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T13:46:24.320 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T13:46:24.423 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T13:46:24.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T13:46:24.498 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T13:46:24.566 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T13:46:24.605 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T13:46:24.666 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T13:46:24.734 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T13:46:24.913 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-03-26T13:46:24.915 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-03-26T13:46:24.927 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T13:46:24.978 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T13:46:25.016 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T13:46:25.076 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T13:46:25.240 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T13:46:25.273 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T13:46:25.317 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T13:46:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T13:46:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T13:46:25.539 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T13:46:25.714 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T13:46:25.849 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T13:46:25.898 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T13:46:25.951 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-03-26T13:46:25.960 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T13:46:26.043 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T13:46:26.069 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T13:46:26.118 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T13:46:26.147 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T13:46:26.192 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T13:46:26.268 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T13:46:26.294 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T13:46:26.352 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T13:46:26.391 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T13:46:26.447 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T13:46:26.488 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T13:46:26.527 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T13:46:26.566 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T13:46:26.576 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T13:46:26.606 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T13:46:26.644 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T13:46:26.685 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T13:46:26.731 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T13:46:26.788 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T13:46:26.839 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T13:46:26.890 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T13:46:27.012 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T13:46:27.015 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T13:46:27.052 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T13:46:27.076 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:46:27.086 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T13:46:27.139 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T13:46:27.181 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T13:46:27.262 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:46:27.288 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:46:27.323 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T13:46:27.409 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:46:27.444 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T13:46:27.448 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T13:46:27.503 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T13:46:27.539 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T13:46:27.550 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T13:46:27.595 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T13:46:27.735 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T13:46:27.805 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T13:46:27.870 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T13:46:27.927 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T13:46:27.952 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:46:28.102 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:46:28.128 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T13:46:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T13:46:28.298 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T13:46:28.330 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T13:46:28.402 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T13:46:28.732 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T13:46:28.767 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T13:46:28.789 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T13:46:28.812 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T13:46:28.846 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T13:46:28.847 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T13:46:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T13:46:29.121 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T13:46:29.157 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T13:46:29.288 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T13:46:29.329 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T13:46:29.451 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T13:46:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T13:46:29.568 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T13:46:29.641 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:29.682 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T13:46:29.690 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:30.052 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:30.239 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T13:46:30.374 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T13:46:30.405 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T13:46:30.433 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T13:46:30.485 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T13:46:30.486 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T13:46:30.589 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T13:46:30.620 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T13:46:30.662 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T13:46:30.705 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T13:46:30.796 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T13:46:30.852 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T13:46:30.915 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T13:46:30.984 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T13:46:31.025 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T13:46:31.146 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T13:46:31.186 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T13:46:31.236 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T13:46:31.268 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T13:46:31.527 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T13:46:31.588 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T13:46:31.688 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T13:46:31.874 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T13:46:31.940 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T13:46:32.117 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-03-26T13:46:32.134 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T13:46:32.222 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T13:46:32.222 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-26T13:46:32.259 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T13:46:32.271 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-26T13:46:32.290 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T13:46:32.309 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T13:46:32.329 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T13:46:32.353 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-26T13:46:32.438 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-26T13:46:32.533 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-26T13:46:32.583 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-26T13:46:32.615 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-26T13:46:32.632 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T13:46:32.703 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T13:46:32.704 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-26T13:46:32.728 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T13:46:32.737 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T13:46:32.841 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T13:46:32.856 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T13:46:32.885 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T13:46:32.888 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T13:46:32.914 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T13:46:32.940 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T13:46:32.983 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T13:46:33.048 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T13:46:33.106 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T13:46:33.115 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-26T13:46:33.146 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T13:46:33.163 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:46:33.313 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:46:33.356 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:46:33.450 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T13:46:33.480 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:46:33.482 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T13:46:33.513 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T13:46:33.557 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T13:46:33.563 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T13:46:33.575 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-26T13:46:33.583 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T13:46:33.837 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T13:46:33.864 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-26T13:46:33.893 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T13:46:33.919 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-26T13:46:33.947 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T13:46:33.970 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:46:34.124 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:46:34.145 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T13:46:34.162 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T13:46:34.223 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T13:46:34.252 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 17/198 2024-03-26T13:46:34.298 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-26T13:46:34.303 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T13:46:34.322 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T13:46:34.384 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-26T13:46:34.403 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T13:46:34.439 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T13:46:34.479 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T13:46:34.582 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T13:46:34.629 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T13:46:34.693 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T13:46:34.698 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T13:46:34.740 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-26T13:46:34.758 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T13:46:34.781 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-26T13:46:34.951 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T13:46:35.010 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T13:46:35.048 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T13:46:35.104 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T13:46:35.112 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-26T13:46:35.234 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T13:46:35.280 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T13:46:35.294 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T13:46:35.352 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T13:46:35.368 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-26T13:46:35.387 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T13:46:35.420 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T13:46:35.574 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:35.574 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T13:46:35.633 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T13:46:35.908 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T13:46:35.955 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T13:46:35.980 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-26T13:46:36.018 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T13:46:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T13:46:36.122 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:36.155 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:36.159 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-26T13:46:36.293 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-26T13:46:36.330 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-26T13:46:36.366 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-26T13:46:36.368 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T13:46:36.426 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T13:46:36.426 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-26T13:46:36.538 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-26T13:46:36.606 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-26T13:46:36.638 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T13:46:36.655 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-26T13:46:36.690 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T13:46:36.730 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T13:46:36.746 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-26T13:46:36.769 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T13:46:36.782 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:36.794 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-26T13:46:36.811 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T13:46:36.825 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T13:46:36.855 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-26T13:46:36.882 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T13:46:36.899 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-26T13:46:36.931 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-26T13:46:36.950 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T13:46:36.958 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T13:46:36.958 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T13:46:36.958 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T13:46:36.958 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:37.000 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T13:46:37.008 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T13:46:37.042 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-26T13:46:37.056 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T13:46:37.077 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 39/198 2024-03-26T13:46:37.125 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 40/198 2024-03-26T13:46:37.149 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 41/198 2024-03-26T13:46:37.184 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T13:46:37.254 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-26T13:46:37.257 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T13:46:37.306 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T13:46:37.310 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-26T13:46:37.373 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T13:46:37.409 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-26T13:46:37.418 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T13:46:37.564 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T13:46:37.585 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-26T13:46:37.682 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T13:46:37.738 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T13:46:37.781 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T13:46:37.838 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T13:46:37.909 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-26T13:46:37.978 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T13:46:38.073 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T13:46:38.632 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-26T13:46:38.727 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-26T13:46:38.844 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-26T13:46:38.938 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T13:46:38.966 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-26T13:46:39.016 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-26T13:46:39.048 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T13:46:39.088 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-26T13:46:39.105 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T13:46:39.138 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T13:46:39.148 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 53/198 2024-03-26T13:46:39.195 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-26T13:46:39.222 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T13:46:39.344 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T13:46:39.381 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T13:46:39.609 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-26T13:46:39.614 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T13:46:39.652 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T13:46:39.745 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T13:46:39.868 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-26T13:46:39.959 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-26T13:46:40.007 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-26T13:46:40.203 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-26T13:46:40.255 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-26T13:46:40.328 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-26T13:46:40.415 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-26T13:46:40.455 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-26T13:46:40.510 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-26T13:46:40.555 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T13:46:40.580 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-26T13:46:40.619 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-26T13:46:40.684 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-26T13:46:40.689 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T13:46:40.750 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-26T13:46:40.940 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-26T13:46:40.992 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-26T13:46:41.030 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-26T13:46:41.086 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-26T13:46:41.215 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-26T13:46:41.261 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-26T13:46:41.297 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-26T13:46:41.327 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-26T13:46:41.360 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-26T13:46:41.495 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-26T13:46:41.667 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-26T13:46:41.801 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-26T13:46:41.847 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-26T13:46:41.909 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-26T13:46:41.983 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-26T13:46:42.229 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-26T13:46:42.233 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T13:46:42.285 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-26T13:46:42.341 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T13:46:42.379 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T13:46:42.410 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T13:46:42.451 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T13:46:42.460 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-26T13:46:42.506 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-26T13:46:42.539 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-26T13:46:42.576 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-26T13:46:42.617 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-26T13:46:42.694 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-26T13:46:42.758 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-26T13:46:42.808 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-26T13:46:42.826 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T13:46:42.848 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-26T13:46:42.856 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T13:46:42.971 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-26T13:46:42.975 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T13:46:43.010 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-26T13:46:43.011 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T13:46:43.044 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-26T13:46:43.106 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T13:46:43.108 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-26T13:46:43.147 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-26T13:46:43.302 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-26T13:46:43.418 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-26T13:46:43.469 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-26T13:46:43.505 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-26T13:46:43.560 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-26T13:46:43.645 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T13:46:43.694 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T13:46:43.701 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-26T13:46:43.757 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T13:46:43.786 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T13:46:43.795 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-26T13:46:44.748 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-26T13:46:44.804 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-26T13:46:44.855 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-26T13:46:44.913 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-26T13:46:45.023 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-26T13:46:45.063 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-26T13:46:45.293 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-26T13:46:45.326 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-26T13:46:45.423 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-26T13:46:46.209 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-26T13:46:46.329 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-26T13:46:46.463 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:46.531 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:47.176 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:47.221 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T13:46:47.405 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T13:46:47.405 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T13:46:47.406 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T13:46:47.406 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:46:47.455 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T13:46:47.849 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-26T13:46:47.930 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-26T13:46:47.967 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-26T13:46:47.998 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-26T13:46:48.038 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-26T13:46:48.455 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-26T13:46:48.688 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T13:46:48.821 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 124/198 2024-03-26T13:46:48.846 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 125/198 2024-03-26T13:46:48.907 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-26T13:46:49.465 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-26T13:46:49.499 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T13:46:49.600 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 128/198 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-03-26T13:46:49.626 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:49.627 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 129/198 2024-03-26T13:46:49.862 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T13:46:49.911 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T13:46:49.911 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:49.911 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T13:46:49.911 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:51.534 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T13:46:51.615 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T13:46:51.615 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:51.615 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T13:46:51.615 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T13:46:51.615 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T13:46:51.616 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:51.883 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T13:46:51.950 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T13:46:51.951 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:51.951 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T13:46:51.951 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T13:46:51.951 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T13:46:51.951 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:52.270 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:52.308 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:52.468 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T13:46:52.545 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T13:46:52.545 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:52.546 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T13:46:52.546 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T13:46:52.546 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T13:46:52.546 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:52.661 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T13:46:52.930 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 130/198 2024-03-26T13:46:52.978 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T13:46:53.114 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 131/198 2024-03-26T13:46:53.114 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-26T13:46:53.114 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-26T13:46:53.114 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:46:53.154 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T13:46:53.295 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T13:46:53.328 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T13:46:53.377 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T13:46:53.476 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T13:46:53.507 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T13:46:53.835 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T13:46:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T13:46:55.008 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T13:46:55.052 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T13:46:55.194 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T13:46:55.371 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T13:46:55.402 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T13:46:55.477 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T13:46:55.477 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:55.478 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T13:46:55.478 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T13:46:55.478 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T13:46:55.478 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:55.528 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T13:46:55.563 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T13:46:55.627 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T13:46:56.569 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T13:46:56.605 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T13:46:56.671 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T13:46:56.671 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:56.671 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T13:46:56.671 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T13:46:56.671 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T13:46:56.672 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:56.730 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T13:46:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T13:46:56.820 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:56.820 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T13:46:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:56.898 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:46:56.996 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:46:56.997 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:56.997 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T13:46:56.997 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T13:46:56.997 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T13:46:56.997 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:46:57.373 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T13:46:57.453 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T13:46:57.453 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:46:57.453 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T13:46:57.454 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T13:46:57.454 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T13:46:57.454 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:47:00.448 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-03-26T13:47:00.449 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:00.651 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T13:47:00.692 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T13:47:00.692 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:00.692 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T13:47:00.692 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:02.280 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T13:47:02.297 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T13:47:02.360 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T13:47:02.360 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:02.360 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T13:47:02.360 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T13:47:02.360 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T13:47:02.360 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:02.361 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T13:47:02.394 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T13:47:02.432 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T13:47:02.613 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T13:47:02.684 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T13:47:02.684 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:02.684 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T13:47:02.684 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T13:47:02.685 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T13:47:02.685 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:02.711 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T13:47:02.830 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T13:47:02.879 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T13:47:02.932 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T13:47:02.986 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T13:47:03.031 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T13:47:03.065 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T13:47:03.102 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T13:47:03.155 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T13:47:03.172 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T13:47:03.241 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T13:47:03.241 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:03.241 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T13:47:03.241 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T13:47:03.241 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T13:47:03.241 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:03.258 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T13:47:03.357 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T13:47:03.378 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T13:47:03.429 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T13:47:04.013 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T13:47:04.066 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T13:47:04.118 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T13:47:04.219 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T13:47:04.253 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:04.276 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T13:47:04.301 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:04.361 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:04.609 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T13:47:04.638 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T13:47:05.679 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T13:47:05.754 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T13:47:05.788 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T13:47:05.932 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T13:47:05.957 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T13:47:06.049 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 132/198 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-03-26T13:47:06.056 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:06.110 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T13:47:06.149 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T13:47:06.198 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T13:47:06.241 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T13:47:06.245 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T13:47:06.245 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:06.245 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T13:47:06.245 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T13:47:06.245 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T13:47:06.245 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:06.266 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T13:47:06.283 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T13:47:06.299 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T13:47:06.301 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T13:47:06.332 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 133/198 2024-03-26T13:47:06.332 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:06.332 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-26T13:47:06.332 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:06.337 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T13:47:06.359 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T13:47:06.359 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T13:47:06.392 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T13:47:06.504 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T13:47:06.546 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T13:47:06.547 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T13:47:06.668 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T13:47:06.712 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T13:47:06.767 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T13:47:06.808 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T13:47:06.866 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T13:47:06.933 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T13:47:07.012 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T13:47:07.056 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T13:47:07.089 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T13:47:07.123 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T13:47:07.156 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T13:47:07.198 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T13:47:07.235 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:07.235 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:07.350 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T13:47:07.410 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T13:47:07.506 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T13:47:07.506 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:07.506 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T13:47:07.507 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T13:47:07.507 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T13:47:07.507 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:07.561 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T13:47:07.564 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:07.564 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T13:47:07.661 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T13:47:07.661 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:07.661 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T13:47:07.661 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:07.704 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T13:47:07.734 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T13:47:07.745 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:07.789 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T13:47:07.837 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T13:47:07.850 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:07.850 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:07.850 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T13:47:07.850 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T13:47:07.850 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T13:47:07.851 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:07.877 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T13:47:07.912 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T13:47:07.986 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 134/198 2024-03-26T13:47:07.986 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:07.986 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-26T13:47:07.986 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T13:47:07.986 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-26T13:47:07.986 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:07.996 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T13:47:08.238 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T13:47:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T13:47:08.243 INFO:teuthology.orchestra.run.smithi137.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T13:47:08.243 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:47:08.246 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T13:47:08.330 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 135/198 2024-03-26T13:47:08.330 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:08.331 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-26T13:47:08.331 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T13:47:08.331 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-26T13:47:08.331 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:08.331 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T13:47:08.332 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:08.332 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T13:47:08.332 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T13:47:08.332 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T13:47:08.332 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:08.847 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T13:47:08.918 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 136/198 2024-03-26T13:47:08.918 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:08.918 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-26T13:47:08.918 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T13:47:08.919 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-26T13:47:08.919 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:09.040 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-26T13:47:09.671 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T13:47:09.698 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 138/198 2024-03-26T13:47:09.749 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 139/198 2024-03-26T13:47:09.849 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 140/198 2024-03-26T13:47:09.879 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T13:47:10.190 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 141/198 2024-03-26T13:47:10.221 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T13:47:11.304 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 142/198 2024-03-26T13:47:11.338 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T13:47:11.480 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 143/198 2024-03-26T13:47:11.651 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 144/198 2024-03-26T13:47:11.682 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T13:47:11.747 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 145/198 2024-03-26T13:47:11.747 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:11.747 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-26T13:47:11.747 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T13:47:11.747 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-26T13:47:11.747 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:11.807 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T13:47:11.844 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 146/198 2024-03-26T13:47:11.897 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 147/198 2024-03-26T13:47:12.832 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-26T13:47:12.868 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T13:47:12.946 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 149/198 2024-03-26T13:47:12.946 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:12.946 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-26T13:47:12.946 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T13:47:12.946 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-26T13:47:12.946 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:13.002 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T13:47:13.027 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T13:47:13.028 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T13:47:13.029 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T13:47:13.046 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T13:47:13.047 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T13:47:13.048 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T13:47:13.049 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T13:47:13.050 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T13:47:13.051 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T13:47:13.052 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T13:47:13.053 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T13:47:13.054 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T13:47:13.055 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T13:47:13.056 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T13:47:13.057 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T13:47:13.058 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T13:47:13.059 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T13:47:13.083 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 150/198 2024-03-26T13:47:13.083 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:13.083 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-26T13:47:13.083 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:13.109 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T13:47:13.154 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T13:47:13.161 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:13.207 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T13:47:13.242 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:13.242 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:13.242 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-26T13:47:13.242 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T13:47:13.242 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-26T13:47:13.242 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:13.245 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T13:47:13.524 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T13:47:13.590 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T13:47:13.658 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 152/198 2024-03-26T13:47:13.658 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-26T13:47:13.658 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-26T13:47:13.658 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T13:47:13.658 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-26T13:47:13.658 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:13.678 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T13:47:13.725 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T13:47:13.779 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T13:47:13.850 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T13:47:13.896 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T13:47:13.956 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T13:47:14.009 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T13:47:14.060 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T13:47:14.149 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T13:47:14.259 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T13:47:14.301 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T13:47:15.195 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:15.239 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:15.305 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T13:47:15.425 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:15.426 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.427 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.428 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T13:47:15.429 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: jq-1.6-16.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.430 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T13:47:15.431 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T13:47:15.432 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T13:47:15.433 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T13:47:15.434 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T13:47:15.435 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:15.436 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:47:15.437 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-03-26T13:47:16.065 DEBUG:teuthology.parallel:result is None 2024-03-26T13:47:16.638 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T13:47:16.737 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T13:47:16.793 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T13:47:16.926 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T13:47:16.966 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T13:47:16.999 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T13:47:17.066 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T13:47:17.110 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T13:47:17.111 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T13:47:17.255 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T13:47:17.304 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T13:47:17.305 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T13:47:17.457 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T13:47:17.511 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T13:47:17.558 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T13:47:17.591 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T13:47:17.623 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T13:47:17.693 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T13:47:17.762 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T13:47:17.803 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T13:47:17.861 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T13:47:17.895 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T13:47:17.928 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T13:47:17.962 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T13:47:17.998 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:17.999 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:18.338 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T13:47:18.444 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T13:47:18.485 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T13:47:18.489 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 154/198 2024-03-26T13:47:18.514 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T13:47:18.524 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 155/198 2024-03-26T13:47:18.563 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-26T13:47:18.569 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T13:47:18.626 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T13:47:18.666 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T13:47:18.795 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T13:47:18.827 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 157/198 2024-03-26T13:47:18.961 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-26T13:47:19.009 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 159/198 2024-03-26T13:47:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T13:47:19.051 INFO:teuthology.orchestra.run.smithi159.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T13:47:19.051 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:19.064 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-26T13:47:19.159 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-26T13:47:19.220 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-26T13:47:19.256 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 163/198 2024-03-26T13:47:19.292 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-26T13:47:19.344 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-26T13:47:19.441 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-26T13:47:19.542 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-26T13:47:19.585 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-26T13:47:20.461 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:20.516 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:20.572 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-26T13:47:21.886 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 170/198 2024-03-26T13:47:21.980 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-26T13:47:22.033 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-26T13:47:22.155 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-26T13:47:22.190 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-26T13:47:22.223 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-26T13:47:22.257 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-26T13:47:22.306 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-26T13:47:22.306 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T13:47:22.461 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-26T13:47:22.503 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-26T13:47:22.504 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T13:47:22.633 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-26T13:47:22.677 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-26T13:47:22.724 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-26T13:47:22.757 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-26T13:47:22.781 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-26T13:47:22.849 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-26T13:47:22.920 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-26T13:47:22.961 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-26T13:47:22.994 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-26T13:47:23.027 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-26T13:47:23.061 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-26T13:47:23.094 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-26T13:47:23.131 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:23.132 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:23.470 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-26T13:47:23.470 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T13:47:23.600 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-26T13:47:23.639 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-26T13:47:23.694 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-26T13:47:23.725 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-26T13:47:23.765 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-26T13:47:23.893 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T13:47:23.893 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T13:47:23.893 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T13:47:23.893 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T13:47:23.894 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T13:47:23.895 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T13:47:23.910 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T13:47:23.911 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T13:47:23.912 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T13:47:23.913 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T13:47:23.914 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T13:47:23.915 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T13:47:23.916 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T13:47:23.917 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T13:47:23.918 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T13:47:23.919 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T13:47:23.920 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T13:47:24.126 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-26T13:47:24.126 INFO:teuthology.orchestra.run.smithi179.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-26T13:47:24.127 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:26.506 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T13:47:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:26.507 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-03-26T13:47:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:26.507 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.508 INFO:teuthology.orchestra.run.smithi159.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.509 INFO:teuthology.orchestra.run.smithi159.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T13:47:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.511 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.512 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T13:47:26.513 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: jq-1.6-16.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.514 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.515 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T13:47:26.516 INFO:teuthology.orchestra.run.smithi159.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T13:47:26.517 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.518 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T13:47:26.519 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T13:47:26.520 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T13:47:26.521 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T13:47:26.522 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T13:47:26.523 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T13:47:26.524 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T13:47:26.525 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T13:47:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T13:47:26.526 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-03-26T13:47:26.526 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-03-26T13:47:27.194 DEBUG:teuthology.parallel:result is None 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 1/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 3/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 4/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2145.g3be9a 5/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 6/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 7/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 8/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 9/198 2024-03-26T13:47:29.035 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 10/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 11/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 12/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 13/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 14/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_ 15/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 16/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 17/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_6 18/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86 19/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 20/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el 21/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9. 22/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_6 23/198 2024-03-26T13:47:29.036 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 24/198 2024-03-26T13:47:29.037 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 25/198 2024-03-26T13:47:29.053 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 26/198 2024-03-26T13:47:29.053 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 27/198 2024-03-26T13:47:29.053 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 28/198 2024-03-26T13:47:29.053 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 29/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9. 30/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noa 31/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.n 32/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be 33/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el 34/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 35/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.e 36/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 37/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 38/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 39/198 2024-03-26T13:47:29.054 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-26T13:47:29.055 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-26T13:47:29.056 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-26T13:47:29.057 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-26T13:47:29.058 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-26T13:47:29.059 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-26T13:47:29.060 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-26T13:47:29.061 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-26T13:47:29.062 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 151/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 153/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-26T13:47:29.063 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-26T13:47:29.064 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-26T13:47:29.065 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-26T13:47:29.066 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-26T13:47:31.484 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: acl-2.3.1-4.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-26T13:47:31.485 INFO:teuthology.orchestra.run.smithi179.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.486 INFO:teuthology.orchestra.run.smithi179.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.2-8.el9.noarch 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.7.4-5.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.487 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.488 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:19.0.0-2145.g3be9a5c9.el9.noarch 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.4-9.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-26T13:47:31.489 INFO:teuthology.orchestra.run.smithi179.stdout: jq-1.6-16.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.490 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-26T13:47:31.491 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-13.14-1.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-26T13:47:31.492 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-26T13:47:31.493 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-26T13:47:31.494 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-26T13:47:31.495 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-26T13:47:31.496 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.497 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-2145.g3be9a5c9.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-26T13:47:31.498 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:47:31.499 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-03-26T13:47:32.112 DEBUG:teuthology.parallel:result is None 2024-03-26T13:47:32.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:47:32.328 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T13:47:32.399 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T13:47:32.400 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T13:47:32.400 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T13:47:32.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:47:32.615 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T13:47:32.674 INFO:teuthology.orchestra.run.smithi159.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T13:47:32.675 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T13:47:32.675 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T13:47:32.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:47:32.813 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-26T13:47:32.878 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-2145.g3be9a5c9.el9 2024-03-26T13:47:32.878 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2145.g3be9a5c9.el9 2024-03-26T13:47:32.878 INFO:teuthology.task.install:The correct ceph version 19.0.0-2145.g3be9a5c9 is installed. 2024-03-26T13:47:32.880 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-26T13:47:32.880 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:47:32.881 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T13:47:32.983 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:47:32.984 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T13:47:33.083 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:47:33.083 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-26T13:47:33.171 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-26T13:47:33.171 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:47:33.172 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T13:47:33.270 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T13:47:33.407 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:47:33.407 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T13:47:33.504 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T13:47:33.641 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:47:33.641 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-03-26T13:47:33.715 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-26T13:47:33.826 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-26T13:47:33.826 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:47:33.826 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T13:47:33.924 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T13:47:34.060 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:47:34.060 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T13:47:34.153 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T13:47:34.292 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:47:34.292 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-26T13:47:34.365 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-26T13:47:34.476 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-26T13:47:34.476 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:47:34.476 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T13:47:34.571 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T13:47:34.665 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:47:34.665 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T13:47:34.765 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T13:47:34.860 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:47:34.860 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-03-26T13:47:34.943 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-26T13:47:35.054 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-26T13:47:35.143 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T13:47:35.143 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:47:35.143 INFO:tasks.cephadm:Cluster fsid is 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:47:35.143 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-26T13:47:35.144 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.137', 'mon.b': '172.21.15.159', 'mon.c': '172.21.15.179'} 2024-03-26T13:47:35.144 INFO:tasks.cephadm:First mon is mon.a on smithi137 2024-03-26T13:47:35.144 INFO:tasks.cephadm:First mgr is x 2024-03-26T13:47:35.144 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-26T13:47:35.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-03-26T13:47:35.242 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-03-26T13:47:35.343 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-03-26T13:47:35.430 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-26T13:47:35.431 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:47:35.565 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3be9a5c9bc793e11e2800a8c0c696e8b46742033', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77877/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2145-g3be9a5c9', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2145.g3be9a5c9'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-22 05:51:42.267384', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-main-20240318.032620', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-26T13:47:35.699 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-mchangir-testing-main-20240318.032620, sha1 3be9a5c9bc793e11e2800a8c0c696e8b46742033 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:47:35.700 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm 2024-03-26T13:47:35.700 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:36.684 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 13:47 /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:36.684 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:37.469 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 13:47 /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:37.470 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-main-20240318.032620/3be9a5c9bc793e11e2800a8c0c696e8b46742033/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:38.506 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 767371 Mar 26 13:47 /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:38.506 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:38.565 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:38.623 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-26T13:47:38.698 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 on all hosts... 2024-03-26T13:47:38.699 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T13:47:38.706 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T13:47:38.713 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 pull 2024-03-26T13:47:39.873 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T13:47:39.957 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T13:47:40.048 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T13:48:21.044 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T13:48:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T13:48:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T13:48:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-03-26T13:48:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T13:48:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T13:48:21.045 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-03-26T13:48:32.455 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-03-26T13:48:33.843 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-26T13:48:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T13:48:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2", 2024-03-26T13:48:33.843 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-03-26T13:48:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc" 2024-03-26T13:48:33.844 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-03-26T13:48:33.844 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-26T13:48:33.897 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-03-26T13:48:34.000 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-03-26T13:48:34.099 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-03-26T13:48:34.204 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-03-26T13:48:34.300 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-03-26T13:48:34.397 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-03-26T13:48:34.494 INFO:tasks.cephadm:Writing seed config... 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-26T13:48:34.495 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-26T13:48:34.496 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-26T13:48:34.497 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-26T13:48:34.498 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-26T13:48:34.498 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-03-26T13:48:34.499 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:48:34.499 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-26T13:48:34.555 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 66aee2a0-eb77-11ee-95d0-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-26T13:48:34.555 DEBUG:teuthology.orchestra.run.smithi137:mon.a> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service 2024-03-26T13:48:34.604 DEBUG:teuthology.orchestra.run.smithi137:mgr.x> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x.service 2024-03-26T13:48:34.650 INFO:tasks.cephadm:Bootstrapping... 2024-03-26T13:48:34.650 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 -v bootstrap --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.137 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-26T13:48:34.921 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-03-26T13:48:34.921 INFO:teuthology.orchestra.run.smithi137.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033', '-v', 'bootstrap', '--fsid', '66aee2a0-eb77-11ee-95d0-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.137', '--skip-admin-label'] 2024-03-26T13:48:34.974 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T13:48:34.975 INFO:teuthology.orchestra.run.smithi137.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-26T13:48:34.975 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying podman|docker is present... 2024-03-26T13:48:35.024 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T13:48:35.024 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying lvm2 is present... 2024-03-26T13:48:35.025 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying time synchronization is in place... 2024-03-26T13:48:35.044 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T13:48:35.044 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T13:48:35.062 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T13:48:35.063 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-03-26T13:48:35.087 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout enabled 2024-03-26T13:48:35.109 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout active 2024-03-26T13:48:35.109 INFO:teuthology.orchestra.run.smithi137.stdout:Unit chronyd.service is enabled and running 2024-03-26T13:48:35.110 INFO:teuthology.orchestra.run.smithi137.stdout:Repeating the final host check... 2024-03-26T13:48:35.167 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout 4.9.4-dev 2024-03-26T13:48:35.167 INFO:teuthology.orchestra.run.smithi137.stdout:podman (/bin/podman) version 4.9.4 is present 2024-03-26T13:48:35.167 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl is present 2024-03-26T13:48:35.167 INFO:teuthology.orchestra.run.smithi137.stdout:lvcreate is present 2024-03-26T13:48:35.184 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-26T13:48:35.184 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-26T13:48:35.203 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-26T13:48:35.203 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-03-26T13:48:35.227 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout enabled 2024-03-26T13:48:35.248 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout active 2024-03-26T13:48:35.248 INFO:teuthology.orchestra.run.smithi137.stdout:Unit chronyd.service is enabled and running 2024-03-26T13:48:35.249 INFO:teuthology.orchestra.run.smithi137.stdout:Host looks OK 2024-03-26T13:48:35.249 INFO:teuthology.orchestra.run.smithi137.stdout:Cluster fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:35.249 INFO:teuthology.orchestra.run.smithi137.stdout:Acquiring lock 139727853469120 on /run/cephadm/66aee2a0-eb77-11ee-95d0-87774f69a715.lock 2024-03-26T13:48:35.250 INFO:teuthology.orchestra.run.smithi137.stdout:Lock 139727853469120 acquired on /run/cephadm/66aee2a0-eb77-11ee-95d0-87774f69a715.lock 2024-03-26T13:48:35.251 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying IP 172.21.15.137 port 3300 ... 2024-03-26T13:48:35.252 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying IP 172.21.15.137 port 6789 ... 2024-03-26T13:48:35.253 INFO:teuthology.orchestra.run.smithi137.stdout:Base mon IP(s) is [172.21.15.137:3300, 172.21.15.137:6789], mon addrv is [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-03-26T13:48:35.268 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.137 metric 100 2024-03-26T13:48:35.268 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.137 metric 100 2024-03-26T13:48:35.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-26T13:48:35.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-26T13:48:35.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1725sec hoplimit 64 pref medium 2024-03-26T13:48:35.287 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-26T13:48:35.287 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-26T13:48:35.287 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T13:48:35.287 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-26T13:48:35.287 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-03-26T13:48:35.287 INFO:teuthology.orchestra.run.smithi137.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-26T13:48:35.289 INFO:teuthology.orchestra.run.smithi137.stdout:Mon IP `172.21.15.137` is in CIDR network `172.21.0.0/20` 2024-03-26T13:48:35.289 INFO:teuthology.orchestra.run.smithi137.stdout:Mon IP `172.21.15.137` is in CIDR network `172.21.0.0/20` 2024-03-26T13:48:35.289 INFO:teuthology.orchestra.run.smithi137.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-26T13:48:35.290 INFO:teuthology.orchestra.run.smithi137.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-26T13:48:35.291 INFO:teuthology.orchestra.run.smithi137.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stdout c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033... 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Getting image source signatures 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying blob sha256:4dd83c6ee8921fe17396d77c31e205985ae94c87d3f752465b5f04aaf9ae340a 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Copying config sha256:c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 2024-03-26T13:48:36.997 INFO:teuthology.orchestra.run.smithi137.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-26T13:48:38.132 INFO:teuthology.orchestra.run.smithi137.stdout:ceph: stdout ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-26T13:48:38.132 INFO:teuthology.orchestra.run.smithi137.stdout:Ceph version: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev) 2024-03-26T13:48:38.132 INFO:teuthology.orchestra.run.smithi137.stdout:Extracting ceph user uid/gid from container image... 2024-03-26T13:48:39.212 INFO:teuthology.orchestra.run.smithi137.stdout:stat: stdout 167 167 2024-03-26T13:48:39.213 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial keys... 2024-03-26T13:48:40.188 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQA30gJm7zs1KBAAryuUAbt5HjoD42uz5KXlOA== 2024-03-26T13:48:41.313 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQA40gJmsAF7KhAAMgjQY1UaJ8cdBFHVlNqR5g== 2024-03-26T13:48:42.423 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph-authtool: stdout AQA50gJm84w9LhAAPNjOOuB5zsWLYJLvP9HBKg== 2024-03-26T13:48:42.427 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial monmap... 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:monmaptool for a [v2:172.21.15.137:3300,v1:172.21.15.137:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:setting min_mon_release = quincy 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: set fsid to 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:43.537 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-26T13:48:43.538 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:48:43.538 INFO:teuthology.orchestra.run.smithi137.stdout:Creating mon... 2024-03-26T13:48:44.599 INFO:teuthology.orchestra.run.smithi137.stdout:create mon.a on 2024-03-26T13:48:44.931 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-26T13:48:45.166 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-26T13:48:45.450 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715.target. 2024-03-26T13:48:45.450 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715.target → /etc/systemd/system/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715.target. 2024-03-26T13:48:45.830 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a 2024-03-26T13:48:45.831 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to reset failed state of unit ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service: Unit ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service not loaded. 2024-03-26T13:48:46.124 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715.target.wants/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service → /etc/systemd/system/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@.service. 2024-03-26T13:48:46.763 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 podman[31146]: 2024-03-26 13:48:46.525421638 +0000 UTC m=+0.134899550 container create 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T13:48:46.763 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 podman[31146]: 2024-03-26 13:48:46.770509268 +0000 UTC m=+0.379987187 container init 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T13:48:46.876 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T13:48:46.876 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-03-26T13:48:46.891 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T13:48:46.891 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-03-26T13:48:46.891 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-03-26T13:48:46.892 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T13:48:46.892 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mon to start... 2024-03-26T13:48:46.892 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mon... 2024-03-26T13:48:47.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 podman[31146]: 2024-03-26 13:48:46.779410084 +0000 UTC m=+0.388888002 container start 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:48:47.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 bash[31146]: 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae 2024-03-26T13:48:47.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 ceph-mon[31164]: mkfs 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:47.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 ceph-mon[31164]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T13:48:47.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:46 smithi137 systemd[1]: Started Ceph mon.a for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:48:48.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:47 smithi137 ceph-mon[31164]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T13:48:48.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:47 smithi137 ceph-mon[31164]: monmap e1: 1 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:48:48.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:47 smithi137 ceph-mon[31164]: fsmap 2024-03-26T13:48:48.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:47 smithi137 ceph-mon[31164]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T13:48:48.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:47 smithi137 ceph-mon[31164]: mgrmap e1: no daemons active 2024-03-26T13:48:48.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:47 smithi137 ceph-mon[31164]: from='client.? 172.21.15.137:0/1730048342' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-26T13:48:48.290 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout cluster: 2024-03-26T13:48:48.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout id: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:48.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-26T13:48:48.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:48.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout services: 2024-03-26T13:48:48.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.667229s) 2024-03-26T13:48:48.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-26T13:48:48.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-26T13:48:48.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:48.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout data: 2024-03-26T13:48:48.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-26T13:48:48.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-26T13:48:48.301 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout pgs: 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.429+0000 7f710e000700 1 Processor -- start 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.429+0000 7f710e000700 1 -- start start 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.429+0000 7f710e000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f7108109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.429+0000 7f710e000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710810a580 con 0x7f7108109b90 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.430+0000 7f7107400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f7108109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.430+0000 7f7107400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f7108109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34914/0 (socket says 172.21.15.137:34914) 2024-03-26T13:48:48.302 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.431+0000 7f7107400700 1 -- 172.21.15.137:0/438640680 learned_addr learned my addr 172.21.15.137:0/438640680 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.431+0000 7f7107400700 1 -- 172.21.15.137:0/438640680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710810ad90 con 0x7f7108109b90 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.431+0000 7f7107400700 1 --2- 172.21.15.137:0/438640680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f7108109fb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70f80098a0 tx=0x7f70f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72a4280524267d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.431+0000 7f7106a00700 1 -- 172.21.15.137:0/438640680 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70f8004160 con 0x7f7108109b90 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.431+0000 7f7106a00700 1 -- 172.21.15.137:0/438640680 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8004300 con 0x7f7108109b90 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.431+0000 7f7106a00700 1 -- 172.21.15.137:0/438640680 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70f80036a0 con 0x7f7108109b90 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 -- 172.21.15.137:0/438640680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 msgr2=0x7f7108109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 --2- 172.21.15.137:0/438640680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f7108109fb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f70f80098a0 tx=0x7f70f8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:48:48.303 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 -- 172.21.15.137:0/438640680 shutdown_connections 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 --2- 172.21.15.137:0/438640680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f7108109fb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 -- 172.21.15.137:0/438640680 >> 172.21.15.137:0/438640680 conn(0x7f710807d020 msgr2=0x7f710807d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 -- 172.21.15.137:0/438640680 shutdown_connections 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.432+0000 7f710e000700 1 -- 172.21.15.137:0/438640680 wait complete. 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.433+0000 7f710e000700 1 Processor -- start 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.434+0000 7f710e000700 1 -- start start 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.434+0000 7f710e000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f710819d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.434+0000 7f710e000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710810b2c0 con 0x7f7108109b90 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.434+0000 7f7107400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f710819d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:48.304 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.434+0000 7f7107400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f710819d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34924/0 (socket says 172.21.15.137:34924) 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.434+0000 7f7107400700 1 -- 172.21.15.137:0/1730048342 learned_addr learned my addr 172.21.15.137:0/1730048342 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f7107400700 1 -- 172.21.15.137:0/1730048342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710819d810 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f7107400700 1 --2- 172.21.15.137:0/1730048342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f710819d2d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000c00 tx=0x7f70f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70f8003db0 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f8011630 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70f8011950 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710819daa0 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.435+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71081a0790 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.436+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f70f801a460 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.436+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f70f801b780 con 0x7f7108109b90 2024-03-26T13:48:48.305 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.436+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71080637c0 con 0x7f7108109b90 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.440+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f70f8031020 con 0x7f7108109b90 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.518+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f710802dd30 con 0x7f7108109b90 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.519+0000 7f7104c00700 1 -- 172.21.15.137:0/1730048342 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f70f801ba20 con 0x7f7108109b90 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.520+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 msgr2=0x7f710819d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.520+0000 7f710e000700 1 --2- 172.21.15.137:0/1730048342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f710819d2d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f70f8000c00 tx=0x7f70f8004750 comp rx=0 tx=0).stop 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.520+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 shutdown_connections 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.521+0000 7f710e000700 1 --2- 172.21.15.137:0/1730048342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7108109b90 0x7f710819d2d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.521+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 >> 172.21.15.137:0/1730048342 conn(0x7f710807d020 msgr2=0x7f7108193f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:48.306 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.521+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 shutdown_connections 2024-03-26T13:48:48.307 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:47.521+0000 7f710e000700 1 -- 172.21.15.137:0/1730048342 wait complete. 2024-03-26T13:48:48.307 INFO:teuthology.orchestra.run.smithi137.stdout:mon is available 2024-03-26T13:48:48.307 INFO:teuthology.orchestra.run.smithi137.stdout:Assimilating anything we can from ceph.conf... 2024-03-26T13:48:49.268 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:49 smithi137 ceph-mon[31164]: from='client.? 172.21.15.137:0/3881076900' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T13:48:49.268 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:49 smithi137 ceph-mon[31164]: from='client.? 172.21.15.137:0/3881076900' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-26T13:48:49.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:49.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T13:48:49.539 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout fsid = 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.137:3300,v1:172.21.15.137:6789] 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:49.543 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.889+0000 7f3f57e00700 1 Processor -- start 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.889+0000 7f3f57e00700 1 -- start start 2024-03-26T13:48:49.544 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.889+0000 7f3f57e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f5807eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.889+0000 7f3f57e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5807f170 con 0x7f3f5807e780 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.890+0000 7f3f56a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f5807eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.890+0000 7f3f56a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f5807eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43584/0 (socket says 172.21.15.137:43584) 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.890+0000 7f3f56a00700 1 -- 172.21.15.137:0/262193642 learned_addr learned my addr 172.21.15.137:0/262193642 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.890+0000 7f3f56a00700 1 -- 172.21.15.137:0/262193642 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5807f9d0 con 0x7f3f5807e780 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.891+0000 7f3f56a00700 1 --2- 172.21.15.137:0/262193642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f5807eba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3f480098a0 tx=0x7f3f48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94730f61ca5680f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.891+0000 7f3f55600700 1 -- 172.21.15.137:0/262193642 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f48004160 con 0x7f3f5807e780 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.891+0000 7f3f55600700 1 -- 172.21.15.137:0/262193642 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48004300 con 0x7f3f5807e780 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.891+0000 7f3f55600700 1 -- 172.21.15.137:0/262193642 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f480036a0 con 0x7f3f5807e780 2024-03-26T13:48:49.545 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.891+0000 7f3f57e00700 1 -- 172.21.15.137:0/262193642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 msgr2=0x7f3f5807eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:49.549 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.891+0000 7f3f57e00700 1 --2- 172.21.15.137:0/262193642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f5807eba0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3f480098a0 tx=0x7f3f48009bb0 comp rx=0 tx=0).stop 2024-03-26T13:48:49.549 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.892+0000 7f3f57e00700 1 -- 172.21.15.137:0/262193642 shutdown_connections 2024-03-26T13:48:49.549 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.892+0000 7f3f57e00700 1 --2- 172.21.15.137:0/262193642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f5807eba0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:49.549 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.892+0000 7f3f57e00700 1 -- 172.21.15.137:0/262193642 >> 172.21.15.137:0/262193642 conn(0x7f3f5807cd90 msgr2=0x7f3f5807d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.892+0000 7f3f57e00700 1 -- 172.21.15.137:0/262193642 shutdown_connections 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.892+0000 7f3f57e00700 1 -- 172.21.15.137:0/262193642 wait complete. 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.893+0000 7f3f57e00700 1 Processor -- start 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.893+0000 7f3f57e00700 1 -- start start 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.893+0000 7f3f57e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f581a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.893+0000 7f3f57e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5807ff00 con 0x7f3f5807e780 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f56a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f581a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f56a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f581a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43594/0 (socket says 172.21.15.137:43594) 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f56a00700 1 -- 172.21.15.137:0/3881076900 learned_addr learned my addr 172.21.15.137:0/3881076900 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f56a00700 1 -- 172.21.15.137:0/3881076900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f581a1a40 con 0x7f3f5807e780 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f56a00700 1 --2- 172.21.15.137:0/3881076900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f581a1500 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3f48000c00 tx=0x7f3f48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:49.550 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f48003db0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.894+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48011630 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.895+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f581a1cd0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.895+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f48011950 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.895+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f581a49c0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.895+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3f48011af0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.895+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f4801c450 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.896+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f580637c0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.899+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3f48032020 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.974+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3f5802dbb0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.983+0000 7f3f4f400700 1 -- 172.21.15.137:0/3881076900 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f3f48011de0 con 0x7f3f5807e780 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 msgr2=0x7f3f581a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:49.551 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 --2- 172.21.15.137:0/3881076900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f581a1500 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3f48000c00 tx=0x7f3f48004750 comp rx=0 tx=0).stop 2024-03-26T13:48:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 shutdown_connections 2024-03-26T13:48:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 --2- 172.21.15.137:0/3881076900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3f5807e780 0x7f3f581a1500 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 >> 172.21.15.137:0/3881076900 conn(0x7f3f5807cd90 msgr2=0x7f3f581077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 shutdown_connections 2024-03-26T13:48:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:48.985+0000 7f3f57e00700 1 -- 172.21.15.137:0/3881076900 wait complete. 2024-03-26T13:48:49.552 INFO:teuthology.orchestra.run.smithi137.stdout:Generating new minimal ceph.conf... 2024-03-26T13:48:50.517 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:50 smithi137 ceph-mon[31164]: from='client.? 172.21.15.137:0/3250554052' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:48:50.832 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.135+0000 7f6bb1a00700 1 Processor -- start 2024-03-26T13:48:50.832 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.135+0000 7f6bb1a00700 1 -- start start 2024-03-26T13:48:50.832 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.135+0000 7f6bb1a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac081de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.135+0000 7f6bb1a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0823b0 con 0x7f6bac0819c0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.136+0000 7f6baaa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac081de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.136+0000 7f6baaa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac081de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43596/0 (socket says 172.21.15.137:43596) 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.136+0000 7f6baaa00700 1 -- 172.21.15.137:0/3287132026 learned_addr learned my addr 172.21.15.137:0/3287132026 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.137+0000 7f6baaa00700 1 -- 172.21.15.137:0/3287132026 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac10f1a0 con 0x7f6bac0819c0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.137+0000 7f6baaa00700 1 --2- 172.21.15.137:0/3287132026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac081de0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0098a0 tx=0x7f6b9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f823daa8163834 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.137+0000 7f6baa000700 1 -- 172.21.15.137:0/3287132026 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c004160 con 0x7f6bac0819c0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.137+0000 7f6baa000700 1 -- 172.21.15.137:0/3287132026 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f6b9c004300 con 0x7f6bac0819c0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.137+0000 7f6baa000700 1 -- 172.21.15.137:0/3287132026 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c011a80 con 0x7f6bac0819c0 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3287132026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 msgr2=0x7f6bac081de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 --2- 172.21.15.137:0/3287132026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac081de0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0098a0 tx=0x7f6b9c009bb0 comp rx=0 tx=0).stop 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3287132026 shutdown_connections 2024-03-26T13:48:50.833 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 --2- 172.21.15.137:0/3287132026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac081de0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3287132026 >> 172.21.15.137:0/3287132026 conn(0x7f6bac07d020 msgr2=0x7f6bac07f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3287132026 shutdown_connections 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.138+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3287132026 wait complete. 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.139+0000 7f6bb1a00700 1 Processor -- start 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6bb1a00700 1 -- start start 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6bb1a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6bb1a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1123f0 con 0x7f6bac0819c0 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6baaa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6baaa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac1a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43600/0 (socket says 172.21.15.137:43600) 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6baaa00700 1 -- 172.21.15.137:0/3250554052 learned_addr learned my addr 172.21.15.137:0/3250554052 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.140+0000 7f6baaa00700 1 -- 172.21.15.137:0/3250554052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac1a2430 con 0x7f6bac0819c0 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.141+0000 7f6baaa00700 1 --2- 172.21.15.137:0/3250554052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac1a1ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006af0 tx=0x7f6b9c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:50.834 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.141+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c003930 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.141+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f6b9c003ad0 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.141+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b9c0195e0 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.141+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac1a26c0 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.141+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac1a53b0 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.142+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6b9c019c20 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.142+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6b9c024730 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.143+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac0637c0 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.145+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6b9c043dd0 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.221+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6bac1a5580 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.221+0000 7f6ba3400700 1 -- 172.21.15.137:0/3250554052 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7f6b9c032030 con 0x7f6bac0819c0 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.222+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 msgr2=0x7f6bac1a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.222+0000 7f6bb1a00700 1 --2- 172.21.15.137:0/3250554052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac1a1ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006af0 tx=0x7f6b9c011b70 comp rx=0 tx=0).stop 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.223+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 shutdown_connections 2024-03-26T13:48:50.835 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.223+0000 7f6bb1a00700 1 --2- 172.21.15.137:0/3250554052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bac0819c0 0x7f6bac1a1ef0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:50.836 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.223+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 >> 172.21.15.137:0/3250554052 conn(0x7f6bac07d020 msgr2=0x7f6bac07db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:50.836 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.223+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 shutdown_connections 2024-03-26T13:48:50.836 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:50.223+0000 7f6bb1a00700 1 -- 172.21.15.137:0/3250554052 wait complete. 2024-03-26T13:48:50.836 INFO:teuthology.orchestra.run.smithi137.stdout:Restarting the monitor... 2024-03-26T13:48:51.266 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:50 smithi137 systemd[1]: Stopping Ceph mon.a for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:48:51.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a[31160]: 2024-03-26T13:48:51.304+0000 7f698ba00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T13:48:51.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a[31160]: 2024-03-26T13:48:51.304+0000 7f698ba00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-26T13:48:51.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:51 smithi137 podman[31407]: 2024-03-26 13:48:51.407169513 +0000 UTC m=+0.218597525 container died 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T13:48:51.910 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:51 smithi137 podman[31407]: 2024-03-26 13:48:51.825496015 +0000 UTC m=+0.636924064 container cleanup 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=) 2024-03-26T13:48:51.910 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:51 smithi137 bash[31407]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a 2024-03-26T13:48:52.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 podman[31422]: 2024-03-26 13:48:52.057396102 +0000 UTC m=+0.635534464 container remove 81daf16c6023e752ef63ce155fd64b5516ddd3e8ea645a1329562e7fa55893ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T13:48:52.776 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 systemd[1]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-26T13:48:52.776 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 systemd[1]: Stopped Ceph mon.a for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:48:52.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 systemd[1]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service: Consumed 1.801s CPU time. 2024-03-26T13:48:52.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 systemd[1]: Starting Ceph mon.a for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:48:53.178 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 podman[31520]: 2024-03-26 13:48:52.799959282 +0000 UTC m=+0.035400210 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:48:53.179 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:52 smithi137 podman[31520]: 2024-03-26 13:48:52.916645631 +0000 UTC m=+0.152086582 container create 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:48:53.179 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 podman[31520]: 2024-03-26 13:48:53.186580489 +0000 UTC m=+0.422021461 container init 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T13:48:53.278 INFO:teuthology.orchestra.run.smithi137.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-26T13:48:53.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 podman[31520]: 2024-03-26 13:48:53.197031214 +0000 UTC m=+0.432472149 container start 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T13:48:53.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T13:48:53.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T13:48:53.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: pidfile_write: ignore empty --pid-file 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: load: jerasure load: lrc 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: RocksDB version: 7.9.2 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Git sha 0 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: DB SUMMARY 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: DB Session ID: 89MIEB45F8QLHXSQHU8H 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: CURRENT file: CURRENT 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100258 ; 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.error_if_exists: 0 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.create_if_missing: 0 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.paranoid_checks: 1 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T13:48:53.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.env: 0x562e8a9a6b20 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.info_log: 0x562e8b761960 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.statistics: (nil) 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.use_fsync: 0 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_log_file_size: 0 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T13:48:53.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.allow_fallocate: 1 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.use_direct_reads: 0 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.db_log_dir: 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.wal_dir: 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T13:48:53.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.write_buffer_manager: 0x562e8b7341e0 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T13:48:53.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.unordered_write: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.row_cache: None 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.wal_filter: None 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.two_write_queues: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.wal_compression: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.atomic_flush: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T13:48:53.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.log_readahead_size: 0 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_background_jobs: 2 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_background_compactions: -1 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_subcompactions: 1 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T13:48:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_open_files: -1 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_background_flushes: -1 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Compression algorithms supported: 2024-03-26T13:48:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kZSTD supported: 0 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kXpressCompression supported: 0 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kZlibCompression supported: 1 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kSnappyCompression supported: 1 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kLZ4Compression supported: 1 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: kBZip2Compression supported: 0 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.merge_operator: 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_filter: None 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T13:48:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e8b761a60) 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: index_type: 0 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: data_block_index_type: 0 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: index_shortening: 1 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: checksum: 4 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: no_block_cache: 0 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache: 0x562e8b767090 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-03-26T13:48:53.537 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache_options: 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: capacity : 536870912 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: num_shard_bits : 4 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: strict_capacity_limit : 0 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_cache_compressed: (nil) 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: persistent_cache: (nil) 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_size: 4096 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_size_deviation: 10 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_restart_interval: 16 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: index_block_restart_interval: 1 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: metadata_block_size: 4096 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: partition_filters: 0 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: use_delta_encoding: 1 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: filter_policy: bloomfilter 2024-03-26T13:48:53.538 INFO:journalctl@ceph.mon.a.smithi137.stdout: whole_key_filtering: 1 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: verify_compression: 0 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: format_version: 5 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: enable_index_compression: 1 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: block_align: 0 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: max_auto_readahead_size: 262144 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: prepopulate_block_cache: 0 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: initial_auto_readahead_size: 8192 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression: NoCompression 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.num_levels: 7 2024-03-26T13:48:53.539 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T13:48:53.540 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T13:48:53.540 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T13:48:53.540 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T13:48:53.540 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T13:48:53.541 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T13:48:53.542 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T13:48:53.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.table_properties_collectors: 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.inplace_update_support: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.bloom_locality: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.max_successive_merges: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.ttl: 2592000 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.enable_blob_files: false 2024-03-26T13:48:53.544 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.min_blob_size: 0 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 08a72a0a-102c-4ded-af9b-50e7add8f05b 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711460933253565, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711460933255718, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 93451, "index_size": 289, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13799, "raw_average_key_size": 50, "raw_value_size": 85920, "raw_average_value_size": 312, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711460933, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "08a72a0a-102c-4ded-af9b-50e7add8f05b", "db_session_id": "89MIEB45F8QLHXSQHU8H", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711460933255899, "job": 1, "event": "recovery_finished"} 2024-03-26T13:48:53.545 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e8b85a000 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: DB pointer 0x562e8b844000 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** DB Stats ** 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** Compaction Stats [default] ** 2024-03-26T13:48:53.546 INFO:journalctl@ceph.mon.a.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: L0 2/0 95.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: Sum 2/0 95.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 61.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** Compaction Stats [default] ** 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 61.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T13:48:53.547 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: Cumulative compaction: 0.00 GB write, 11.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: Interval compaction: 0.00 GB write, 11.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: Block cache BinnedLRUCache@0x562e8b767090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 2.6e-05 secs_since: 0 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: starting mon.a rank 0 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???) e1 preinit fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).mds e1 new map 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).mds e1 print_map 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: e1 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T13:48:53.548 INFO:journalctl@ceph.mon.a.smithi137.stdout: legacy client fscid: -1 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout: 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout: No filesystems configured 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 bash[31520]: 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 systemd[1]: Started Ceph mon.a for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: monmap e1: 1 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: fsmap 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: osdmap e1: 0 total, 0 up, 0 in 2024-03-26T13:48:53.549 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:53 smithi137 ceph-mon[31539]: mgrmap e1: no daemons active 2024-03-26T13:48:54.489 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.807+0000 7f10fda00700 1 Processor -- start 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.807+0000 7f10fda00700 1 -- start start 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.807+0000 7f10fda00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f8109de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.807+0000 7f10fda00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f810a3b0 con 0x7f10f81099c0 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.808+0000 7f10f7e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f8109de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.808+0000 7f10f7e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f8109de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43604/0 (socket says 172.21.15.137:43604) 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.808+0000 7f10f7e00700 1 -- 172.21.15.137:0/2542910098 learned_addr learned my addr 172.21.15.137:0/2542910098 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.809+0000 7f10f7e00700 1 -- 172.21.15.137:0/2542910098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f810abc0 con 0x7f10f81099c0 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.809+0000 7f10f7e00700 1 --2- 172.21.15.137:0/2542910098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f8109de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f10e80098a0 tx=0x7f10e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650a4184f47faf55 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.809+0000 7f10f6a00700 1 -- 172.21.15.137:0/2542910098 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e8004160 con 0x7f10f81099c0 2024-03-26T13:48:54.490 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.809+0000 7f10f6a00700 1 -- 172.21.15.137:0/2542910098 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f10e8004300 con 0x7f10f81099c0 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.809+0000 7f10f6a00700 1 -- 172.21.15.137:0/2542910098 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e8011a80 con 0x7f10f81099c0 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.810+0000 7f10fda00700 1 -- 172.21.15.137:0/2542910098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 msgr2=0x7f10f8109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.810+0000 7f10fda00700 1 --2- 172.21.15.137:0/2542910098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f8109de0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f10e80098a0 tx=0x7f10e8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.810+0000 7f10fda00700 1 -- 172.21.15.137:0/2542910098 shutdown_connections 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.810+0000 7f10fda00700 1 --2- 172.21.15.137:0/2542910098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f8109de0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.810+0000 7f10fda00700 1 -- 172.21.15.137:0/2542910098 >> 172.21.15.137:0/2542910098 conn(0x7f10f807cd90 msgr2=0x7f10f807d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.810+0000 7f10fda00700 1 -- 172.21.15.137:0/2542910098 shutdown_connections 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.811+0000 7f10fda00700 1 -- 172.21.15.137:0/2542910098 wait complete. 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.811+0000 7f10fda00700 1 Processor -- start 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.812+0000 7f10fda00700 1 -- start start 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.812+0000 7f10fda00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.812+0000 7f10fda00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f810df70 con 0x7f10f81099c0 2024-03-26T13:48:54.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.812+0000 7f10f7e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.812+0000 7f10f7e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f819d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43608/0 (socket says 172.21.15.137:43608) 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.812+0000 7f10f7e00700 1 -- 172.21.15.137:0/1386152490 learned_addr learned my addr 172.21.15.137:0/1386152490 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10f7e00700 1 -- 172.21.15.137:0/1386152490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f819de50 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10f7e00700 1 --2- 172.21.15.137:0/1386152490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f819d910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000c00 tx=0x7f10e8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e80039f0 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f10e8003b90 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10fda00700 1 -- 172.21.15.137:0/1386152490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f819e0e0 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e801a740 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.813+0000 7f10fda00700 1 -- 172.21.15.137:0/1386152490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f81a0dd0 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.814+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f10e802d3f0 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.814+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f10e8019700 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.814+0000 7f10fda00700 1 -- 172.21.15.137:0/1386152490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f80637c0 con 0x7f10f81099c0 2024-03-26T13:48:54.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.817+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f10e8024af0 con 0x7f10f81099c0 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.888+0000 7f10fda00700 1 -- 172.21.15.137:0/1386152490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f10f81a0fa0 con 0x7f10f81099c0 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.892+0000 7f10f4c00700 1 -- 172.21.15.137:0/1386152490 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f10e8032030 con 0x7f10f81099c0 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 -- 172.21.15.137:0/1386152490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 msgr2=0x7f10f819d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 --2- 172.21.15.137:0/1386152490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f819d910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f10e8000c00 tx=0x7f10e8011b00 comp rx=0 tx=0).stop 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 -- 172.21.15.137:0/1386152490 shutdown_connections 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 --2- 172.21.15.137:0/1386152490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f10f81099c0 0x7f10f819d910 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 -- 172.21.15.137:0/1386152490 >> 172.21.15.137:0/1386152490 conn(0x7f10f807cd90 msgr2=0x7f10f8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 -- 172.21.15.137:0/1386152490 shutdown_connections 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:53.893+0000 7f10ee000700 1 -- 172.21.15.137:0/1386152490 wait complete. 2024-03-26T13:48:54.493 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-26T13:48:54.515 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-26T13:48:54.515 INFO:teuthology.orchestra.run.smithi137.stdout:Creating mgr... 2024-03-26T13:48:54.517 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-26T13:48:54.518 INFO:teuthology.orchestra.run.smithi137.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-26T13:48:54.946 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x 2024-03-26T13:48:54.946 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Failed to reset failed state of unit ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x.service: Unit ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x.service not loaded. 2024-03-26T13:48:55.200 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715.target.wants/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@.service. 2024-03-26T13:48:55.230 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:54 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1386152490' entity='client.admin' 2024-03-26T13:48:55.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 systemd[1]: Starting Ceph mgr.x for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:48:55.878 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 podman[31753]: 2024-03-26 13:48:55.500770569 +0000 UTC m=+0.031555844 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T13:48:55.878 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 podman[31753]: 2024-03-26 13:48:55.601290929 +0000 UTC m=+0.132076235 container create fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:48:55.878 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 podman[31753]: 2024-03-26 13:48:55.821678854 +0000 UTC m=+0.352464200 container init fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-26T13:48:55.878 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 podman[31753]: 2024-03-26 13:48:55.832271113 +0000 UTC m=+0.363056385 container start fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-26T13:48:55.878 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 bash[31753]: fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 2024-03-26T13:48:55.907 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T13:48:55.907 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-03-26T13:48:55.923 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T13:48:55.923 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-03-26T13:48:55.923 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-03-26T13:48:55.923 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to enable service . firewalld.service is not available 2024-03-26T13:48:55.944 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T13:48:55.944 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-03-26T13:48:55.960 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T13:48:55.960 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-03-26T13:48:55.960 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-03-26T13:48:55.960 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-26T13:48:55.960 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr to start... 2024-03-26T13:48:55.960 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr... 2024-03-26T13:48:56.232 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:55 smithi137 systemd[1]: Started Ceph mgr.x for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:48:57.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:48:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/366363087' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T13:48:57.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:56 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:56.594+0000 7f75dbf10200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T13:48:57.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:56 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:56.723+0000 7f75dbf10200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T13:48:57.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:48:57.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:48:57.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T13:48:57.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T13:48:57.353 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T13:48:57.354 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T13:48:57.355 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T13:48:57.356 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T13:48:57.356 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T13:48:57.365 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T13:48:57.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T13:48:57.367 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T13:48:46.855492+0000", 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:48:57.368 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.603+0000 7f2fde800700 1 Processor -- start 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.603+0000 7f2fde800700 1 -- start start 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.603+0000 7f2fde800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd8106d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.603+0000 7f2fde800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81072d0 con 0x7f2fd81068e0 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.604+0000 7f2fd7400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd8106d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.604+0000 7f2fd7400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd8106d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43636/0 (socket says 172.21.15.137:43636) 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.604+0000 7f2fd7400700 1 -- 172.21.15.137:0/1137937528 learned_addr learned my addr 172.21.15.137:0/1137937528 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.604+0000 7f2fd7400700 1 -- 172.21.15.137:0/1137937528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8107ae0 con 0x7f2fd81068e0 2024-03-26T13:48:57.369 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.605+0000 7f2fd7400700 1 --2- 172.21.15.137:0/1137937528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd8106d00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80098a0 tx=0x7f2fc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423a042f54ea8d1d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.605+0000 7f2fd6000700 1 -- 172.21.15.137:0/1137937528 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fc8004160 con 0x7f2fd81068e0 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.605+0000 7f2fd6000700 1 -- 172.21.15.137:0/1137937528 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2fc8004300 con 0x7f2fd81068e0 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.605+0000 7f2fd6000700 1 -- 172.21.15.137:0/1137937528 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fc8011a80 con 0x7f2fd81068e0 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.606+0000 7f2fde800700 1 -- 172.21.15.137:0/1137937528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 msgr2=0x7f2fd8106d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.606+0000 7f2fde800700 1 --2- 172.21.15.137:0/1137937528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd8106d00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80098a0 tx=0x7f2fc8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.606+0000 7f2fde800700 1 -- 172.21.15.137:0/1137937528 shutdown_connections 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.606+0000 7f2fde800700 1 --2- 172.21.15.137:0/1137937528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd8106d00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.606+0000 7f2fde800700 1 -- 172.21.15.137:0/1137937528 >> 172.21.15.137:0/1137937528 conn(0x7f2fd8101ea0 msgr2=0x7f2fd81042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.607+0000 7f2fde800700 1 -- 172.21.15.137:0/1137937528 shutdown_connections 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.607+0000 7f2fde800700 1 -- 172.21.15.137:0/1137937528 wait complete. 2024-03-26T13:48:57.370 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.608+0000 7f2fde800700 1 Processor -- start 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.608+0000 7f2fde800700 1 -- start start 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.609+0000 7f2fde800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.609+0000 7f2fde800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd810ae90 con 0x7f2fd81068e0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.609+0000 7f2fd7400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.609+0000 7f2fd7400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43652/0 (socket says 172.21.15.137:43652) 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.609+0000 7f2fd7400700 1 -- 172.21.15.137:0/366363087 learned_addr learned my addr 172.21.15.137:0/366363087 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.609+0000 7f2fd7400700 1 -- 172.21.15.137:0/366363087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd819beb0 con 0x7f2fd81068e0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.610+0000 7f2fd7400700 1 --2- 172.21.15.137:0/366363087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd819b970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8000c00 tx=0x7f2fc8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.610+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fc80039c0 con 0x7f2fd81068e0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.610+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2fc8003b60 con 0x7f2fd81068e0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.610+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fc801a6f0 con 0x7f2fd81068e0 2024-03-26T13:48:57.371 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.610+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd819c140 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.610+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd819ee30 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.611+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2fc801ac20 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.611+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2fc8019700 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.611+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd80637c0 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.614+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2fc8004300 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.692+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2fd802dc70 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.693+0000 7f2fcfe00700 1 -- 172.21.15.137:0/366363087 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f2fc8032030 con 0x7f2fd81068e0 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.693+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 msgr2=0x7f2fd819b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.694+0000 7f2fde800700 1 --2- 172.21.15.137:0/366363087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd819b970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8000c00 tx=0x7f2fc8004290 comp rx=0 tx=0).stop 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.694+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 shutdown_connections 2024-03-26T13:48:57.372 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.694+0000 7f2fde800700 1 --2- 172.21.15.137:0/366363087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fd81068e0 0x7f2fd819b970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:48:57.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.694+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 >> 172.21.15.137:0/366363087 conn(0x7f2fd8101ea0 msgr2=0x7f2fd8102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:48:57.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.694+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 shutdown_connections 2024-03-26T13:48:57.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:56.694+0000 7f2fde800700 1 -- 172.21.15.137:0/366363087 wait complete. 2024-03-26T13:48:57.373 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (1/15)... 2024-03-26T13:48:57.645 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:57 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:57.533+0000 7f75dbf10200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T13:48:57.956 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:57 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:57.653+0000 7f75dbf10200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T13:48:57.956 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:57 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:57.836+0000 7f75dbf10200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T13:48:57.956 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:57 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:57.963+0000 7f75dbf10200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T13:48:58.277 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:58.070+0000 7f75dbf10200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T13:48:58.758 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:58.396+0000 7f75dbf10200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T13:48:58.758 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:58.534+0000 7f75dbf10200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T13:48:58.758 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:58.643+0000 7f75dbf10200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T13:48:59.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:58.764+0000 7f75dbf10200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T13:48:59.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:48:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:48:58.979+0000 7f75dbf10200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T13:49:00.357 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:00 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2010416160' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T13:49:00.357 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:00 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:00.297+0000 7f75dbf10200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T13:49:00.673 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:00 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:00.468+0000 7f75dbf10200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T13:49:00.688 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T13:49:00.689 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T13:49:00.690 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T13:49:00.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T13:49:00.698 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:49:00.699 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T13:48:46.855492+0000", 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.932+0000 7f8a25400700 1 Processor -- start 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.933+0000 7f8a25400700 1 -- start start 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.933+0000 7f8a25400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.933+0000 7f8a25400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2007c900 con 0x7f8a2007ded0 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.933+0000 7f8a1e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.933+0000 7f8a1e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54586/0 (socket says 172.21.15.137:54586) 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.933+0000 7f8a1e800700 1 -- 172.21.15.137:0/3697251478 learned_addr learned my addr 172.21.15.137:0/3697251478 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:00.700 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.934+0000 7f8a1e800700 1 -- 172.21.15.137:0/3697251478 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2007ca80 con 0x7f8a2007ded0 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.934+0000 7f8a1e800700 1 --2- 172.21.15.137:0/3697251478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2007c330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a100098a0 tx=0x7f8a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f492c6ffce48a1e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.934+0000 7f8a1d400700 1 -- 172.21.15.137:0/3697251478 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a10004160 con 0x7f8a2007ded0 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.934+0000 7f8a1d400700 1 -- 172.21.15.137:0/3697251478 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8a10004300 con 0x7f8a2007ded0 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.934+0000 7f8a1d400700 1 -- 172.21.15.137:0/3697251478 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a10011a80 con 0x7f8a2007ded0 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 -- 172.21.15.137:0/3697251478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 msgr2=0x7f8a2007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 --2- 172.21.15.137:0/3697251478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2007c330 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8a100098a0 tx=0x7f8a10009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 -- 172.21.15.137:0/3697251478 shutdown_connections 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 --2- 172.21.15.137:0/3697251478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2007c330 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 -- 172.21.15.137:0/3697251478 >> 172.21.15.137:0/3697251478 conn(0x7f8a20102fa0 msgr2=0x7f8a201053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 -- 172.21.15.137:0/3697251478 shutdown_connections 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.935+0000 7f8a25400700 1 -- 172.21.15.137:0/3697251478 wait complete. 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.936+0000 7f8a25400700 1 Processor -- start 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.936+0000 7f8a25400700 1 -- start start 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.937+0000 7f8a25400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.937+0000 7f8a25400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2010df70 con 0x7f8a2007ded0 2024-03-26T13:49:00.701 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.937+0000 7f8a1e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.937+0000 7f8a1e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54592/0 (socket says 172.21.15.137:54592) 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.937+0000 7f8a1e800700 1 -- 172.21.15.137:0/2010416160 learned_addr learned my addr 172.21.15.137:0/2010416160 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.937+0000 7f8a1e800700 1 -- 172.21.15.137:0/2010416160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2019e090 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a1e800700 1 --2- 172.21.15.137:0/2010416160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2019db50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a10000c00 tx=0x7f8a10004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a100039c0 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8a10003b60 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a1001a6f0 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2019e320 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a201a1010 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8a1001ac20 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.938+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8a10019700 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.939+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200637c0 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:48:59.941+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8a10004300 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.018+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8a201a11e0 con 0x7f8a2007ded0 2024-03-26T13:49:00.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.019+0000 7f8a17400700 1 -- 172.21.15.137:0/2010416160 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f8a10024a40 con 0x7f8a2007ded0 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 msgr2=0x7f8a2019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 --2- 172.21.15.137:0/2010416160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2019db50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a10000c00 tx=0x7f8a10004290 comp rx=0 tx=0).stop 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 shutdown_connections 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 --2- 172.21.15.137:0/2010416160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a2007ded0 0x7f8a2019db50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 >> 172.21.15.137:0/2010416160 conn(0x7f8a20102fa0 msgr2=0x7f8a20103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 shutdown_connections 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:00.020+0000 7f8a25400700 1 -- 172.21.15.137:0/2010416160 wait complete. 2024-03-26T13:49:00.703 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (2/15)... 2024-03-26T13:49:00.996 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:00 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:00.887+0000 7f75dbf10200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T13:49:00.996 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:01 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:01.002+0000 7f75dbf10200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T13:49:01.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:01 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:01.254+0000 7f75dbf10200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:49:01.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:01 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:01.495+0000 7f75dbf10200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T13:49:02.175 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:01 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:01.864+0000 7f75dbf10200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:49:02.175 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:02 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:02.182+0000 7f75dbf10200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T13:49:02.776 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:02 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:02.494+0000 7f75dbf10200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T13:49:03.711 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:03 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2060481632' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T13:49:03.712 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:03 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:03.579+0000 7f75dbf10200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T13:49:04.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:49:04.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:04.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T13:49:04.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T13:49:04.010 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T13:49:04.018 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T13:49:04.018 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T13:49:04.018 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T13:49:04.018 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.018 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T13:48:46.855492+0000", 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:04.020 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.267+0000 7fe1ed600700 1 Processor -- start 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.267+0000 7fe1ed600700 1 -- start start 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.267+0000 7fe1ed600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e8106d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.267+0000 7fe1ed600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8107320 con 0x7fe1e8106930 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.268+0000 7fe1e6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e8106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.268+0000 7fe1e6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e8106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54606/0 (socket says 172.21.15.137:54606) 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.268+0000 7fe1e6a00700 1 -- 172.21.15.137:0/81102164 learned_addr learned my addr 172.21.15.137:0/81102164 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.268+0000 7fe1e6a00700 1 -- 172.21.15.137:0/81102164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8107b30 con 0x7fe1e8106930 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.269+0000 7fe1e6a00700 1 --2- 172.21.15.137:0/81102164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e8106d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098a0 tx=0x7fe1d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec946bf4563c945 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.269+0000 7fe1e5600700 1 -- 172.21.15.137:0/81102164 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8004160 con 0x7fe1e8106930 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.269+0000 7fe1e5600700 1 -- 172.21.15.137:0/81102164 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe1d8004300 con 0x7fe1e8106930 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.269+0000 7fe1e5600700 1 -- 172.21.15.137:0/81102164 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d8011a80 con 0x7fe1e8106930 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 -- 172.21.15.137:0/81102164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 msgr2=0x7fe1e8106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:04.021 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 --2- 172.21.15.137:0/81102164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e8106d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1d80098a0 tx=0x7fe1d8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 -- 172.21.15.137:0/81102164 shutdown_connections 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 --2- 172.21.15.137:0/81102164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e8106d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 -- 172.21.15.137:0/81102164 >> 172.21.15.137:0/81102164 conn(0x7fe1e8101ef0 msgr2=0x7fe1e8104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 -- 172.21.15.137:0/81102164 shutdown_connections 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.270+0000 7fe1ed600700 1 -- 172.21.15.137:0/81102164 wait complete. 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.271+0000 7fe1ed600700 1 Processor -- start 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.271+0000 7fe1ed600700 1 -- start start 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.271+0000 7fe1ed600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.271+0000 7fe1ed600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e810aee0 con 0x7fe1e8106930 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.272+0000 7fe1e6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.272+0000 7fe1e6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e819b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54622/0 (socket says 172.21.15.137:54622) 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.272+0000 7fe1e6a00700 1 -- 172.21.15.137:0/2060481632 learned_addr learned my addr 172.21.15.137:0/2060481632 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.272+0000 7fe1e6a00700 1 -- 172.21.15.137:0/2060481632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e819bf10 con 0x7fe1e8106930 2024-03-26T13:49:04.022 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.272+0000 7fe1e6a00700 1 --2- 172.21.15.137:0/2060481632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e819b9d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8000c00 tx=0x7fe1d8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d80039c0 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe1d8003b60 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1d801a6f0 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e819c1a0 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e819ee90 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe1d801ac20 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.273+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe1d8019700 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.274+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e80637c0 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.277+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe1d8004300 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.356+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe1e802dd20 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.357+0000 7fe1df400700 1 -- 172.21.15.137:0/2060481632 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fe1d8024a40 con 0x7fe1e8106930 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 msgr2=0x7fe1e819b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:04.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 --2- 172.21.15.137:0/2060481632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e819b9d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8000c00 tx=0x7fe1d8004290 comp rx=0 tx=0).stop 2024-03-26T13:49:04.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 shutdown_connections 2024-03-26T13:49:04.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 --2- 172.21.15.137:0/2060481632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1e8106930 0x7fe1e819b9d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:04.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 >> 172.21.15.137:0/2060481632 conn(0x7fe1e8101ef0 msgr2=0x7fe1e8102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:04.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 shutdown_connections 2024-03-26T13:49:04.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:03.358+0000 7fe1ed600700 1 -- 172.21.15.137:0/2060481632 wait complete. 2024-03-26T13:49:04.024 INFO:teuthology.orchestra.run.smithi137.stdout:mgr not available, waiting (3/15)... 2024-03-26T13:49:04.685 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:04.283+0000 7f75dbf10200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T13:49:04.685 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:04.412+0000 7f75dbf10200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:04.691+0000 7f75dbf10200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: Activating manager daemon x 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: mgrmap e2: x(active, starting, since 0.00959519s) 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T13:49:05.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:49:05.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T13:49:05.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: Manager daemon x is now available 2024-03-26T13:49:05.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T13:49:05.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T13:49:05.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' 2024-03-26T13:49:05.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:04 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' 2024-03-26T13:49:06.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:05 smithi137 ceph-mon[31539]: from='mgr.14100 172.21.15.137:0/2495387637' entity='mgr.x' 2024-03-26T13:49:06.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:05 smithi137 ceph-mon[31539]: mgrmap e3: x(active, since 1.01862s) 2024-03-26T13:49:07.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:49:07.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:07.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T13:49:07.591 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "health": { 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 0 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-26T13:49:07.592 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "a" 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:07.593 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-26T13:49:07.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-26T13:49:07.594 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-26T13:49:07.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-26T13:49:07.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-26T13:49:07.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-26T13:49:07.607 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-26T13:49:07.608 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-26T13:49:07.609 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "restful" 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ], 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "modified": "2024-03-26T13:48:46.855492+0000", 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout }, 2024-03-26T13:49:07.610 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.640+0000 7f4923e00700 1 Processor -- start 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.640+0000 7f4923e00700 1 -- start start 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.641+0000 7f4923e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c109ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.641+0000 7f4923e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10a5c0 con 0x7f491c109bd0 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.641+0000 7f4921400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c109ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.641+0000 7f4921400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c109ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54696/0 (socket says 172.21.15.137:54696) 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.641+0000 7f4921400700 1 -- 172.21.15.137:0/2661120548 learned_addr learned my addr 172.21.15.137:0/2661120548 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.642+0000 7f4921400700 1 -- 172.21.15.137:0/2661120548 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c10add0 con 0x7f491c109bd0 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.642+0000 7f4921400700 1 --2- 172.21.15.137:0/2661120548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c109ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49100098a0 tx=0x7f4910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd744d08e70d42d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:07.611 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.642+0000 7f491be00700 1 -- 172.21.15.137:0/2661120548 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4910004160 con 0x7f491c109bd0 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.642+0000 7f491be00700 1 -- 172.21.15.137:0/2661120548 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f4910004300 con 0x7f491c109bd0 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.642+0000 7f491be00700 1 -- 172.21.15.137:0/2661120548 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4910011a80 con 0x7f491c109bd0 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 -- 172.21.15.137:0/2661120548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 msgr2=0x7f491c109ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 --2- 172.21.15.137:0/2661120548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c109ff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f49100098a0 tx=0x7f4910009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 -- 172.21.15.137:0/2661120548 shutdown_connections 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 --2- 172.21.15.137:0/2661120548 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c109ff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 -- 172.21.15.137:0/2661120548 >> 172.21.15.137:0/2661120548 conn(0x7f491c07cfc0 msgr2=0x7f491c07d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 -- 172.21.15.137:0/2661120548 shutdown_connections 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.643+0000 7f4923e00700 1 -- 172.21.15.137:0/2661120548 wait complete. 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.644+0000 7f4923e00700 1 Processor -- start 2024-03-26T13:49:07.612 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.644+0000 7f4923e00700 1 -- start start 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4923e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4923e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c10e180 con 0x7f491c109bd0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4921400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4921400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54710/0 (socket says 172.21.15.137:54710) 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4921400700 1 -- 172.21.15.137:0/3254103810 learned_addr learned my addr 172.21.15.137:0/3254103810 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4921400700 1 -- 172.21.15.137:0/3254103810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491c19e0a0 con 0x7f491c109bd0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.645+0000 7f4921400700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c19db60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4910000c00 tx=0x7f4910004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.646+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49100039c0 con 0x7f491c109bd0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.646+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f4910003b60 con 0x7f491c109bd0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.646+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f491001a6f0 con 0x7f491c109bd0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.646+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c19e330 con 0x7f491c109bd0 2024-03-26T13:49:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.646+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c1a1020 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.647+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c0637c0 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.647+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 3) v1 ==== 45989+0+0 (secure 0 0 0) 0x7f4910024a40 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.648+0000 7f491a000700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f4904038e00 0x7f490403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.648+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f491004cf20 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.648+0000 7f4920a00700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f4904038e00 0x7f490403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.649+0000 7f4920a00700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f4904038e00 0x7f490403b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4908006fd0 tx=0x7f4908006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.654+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4910004300 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.759+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f491002d770 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.963+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f491c1a11f0 con 0x7f491c109bd0 2024-03-26T13:49:07.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.964+0000 7f491a000700 1 -- 172.21.15.137:0/3254103810 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f491c1a11f0 con 0x7f491c109bd0 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f4904038e00 msgr2=0x7f490403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f4904038e00 0x7f490403b2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4908006fd0 tx=0x7f4908006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 msgr2=0x7f491c19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c19db60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4910000c00 tx=0x7f4910004290 comp rx=0 tx=0).stop 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 shutdown_connections 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f4904038e00 0x7f490403b2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 --2- 172.21.15.137:0/3254103810 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f491c109bd0 0x7f491c19db60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 >> 172.21.15.137:0/3254103810 conn(0x7f491c07cfc0 msgr2=0x7f491c1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 shutdown_connections 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:06.967+0000 7f4923e00700 1 -- 172.21.15.137:0/3254103810 wait complete. 2024-03-26T13:49:07.615 INFO:teuthology.orchestra.run.smithi137.stdout:mgr is available 2024-03-26T13:49:07.848 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:07 smithi137 ceph-mon[31539]: mgrmap e4: x(active, since 2s) 2024-03-26T13:49:07.848 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:07 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3254103810' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-26T13:49:09.011 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2578757662' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [global] 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout fsid = 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-26T13:49:09.366 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout [osd] 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.164+0000 7f96e1000700 1 Processor -- start 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.164+0000 7f96e1000700 1 -- start start 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.165+0000 7f96e1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc106b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:09.373 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.165+0000 7f96e1000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc107110 con 0x7f96dc106720 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.165+0000 7f96db400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc106b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.165+0000 7f96db400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc106b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48100/0 (socket says 172.21.15.137:48100) 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.165+0000 7f96db400700 1 -- 172.21.15.137:0/2136854059 learned_addr learned my addr 172.21.15.137:0/2136854059 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.166+0000 7f96db400700 1 -- 172.21.15.137:0/2136854059 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc107920 con 0x7f96dc106720 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.166+0000 7f96db400700 1 --2- 172.21.15.137:0/2136854059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc106b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00bb40 tx=0x7f96cc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681080ae86a3c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.166+0000 7f96da000700 1 -- 172.21.15.137:0/2136854059 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96cc00dbe0 con 0x7f96dc106720 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.166+0000 7f96da000700 1 -- 172.21.15.137:0/2136854059 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f96cc0044c0 con 0x7f96dc106720 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.166+0000 7f96da000700 1 -- 172.21.15.137:0/2136854059 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96cc010470 con 0x7f96dc106720 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 -- 172.21.15.137:0/2136854059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 msgr2=0x7f96dc106b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 --2- 172.21.15.137:0/2136854059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc106b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00bb40 tx=0x7f96cc00be50 comp rx=0 tx=0).stop 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 -- 172.21.15.137:0/2136854059 shutdown_connections 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 --2- 172.21.15.137:0/2136854059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc106b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:09.374 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 -- 172.21.15.137:0/2136854059 >> 172.21.15.137:0/2136854059 conn(0x7f96dc101d40 msgr2=0x7f96dc104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 -- 172.21.15.137:0/2136854059 shutdown_connections 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.167+0000 7f96e1000700 1 -- 172.21.15.137:0/2136854059 wait complete. 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.168+0000 7f96e1000700 1 Processor -- start 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.168+0000 7f96e1000700 1 -- start start 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.169+0000 7f96e1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.169+0000 7f96e1000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc07afe0 con 0x7f96dc106720 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.169+0000 7f96db400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.169+0000 7f96db400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc19d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48112/0 (socket says 172.21.15.137:48112) 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.169+0000 7f96db400700 1 -- 172.21.15.137:0/2578757662 learned_addr learned my addr 172.21.15.137:0/2578757662 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.169+0000 7f96db400700 1 -- 172.21.15.137:0/2578757662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc19df10 con 0x7f96dc106720 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.170+0000 7f96db400700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc19d9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00b340 tx=0x7f96cc003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.170+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96cc010720 con 0x7f96dc106720 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.170+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f96cc01d070 con 0x7f96dc106720 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.170+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc19e1a0 con 0x7f96dc106720 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.170+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96cc021ac0 con 0x7f96dc106720 2024-03-26T13:49:09.375 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.170+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc1a0e90 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.171+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f96cc01f030 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.171+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0637c0 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.172+0000 7f96d3e00700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f96c0038ed0 0x7f96c003b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.172+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f96cc04cc80 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.172+0000 7f96daa00700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f96c0038ed0 0x7f96c003b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.173+0000 7f96daa00700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f96c0038ed0 0x7f96c003b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96c8006fd0 tx=0x7f96c8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.177+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f96cc015070 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.397+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f96dc1a1060 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.399+0000 7f96d3e00700 1 -- 172.21.15.137:0/2578757662 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f96dc1a1060 con 0x7f96dc106720 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f96c0038ed0 msgr2=0x7f96c003b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f96c0038ed0 0x7f96c003b390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96c8006fd0 tx=0x7f96c8006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 msgr2=0x7f96dc19d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc19d9d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00b340 tx=0x7f96cc003fe0 comp rx=0 tx=0).stop 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 shutdown_connections 2024-03-26T13:49:09.376 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f96c0038ed0 0x7f96c003b390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:09.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 --2- 172.21.15.137:0/2578757662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96dc106720 0x7f96dc19d9d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:09.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 >> 172.21.15.137:0/2578757662 conn(0x7f96dc101d40 msgr2=0x7f96dc194010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:09.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 shutdown_connections 2024-03-26T13:49:09.377 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:08.403+0000 7f96e1000700 1 -- 172.21.15.137:0/2578757662 wait complete. 2024-03-26T13:49:09.377 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling cephadm module... 2024-03-26T13:49:10.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/703211791' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-26T13:49:10.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:10 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: ignoring --setuser ceph since I am not root 2024-03-26T13:49:10.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:10 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: ignoring --setgroup ceph since I am not root 2024-03-26T13:49:10.998 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.938+0000 7f2777800700 1 Processor -- start 2024-03-26T13:49:10.998 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.938+0000 7f2777800700 1 -- start start 2024-03-26T13:49:10.998 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.939+0000 7f2777800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:10.998 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.939+0000 7f2777800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277007c8c0 con 0x7f277007de90 2024-03-26T13:49:10.999 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.939+0000 7f2774e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:10.999 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.939+0000 7f2774e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277007c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48124/0 (socket says 172.21.15.137:48124) 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.939+0000 7f2774e00700 1 -- 172.21.15.137:0/1204780634 learned_addr learned my addr 172.21.15.137:0/1204780634 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.939+0000 7f2774e00700 1 -- 172.21.15.137:0/1204780634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277007ca40 con 0x7f277007de90 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.940+0000 7f2774e00700 1 --2- 172.21.15.137:0/1204780634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277007c2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27640098a0 tx=0x7f2764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87659cdb22a5bebb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.940+0000 7f276f400700 1 -- 172.21.15.137:0/1204780634 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2764004160 con 0x7f277007de90 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.940+0000 7f276f400700 1 -- 172.21.15.137:0/1204780634 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2764004300 con 0x7f277007de90 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.940+0000 7f276f400700 1 -- 172.21.15.137:0/1204780634 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2764011a80 con 0x7f277007de90 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 -- 172.21.15.137:0/1204780634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 msgr2=0x7f277007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 --2- 172.21.15.137:0/1204780634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277007c2f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f27640098a0 tx=0x7f2764009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 -- 172.21.15.137:0/1204780634 shutdown_connections 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 --2- 172.21.15.137:0/1204780634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277007c2f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 -- 172.21.15.137:0/1204780634 >> 172.21.15.137:0/1204780634 conn(0x7f2770102ff0 msgr2=0x7f2770105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:11.005 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 -- 172.21.15.137:0/1204780634 shutdown_connections 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.941+0000 7f2777800700 1 -- 172.21.15.137:0/1204780634 wait complete. 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.942+0000 7f2777800700 1 Processor -- start 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.942+0000 7f2777800700 1 -- start start 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.943+0000 7f2777800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.943+0000 7f2777800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277010dfc0 con 0x7f277007de90 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.943+0000 7f2774e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.943+0000 7f2774e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277019db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48140/0 (socket says 172.21.15.137:48140) 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.943+0000 7f2774e00700 1 -- 172.21.15.137:0/703211791 learned_addr learned my addr 172.21.15.137:0/703211791 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.943+0000 7f2774e00700 1 -- 172.21.15.137:0/703211791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277019e090 con 0x7f277007de90 2024-03-26T13:49:11.006 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.944+0000 7f2774e00700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277019db50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2764011a50 tx=0x7f2764004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.944+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27640039f0 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.944+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2764003b90 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.944+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277019e320 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.944+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f276401a600 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.944+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701a1010 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.945+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f276401a8b0 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.945+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27700637c0 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.945+0000 7f276d600700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f2758038f20 0x7f275803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.946+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f276404d360 con 0x7f277007de90 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.946+0000 7f276fe00700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f2758038f20 0x7f275803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.947+0000 7f276fe00700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f2758038f20 0x7f275803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f275c006fd0 tx=0x7f275c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:11.007 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:09.951+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f276404cd80 con 0x7f277007de90 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.210+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f27701a11e0 con 0x7f277007de90 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.264+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f27701a11e0 con 0x7f277007de90 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.267+0000 7f276d600700 1 -- 172.21.15.137:0/703211791 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f2764024a40 con 0x7f277007de90 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f2758038f20 msgr2=0x7f275803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f2758038f20 0x7f275803b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f275c006fd0 tx=0x7f275c006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 msgr2=0x7f277019db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277019db50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2764011a50 tx=0x7f2764004290 comp rx=0 tx=0).stop 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 shutdown_connections 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f2758038f20 0x7f275803b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 --2- 172.21.15.137:0/703211791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f277007de90 0x7f277019db50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.268+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 >> 172.21.15.137:0/703211791 conn(0x7f2770102ff0 msgr2=0x7f2770103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:11.008 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.269+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 shutdown_connections 2024-03-26T13:49:11.009 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:10.269+0000 7f2777800700 1 -- 172.21.15.137:0/703211791 wait complete. 2024-03-26T13:49:11.265 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:11 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:11.043+0000 7f581b3c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T13:49:11.265 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:11 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:11.172+0000 7f581b3c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T13:49:11.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:11 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/703211791' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-26T13:49:11.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:11 smithi137 ceph-mon[31539]: mgrmap e5: x(active, since 5s) 2024-03-26T13:49:12.233 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:11 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:11.966+0000 7f581b3c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T13:49:12.233 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:12.088+0000 7f581b3c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T13:49:12.500 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2134097797' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T13:49:12.501 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:12.273+0000 7f581b3c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T13:49:12.501 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:12.398+0000 7f581b3c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T13:49:12.501 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:12.508+0000 7f581b3c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T13:49:12.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.635+0000 7f858fc00700 1 Processor -- start 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.635+0000 7f858fc00700 1 -- start start 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.635+0000 7f858fc00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f8588107e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.635+0000 7f858fc00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588108450 con 0x7f8588107a60 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.636+0000 7f858d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f8588107e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.636+0000 7f858d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f8588107e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48166/0 (socket says 172.21.15.137:48166) 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.636+0000 7f858d200700 1 -- 172.21.15.137:0/56813836 learned_addr learned my addr 172.21.15.137:0/56813836 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.636+0000 7f858d200700 1 -- 172.21.15.137:0/56813836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588108c10 con 0x7f8588107a60 2024-03-26T13:49:12.579 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.637+0000 7f858d200700 1 --2- 172.21.15.137:0/56813836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f8588107e80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f857c0098a0 tx=0x7f857c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7ab6dec710ce94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.637+0000 7f8587400700 1 -- 172.21.15.137:0/56813836 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f857c004160 con 0x7f8588107a60 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.637+0000 7f8587400700 1 -- 172.21.15.137:0/56813836 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f857c004300 con 0x7f8588107a60 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.637+0000 7f8587400700 1 -- 172.21.15.137:0/56813836 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f857c011a80 con 0x7f8588107a60 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 -- 172.21.15.137:0/56813836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 msgr2=0x7f8588107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 --2- 172.21.15.137:0/56813836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f8588107e80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f857c0098a0 tx=0x7f857c009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 -- 172.21.15.137:0/56813836 shutdown_connections 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 --2- 172.21.15.137:0/56813836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f8588107e80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 -- 172.21.15.137:0/56813836 >> 172.21.15.137:0/56813836 conn(0x7f8588103020 msgr2=0x7f8588105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 -- 172.21.15.137:0/56813836 shutdown_connections 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.638+0000 7f858fc00700 1 -- 172.21.15.137:0/56813836 wait complete. 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.639+0000 7f858fc00700 1 Processor -- start 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.639+0000 7f858fc00700 1 -- start start 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.639+0000 7f858fc00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f858819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:12.580 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.639+0000 7f858fc00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858810bfc0 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.639+0000 7f858d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f858819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.640+0000 7f858d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f858819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48168/0 (socket says 172.21.15.137:48168) 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.640+0000 7f858d200700 1 -- 172.21.15.137:0/2134097797 learned_addr learned my addr 172.21.15.137:0/2134097797 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.640+0000 7f858d200700 1 -- 172.21.15.137:0/2134097797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858819e100 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.640+0000 7f858d200700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f858819dbc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f857c000c00 tx=0x7f857c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.640+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f857c0039c0 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.641+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f857c003b60 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.641+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858819e390 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.641+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f857c01a6f0 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.641+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85881a1080 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.642+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f857c01a890 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.642+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85880637c0 con 0x7f8588107a60 2024-03-26T13:49:12.581 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.642+0000 7f8585600700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 0x7f857003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.642+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f857c04d220 con 0x7f8588107a60 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.642+0000 7f8587e00700 1 -- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 msgr2=0x7f857003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.642+0000 7f8587e00700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 0x7f857003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.647+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f857c00bc10 con 0x7f8588107a60 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.843+0000 7f8587e00700 1 -- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 msgr2=0x7f857003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.843+0000 7f8587e00700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 0x7f857003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.939+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f85881a1250 con 0x7f8588107a60 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.940+0000 7f8585600700 1 -- 172.21.15.137:0/2134097797 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f85881a1250 con 0x7f8588107a60 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.942+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 msgr2=0x7f857003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.942+0000 7f858fc00700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 0x7f857003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.942+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 msgr2=0x7f858819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.942+0000 7f858fc00700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f858819dbc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f857c000c00 tx=0x7f857c004290 comp rx=0 tx=0).stop 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.943+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 shutdown_connections 2024-03-26T13:49:12.582 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.943+0000 7f858fc00700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7f8570038f70 0x7f857003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:12.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.943+0000 7f858fc00700 1 --2- 172.21.15.137:0/2134097797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8588107a60 0x7f858819dbc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:12.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.943+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 >> 172.21.15.137:0/2134097797 conn(0x7f8588103020 msgr2=0x7f8588103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:12.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.943+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 shutdown_connections 2024-03-26T13:49:12.583 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:11.943+0000 7f858fc00700 1 -- 172.21.15.137:0/2134097797 wait complete. 2024-03-26T13:49:12.583 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for the mgr to restart... 2024-03-26T13:49:12.583 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr epoch 5... 2024-03-26T13:49:13.105 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:12.852+0000 7f581b3c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T13:49:13.105 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:12.995+0000 7f581b3c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T13:49:13.105 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:13 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:13.112+0000 7f581b3c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T13:49:13.447 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:13 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:13.238+0000 7f581b3c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T13:49:13.448 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:13 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:13.455+0000 7f581b3c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T13:49:14.880 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:14.718+0000 7f581b3c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T13:49:14.880 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:14.887+0000 7f581b3c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T13:49:15.661 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:15.302+0000 7f581b3c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T13:49:15.661 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:15.418+0000 7f581b3c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T13:49:15.661 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:15.668+0000 7f581b3c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:49:16.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:15.909+0000 7f581b3c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T13:49:16.527 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:16.279+0000 7f581b3c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:49:16.888 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:16.597+0000 7f581b3c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T13:49:16.888 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:16.895+0000 7f581b3c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T13:49:18.277 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:17.901+0000 7f581b3c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T13:49:18.990 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:18.592+0000 7f581b3c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T13:49:18.990 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:18.719+0000 7f581b3c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T13:49:19.277 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:18.998+0000 7f581b3c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T13:49:19.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: Active manager daemon x restarted 2024-03-26T13:49:19.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: Activating manager daemon x 2024-03-26T13:49:19.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: osdmap e2: 0 total, 0 up, 0 in 2024-03-26T13:49:19.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: mgrmap e6: x(active, starting, since 0.00882728s) 2024-03-26T13:49:19.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:49:19.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T13:49:19.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T13:49:19.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T13:49:19.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T13:49:19.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: Manager daemon x is now available 2024-03-26T13:49:19.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:19 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: Found migration_current of "None". Setting to last migration. 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T13:49:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:20.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:20.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:20 smithi137 ceph-mon[31539]: mgrmap e7: x(active, since 1.01351s) 2024-03-26T13:49:20.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:20.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-26T13:49:20.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T13:49:20.740 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.141+0000 7fcafae00700 1 Processor -- start 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.141+0000 7fcafae00700 1 -- start start 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.142+0000 7fcafae00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf407c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.142+0000 7fcafae00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf407c8c0 con 0x7fcaf407de90 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.142+0000 7fcaf3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf407c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.142+0000 7fcaf3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf407c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48184/0 (socket says 172.21.15.137:48184) 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.142+0000 7fcaf3e00700 1 -- 172.21.15.137:0/2060059615 learned_addr learned my addr 172.21.15.137:0/2060059615 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.143+0000 7fcaf3e00700 1 -- 172.21.15.137:0/2060059615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf407ca40 con 0x7fcaf407de90 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.143+0000 7fcaf3e00700 1 --2- 172.21.15.137:0/2060059615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf407c2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcae40098a0 tx=0x7fcae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e09f1a4c588afc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:20.741 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.143+0000 7fcaf2a00700 1 -- 172.21.15.137:0/2060059615 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae4004160 con 0x7fcaf407de90 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.143+0000 7fcaf2a00700 1 -- 172.21.15.137:0/2060059615 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fcae4004300 con 0x7fcaf407de90 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.143+0000 7fcaf2a00700 1 -- 172.21.15.137:0/2060059615 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae4011a80 con 0x7fcaf407de90 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 -- 172.21.15.137:0/2060059615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 msgr2=0x7fcaf407c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 --2- 172.21.15.137:0/2060059615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf407c2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcae40098a0 tx=0x7fcae4009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 -- 172.21.15.137:0/2060059615 shutdown_connections 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 --2- 172.21.15.137:0/2060059615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf407c2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 -- 172.21.15.137:0/2060059615 >> 172.21.15.137:0/2060059615 conn(0x7fcaf4102fa0 msgr2=0x7fcaf41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 -- 172.21.15.137:0/2060059615 shutdown_connections 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.144+0000 7fcafae00700 1 -- 172.21.15.137:0/2060059615 wait complete. 2024-03-26T13:49:20.742 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.145+0000 7fcafae00700 1 Processor -- start 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.145+0000 7fcafae00700 1 -- start start 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.145+0000 7fcafae00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf419db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.145+0000 7fcafae00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf410dfb0 con 0x7fcaf407de90 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.146+0000 7fcaf3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf419db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.146+0000 7fcaf3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf419db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48196/0 (socket says 172.21.15.137:48196) 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.146+0000 7fcaf3e00700 1 -- 172.21.15.137:0/3846837918 learned_addr learned my addr 172.21.15.137:0/3846837918 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.146+0000 7fcaf3e00700 1 -- 172.21.15.137:0/3846837918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf419e040 con 0x7fcaf407de90 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.146+0000 7fcaf3e00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf419db00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcae4011a50 tx=0x7fcae4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.147+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae40039f0 con 0x7fcaf407de90 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.147+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf419e2d0 con 0x7fcaf407de90 2024-03-26T13:49:20.743 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.147+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fcae4003b90 con 0x7fcaf407de90 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.147+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcae401a600 con 0x7fcaf407de90 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.147+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf41a0fc0 con 0x7fcaf407de90 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.148+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fcae401a900 con 0x7fcaf407de90 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.148+0000 7fcaf0c00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.148+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 --> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcad803baf0 con 0x7fcad8038f20 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.148+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcae404d2f0 con 0x7fcaf407de90 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.148+0000 7fcaf3400700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 msgr2=0x7fcad803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.148+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.349+0000 7fcaf3400700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 msgr2=0x7fcad803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:20.744 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.349+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.750+0000 7fcaf3400700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 msgr2=0x7fcad803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:13.750+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:14.551+0000 7fcaf3400700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 msgr2=0x7fcad803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:14.551+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:16.153+0000 7fcaf3400700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 msgr2=0x7fcad803b3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1955125876 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:16.153+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:19.007+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fcae4024a40 con 0x7fcaf407de90 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:19.007+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 msgr2=0x7fcad803b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:19.007+0000 7fcaf0c00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.008+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fcae404f210 con 0x7fcaf407de90 2024-03-26T13:49:20.745 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.008+0000 7fcaf0c00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcad803c810 0x7fcad803ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.008+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcae4012ea0 con 0x7fcad803c810 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.010+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcad803c810 0x7fcad803ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.012+0000 7fcaf3400700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcad803c810 0x7fcad803ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcae0003a10 tx=0x7fcae0009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.013+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fcae4012ea0 con 0x7fcad803c810 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.023+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcaf40637c0 con 0x7fcad803c810 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcaf0c00700 1 -- 172.21.15.137:0/3846837918 <== mgr.14120 v2:172.21.15.137:6800/1621047019 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fcaf40637c0 con 0x7fcad803c810 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcad803c810 msgr2=0x7fcad803ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcad803c810 0x7fcad803ec00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcae0003a10 tx=0x7fcae0009290 comp rx=0 tx=0).stop 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 msgr2=0x7fcaf419db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf419db00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcae4011a50 tx=0x7fcae4004290 comp rx=0 tx=0).stop 2024-03-26T13:49:20.746 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 shutdown_connections 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcad803c810 0x7fcad803ec00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaf407de90 0x7fcaf419db00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 --2- 172.21.15.137:0/3846837918 >> [v2:172.21.15.137:6800/1955125876,v1:172.21.15.137:6801/1955125876] conn(0x7fcad8038f20 0x7fcad803b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.024+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 >> 172.21.15.137:0/3846837918 conn(0x7fcaf4102fa0 msgr2=0x7fcaf4103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.025+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 shutdown_connections 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:20.025+0000 7fcafae00700 1 -- 172.21.15.137:0/3846837918 wait complete. 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:mgr epoch 5 is available 2024-03-26T13:49:20.747 INFO:teuthology.orchestra.run.smithi137.stdout:Setting orchestrator backend to cephadm... 2024-03-26T13:49:21.269 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T13:49:21.269 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T13:49:21.269 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:20] ENGINE Bus STARTING 2024-03-26T13:49:21.269 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:20] ENGINE Serving on http://172.21.15.137:8765 2024-03-26T13:49:21.270 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:20] ENGINE Serving on https://172.21.15.137:7150 2024-03-26T13:49:21.270 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:20] ENGINE Bus STARTED 2024-03-26T13:49:21.270 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:21 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:22.272 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:22 smithi137 ceph-mon[31539]: mgrmap e8: x(active, since 2s) 2024-03-26T13:49:22.273 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:22 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:22.273 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:22 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:22.284 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.337+0000 7fd4fd200700 1 Processor -- start 2024-03-26T13:49:22.284 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.337+0000 7fd4fd200700 1 -- start start 2024-03-26T13:49:22.284 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.338+0000 7fd4fd200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f8109fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:22.285 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.338+0000 7fd4fd200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f810a5a0 con 0x7fd4f8109bb0 2024-03-26T13:49:22.285 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.338+0000 7fd4f5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f8109fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:22.286 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.338+0000 7fd4f5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f8109fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48070/0 (socket says 172.21.15.137:48070) 2024-03-26T13:49:22.286 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.338+0000 7fd4f5e00700 1 -- 172.21.15.137:0/798631490 learned_addr learned my addr 172.21.15.137:0/798631490 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:22.286 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.339+0000 7fd4f5e00700 1 -- 172.21.15.137:0/798631490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f810adb0 con 0x7fd4f8109bb0 2024-03-26T13:49:22.286 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.339+0000 7fd4f5e00700 1 --2- 172.21.15.137:0/798631490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f8109fd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80098a0 tx=0x7fd4e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3cedab633c24b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:22.287 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.339+0000 7fd4f5400700 1 -- 172.21.15.137:0/798631490 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4e8004160 con 0x7fd4f8109bb0 2024-03-26T13:49:22.287 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.339+0000 7fd4f5400700 1 -- 172.21.15.137:0/798631490 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd4e8004300 con 0x7fd4f8109bb0 2024-03-26T13:49:22.287 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.339+0000 7fd4f5400700 1 -- 172.21.15.137:0/798631490 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4e8011a80 con 0x7fd4f8109bb0 2024-03-26T13:49:22.287 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 -- 172.21.15.137:0/798631490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 msgr2=0x7fd4f8109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:22.287 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 --2- 172.21.15.137:0/798631490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f8109fd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80098a0 tx=0x7fd4e8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:22.288 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 -- 172.21.15.137:0/798631490 shutdown_connections 2024-03-26T13:49:22.288 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 --2- 172.21.15.137:0/798631490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f8109fd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:22.288 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 -- 172.21.15.137:0/798631490 >> 172.21.15.137:0/798631490 conn(0x7fd4f807d000 msgr2=0x7fd4f807d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:22.289 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 -- 172.21.15.137:0/798631490 shutdown_connections 2024-03-26T13:49:22.289 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.340+0000 7fd4fd200700 1 -- 172.21.15.137:0/798631490 wait complete. 2024-03-26T13:49:22.289 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.342+0000 7fd4fd200700 1 Processor -- start 2024-03-26T13:49:22.289 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.342+0000 7fd4fd200700 1 -- start start 2024-03-26T13:49:22.289 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.342+0000 7fd4fd200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:22.290 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.342+0000 7fd4fd200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f810e160 con 0x7fd4f8109bb0 2024-03-26T13:49:22.290 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.343+0000 7fd4f5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:22.290 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.343+0000 7fd4f5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48078/0 (socket says 172.21.15.137:48078) 2024-03-26T13:49:22.290 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.343+0000 7fd4f5e00700 1 -- 172.21.15.137:0/3780636886 learned_addr learned my addr 172.21.15.137:0/3780636886 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:22.291 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.343+0000 7fd4f5e00700 1 -- 172.21.15.137:0/3780636886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f819e0a0 con 0x7fd4f8109bb0 2024-03-26T13:49:22.291 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.343+0000 7fd4f5e00700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f819db60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8000c00 tx=0x7fd4e8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:22.291 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.344+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4e80039c0 con 0x7fd4f8109bb0 2024-03-26T13:49:22.291 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.344+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd4e8003b60 con 0x7fd4f8109bb0 2024-03-26T13:49:22.291 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.344+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4e801a6f0 con 0x7fd4f8109bb0 2024-03-26T13:49:22.292 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.344+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f819e330 con 0x7fd4f8109bb0 2024-03-26T13:49:22.292 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.344+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f81a1020 con 0x7fd4f8109bb0 2024-03-26T13:49:22.292 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.345+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd4e8024a40 con 0x7fd4f8109bb0 2024-03-26T13:49:22.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.345+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f80637c0 con 0x7fd4f8109bb0 2024-03-26T13:49:22.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.345+0000 7fd4eea00700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fd4dc038f70 0x7fd4dc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:22.293 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.346+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd4e804d250 con 0x7fd4f8109bb0 2024-03-26T13:49:22.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.346+0000 7fd4efe00700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fd4dc038f70 0x7fd4dc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:22.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.347+0000 7fd4efe00700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fd4dc038f70 0x7fd4dc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:22.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.352+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4e8004300 con 0x7fd4f8109bb0 2024-03-26T13:49:22.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.593+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4f8107de0 con 0x7fd4dc038f70 2024-03-26T13:49:22.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.606+0000 7fd4eea00700 1 -- 172.21.15.137:0/3780636886 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd4f8107de0 con 0x7fd4dc038f70 2024-03-26T13:49:22.294 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.609+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fd4dc038f70 msgr2=0x7fd4dc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:22.295 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.609+0000 7fd4fd200700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fd4dc038f70 0x7fd4dc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006fd0 tx=0x7fd4e0006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:22.295 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.609+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 msgr2=0x7fd4f819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:22.295 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.609+0000 7fd4fd200700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f819db60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8000c00 tx=0x7fd4e8004290 comp rx=0 tx=0).stop 2024-03-26T13:49:22.295 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.609+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 shutdown_connections 2024-03-26T13:49:22.295 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.609+0000 7fd4fd200700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fd4dc038f70 0x7fd4dc03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:22.296 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.610+0000 7fd4fd200700 1 --2- 172.21.15.137:0/3780636886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4f8109bb0 0x7fd4f819db60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:22.296 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.610+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 >> 172.21.15.137:0/3780636886 conn(0x7fd4f807d000 msgr2=0x7fd4f81076d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:22.296 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.610+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 shutdown_connections 2024-03-26T13:49:22.296 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:21.610+0000 7fd4fd200700 1 -- 172.21.15.137:0/3780636886 wait complete. 2024-03-26T13:49:23.112 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:23 smithi137 ceph-mon[31539]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:23.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.859+0000 7f2250e00700 1 Processor -- start 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.859+0000 7f2250e00700 1 -- start start 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.860+0000 7f2250e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c106700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.860+0000 7f2250e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c07be20 con 0x7f224c1062e0 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.860+0000 7f2249e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.860+0000 7f2249e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48090/0 (socket says 172.21.15.137:48090) 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.860+0000 7f2249e00700 1 -- 172.21.15.137:0/2172007987 learned_addr learned my addr 172.21.15.137:0/2172007987 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.861+0000 7f2249e00700 1 -- 172.21.15.137:0/2172007987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c106c40 con 0x7f224c1062e0 2024-03-26T13:49:23.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.861+0000 7f2249e00700 1 --2- 172.21.15.137:0/2172007987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c106700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f223c0098a0 tx=0x7f223c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8708c202852433 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.861+0000 7f2248a00700 1 -- 172.21.15.137:0/2172007987 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c004160 con 0x7f224c1062e0 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2248a00700 1 -- 172.21.15.137:0/2172007987 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f223c004300 con 0x7f224c1062e0 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2248a00700 1 -- 172.21.15.137:0/2172007987 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c011a80 con 0x7f224c1062e0 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2250e00700 1 -- 172.21.15.137:0/2172007987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 msgr2=0x7f224c106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2250e00700 1 --2- 172.21.15.137:0/2172007987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c106700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f223c0098a0 tx=0x7f223c009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2250e00700 1 -- 172.21.15.137:0/2172007987 shutdown_connections 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2250e00700 1 --2- 172.21.15.137:0/2172007987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c106700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2250e00700 1 -- 172.21.15.137:0/2172007987 >> 172.21.15.137:0/2172007987 conn(0x7f224c101ea0 msgr2=0x7f224c1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.862+0000 7f2250e00700 1 -- 172.21.15.137:0/2172007987 shutdown_connections 2024-03-26T13:49:23.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.863+0000 7f2250e00700 1 -- 172.21.15.137:0/2172007987 wait complete. 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.863+0000 7f2250e00700 1 Processor -- start 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.864+0000 7f2250e00700 1 -- start start 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.864+0000 7f2250e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.864+0000 7f2250e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c107c70 con 0x7f224c1062e0 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.864+0000 7f2249e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.864+0000 7f2249e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c19db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48096/0 (socket says 172.21.15.137:48096) 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.864+0000 7f2249e00700 1 -- 172.21.15.137:0/4039300515 learned_addr learned my addr 172.21.15.137:0/4039300515 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2249e00700 1 -- 172.21.15.137:0/4039300515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c19e070 con 0x7f224c1062e0 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2249e00700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c19db30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f223c011a50 tx=0x7f223c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c0039f0 con 0x7f224c1062e0 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f223c003b90 con 0x7f224c1062e0 2024-03-26T13:49:23.888 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f223c01a600 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c19e300 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.865+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c1a0ff0 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.867+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f223c024a40 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.867+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c0637c0 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.867+0000 7f2242a00700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f2230038f70 0x7f223003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.867+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f223c04d390 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.868+0000 7f2249400700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f2230038f70 0x7f223003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.868+0000 7f2249400700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f2230038f70 0x7f223003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2238006fd0 tx=0x7f2238006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:22.872+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f223c04cce0 con 0x7f224c1062e0 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.099+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f224c0008d0 con 0x7f2230038f70 2024-03-26T13:49:23.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.100+0000 7f2242a00700 1 -- 172.21.15.137:0/4039300515 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f224c0008d0 con 0x7f2230038f70 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f2230038f70 msgr2=0x7f223003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f2230038f70 0x7f223003b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2238006fd0 tx=0x7f2238006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 msgr2=0x7f224c19db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c19db30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f223c011a50 tx=0x7f223c004290 comp rx=0 tx=0).stop 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 shutdown_connections 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f2230038f70 0x7f223003b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 --2- 172.21.15.137:0/4039300515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f224c1062e0 0x7f224c19db30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 >> 172.21.15.137:0/4039300515 conn(0x7f224c101ea0 msgr2=0x7f224c194750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.103+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 shutdown_connections 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:23.104+0000 7f2250e00700 1 -- 172.21.15.137:0/4039300515 wait complete. 2024-03-26T13:49:23.890 INFO:teuthology.orchestra.run.smithi137.stdout:Generating ssh key... 2024-03-26T13:49:24.198 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-mon[31539]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:25.278 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: Generating public/private rsa key pair. 2024-03-26T13:49:25.278 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: Your identification has been saved in /tmp/tmpjoagkxzq/key. 2024-03-26T13:49:25.278 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: Your public key has been saved in /tmp/tmpjoagkxzq/key.pub. 2024-03-26T13:49:25.278 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: The key fingerprint is: 2024-03-26T13:49:25.278 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: SHA256:bfP8GIJEM4H+NYZ8yvhG2D6VprZ89TKBb7WPno3pavs ceph-66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: The key's randomart image is: 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: +---[RSA 3072]----+ 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | .. | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | . . | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | . .+. | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | ..o+= | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | *S=++ | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | o.*o=+o . | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | +.+.o++ . | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | .B ..*+o* | 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: | oo+ o+OE.o| 2024-03-26T13:49:25.279 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: +----[SHA256]-----+ 2024-03-26T13:49:25.668 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.487+0000 7f5044400700 1 Processor -- start 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.487+0000 7f5044400700 1 -- start start 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.488+0000 7f5044400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c107e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.488+0000 7f5044400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c1083d0 con 0x7f503c1079e0 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.488+0000 7f5041a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c107e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.488+0000 7f5041a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c107e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48110/0 (socket says 172.21.15.137:48110) 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.488+0000 7f5041a00700 1 -- 172.21.15.137:0/2359472911 learned_addr learned my addr 172.21.15.137:0/2359472911 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.489+0000 7f5041a00700 1 -- 172.21.15.137:0/2359472911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c108be0 con 0x7f503c1079e0 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.489+0000 7f5041a00700 1 --2- 172.21.15.137:0/2359472911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c107e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50300098a0 tx=0x7f5030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11875f4056dfddc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.489+0000 7f503be00700 1 -- 172.21.15.137:0/2359472911 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5030004160 con 0x7f503c1079e0 2024-03-26T13:49:25.669 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f503be00700 1 -- 172.21.15.137:0/2359472911 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f5030004300 con 0x7f503c1079e0 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f503be00700 1 -- 172.21.15.137:0/2359472911 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5030011a80 con 0x7f503c1079e0 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 -- 172.21.15.137:0/2359472911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 msgr2=0x7f503c107e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 --2- 172.21.15.137:0/2359472911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c107e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50300098a0 tx=0x7f5030009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 -- 172.21.15.137:0/2359472911 shutdown_connections 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 --2- 172.21.15.137:0/2359472911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c107e00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 -- 172.21.15.137:0/2359472911 >> 172.21.15.137:0/2359472911 conn(0x7f503c102fa0 msgr2=0x7f503c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 -- 172.21.15.137:0/2359472911 shutdown_connections 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.490+0000 7f5044400700 1 -- 172.21.15.137:0/2359472911 wait complete. 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.491+0000 7f5044400700 1 Processor -- start 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.492+0000 7f5044400700 1 -- start start 2024-03-26T13:49:25.670 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.492+0000 7f5044400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.492+0000 7f5044400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c10bf90 con 0x7f503c1079e0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.492+0000 7f5041a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.492+0000 7f5041a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48114/0 (socket says 172.21.15.137:48114) 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.492+0000 7f5041a00700 1 -- 172.21.15.137:0/223739175 learned_addr learned my addr 172.21.15.137:0/223739175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f5041a00700 1 -- 172.21.15.137:0/223739175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c19e0c0 con 0x7f503c1079e0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f5041a00700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c19db80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5030000c00 tx=0x7f5030004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50300039c0 con 0x7f503c1079e0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f5030003b60 con 0x7f503c1079e0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f503001a6f0 con 0x7f503c1079e0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c19e350 con 0x7f503c1079e0 2024-03-26T13:49:25.671 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.493+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c1a1040 con 0x7f503c1079e0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.494+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f5030024a40 con 0x7f503c1079e0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.494+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c0637c0 con 0x7f503c1079e0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.495+0000 7f503a000700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f5024038f20 0x7f502403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.495+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f503004d1d0 con 0x7f503c1079e0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.495+0000 7f5041000700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f5024038f20 0x7f502403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.496+0000 7f5041000700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f5024038f20 0x7f502403b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5028006fd0 tx=0x7f5028006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.501+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5030004300 con 0x7f503c1079e0 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.723+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f503c104140 con 0x7f5024038f20 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.968+0000 7f503a000700 1 -- 172.21.15.137:0/223739175 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f503c104140 con 0x7f5024038f20 2024-03-26T13:49:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f5024038f20 msgr2=0x7f502403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f5024038f20 0x7f502403b3e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5028006fd0 tx=0x7f5028006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 msgr2=0x7f503c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c19db80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5030000c00 tx=0x7f5030004290 comp rx=0 tx=0).stop 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 shutdown_connections 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f5024038f20 0x7f502403b3e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 --2- 172.21.15.137:0/223739175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f503c1079e0 0x7f503c19db80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.971+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 >> 172.21.15.137:0/223739175 conn(0x7f503c102fa0 msgr2=0x7f503c103a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.972+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 shutdown_connections 2024-03-26T13:49:25.673 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:24.972+0000 7f5044400700 1 -- 172.21.15.137:0/223739175 wait complete. 2024-03-26T13:49:26.253 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:25 smithi137 ceph-mon[31539]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:26.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:25 smithi137 ceph-mon[31539]: Generating ssh key... 2024-03-26T13:49:26.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:25 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:26.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:25 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:27.149 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.261+0000 7fe519800700 1 Processor -- start 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.261+0000 7fe519800700 1 -- start start 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.261+0000 7fe519800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe514109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.261+0000 7fe519800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51410a580 con 0x7fe514109b90 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.262+0000 7fe512a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe514109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.262+0000 7fe512a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe514109fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48120/0 (socket says 172.21.15.137:48120) 2024-03-26T13:49:27.151 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.262+0000 7fe512a00700 1 -- 172.21.15.137:0/1325903986 learned_addr learned my addr 172.21.15.137:0/1325903986 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.263+0000 7fe512a00700 1 -- 172.21.15.137:0/1325903986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51410ad90 con 0x7fe514109b90 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.263+0000 7fe512a00700 1 --2- 172.21.15.137:0/1325903986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe514109fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5040098a0 tx=0x7fe504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c833a7885111b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.263+0000 7fe511600700 1 -- 172.21.15.137:0/1325903986 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe504004160 con 0x7fe514109b90 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.263+0000 7fe511600700 1 -- 172.21.15.137:0/1325903986 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe504004300 con 0x7fe514109b90 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.263+0000 7fe511600700 1 -- 172.21.15.137:0/1325903986 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe504011a80 con 0x7fe514109b90 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.264+0000 7fe519800700 1 -- 172.21.15.137:0/1325903986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 msgr2=0x7fe514109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.264+0000 7fe519800700 1 --2- 172.21.15.137:0/1325903986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe514109fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5040098a0 tx=0x7fe504009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:27.156 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.264+0000 7fe519800700 1 -- 172.21.15.137:0/1325903986 shutdown_connections 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.264+0000 7fe519800700 1 --2- 172.21.15.137:0/1325903986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe514109fb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.264+0000 7fe519800700 1 -- 172.21.15.137:0/1325903986 >> 172.21.15.137:0/1325903986 conn(0x7fe51407d020 msgr2=0x7fe51407d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.265+0000 7fe519800700 1 -- 172.21.15.137:0/1325903986 shutdown_connections 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.265+0000 7fe519800700 1 -- 172.21.15.137:0/1325903986 wait complete. 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.266+0000 7fe519800700 1 Processor -- start 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.266+0000 7fe519800700 1 -- start start 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe519800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe51419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe519800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51410e140 con 0x7fe514109b90 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe512a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe51419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe512a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe51419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48126/0 (socket says 172.21.15.137:48126) 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe512a00700 1 -- 172.21.15.137:0/492955489 learned_addr learned my addr 172.21.15.137:0/492955489 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe512a00700 1 -- 172.21.15.137:0/492955489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51419e0b0 con 0x7fe514109b90 2024-03-26T13:49:27.157 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.267+0000 7fe512a00700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe51419db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe504000c00 tx=0x7fe504004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.268+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5040039c0 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.268+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fe504003b60 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.268+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51419e340 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.268+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe50401a6f0 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.268+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5141a1030 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.269+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fe504024a40 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.269+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5140637c0 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.270+0000 7fe50b400700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fe4f8038f70 0x7fe4f803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.270+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe50404d250 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.270+0000 7fe512000700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fe4f8038f70 0x7fe4f803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.271+0000 7fe512000700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fe4f8038f70 0x7fe4f803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe500006fd0 tx=0x7fe500006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.275+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe504004300 con 0x7fe514109b90 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.497+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe514107c60 con 0x7fe4f8038f70 2024-03-26T13:49:27.158 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.499+0000 7fe50b400700 1 -- 172.21.15.137:0/492955489 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe514107c60 con 0x7fe4f8038f70 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.502+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fe4f8038f70 msgr2=0x7fe4f803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.502+0000 7fe519800700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fe4f8038f70 0x7fe4f803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe500006fd0 tx=0x7fe500006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.502+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 msgr2=0x7fe51419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.502+0000 7fe519800700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe51419db70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe504000c00 tx=0x7fe504004290 comp rx=0 tx=0).stop 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.503+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 shutdown_connections 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.503+0000 7fe519800700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fe4f8038f70 0x7fe4f803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.503+0000 7fe519800700 1 --2- 172.21.15.137:0/492955489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe514109b90 0x7fe51419db70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.503+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 >> 172.21.15.137:0/492955489 conn(0x7fe51407d020 msgr2=0x7fe514107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.503+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 shutdown_connections 2024-03-26T13:49:27.159 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:26.503+0000 7fe519800700 1 -- 172.21.15.137:0/492955489 wait complete. 2024-03-26T13:49:27.160 INFO:teuthology.orchestra.run.smithi137.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-26T13:49:27.160 INFO:teuthology.orchestra.run.smithi137.stdout:Adding key to root@localhost authorized_keys... 2024-03-26T13:49:27.160 INFO:teuthology.orchestra.run.smithi137.stdout:Adding host smithi137... 2024-03-26T13:49:27.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:27 smithi137 ceph-mon[31539]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:28.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:28 smithi137 ceph-mon[31539]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "addr": "172.21.15.137", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:30.631 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:30 smithi137 ceph-mon[31539]: Deploying cephadm binary to smithi137 2024-03-26T13:49:34.927 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Added host 'smithi137' with addr '172.21.15.137' 2024-03-26T13:49:34.927 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.768+0000 7f89b3a00700 1 Processor -- start 2024-03-26T13:49:34.927 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.769+0000 7f89b3a00700 1 -- start start 2024-03-26T13:49:34.927 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.769+0000 7f89b3a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac10a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.769+0000 7f89b3a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac07be80 con 0x7f89ac1084a0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.769+0000 7f89b1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac10a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.769+0000 7f89b1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac10a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48130/0 (socket says 172.21.15.137:48130) 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.769+0000 7f89b1000700 1 -- 172.21.15.137:0/2353624447 learned_addr learned my addr 172.21.15.137:0/2353624447 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.770+0000 7f89b1000700 1 -- 172.21.15.137:0/2353624447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac10add0 con 0x7f89ac1084a0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.770+0000 7f89b1000700 1 --2- 172.21.15.137:0/2353624447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac10a890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89a00098a0 tx=0x7f89a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8ce47b0da7bcca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.770+0000 7f89ab400700 1 -- 172.21.15.137:0/2353624447 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89a0004160 con 0x7f89ac1084a0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.771+0000 7f89ab400700 1 -- 172.21.15.137:0/2353624447 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f89a0004300 con 0x7f89ac1084a0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.771+0000 7f89ab400700 1 -- 172.21.15.137:0/2353624447 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89a0011a80 con 0x7f89ac1084a0 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.771+0000 7f89b3a00700 1 -- 172.21.15.137:0/2353624447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 msgr2=0x7f89ac10a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.771+0000 7f89b3a00700 1 --2- 172.21.15.137:0/2353624447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac10a890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89a00098a0 tx=0x7f89a0009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.772+0000 7f89b3a00700 1 -- 172.21.15.137:0/2353624447 shutdown_connections 2024-03-26T13:49:34.928 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.772+0000 7f89b3a00700 1 --2- 172.21.15.137:0/2353624447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac10a890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.772+0000 7f89b3a00700 1 -- 172.21.15.137:0/2353624447 >> 172.21.15.137:0/2353624447 conn(0x7f89ac101ef0 msgr2=0x7f89ac104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.772+0000 7f89b3a00700 1 -- 172.21.15.137:0/2353624447 shutdown_connections 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.772+0000 7f89b3a00700 1 -- 172.21.15.137:0/2353624447 wait complete. 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.773+0000 7f89b3a00700 1 Processor -- start 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.773+0000 7f89b3a00700 1 -- start start 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.774+0000 7f89b3a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.774+0000 7f89b3a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89ac10bd90 con 0x7f89ac1084a0 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.774+0000 7f89b1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.774+0000 7f89b1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48138/0 (socket says 172.21.15.137:48138) 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.774+0000 7f89b1000700 1 -- 172.21.15.137:0/2866302143 learned_addr learned my addr 172.21.15.137:0/2866302143 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.774+0000 7f89b1000700 1 -- 172.21.15.137:0/2866302143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac108260 con 0x7f89ac1084a0 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.775+0000 7f89b1000700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac107d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89a0011a50 tx=0x7f89a0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.775+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89a00039f0 con 0x7f89ac1084a0 2024-03-26T13:49:34.929 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.775+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f89a0003b90 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.775+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89a001a600 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.775+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89ac106450 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.775+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ac106990 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.776+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f89a0024a40 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.776+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89ac0637c0 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.777+0000 7f89a9600700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f8994038f70 0x7f899403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.777+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f89a004d2d0 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.777+0000 7f89abe00700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f8994038f70 0x7f899403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.778+0000 7f89abe00700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f8994038f70 0x7f899403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8998006fd0 tx=0x7f8998006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:27.782+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f89a002d770 con 0x7f89ac1084a0 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:28.007+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "addr": "172.21.15.137", "target": ["mon-mgr", ""]}) v1 -- 0x7f89ac103160 con 0x7f8994038f70 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.212+0000 7f89a9600700 1 -- 172.21.15.137:0/2866302143 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f89ac103160 con 0x7f8994038f70 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f8994038f70 msgr2=0x7f899403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:34.930 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f8994038f70 0x7f899403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8998006fd0 tx=0x7f8998006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 msgr2=0x7f89ac107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac107d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89a0011a50 tx=0x7f89a0004290 comp rx=0 tx=0).stop 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 shutdown_connections 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f8994038f70 0x7f899403b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 --2- 172.21.15.137:0/2866302143 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89ac1084a0 0x7f89ac107d20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 >> 172.21.15.137:0/2866302143 conn(0x7f89ac101ef0 msgr2=0x7f89ac102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 shutdown_connections 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:34.216+0000 7f89b3a00700 1 -- 172.21.15.137:0/2866302143 wait complete. 2024-03-26T13:49:34.931 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying unmanaged mon service... 2024-03-26T13:49:35.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:35 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:35.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:35 smithi137 ceph-mon[31539]: Added host smithi137 2024-03-26T13:49:35.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:35 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:36.990 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-26T13:49:36.990 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.569+0000 7fb2db600700 1 Processor -- start 2024-03-26T13:49:36.990 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.569+0000 7fb2db600700 1 -- start start 2024-03-26T13:49:36.990 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.570+0000 7fb2db600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:36.990 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.570+0000 7fb2db600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d407c8a0 con 0x7fb2d407de70 2024-03-26T13:49:36.990 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.570+0000 7fb2d8c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.570+0000 7fb2d8c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45392/0 (socket says 172.21.15.137:45392) 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.570+0000 7fb2d8c00700 1 -- 172.21.15.137:0/1253259402 learned_addr learned my addr 172.21.15.137:0/1253259402 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.571+0000 7fb2d8c00700 1 -- 172.21.15.137:0/1253259402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d407ca20 con 0x7fb2d407de70 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.571+0000 7fb2d8c00700 1 --2- 172.21.15.137:0/1253259402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d407c2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80098a0 tx=0x7fb2c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d91a91575c309f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.571+0000 7fb2d3400700 1 -- 172.21.15.137:0/1253259402 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8004160 con 0x7fb2d407de70 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.571+0000 7fb2d3400700 1 -- 172.21.15.137:0/1253259402 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fb2c8004300 con 0x7fb2d407de70 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.571+0000 7fb2d3400700 1 -- 172.21.15.137:0/1253259402 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8011a80 con 0x7fb2d407de70 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 -- 172.21.15.137:0/1253259402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 msgr2=0x7fb2d407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 --2- 172.21.15.137:0/1253259402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d407c2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80098a0 tx=0x7fb2c8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 -- 172.21.15.137:0/1253259402 shutdown_connections 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 --2- 172.21.15.137:0/1253259402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d407c2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 -- 172.21.15.137:0/1253259402 >> 172.21.15.137:0/1253259402 conn(0x7fb2d4102ff0 msgr2=0x7fb2d4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:36.991 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 -- 172.21.15.137:0/1253259402 shutdown_connections 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.572+0000 7fb2db600700 1 -- 172.21.15.137:0/1253259402 wait complete. 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2db600700 1 Processor -- start 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2db600700 1 -- start start 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2db600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2db600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d410df80 con 0x7fb2d407de70 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2d8c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2d8c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45396/0 (socket says 172.21.15.137:45396) 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.574+0000 7fb2d8c00700 1 -- 172.21.15.137:0/3192361893 learned_addr learned my addr 172.21.15.137:0/3192361893 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2d8c00700 1 -- 172.21.15.137:0/3192361893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d419e090 con 0x7fb2d407de70 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2d8c00700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d419db50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8011a50 tx=0x7fb2c8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c80039f0 con 0x7fb2d407de70 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fb2c8003b90 con 0x7fb2d407de70 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d419e320 con 0x7fb2d407de70 2024-03-26T13:49:36.992 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c801a600 con 0x7fb2d407de70 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.575+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d41a1010 con 0x7fb2d407de70 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.577+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fb2c8024440 con 0x7fb2d407de70 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.577+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d40637c0 con 0x7fb2d407de70 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.577+0000 7fb2d1600700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fb2bc038f20 0x7fb2bc03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.577+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb2c804d390 con 0x7fb2d407de70 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.577+0000 7fb2d3e00700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fb2bc038f20 0x7fb2bc03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.578+0000 7fb2d3e00700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fb2bc038f20 0x7fb2bc03b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0006fd0 tx=0x7fb2c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.583+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb2c804cce0 con 0x7fb2d407de70 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.832+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d41042b0 con 0x7fb2bc038f20 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.839+0000 7fb2d1600700 1 -- 172.21.15.137:0/3192361893 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb2d41042b0 con 0x7fb2bc038f20 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.842+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fb2bc038f20 msgr2=0x7fb2bc03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:36.993 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.842+0000 7fb2db600700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fb2bc038f20 0x7fb2bc03b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0006fd0 tx=0x7fb2c0006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.842+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 msgr2=0x7fb2d419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.842+0000 7fb2db600700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d419db50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8011a50 tx=0x7fb2c8004290 comp rx=0 tx=0).stop 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.842+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 shutdown_connections 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.843+0000 7fb2db600700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fb2bc038f20 0x7fb2bc03b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.843+0000 7fb2db600700 1 --2- 172.21.15.137:0/3192361893 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb2d407de70 0x7fb2d419db50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.843+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 >> 172.21.15.137:0/3192361893 conn(0x7fb2d4102ff0 msgr2=0x7fb2d4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.843+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 shutdown_connections 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:35.843+0000 7fb2db600700 1 -- 172.21.15.137:0/3192361893 wait complete. 2024-03-26T13:49:36.994 INFO:teuthology.orchestra.run.smithi137.stdout:Deploying unmanaged mgr service... 2024-03-26T13:49:37.143 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:36 smithi137 ceph-mon[31539]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:37.143 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:36 smithi137 ceph-mon[31539]: Saving service mon spec with placement count:5 2024-03-26T13:49:37.143 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:36 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.651+0000 7faa8de00700 1 Processor -- start 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.651+0000 7faa8de00700 1 -- start start 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.652+0000 7faa8de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.652+0000 7faa8de00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8807c8c0 con 0x7faa8807de90 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.652+0000 7faa86a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.652+0000 7faa86a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45404/0 (socket says 172.21.15.137:45404) 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.652+0000 7faa86a00700 1 -- 172.21.15.137:0/703977420 learned_addr learned my addr 172.21.15.137:0/703977420 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:38.614 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.653+0000 7faa86a00700 1 -- 172.21.15.137:0/703977420 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8807ca40 con 0x7faa8807de90 2024-03-26T13:49:38.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.653+0000 7faa86a00700 1 --2- 172.21.15.137:0/703977420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8807c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa780098a0 tx=0x7faa78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc4142450fedd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:38.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.653+0000 7faa86000700 1 -- 172.21.15.137:0/703977420 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa78004160 con 0x7faa8807de90 2024-03-26T13:49:38.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.653+0000 7faa86000700 1 -- 172.21.15.137:0/703977420 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7faa78004300 con 0x7faa8807de90 2024-03-26T13:49:38.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.653+0000 7faa86000700 1 -- 172.21.15.137:0/703977420 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa78011a80 con 0x7faa8807de90 2024-03-26T13:49:38.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 -- 172.21.15.137:0/703977420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 msgr2=0x7faa8807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:38.615 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 --2- 172.21.15.137:0/703977420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8807c2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7faa780098a0 tx=0x7faa78009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:38.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 -- 172.21.15.137:0/703977420 shutdown_connections 2024-03-26T13:49:38.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 --2- 172.21.15.137:0/703977420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8807c2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:38.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 -- 172.21.15.137:0/703977420 >> 172.21.15.137:0/703977420 conn(0x7faa88102ff0 msgr2=0x7faa88105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:38.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 -- 172.21.15.137:0/703977420 shutdown_connections 2024-03-26T13:49:38.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.654+0000 7faa8de00700 1 -- 172.21.15.137:0/703977420 wait complete. 2024-03-26T13:49:38.619 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.655+0000 7faa8de00700 1 Processor -- start 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.655+0000 7faa8de00700 1 -- start start 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa8de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa8de00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8810dfc0 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa86a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa86a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45410/0 (socket says 172.21.15.137:45410) 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa86a00700 1 -- 172.21.15.137:0/3080047845 learned_addr learned my addr 172.21.15.137:0/3080047845 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa86a00700 1 -- 172.21.15.137:0/3080047845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8819e090 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.656+0000 7faa86a00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8819db50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa78011a50 tx=0x7faa78004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.657+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa780039f0 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.657+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7faa78003b90 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.657+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8819e320 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.657+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa7801a600 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.657+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa881a1010 con 0x7faa8807de90 2024-03-26T13:49:38.620 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.658+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7faa78024a40 con 0x7faa8807de90 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.658+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa880637c0 con 0x7faa8807de90 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.658+0000 7faa8ca00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7faa6c038f70 0x7faa6c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.658+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faa7804d280 con 0x7faa8807de90 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.659+0000 7faa7fe00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7faa6c038f70 0x7faa6c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.659+0000 7faa7fe00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7faa6c038f70 0x7faa6c03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa70006fd0 tx=0x7faa70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.664+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faa7802d770 con 0x7faa8807de90 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.853+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mon.0 v2:172.21.15.137:3300/0 7 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa78018710 con 0x7faa8807de90 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.907+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7faa88104270 con 0x7faa6c038f70 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.915+0000 7faa8ca00700 1 -- 172.21.15.137:0/3080047845 <== mgr.14120 v2:172.21.15.137:6800/1621047019 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faa88104270 con 0x7faa6c038f70 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7faa6c038f70 msgr2=0x7faa6c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7faa6c038f70 0x7faa6c03b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faa70006fd0 tx=0x7faa70006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 msgr2=0x7faa8819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8819db50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7faa78011a50 tx=0x7faa78004290 comp rx=0 tx=0).stop 2024-03-26T13:49:38.621 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 shutdown_connections 2024-03-26T13:49:38.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7faa6c038f70 0x7faa6c03b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:38.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 --2- 172.21.15.137:0/3080047845 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faa8807de90 0x7faa8819db50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:38.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 >> 172.21.15.137:0/3080047845 conn(0x7faa88102ff0 msgr2=0x7faa88103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:38.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 shutdown_connections 2024-03-26T13:49:38.622 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:37.918+0000 7faa8de00700 1 -- 172.21.15.137:0/3080047845 wait complete. 2024-03-26T13:49:38.900 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:38 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:38.900 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:38 smithi137 ceph-mon[31539]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:38.900 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:38 smithi137 ceph-mon[31539]: Saving service mgr spec with placement count:2 2024-03-26T13:49:38.900 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:38 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:38.900 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:38 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:40.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:40 smithi137 ceph-mon[31539]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:49:40.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:40 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1496351597' entity='client.admin' 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.218+0000 7f54e9200700 1 Processor -- start 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.218+0000 7f54e9200700 1 -- start start 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.219+0000 7f54e9200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e4107e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.219+0000 7f54e9200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41083f0 con 0x7f54e4107a00 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.219+0000 7f54e1e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e4107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.219+0000 7f54e1e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e4107e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59734/0 (socket says 172.21.15.137:59734) 2024-03-26T13:49:40.806 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.219+0000 7f54e1e00700 1 -- 172.21.15.137:0/2948564326 learned_addr learned my addr 172.21.15.137:0/2948564326 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.220+0000 7f54e1e00700 1 -- 172.21.15.137:0/2948564326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e4108c00 con 0x7f54e4107a00 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.220+0000 7f54e1e00700 1 --2- 172.21.15.137:0/2948564326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e4107e20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b33b50ab29fb9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.221+0000 7f54e0a00700 1 -- 172.21.15.137:0/2948564326 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d4004160 con 0x7f54e4107a00 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.221+0000 7f54e0a00700 1 -- 172.21.15.137:0/2948564326 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54d4004300 con 0x7f54e4107a00 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.221+0000 7f54e0a00700 1 -- 172.21.15.137:0/2948564326 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d4011a50 con 0x7f54e4107a00 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 -- 172.21.15.137:0/2948564326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 msgr2=0x7f54e4107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 --2- 172.21.15.137:0/2948564326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e4107e20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54d40098a0 tx=0x7f54d4009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 -- 172.21.15.137:0/2948564326 shutdown_connections 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 --2- 172.21.15.137:0/2948564326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e4107e20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 -- 172.21.15.137:0/2948564326 >> 172.21.15.137:0/2948564326 conn(0x7f54e4102fa0 msgr2=0x7f54e41053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 -- 172.21.15.137:0/2948564326 shutdown_connections 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.222+0000 7f54e9200700 1 -- 172.21.15.137:0/2948564326 wait complete. 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.223+0000 7f54e9200700 1 Processor -- start 2024-03-26T13:49:40.807 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.224+0000 7f54e9200700 1 -- start start 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.224+0000 7f54e9200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e407db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.224+0000 7f54e9200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e410bfb0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.224+0000 7f54e1e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e407db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.224+0000 7f54e1e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e407db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59746/0 (socket says 172.21.15.137:59746) 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.224+0000 7f54e1e00700 1 -- 172.21.15.137:0/1496351597 learned_addr learned my addr 172.21.15.137:0/1496351597 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54e1e00700 1 -- 172.21.15.137:0/1496351597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e407e0c0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54e1e00700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e407db80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54d4011a20 tx=0x7f54d4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d4003af0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54d401a8e0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e407c2b0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d4024860 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.225+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e407c7f0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.226+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f54d4022020 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.226+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e40637c0 con 0x7f54e4107a00 2024-03-26T13:49:40.808 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.227+0000 7f54daa00700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f54c8038f20 0x7f54c803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.227+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f54d404d160 con 0x7f54e4107a00 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.227+0000 7f54e1400700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f54c8038f20 0x7f54c803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.228+0000 7f54e1400700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f54c8038f20 0x7f54c803b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f54d0006fd0 tx=0x7f54d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.233+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f54d400b3e0 con 0x7f54e4107a00 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.458+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f54e407d090 con 0x7f54e4107a00 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.461+0000 7f54daa00700 1 -- 172.21.15.137:0/1496351597 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f54d40172b0 con 0x7f54e4107a00 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.464+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f54c8038f20 msgr2=0x7f54c803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.464+0000 7f54e9200700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f54c8038f20 0x7f54c803b3e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f54d0006fd0 tx=0x7f54d0006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.464+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 msgr2=0x7f54e407db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.464+0000 7f54e9200700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e407db80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f54d4011a20 tx=0x7f54d4011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.465+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 shutdown_connections 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.465+0000 7f54e9200700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f54c8038f20 0x7f54c803b3e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.465+0000 7f54e9200700 1 --2- 172.21.15.137:0/1496351597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54e4107a00 0x7f54e407db80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.465+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 >> 172.21.15.137:0/1496351597 conn(0x7f54e4102fa0 msgr2=0x7f54e4103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:40.809 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.465+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 shutdown_connections 2024-03-26T13:49:40.810 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:39.465+0000 7f54e9200700 1 -- 172.21.15.137:0/1496351597 wait complete. 2024-03-26T13:49:42.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:41 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:42.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:41 smithi137 ceph-mon[31539]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:49:42.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3926909350' entity='client.admin' 2024-03-26T13:49:42.326 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.401+0000 7fad3ea00700 1 Processor -- start 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.401+0000 7fad3ea00700 1 -- start start 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.401+0000 7fad3ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3810a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.401+0000 7fad3ea00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3807be40 con 0x7fad381084e0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.402+0000 7fad37e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3810a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.402+0000 7fad37e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3810a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59754/0 (socket says 172.21.15.137:59754) 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.402+0000 7fad37e00700 1 -- 172.21.15.137:0/2214417960 learned_addr learned my addr 172.21.15.137:0/2214417960 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.402+0000 7fad37e00700 1 -- 172.21.15.137:0/2214417960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3810ae10 con 0x7fad381084e0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.403+0000 7fad37e00700 1 --2- 172.21.15.137:0/2214417960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3810a8d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad280098a0 tx=0x7fad28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9df58ebc513e831 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.403+0000 7fad36a00700 1 -- 172.21.15.137:0/2214417960 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad28004160 con 0x7fad381084e0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.403+0000 7fad36a00700 1 -- 172.21.15.137:0/2214417960 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fad28004300 con 0x7fad381084e0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.403+0000 7fad36a00700 1 -- 172.21.15.137:0/2214417960 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad28011a50 con 0x7fad381084e0 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 -- 172.21.15.137:0/2214417960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 msgr2=0x7fad3810a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:42.328 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 --2- 172.21.15.137:0/2214417960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3810a8d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad280098a0 tx=0x7fad28009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:42.329 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 -- 172.21.15.137:0/2214417960 shutdown_connections 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 --2- 172.21.15.137:0/2214417960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3810a8d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 -- 172.21.15.137:0/2214417960 >> 172.21.15.137:0/2214417960 conn(0x7fad38101ef0 msgr2=0x7fad38104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 -- 172.21.15.137:0/2214417960 shutdown_connections 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.404+0000 7fad3ea00700 1 -- 172.21.15.137:0/2214417960 wait complete. 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.405+0000 7fad3ea00700 1 Processor -- start 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.405+0000 7fad3ea00700 1 -- start start 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.406+0000 7fad3ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.406+0000 7fad3ea00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad381070a0 con 0x7fad381084e0 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.406+0000 7fad37e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:42.333 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.406+0000 7fad37e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59756/0 (socket says 172.21.15.137:59756) 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.406+0000 7fad37e00700 1 -- 172.21.15.137:0/3926909350 learned_addr learned my addr 172.21.15.137:0/3926909350 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.406+0000 7fad37e00700 1 -- 172.21.15.137:0/3926909350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3819e090 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.407+0000 7fad37e00700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3819db50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad28011a20 tx=0x7fad28011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.407+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad280039c0 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.407+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fad28003b60 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.407+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3819e320 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.407+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad2801a6b0 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.407+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad381a1010 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.408+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fad28024430 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.408+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad380637c0 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.409+0000 7fad34c00700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fad1c038f20 0x7fad1c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.409+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fad2804d520 con 0x7fad381084e0 2024-03-26T13:49:42.334 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.409+0000 7fad37400700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fad1c038f20 0x7fad1c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.410+0000 7fad37400700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fad1c038f20 0x7fad1c03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad24006fd0 tx=0x7fad24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.414+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad28024720 con 0x7fad381084e0 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.638+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fad3802dd30 con 0x7fad381084e0 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.644+0000 7fad34c00700 1 -- 172.21.15.137:0/3926909350 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fad2804ce70 con 0x7fad381084e0 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fad1c038f20 msgr2=0x7fad1c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fad1c038f20 0x7fad1c03b3e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad24006fd0 tx=0x7fad24006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 msgr2=0x7fad3819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3819db50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad28011a20 tx=0x7fad28011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 shutdown_connections 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fad1c038f20 0x7fad1c03b3e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 --2- 172.21.15.137:0/3926909350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad381084e0 0x7fad3819db50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.647+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 >> 172.21.15.137:0/3926909350 conn(0x7fad38101ef0 msgr2=0x7fad38102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.648+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 shutdown_connections 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:41.648+0000 7fad3ea00700 1 -- 172.21.15.137:0/3926909350 wait complete. 2024-03-26T13:49:42.335 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling the dashboard module... 2024-03-26T13:49:44.075 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:43 smithi137 ceph-mon[31539]: from='mgr.14120 172.21.15.137:0/840809782' entity='mgr.x' 2024-03-26T13:49:44.075 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:43 smithi137 ceph-mon[31539]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:49:44.075 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:43 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1784028215' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-26T13:49:44.350 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: ignoring --setuser ceph since I am not root 2024-03-26T13:49:44.350 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: ignoring --setgroup ceph since I am not root 2024-03-26T13:49:45.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:44.783+0000 7f8c4395b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T13:49:45.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:44.906+0000 7f8c4395b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T13:49:45.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:44 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1784028215' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-26T13:49:45.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:44 smithi137 ceph-mon[31539]: mgrmap e9: x(active, since 24s) 2024-03-26T13:49:45.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.921+0000 7f16b5e00700 1 Processor -- start 2024-03-26T13:49:45.491 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.921+0000 7f16b5e00700 1 -- start start 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.921+0000 7f16b5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b0107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.921+0000 7f16b5e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b0108460 con 0x7f16b0107a70 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.922+0000 7f16aea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b0107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.922+0000 7f16aea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b0107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59766/0 (socket says 172.21.15.137:59766) 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.922+0000 7f16aea00700 1 -- 172.21.15.137:0/3953554033 learned_addr learned my addr 172.21.15.137:0/3953554033 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.923+0000 7f16aea00700 1 -- 172.21.15.137:0/3953554033 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0108c70 con 0x7f16b0107a70 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.923+0000 7f16aea00700 1 --2- 172.21.15.137:0/3953554033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b0107e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16a00098a0 tx=0x7f16a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7b3ba8572a3add server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.923+0000 7f16ae000700 1 -- 172.21.15.137:0/3953554033 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a0004160 con 0x7f16b0107a70 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.923+0000 7f16ae000700 1 -- 172.21.15.137:0/3953554033 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16a0004300 con 0x7f16b0107a70 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.923+0000 7f16ae000700 1 -- 172.21.15.137:0/3953554033 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a0011a50 con 0x7f16b0107a70 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.924+0000 7f16b5e00700 1 -- 172.21.15.137:0/3953554033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 msgr2=0x7f16b0107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.924+0000 7f16b5e00700 1 --2- 172.21.15.137:0/3953554033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b0107e90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f16a00098a0 tx=0x7f16a0009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.924+0000 7f16b5e00700 1 -- 172.21.15.137:0/3953554033 shutdown_connections 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.924+0000 7f16b5e00700 1 --2- 172.21.15.137:0/3953554033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b0107e90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.924+0000 7f16b5e00700 1 -- 172.21.15.137:0/3953554033 >> 172.21.15.137:0/3953554033 conn(0x7f16b0103050 msgr2=0x7f16b0105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.924+0000 7f16b5e00700 1 -- 172.21.15.137:0/3953554033 shutdown_connections 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.925+0000 7f16b5e00700 1 -- 172.21.15.137:0/3953554033 wait complete. 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.925+0000 7f16b5e00700 1 Processor -- start 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.926+0000 7f16b5e00700 1 -- start start 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.926+0000 7f16b5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.926+0000 7f16b5e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b010c020 con 0x7f16b0107a70 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.926+0000 7f16aea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:45.493 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.926+0000 7f16aea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59774/0 (socket says 172.21.15.137:59774) 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.926+0000 7f16aea00700 1 -- 172.21.15.137:0/1784028215 learned_addr learned my addr 172.21.15.137:0/1784028215 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.927+0000 7f16aea00700 1 -- 172.21.15.137:0/1784028215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b019e0a0 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.927+0000 7f16aea00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b019db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16a0011a20 tx=0x7f16a0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.927+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a0003af0 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.927+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16a001a8e0 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.927+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b019e330 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.927+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a0024860 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.928+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b01a1020 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.929+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f16a0022020 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.929+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b010c1a0 con 0x7f16b0107a70 2024-03-26T13:49:45.494 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.929+0000 7f16b4a00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f1694038f20 0x7f169403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.929+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16a004d2a0 con 0x7f16b0107a70 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.929+0000 7f16a7e00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f1694038f20 0x7f169403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.930+0000 7f16a7e00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f1694038f20 0x7f169403b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1698006fd0 tx=0x7f1698006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:42.934+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16a004cbf0 con 0x7f16b0107a70 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.220+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f16b01a15d0 con 0x7f16b0107a70 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.980+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f16a002d910 con 0x7f16b0107a70 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.982+0000 7f16b4a00700 1 -- 172.21.15.137:0/1784028215 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f16a002d3f0 con 0x7f16b0107a70 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f1694038f20 msgr2=0x7f169403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f1694038f20 0x7f169403b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1698006fd0 tx=0x7f1698006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 msgr2=0x7f16b019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:45.495 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b019db60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16a0011a20 tx=0x7f16a0011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:45.496 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 shutdown_connections 2024-03-26T13:49:45.496 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7f1694038f20 0x7f169403b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:45.496 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 --2- 172.21.15.137:0/1784028215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16b0107a70 0x7f16b019db60 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16a0011a20 tx=0x7f16a0011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:45.496 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.985+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 >> 172.21.15.137:0/1784028215 conn(0x7f16b0103050 msgr2=0x7f16b0103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:45.496 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.986+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 shutdown_connections 2024-03-26T13:49:45.496 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:43.986+0000 7f16b5e00700 1 -- 172.21.15.137:0/1784028215 wait complete. 2024-03-26T13:49:45.777 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:45.699+0000 7f8c4395b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T13:49:46.056 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:45.831+0000 7f8c4395b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T13:49:46.056 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.014+0000 7f8c4395b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T13:49:46.316 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.145+0000 7f8c4395b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T13:49:46.316 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.254+0000 7f8c4395b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T13:49:46.591 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4158375094' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-26T13:49:46.846 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.598+0000 7f8c4395b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T13:49:46.846 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.739+0000 7f8c4395b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T13:49:47.126 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.853+0000 7f8c4395b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T13:49:47.126 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:46.975+0000 7f8c4395b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T13:49:47.136 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.115+0000 7ffb3da00700 1 Processor -- start 2024-03-26T13:49:47.142 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.116+0000 7ffb3da00700 1 -- start start 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.116+0000 7ffb3da00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.116+0000 7ffb3da00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3807c8c0 con 0x7ffb3807de90 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.116+0000 7ffb36a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.117+0000 7ffb36a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3807c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59800/0 (socket says 172.21.15.137:59800) 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.117+0000 7ffb36a00700 1 -- 172.21.15.137:0/3161538536 learned_addr learned my addr 172.21.15.137:0/3161538536 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.117+0000 7ffb36a00700 1 -- 172.21.15.137:0/3161538536 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3807ca40 con 0x7ffb3807de90 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.117+0000 7ffb36a00700 1 --2- 172.21.15.137:0/3161538536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3807c2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb280098a0 tx=0x7ffb28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fc170d0eae10da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.118+0000 7ffb35600700 1 -- 172.21.15.137:0/3161538536 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb28004160 con 0x7ffb3807de90 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.118+0000 7ffb35600700 1 -- 172.21.15.137:0/3161538536 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb28004300 con 0x7ffb3807de90 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.118+0000 7ffb35600700 1 -- 172.21.15.137:0/3161538536 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb28011a50 con 0x7ffb3807de90 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.118+0000 7ffb3da00700 1 -- 172.21.15.137:0/3161538536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 msgr2=0x7ffb3807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.118+0000 7ffb3da00700 1 --2- 172.21.15.137:0/3161538536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3807c2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffb280098a0 tx=0x7ffb28009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.119+0000 7ffb3da00700 1 -- 172.21.15.137:0/3161538536 shutdown_connections 2024-03-26T13:49:47.143 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.119+0000 7ffb3da00700 1 --2- 172.21.15.137:0/3161538536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3807c2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.119+0000 7ffb3da00700 1 -- 172.21.15.137:0/3161538536 >> 172.21.15.137:0/3161538536 conn(0x7ffb38102fa0 msgr2=0x7ffb381053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.119+0000 7ffb3da00700 1 -- 172.21.15.137:0/3161538536 shutdown_connections 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.119+0000 7ffb3da00700 1 -- 172.21.15.137:0/3161538536 wait complete. 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb3da00700 1 Processor -- start 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb3da00700 1 -- start start 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb3da00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3819daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb3da00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3810e000 con 0x7ffb3807de90 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb36a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3819daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb36a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3819daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59804/0 (socket says 172.21.15.137:59804) 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.120+0000 7ffb36a00700 1 -- 172.21.15.137:0/4158375094 learned_addr learned my addr 172.21.15.137:0/4158375094 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.121+0000 7ffb36a00700 1 -- 172.21.15.137:0/4158375094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3819dfe0 con 0x7ffb3807de90 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.121+0000 7ffb36a00700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3819daa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffb28011a00 tx=0x7ffb28011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.121+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb28003af0 con 0x7ffb3807de90 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.121+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb2801a8e0 con 0x7ffb3807de90 2024-03-26T13:49:47.144 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.121+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3819e270 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.121+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb28024840 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.122+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb381a0f60 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.123+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7ffb28022020 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.123+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb380637c0 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.123+0000 7ffb2f400700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 0x7ffb1c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.123+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffb2804d1e0 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.123+0000 7ffb36000700 1 -- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 msgr2=0x7ffb1c03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.123+0000 7ffb36000700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 0x7ffb1c03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.129+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffb2802ddb0 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.324+0000 7ffb36000700 1 -- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 msgr2=0x7ffb1c03b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.324+0000 7ffb36000700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 0x7ffb1c03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.421+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ffb381a1510 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.421+0000 7ffb2f400700 1 -- 172.21.15.137:0/4158375094 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7ffb2801a430 con 0x7ffb3807de90 2024-03-26T13:49:47.145 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 msgr2=0x7ffb1c03b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 0x7ffb1c03b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 msgr2=0x7ffb3819daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3819daa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffb28011a00 tx=0x7ffb28011ae0 comp rx=0 tx=0).stop 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 shutdown_connections 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7ffb1c038fd0 0x7ffb1c03b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 --2- 172.21.15.137:0/4158375094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb3807de90 0x7ffb3819daa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 >> 172.21.15.137:0/4158375094 conn(0x7ffb38102fa0 msgr2=0x7ffb38103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 shutdown_connections 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:46.424+0000 7ffb3da00700 1 -- 172.21.15.137:0/4158375094 wait complete. 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for the mgr to restart... 2024-03-26T13:49:47.146 INFO:teuthology.orchestra.run.smithi137.stdout:Waiting for mgr epoch 9... 2024-03-26T13:49:47.437 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:47 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:47.202+0000 7f8c4395b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T13:49:48.693 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:48.531+0000 7f8c4395b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T13:49:48.693 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:48.700+0000 7f8c4395b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T13:49:49.479 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:49.115+0000 7f8c4395b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T13:49:49.479 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:49.233+0000 7f8c4395b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T13:49:49.479 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:49.486+0000 7f8c4395b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:49:50.027 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:49.745+0000 7f8c4395b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T13:49:50.463 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:50.137+0000 7f8c4395b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:49:50.463 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:50.470+0000 7f8c4395b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T13:49:51.026 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:50.777+0000 7f8c4395b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T13:49:52.277 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:51.801+0000 7f8c4395b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T13:49:52.777 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:52.496+0000 7f8c4395b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T13:49:52.777 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:52.624+0000 7f8c4395b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T13:49:53.276 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:49:52.907+0000 7f8c4395b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: Active manager daemon x restarted 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: Activating manager daemon x 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: osdmap e3: 0 total, 0 up, 0 in 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: mgrmap e10: x(active, starting, since 0.00597696s) 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-26T13:49:53.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:52 smithi137 ceph-mon[31539]: Manager daemon x is now available 2024-03-26T13:49:54.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:54.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:49:54.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-26T13:49:54.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-26T13:49:54.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:53 smithi137 ceph-mon[31539]: mgrmap e11: x(active, since 1.0129s) 2024-03-26T13:49:54.565 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout { 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout } 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.757+0000 7fcbcb200700 1 Processor -- start 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.757+0000 7fcbcb200700 1 -- start start 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.757+0000 7fcbcb200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc4107e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.757+0000 7fcbcb200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41083d0 con 0x7fcbc41079e0 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.758+0000 7fcbc3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc4107e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.758+0000 7fcbc3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc4107e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59810/0 (socket says 172.21.15.137:59810) 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.758+0000 7fcbc3e00700 1 -- 172.21.15.137:0/1692085492 learned_addr learned my addr 172.21.15.137:0/1692085492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.758+0000 7fcbc3e00700 1 -- 172.21.15.137:0/1692085492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4108be0 con 0x7fcbc41079e0 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.758+0000 7fcbc3e00700 1 --2- 172.21.15.137:0/1692085492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc4107e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40098a0 tx=0x7fcbb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f622104f1434f271 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:54.574 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.759+0000 7fcbc2a00700 1 -- 172.21.15.137:0/1692085492 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbb4004160 con 0x7fcbc41079e0 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.759+0000 7fcbc2a00700 1 -- 172.21.15.137:0/1692085492 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbb4004300 con 0x7fcbc41079e0 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.759+0000 7fcbc2a00700 1 -- 172.21.15.137:0/1692085492 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbb4011a50 con 0x7fcbc41079e0 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 -- 172.21.15.137:0/1692085492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 msgr2=0x7fcbc4107e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 --2- 172.21.15.137:0/1692085492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc4107e00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40098a0 tx=0x7fcbb4009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 -- 172.21.15.137:0/1692085492 shutdown_connections 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 --2- 172.21.15.137:0/1692085492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc4107e00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 -- 172.21.15.137:0/1692085492 >> 172.21.15.137:0/1692085492 conn(0x7fcbc4102fa0 msgr2=0x7fcbc41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 -- 172.21.15.137:0/1692085492 shutdown_connections 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.760+0000 7fcbcb200700 1 -- 172.21.15.137:0/1692085492 wait complete. 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.761+0000 7fcbcb200700 1 Processor -- start 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.761+0000 7fcbcb200700 1 -- start start 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.761+0000 7fcbcb200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.761+0000 7fcbcb200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc410bf90 con 0x7fcbc41079e0 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc419b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59812/0 (socket says 172.21.15.137:59812) 2024-03-26T13:49:54.575 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc3e00700 1 -- 172.21.15.137:0/3931117834 learned_addr learned my addr 172.21.15.137:0/3931117834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc3e00700 1 -- 172.21.15.137:0/3931117834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc419be10 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc3e00700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc419b8d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4011a20 tx=0x7fcbb4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbb4003af0 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbb401a8e0 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.762+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbb4024860 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.763+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc419c0a0 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.763+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc419ed90 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.764+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fcbb4022020 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.764+0000 7fcbc0c00700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.764+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 --> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcba803bba0 con 0x7fcba8038fd0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.764+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcbb404d140 con 0x7fcbc41079e0 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.764+0000 7fcbc3400700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 msgr2=0x7fcba803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.764+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.965+0000 7fcbc3400700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 msgr2=0x7fcba803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:47.965+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-26T13:49:54.576 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:48.366+0000 7fcbc3400700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 msgr2=0x7fcba803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:48.366+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:49.167+0000 7fcbc3400700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 msgr2=0x7fcba803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:49.167+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:50.769+0000 7fcbc3400700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 msgr2=0x7fcba803b490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.137:6800/1621047019 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:50.769+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:52.915+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7fcbb402d3f0 con 0x7fcbc41079e0 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:52.915+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 msgr2=0x7fcba803b490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:52.915+0000 7fcbc0c00700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.918+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7fcbb404eb20 con 0x7fcbc41079e0 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.918+0000 7fcbc0c00700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcba803ca80 0x7fcba803ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.918+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcbb4017670 con 0x7fcba803ca80 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.919+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcba803ca80 0x7fcba803ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.920+0000 7fcbc3400700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcba803ca80 0x7fcba803ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0003a10 tx=0x7fcbb0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.922+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fcbb4017670 con 0x7fcba803ca80 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.931+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fcbc40637c0 con 0x7fcba803ca80 2024-03-26T13:49:54.577 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.931+0000 7fcbc0c00700 1 -- 172.21.15.137:0/3931117834 <== mgr.14152 v2:172.21.15.137:6800/3167395503 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fcbc40637c0 con 0x7fcba803ca80 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcba803ca80 msgr2=0x7fcba803ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcba803ca80 0x7fcba803ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0003a10 tx=0x7fcbb0009270 comp rx=0 tx=0).stop 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 msgr2=0x7fcbc419b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc419b8d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4011a20 tx=0x7fcbb4011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 shutdown_connections 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcba803ca80 0x7fcba803ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbc41079e0 0x7fcbc419b8d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 --2- 172.21.15.137:0/3931117834 >> [v2:172.21.15.137:6800/1621047019,v1:172.21.15.137:6801/1621047019] conn(0x7fcba8038fd0 0x7fcba803b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 >> 172.21.15.137:0/3931117834 conn(0x7fcbc4102fa0 msgr2=0x7fcbc41039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 shutdown_connections 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:53.932+0000 7fcbcb200700 1 -- 172.21.15.137:0/3931117834 wait complete. 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:mgr epoch 9 is available 2024-03-26T13:49:54.578 INFO:teuthology.orchestra.run.smithi137.stdout:Generating a dashboard self-signed certificate... 2024-03-26T13:49:55.163 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:54 smithi137 ceph-mon[31539]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-26T13:49:55.163 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:54 smithi137 ceph-mon[31539]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-26T13:49:56.154 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:54] ENGINE Bus STARTING 2024-03-26T13:49:56.155 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:54] ENGINE Serving on http://172.21.15.137:8765 2024-03-26T13:49:56.155 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:54] ENGINE Serving on https://172.21.15.137:7150 2024-03-26T13:49:56.155 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: [26/Mar/2024:13:49:54] ENGINE Bus STARTED 2024-03-26T13:49:56.155 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: mgrmap e12: x(active, since 2s) 2024-03-26T13:49:56.155 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:49:56.155 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.170+0000 7fa25fa00700 1 Processor -- start 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.170+0000 7fa25fa00700 1 -- start start 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.170+0000 7fa25fa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25810a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.170+0000 7fa25fa00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25807bec0 con 0x7fa258108450 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.171+0000 7fa25d000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25810a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.171+0000 7fa25d000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25810a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60214/0 (socket says 172.21.15.137:60214) 2024-03-26T13:49:56.514 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.171+0000 7fa25d000700 1 -- 172.21.15.137:0/743773802 learned_addr learned my addr 172.21.15.137:0/743773802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.171+0000 7fa25d000700 1 -- 172.21.15.137:0/743773802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25810ad80 con 0x7fa258108450 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.172+0000 7fa25d000700 1 --2- 172.21.15.137:0/743773802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25810a840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0098a0 tx=0x7fa24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d133d001fc6e8da8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.172+0000 7fa257400700 1 -- 172.21.15.137:0/743773802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa24c004160 con 0x7fa258108450 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.172+0000 7fa257400700 1 -- 172.21.15.137:0/743773802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa24c004300 con 0x7fa258108450 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.172+0000 7fa257400700 1 -- 172.21.15.137:0/743773802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa24c011a50 con 0x7fa258108450 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 -- 172.21.15.137:0/743773802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 msgr2=0x7fa25810a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 --2- 172.21.15.137:0/743773802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25810a840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0098a0 tx=0x7fa24c009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 -- 172.21.15.137:0/743773802 shutdown_connections 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 --2- 172.21.15.137:0/743773802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25810a840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 -- 172.21.15.137:0/743773802 >> 172.21.15.137:0/743773802 conn(0x7fa258101ea0 msgr2=0x7fa2581042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 -- 172.21.15.137:0/743773802 shutdown_connections 2024-03-26T13:49:56.515 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.173+0000 7fa25fa00700 1 -- 172.21.15.137:0/743773802 wait complete. 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.174+0000 7fa25fa00700 1 Processor -- start 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.174+0000 7fa25fa00700 1 -- start start 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.174+0000 7fa25fa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.174+0000 7fa25fa00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25810bd90 con 0x7fa258108450 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.175+0000 7fa25d000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.175+0000 7fa25d000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25819db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60224/0 (socket says 172.21.15.137:60224) 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.175+0000 7fa25d000700 1 -- 172.21.15.137:0/1073713802 learned_addr learned my addr 172.21.15.137:0/1073713802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.175+0000 7fa25d000700 1 -- 172.21.15.137:0/1073713802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25819e050 con 0x7fa258108450 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.175+0000 7fa25d000700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25819db10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa24c011a00 tx=0x7fa24c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:56.516 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.176+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa24c003af0 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.176+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa24c01a8e0 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.176+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25819e2e0 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.176+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa24c024840 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.176+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2581a0fd0 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.177+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2580637c0 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.177+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7fa24c022020 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.178+0000 7fa255600700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa240038f70 0x7fa24003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.178+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa24c04d3d0 con 0x7fa258108450 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.178+0000 7fa257e00700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa240038f70 0x7fa24003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.179+0000 7fa257e00700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa240038f70 0x7fa24003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa244006fd0 tx=0x7fa244006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:56.517 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.183+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa24c00b3e0 con 0x7fa258108450 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.415+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fa258102f90 con 0x7fa240038f70 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.478+0000 7fa255600700 1 -- 172.21.15.137:0/1073713802 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fa258102f90 con 0x7fa240038f70 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa240038f70 msgr2=0x7fa24003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa240038f70 0x7fa24003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa244006fd0 tx=0x7fa244006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 msgr2=0x7fa25819db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25819db10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa24c011a00 tx=0x7fa24c011ae0 comp rx=0 tx=0).stop 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 shutdown_connections 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa240038f70 0x7fa24003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 --2- 172.21.15.137:0/1073713802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa258108450 0x7fa25819db10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 >> 172.21.15.137:0/1073713802 conn(0x7fa258101ea0 msgr2=0x7fa258102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:56.518 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.482+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 shutdown_connections 2024-03-26T13:49:56.519 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:55.483+0000 7fa25fa00700 1 -- 172.21.15.137:0/1073713802 wait complete. 2024-03-26T13:49:56.519 INFO:teuthology.orchestra.run.smithi137.stdout:Creating initial admin user... 2024-03-26T13:49:57.102 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:56 smithi137 ceph-mon[31539]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:57.103 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:49:58.269 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1otIfYUUO6qmMNQOaLdPXO97b7A0TXsF5O2rOCusHvXQtgAnpzsCS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711460997, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-26T13:49:58.269 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.109+0000 7f410f200700 1 Processor -- start 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.109+0000 7f410f200700 1 -- start start 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.110+0000 7f410f200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f4110107c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.110+0000 7f410f200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41101081f0 con 0x7f4110107800 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.110+0000 7f410de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f4110107c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.110+0000 7f410de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f4110107c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60232/0 (socket says 172.21.15.137:60232) 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.110+0000 7f410de00700 1 -- 172.21.15.137:0/2717795033 learned_addr learned my addr 172.21.15.137:0/2717795033 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.111+0000 7f410de00700 1 -- 172.21.15.137:0/2717795033 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110108a00 con 0x7f4110107800 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.111+0000 7f410de00700 1 --2- 172.21.15.137:0/2717795033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f4110107c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41000098a0 tx=0x7f4100009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6130dc4e1a5dac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.111+0000 7f410ca00700 1 -- 172.21.15.137:0/2717795033 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4100004160 con 0x7f4110107800 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410ca00700 1 -- 172.21.15.137:0/2717795033 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4100004300 con 0x7f4110107800 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410ca00700 1 -- 172.21.15.137:0/2717795033 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4100011a50 con 0x7f4110107800 2024-03-26T13:49:58.270 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410f200700 1 -- 172.21.15.137:0/2717795033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 msgr2=0x7f4110107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410f200700 1 --2- 172.21.15.137:0/2717795033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f4110107c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41000098a0 tx=0x7f4100009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410f200700 1 -- 172.21.15.137:0/2717795033 shutdown_connections 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410f200700 1 --2- 172.21.15.137:0/2717795033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f4110107c20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410f200700 1 -- 172.21.15.137:0/2717795033 >> 172.21.15.137:0/2717795033 conn(0x7f4110102dc0 msgr2=0x7f41101051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.112+0000 7f410f200700 1 -- 172.21.15.137:0/2717795033 shutdown_connections 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.113+0000 7f410f200700 1 -- 172.21.15.137:0/2717795033 wait complete. 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.114+0000 7f410f200700 1 Processor -- start 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.114+0000 7f410f200700 1 -- start start 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.114+0000 7f410f200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f41101995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.114+0000 7f410f200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411007c0f0 con 0x7f4110107800 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.114+0000 7f410de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f41101995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f410de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f41101995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60234/0 (socket says 172.21.15.137:60234) 2024-03-26T13:49:58.271 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f410de00700 1 -- 172.21.15.137:0/1835142085 learned_addr learned my addr 172.21.15.137:0/1835142085 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f410de00700 1 -- 172.21.15.137:0/1835142085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4110199ae0 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f410de00700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f41101995a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4100011a20 tx=0x7f4100011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4100003af0 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f410001a8e0 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.115+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4100024860 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.116+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4110199d70 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.116+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411019ca60 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.117+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f4100022020 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.117+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41100637c0 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.117+0000 7f4106a00700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40f403d420 0x7f40f403f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.117+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f410004d260 con 0x7f4110107800 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.118+0000 7f410d400700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40f403d420 0x7f40f403f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:58.272 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.118+0000 7f410d400700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40f403d420 0x7f40f403f8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.123+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4100016260 con 0x7f4110107800 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.362+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f411002dc60 con 0x7f40f403d420 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.606+0000 7f4106a00700 1 -- 172.21.15.137:0/1835142085 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f411002dc60 con 0x7f40f403d420 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40f403d420 msgr2=0x7f40f403f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40f403d420 0x7f40f403f8e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 msgr2=0x7f41101995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f41101995a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4100011a20 tx=0x7f4100011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 shutdown_connections 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40f403d420 0x7f40f403f8e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 --2- 172.21.15.137:0/1835142085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4110107800 0x7f41101995a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 >> 172.21.15.137:0/1835142085 conn(0x7f4110102dc0 msgr2=0x7f4110105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 shutdown_connections 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:57.609+0000 7f410f200700 1 -- 172.21.15.137:0/1835142085 wait complete. 2024-03-26T13:49:58.273 INFO:teuthology.orchestra.run.smithi137.stdout:Fetching dashboard port number... 2024-03-26T13:49:58.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:58 smithi137 ceph-mon[31539]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:49:58.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:58 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:49:58.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:58 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:49:59.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:49:59 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1890631717' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stdout 8443 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.834+0000 7f2ce3e00700 1 Processor -- start 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.834+0000 7f2ce3e00700 1 -- start start 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.834+0000 7f2ce3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.834+0000 7f2ce3e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc07c8a0 con 0x7f2cdc07de70 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.835+0000 7f2ce1400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:59.881 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.835+0000 7f2ce1400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58358/0 (socket says 172.21.15.137:58358) 2024-03-26T13:49:59.884 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.835+0000 7f2ce1400700 1 -- 172.21.15.137:0/3404064218 learned_addr learned my addr 172.21.15.137:0/3404064218 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.835+0000 7f2ce1400700 1 -- 172.21.15.137:0/3404064218 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc07ca20 con 0x7f2cdc07de70 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.836+0000 7f2ce1400700 1 --2- 172.21.15.137:0/3404064218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc07c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098a0 tx=0x7f2cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cebb11d17c3bac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.836+0000 7f2cdbe00700 1 -- 172.21.15.137:0/3404064218 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd0004160 con 0x7f2cdc07de70 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.836+0000 7f2cdbe00700 1 -- 172.21.15.137:0/3404064218 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cd0004300 con 0x7f2cdc07de70 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.836+0000 7f2cdbe00700 1 -- 172.21.15.137:0/3404064218 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd0011a50 con 0x7f2cdc07de70 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 -- 172.21.15.137:0/3404064218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 msgr2=0x7f2cdc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 --2- 172.21.15.137:0/3404064218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc07c2d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098a0 tx=0x7f2cd0009bb0 comp rx=0 tx=0).stop 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 -- 172.21.15.137:0/3404064218 shutdown_connections 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 --2- 172.21.15.137:0/3404064218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc07c2d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 -- 172.21.15.137:0/3404064218 >> 172.21.15.137:0/3404064218 conn(0x7f2cdc102ff0 msgr2=0x7f2cdc105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 -- 172.21.15.137:0/3404064218 shutdown_connections 2024-03-26T13:49:59.885 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.837+0000 7f2ce3e00700 1 -- 172.21.15.137:0/3404064218 wait complete. 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.838+0000 7f2ce3e00700 1 Processor -- start 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.838+0000 7f2ce3e00700 1 -- start start 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.839+0000 7f2ce3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.839+0000 7f2ce3e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc10dfd0 con 0x7f2cdc07de70 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.839+0000 7f2ce1400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.839+0000 7f2ce1400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc19daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58362/0 (socket says 172.21.15.137:58362) 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.839+0000 7f2ce1400700 1 -- 172.21.15.137:0/1890631717 learned_addr learned my addr 172.21.15.137:0/1890631717 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.839+0000 7f2ce1400700 1 -- 172.21.15.137:0/1890631717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc19e030 con 0x7f2cdc07de70 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.840+0000 7f2ce1400700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc19daf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0011a20 tx=0x7f2cd0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.840+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd00039c0 con 0x7f2cdc07de70 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.840+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2cd0003b60 con 0x7f2cdc07de70 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.840+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc19e2c0 con 0x7f2cdc07de70 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.840+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd001a6b0 con 0x7f2cdc07de70 2024-03-26T13:49:59.886 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.840+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc1a0fb0 con 0x7f2cdc07de70 2024-03-26T13:49:59.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.841+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f2cd0024430 con 0x7f2cdc07de70 2024-03-26T13:49:59.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.841+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc0637c0 con 0x7f2cdc07de70 2024-03-26T13:49:59.887 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.842+0000 7f2cda000700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2cc4038f80 0x7f2cc403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.842+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2cd004d5d0 con 0x7f2cdc07de70 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.842+0000 7f2ce0a00700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2cc4038f80 0x7f2cc403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.843+0000 7f2ce0a00700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2cc4038f80 0x7f2cc403b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8006fd0 tx=0x7f2cc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:58.847+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2cd0024a70 con 0x7f2cdc07de70 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.072+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f2cdc1a1560 con 0x7f2cdc07de70 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.073+0000 7f2cda000700 1 -- 172.21.15.137:0/1890631717 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f2cd0024720 con 0x7f2cdc07de70 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2cc4038f80 msgr2=0x7f2cc403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2cc4038f80 0x7f2cc403b440 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8006fd0 tx=0x7f2cc8006e40 comp rx=0 tx=0).stop 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 msgr2=0x7f2cdc19daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc19daf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0011a20 tx=0x7f2cd0011b00 comp rx=0 tx=0).stop 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 shutdown_connections 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2cc4038f80 0x7f2cc403b440 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:59.889 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 --2- 172.21.15.137:0/1890631717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2cdc07de70 0x7f2cdc19daf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:49:59.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.075+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 >> 172.21.15.137:0/1890631717 conn(0x7f2cdc102ff0 msgr2=0x7f2cdc103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:49:59.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.076+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 shutdown_connections 2024-03-26T13:49:59.890 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:49:59.076+0000 7f2ce3e00700 1 -- 172.21.15.137:0/1890631717 wait complete. 2024-03-26T13:49:59.916 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-26T13:49:59.916 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout disabled 2024-03-26T13:49:59.940 INFO:teuthology.orchestra.run.smithi137.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-26T13:49:59.940 INFO:teuthology.orchestra.run.smithi137.stdout:systemctl: stdout inactive 2024-03-26T13:49:59.940 INFO:teuthology.orchestra.run.smithi137.stdout:firewalld.service is not enabled 2024-03-26T13:49:59.940 INFO:teuthology.orchestra.run.smithi137.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-26T13:49:59.944 INFO:teuthology.orchestra.run.smithi137.stdout:Ceph Dashboard is now available at: 2024-03-26T13:49:59.944 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:49:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: URL: https://smithi137.front.sepia.ceph.com:8443/ 2024-03-26T13:49:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: User: admin 2024-03-26T13:49:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: Password: h6230e7c74 2024-03-26T13:49:59.945 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:49:59.945 INFO:teuthology.orchestra.run.smithi137.stdout:Saving cluster configuration to /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config directory 2024-03-26T13:49:59.946 INFO:teuthology.orchestra.run.smithi137.stdout:Enabling autotune for osd_memory_target 2024-03-26T13:50:00.658 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:00 smithi137 ceph-mon[31539]: mgrmap e13: x(active, since 6s) 2024-03-26T13:50:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.665+0000 7f13bd200700 1 Processor -- start 2024-03-26T13:50:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.665+0000 7f13bd200700 1 -- start start 2024-03-26T13:50:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.666+0000 7f13bd200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b8107dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:02.019 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.666+0000 7f13bd200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b81083a0 con 0x7f13b81079b0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.666+0000 7f13b5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b8107dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.667+0000 7f13b5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b8107dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58364/0 (socket says 172.21.15.137:58364) 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.667+0000 7f13b5e00700 1 -- 172.21.15.137:0/3144891899 learned_addr learned my addr 172.21.15.137:0/3144891899 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.667+0000 7f13b5e00700 1 -- 172.21.15.137:0/3144891899 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b8108bb0 con 0x7f13b81079b0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.667+0000 7f13b5e00700 1 --2- 172.21.15.137:0/3144891899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b8107dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13a80098a0 tx=0x7f13a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e2c0e85a442ee4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.667+0000 7f13b4a00700 1 -- 172.21.15.137:0/3144891899 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a8004160 con 0x7f13b81079b0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13b4a00700 1 -- 172.21.15.137:0/3144891899 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f13a8004300 con 0x7f13b81079b0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13b4a00700 1 -- 172.21.15.137:0/3144891899 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a8011a50 con 0x7f13b81079b0 2024-03-26T13:50:02.023 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 -- 172.21.15.137:0/3144891899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 msgr2=0x7f13b8107dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 --2- 172.21.15.137:0/3144891899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b8107dd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f13a80098a0 tx=0x7f13a8009bb0 comp rx=0 tx=0).stop 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 -- 172.21.15.137:0/3144891899 shutdown_connections 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 --2- 172.21.15.137:0/3144891899 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b8107dd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 -- 172.21.15.137:0/3144891899 >> 172.21.15.137:0/3144891899 conn(0x7f13b8102f70 msgr2=0x7f13b8105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 -- 172.21.15.137:0/3144891899 shutdown_connections 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.668+0000 7f13bd200700 1 -- 172.21.15.137:0/3144891899 wait complete. 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.669+0000 7f13bd200700 1 Processor -- start 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.669+0000 7f13bd200700 1 -- start start 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.670+0000 7f13bd200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b807db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.670+0000 7f13bd200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b810bf60 con 0x7f13b81079b0 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.670+0000 7f13b5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b807db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.670+0000 7f13b5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b807db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58370/0 (socket says 172.21.15.137:58370) 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.670+0000 7f13b5e00700 1 -- 172.21.15.137:0/694083126 learned_addr learned my addr 172.21.15.137:0/694083126 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:02.024 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.670+0000 7f13b5e00700 1 -- 172.21.15.137:0/694083126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b807e0c0 con 0x7f13b81079b0 2024-03-26T13:50:02.025 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.671+0000 7f13b5e00700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b807db80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13a8006b20 tx=0x7f13a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:02.025 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.671+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a80039e0 con 0x7f13b81079b0 2024-03-26T13:50:02.025 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.671+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f13a801a8e0 con 0x7f13b81079b0 2024-03-26T13:50:02.025 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.671+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a80247e0 con 0x7f13b81079b0 2024-03-26T13:50:02.025 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.671+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b807c2b0 con 0x7f13b81079b0 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.671+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b807c7f0 con 0x7f13b81079b0 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.672+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f13a8022020 con 0x7f13b81079b0 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.672+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b80637c0 con 0x7f13b81079b0 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.673+0000 7f13aea00700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f139c039060 0x7f139c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.673+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f13a804d490 con 0x7f13b81079b0 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.673+0000 7f13b5400700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f139c039060 0x7f139c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:02.027 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.673+0000 7f13b5400700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f139c039060 0x7f139c03b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006fd0 tx=0x7f13a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.678+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f13a802c3a0 con 0x7f13b81079b0 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.902+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f13b807cb30 con 0x7f13b81079b0 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.903+0000 7f13aea00700 1 -- 172.21.15.137:0/694083126 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f13b807cb30 con 0x7f13b81079b0 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.905+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f139c039060 msgr2=0x7f139c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.905+0000 7f13bd200700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f139c039060 0x7f139c03b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006fd0 tx=0x7f13a4006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.905+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 msgr2=0x7f13b807db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.905+0000 7f13bd200700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b807db80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f13a8006b20 tx=0x7f13a8011ad0 comp rx=0 tx=0).stop 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.905+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 shutdown_connections 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.906+0000 7f13bd200700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f139c039060 0x7f139c03b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.906+0000 7f13bd200700 1 --2- 172.21.15.137:0/694083126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13b81079b0 0x7f13b807db80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.906+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 >> 172.21.15.137:0/694083126 conn(0x7f13b8102f70 msgr2=0x7f13b8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.906+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 shutdown_connections 2024-03-26T13:50:02.028 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:00.906+0000 7f13bd200700 1 -- 172.21.15.137:0/694083126 wait complete. 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.633+0000 7fd385600700 1 Processor -- start 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.633+0000 7fd385600700 1 -- start start 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.634+0000 7fd385600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd380107e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.634+0000 7fd385600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801083e0 con 0x7fd3801079f0 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.634+0000 7fd37ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd380107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.634+0000 7fd37ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd380107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58384/0 (socket says 172.21.15.137:58384) 2024-03-26T13:50:03.697 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.634+0000 7fd37ea00700 1 -- 172.21.15.137:0/2479941887 learned_addr learned my addr 172.21.15.137:0/2479941887 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:03.702 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.635+0000 7fd37ea00700 1 -- 172.21.15.137:0/2479941887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380108bf0 con 0x7fd3801079f0 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.635+0000 7fd37ea00700 1 --2- 172.21.15.137:0/2479941887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd380107e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e80 tx=0x7fd37000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6d859b790aa304 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.635+0000 7fd37e000700 1 -- 172.21.15.137:0/2479941887 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd370017070 con 0x7fd3801079f0 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.635+0000 7fd37e000700 1 -- 172.21.15.137:0/2479941887 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd37000b8c0 con 0x7fd3801079f0 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.635+0000 7fd37e000700 1 -- 172.21.15.137:0/2479941887 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd370004490 con 0x7fd3801079f0 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 -- 172.21.15.137:0/2479941887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 msgr2=0x7fd380107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 --2- 172.21.15.137:0/2479941887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd380107e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e80 tx=0x7fd37000b330 comp rx=0 tx=0).stop 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 -- 172.21.15.137:0/2479941887 shutdown_connections 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 --2- 172.21.15.137:0/2479941887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd380107e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 -- 172.21.15.137:0/2479941887 >> 172.21.15.137:0/2479941887 conn(0x7fd380102ff0 msgr2=0x7fd380105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 -- 172.21.15.137:0/2479941887 shutdown_connections 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.636+0000 7fd385600700 1 -- 172.21.15.137:0/2479941887 wait complete. 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.637+0000 7fd385600700 1 Processor -- start 2024-03-26T13:50:03.703 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd385600700 1 -- start start 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd385600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd38019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd385600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38010bfa0 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd37ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd38019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd37ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd38019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58396/0 (socket says 172.21.15.137:58396) 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd37ea00700 1 -- 172.21.15.137:0/2186056226 learned_addr learned my addr 172.21.15.137:0/2186056226 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.638+0000 7fd37ea00700 1 -- 172.21.15.137:0/2186056226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38019e0b0 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.639+0000 7fd37ea00700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd38019db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b7b0 tx=0x7fd370003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.639+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd370020020 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.639+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd370012430 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.639+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38019e340 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.639+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd370017040 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.639+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3801a1030 con 0x7fd3801079f0 2024-03-26T13:50:03.704 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.640+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fd370023020 con 0x7fd3801079f0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.640+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3800637c0 con 0x7fd3801079f0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.641+0000 7fd377e00700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd364039010 0x7fd36403b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.641+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd37000f050 con 0x7fd3801079f0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.641+0000 7fd376000700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd364039010 0x7fd36403b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.642+0000 7fd376000700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd364039010 0x7fd36403b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd368006fd0 tx=0x7fd368006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.646+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd37002a020 con 0x7fd3801079f0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.983+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fd3801a1200 con 0x7fd3801079f0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.988+0000 7fd377e00700 1 -- 172.21.15.137:0/2186056226 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fd3801a1200 con 0x7fd3801079f0 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd364039010 msgr2=0x7fd36403b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd364039010 0x7fd36403b4d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd368006fd0 tx=0x7fd368006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 msgr2=0x7fd38019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd38019db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd37000b7b0 tx=0x7fd370003ce0 comp rx=0 tx=0).stop 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 shutdown_connections 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd364039010 0x7fd36403b4d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:03.705 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 --2- 172.21.15.137:0/2186056226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3801079f0 0x7fd38019db70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.991+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 >> 172.21.15.137:0/2186056226 conn(0x7fd380102ff0 msgr2=0x7fd380103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.992+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 shutdown_connections 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr 2024-03-26T13:50:02.992+0000 7fd385600700 1 -- 172.21.15.137:0/2186056226 wait complete. 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:Or, if you are only running a single cluster on this host: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: ceph telemetry on 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.706 INFO:teuthology.orchestra.run.smithi137.stdout:For more information see: 2024-03-26T13:50:03.707 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.707 INFO:teuthology.orchestra.run.smithi137.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-26T13:50:03.707 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:03.707 INFO:teuthology.orchestra.run.smithi137.stdout:Bootstrap complete. 2024-03-26T13:50:03.805 INFO:tasks.cephadm:Fetching config... 2024-03-26T13:50:03.805 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:50:03.805 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-26T13:50:03.854 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-26T13:50:03.854 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:50:03.854 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-26T13:50:03.934 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-26T13:50:03.934 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:50:03.934 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-26T13:50:04.063 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-26T13:50:04.063 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:50:04.063 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-26T13:50:04.156 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-26T13:50:04.156 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T13:50:04.229 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:03 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2186056226' entity='client.admin' 2024-03-26T13:50:04.268 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:50:04.331 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T13:50:04.474 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:50:04.537 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-26T13:50:04.673 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJq5pYwEjKDrxAmV+dv3ZVOWyfxQZeEd+8uRdkR2Vb+5z+gr/4VSSMW/g+rhER36nPi+7BWXQekDkLmyMWbAgfg/3k2TI+D9LGcAMru1OoZKBwu31ufdcNGgOuKmKY7yDQVMeB2XZmsPJHW2keh7eN9DJg18t0q31btB5t1ObZKcqFf4GWoXvEcmBS25hI3y+vEDep8gFIWW9R8qQhwqtXF36JdmM95tPZpSjEA8oudQcj4ZOSTr0ta0ndAyjDpzE2EFcux54bVF/9UWPHrqIqUZfXMxdCEoDZKxil1YN0qa5h5cJn89qYhyYF0nOcVggNs4DZX7LH9ZkhZqlbbIMx/m8l1WxJtFAQ+NrWwutwKxRl+v4xHct8QqLRT85ofEEWavZd9hUrIN2RDXWihtDdEseA+448qpjgY++di0BbB/aefNTLRbsxh49lnalrMxkPOwZE/QSZYBwSPegaGE0qMSpFA+3QGP4O2oH89HjsBE4AhV+8CYC9H21c0e+cM4k= ceph-66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:50:04.733 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-26T13:50:05.143 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:07.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.926+0000 7f6e6fe00700 1 -- 172.21.15.137:0/2479516719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 msgr2=0x7f6e70074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.926+0000 7f6e6fe00700 1 --2- 172.21.15.137:0/2479516719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70074620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009910 tx=0x7f6e60009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.926+0000 7f6e6fe00700 1 -- 172.21.15.137:0/2479516719 shutdown_connections 2024-03-26T13:50:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.926+0000 7f6e6fe00700 1 --2- 172.21.15.137:0/2479516719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70074620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.926+0000 7f6e6fe00700 1 -- 172.21.15.137:0/2479516719 >> 172.21.15.137:0/2479516719 conn(0x7f6e700fcfa0 msgr2=0x7f6e700ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.926+0000 7f6e6fe00700 1 -- 172.21.15.137:0/2479516719 shutdown_connections 2024-03-26T13:50:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.927+0000 7f6e6fe00700 1 -- 172.21.15.137:0/2479516719 wait complete. 2024-03-26T13:50:07.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.928+0000 7f6e6fe00700 1 Processor -- start 2024-03-26T13:50:07.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.928+0000 7f6e6fe00700 1 -- start start 2024-03-26T13:50:07.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.928+0000 7f6e6fe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:07.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.928+0000 7f6e6fe00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70109b20 con 0x7f6e700761c0 2024-03-26T13:50:07.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.928+0000 7f6e6ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:07.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.929+0000 7f6e6ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55158/0 (socket says 172.21.15.137:55158) 2024-03-26T13:50:07.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.929+0000 7f6e6ea00700 1 -- 172.21.15.137:0/126120458 learned_addr learned my addr 172.21.15.137:0/126120458 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:07.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.929+0000 7f6e6ea00700 1 -- 172.21.15.137:0/126120458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e70199770 con 0x7f6e700761c0 2024-03-26T13:50:07.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.929+0000 7f6e6ea00700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70199230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e60005950 tx=0x7f6e60004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.929+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e60026070 con 0x7f6e700761c0 2024-03-26T13:50:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.930+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6e6000bc00 con 0x7f6e700761c0 2024-03-26T13:50:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.930+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e70199a00 con 0x7f6e700761c0 2024-03-26T13:50:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.930+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e600039a0 con 0x7f6e700761c0 2024-03-26T13:50:07.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.930+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7019c6f0 con 0x7f6e700761c0 2024-03-26T13:50:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.931+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7010a2a0 con 0x7f6e700761c0 2024-03-26T13:50:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.931+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f6e60029070 con 0x7f6e700761c0 2024-03-26T13:50:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.932+0000 7f6e74a00700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6e54039020 0x7f6e5403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.932+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e6001e070 con 0x7f6e700761c0 2024-03-26T13:50:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.932+0000 7f6e6e000700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6e54039020 0x7f6e5403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.933+0000 7f6e6e000700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6e54039020 0x7f6e5403b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:07.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:07.938+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6e60013070 con 0x7f6e700761c0 2024-03-26T13:50:08.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.159+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f6e7002db40 con 0x7f6e700761c0 2024-03-26T13:50:08.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.163+0000 7f6e74a00700 1 -- 172.21.15.137:0/126120458 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6e60030020 con 0x7f6e700761c0 2024-03-26T13:50:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6e54039020 msgr2=0x7f6e5403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6e54039020 0x7f6e5403b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c006fd0 tx=0x7f6e5c006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 msgr2=0x7f6e70199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70199230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e60005950 tx=0x7f6e60004fa0 comp rx=0 tx=0).stop 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 shutdown_connections 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6e54039020 0x7f6e5403b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 --2- 172.21.15.137:0/126120458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6e700761c0 0x7f6e70199230 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 >> 172.21.15.137:0/126120458 conn(0x7f6e700fcfa0 msgr2=0x7f6e700fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 shutdown_connections 2024-03-26T13:50:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:08.166+0000 7f6e6fe00700 1 -- 172.21.15.137:0/126120458 wait complete. 2024-03-26T13:50:08.935 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-26T13:50:08.936 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-26T13:50:09.419 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/126120458' entity='client.admin' 2024-03-26T13:50:09.820 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:11.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:11.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:11.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:50:11.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:11.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:50:11.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:11.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 -- 172.21.15.137:0/3971896924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 msgr2=0x7f21240745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:11.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 --2- 172.21.15.137:0/3971896924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f21240745c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2114009910 tx=0x7f2114009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:11.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 -- 172.21.15.137:0/3971896924 shutdown_connections 2024-03-26T13:50:11.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 --2- 172.21.15.137:0/3971896924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f21240745c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:11.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 -- 172.21.15.137:0/3971896924 >> 172.21.15.137:0/3971896924 conn(0x7f21240fd150 msgr2=0x7f21240ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:11.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 -- 172.21.15.137:0/3971896924 shutdown_connections 2024-03-26T13:50:11.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.912+0000 7f2129e00700 1 -- 172.21.15.137:0/3971896924 wait complete. 2024-03-26T13:50:11.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.914+0000 7f2129e00700 1 Processor -- start 2024-03-26T13:50:11.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.914+0000 7f2129e00700 1 -- start start 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.914+0000 7f2129e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f2124199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.914+0000 7f2129e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2124109ce0 con 0x7f2124076160 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.915+0000 7f2122a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f2124199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.915+0000 7f2122a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f2124199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55176/0 (socket says 172.21.15.137:55176) 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.915+0000 7f2122a00700 1 -- 172.21.15.137:0/1969709676 learned_addr learned my addr 172.21.15.137:0/1969709676 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.915+0000 7f2122a00700 1 -- 172.21.15.137:0/1969709676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21241999a0 con 0x7f2124076160 2024-03-26T13:50:11.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.915+0000 7f2122a00700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f2124199460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2114005190 tx=0x7f2114004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:11.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.916+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2114027070 con 0x7f2124076160 2024-03-26T13:50:11.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.916+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2114022470 con 0x7f2124076160 2024-03-26T13:50:11.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.916+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f211400f5f0 con 0x7f2124076160 2024-03-26T13:50:11.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.916+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2124199c30 con 0x7f2124076160 2024-03-26T13:50:11.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.916+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212419c920 con 0x7f2124076160 2024-03-26T13:50:11.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.917+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f211402a070 con 0x7f2124076160 2024-03-26T13:50:11.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.917+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212410a460 con 0x7f2124076160 2024-03-26T13:50:11.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.918+0000 7f2128a00700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2108038fc0 0x7f210803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:11.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.918+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f211401e030 con 0x7f2124076160 2024-03-26T13:50:11.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.918+0000 7f2122000700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2108038fc0 0x7f210803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:11.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.918+0000 7f2122000700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2108038fc0 0x7f210803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2110006fd0 tx=0x7f2110006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:11.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:11.923+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2114013070 con 0x7f2124076160 2024-03-26T13:50:12.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.148+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f212402db40 con 0x7f2108038fc0 2024-03-26T13:50:12.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.155+0000 7f2128a00700 1 -- 172.21.15.137:0/1969709676 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f212402db40 con 0x7f2108038fc0 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2108038fc0 msgr2=0x7f210803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2108038fc0 0x7f210803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2110006fd0 tx=0x7f2110006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 msgr2=0x7f2124199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f2124199460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2114005190 tx=0x7f2114004f80 comp rx=0 tx=0).stop 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 shutdown_connections 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2108038fc0 0x7f210803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 --2- 172.21.15.137:0/1969709676 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2124076160 0x7f2124199460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 >> 172.21.15.137:0/1969709676 conn(0x7f21240fd150 msgr2=0x7f21240fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 shutdown_connections 2024-03-26T13:50:12.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:12.158+0000 7f2129e00700 1 -- 172.21.15.137:0/1969709676 wait complete. 2024-03-26T13:50:12.911 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-03-26T13:50:12.911 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:50:12.911 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-03-26T13:50:12.967 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:50:12.967 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:50:13.062 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-03-26T13:50:13.062 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch host add smithi159 2024-03-26T13:50:13.471 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:13.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:50:13.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:13.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:50:13.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:50:13.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:50:13.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-26T13:50:13.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: Updating smithi137:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:50:13.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:13 smithi137 ceph-mon[31539]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:14.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:14 smithi137 ceph-mon[31539]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:50:14.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:14 smithi137 ceph-mon[31539]: Updating smithi137:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.796+0000 7fca82800700 1 -- 172.21.15.137:0/3697026234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 msgr2=0x7fca7c104390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.796+0000 7fca82800700 1 --2- 172.21.15.137:0/3697026234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c104390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009910 tx=0x7fca6c009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.796+0000 7fca82800700 1 -- 172.21.15.137:0/3697026234 shutdown_connections 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.796+0000 7fca82800700 1 --2- 172.21.15.137:0/3697026234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c104390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.796+0000 7fca82800700 1 -- 172.21.15.137:0/3697026234 >> 172.21.15.137:0/3697026234 conn(0x7fca7c0ff880 msgr2=0x7fca7c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.796+0000 7fca82800700 1 -- 172.21.15.137:0/3697026234 shutdown_connections 2024-03-26T13:50:15.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.797+0000 7fca82800700 1 -- 172.21.15.137:0/3697026234 wait complete. 2024-03-26T13:50:15.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.798+0000 7fca82800700 1 Processor -- start 2024-03-26T13:50:15.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.798+0000 7fca82800700 1 -- start start 2024-03-26T13:50:15.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.798+0000 7fca82800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:15.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.799+0000 7fca82800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c10e230 con 0x7fca7c103fb0 2024-03-26T13:50:15.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.799+0000 7fca7b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:15.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.799+0000 7fca7b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c19f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55202/0 (socket says 172.21.15.137:55202) 2024-03-26T13:50:15.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.799+0000 7fca7b400700 1 -- 172.21.15.137:0/2944158159 learned_addr learned my addr 172.21.15.137:0/2944158159 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:15.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.799+0000 7fca7b400700 1 -- 172.21.15.137:0/2944158159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c19f930 con 0x7fca7c103fb0 2024-03-26T13:50:15.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.799+0000 7fca7b400700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c19f3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca6c006010 tx=0x7fca6c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:15.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.800+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca6c026070 con 0x7fca7c103fb0 2024-03-26T13:50:15.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.800+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca6c00bba0 con 0x7fca7c103fb0 2024-03-26T13:50:15.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.800+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c19fc20 con 0x7fca7c103fb0 2024-03-26T13:50:15.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.800+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca6c0038a0 con 0x7fca7c103fb0 2024-03-26T13:50:15.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.800+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c1995b0 con 0x7fca7c103fb0 2024-03-26T13:50:15.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.801+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fca6c029070 con 0x7fca7c103fb0 2024-03-26T13:50:15.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.801+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c105490 con 0x7fca7c103fb0 2024-03-26T13:50:15.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.801+0000 7fca78c00700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca60038fc0 0x7fca6003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:15.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.802+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca6c01e070 con 0x7fca7c103fb0 2024-03-26T13:50:15.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.802+0000 7fca73e00700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca60038fc0 0x7fca6003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:15.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.802+0000 7fca73e00700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca60038fc0 0x7fca6003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca64006fd0 tx=0x7fca64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:15.807+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca6c013070 con 0x7fca7c103fb0 2024-03-26T13:50:16.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:16.032+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) v1 -- 0x7fca7c19aae0 con 0x7fca60038fc0 2024-03-26T13:50:16.024 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:16.024 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:16.025 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:16.025 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:15 smithi137 ceph-mon[31539]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:17.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:16 smithi137 ceph-mon[31539]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:50:18.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:17 smithi137 ceph-mon[31539]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:19.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:18 smithi137 ceph-mon[31539]: Deploying cephadm binary to smithi159 2024-03-26T13:50:19.928 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:19 smithi137 ceph-mon[31539]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:22.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:21 smithi137 ceph-mon[31539]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:22.569 INFO:teuthology.orchestra.run.smithi137.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-03-26T13:50:22.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.576+0000 7fca78c00700 1 -- 172.21.15.137:0/2944158159 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fca7c19aae0 con 0x7fca60038fc0 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.580+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca60038fc0 msgr2=0x7fca6003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.580+0000 7fca82800700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca60038fc0 0x7fca6003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca64006fd0 tx=0x7fca64006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 msgr2=0x7fca7c19f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c19f3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca6c006010 tx=0x7fca6c005780 comp rx=0 tx=0).stop 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 shutdown_connections 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca60038fc0 0x7fca6003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 --2- 172.21.15.137:0/2944158159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca7c103fb0 0x7fca7c19f3f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 >> 172.21.15.137:0/2944158159 conn(0x7fca7c0ff880 msgr2=0x7fca7c105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 shutdown_connections 2024-03-26T13:50:22.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:22.581+0000 7fca82800700 1 -- 172.21.15.137:0/2944158159 wait complete. 2024-03-26T13:50:23.350 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T13:50:23.789 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:23.927 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:23.928 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:23 smithi137 ceph-mon[31539]: Added host smithi159 2024-03-26T13:50:23.928 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:50:23.928 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:23 smithi137 ceph-mon[31539]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:23.928 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:25.510 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:25.511 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:25 smithi137 ceph-mon[31539]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 -- 172.21.15.137:0/2607238629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 msgr2=0x7f81fc106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 --2- 172.21.15.137:0/2607238629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc106e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009910 tx=0x7f81f0009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 -- 172.21.15.137:0/2607238629 shutdown_connections 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 --2- 172.21.15.137:0/2607238629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc106e20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 -- 172.21.15.137:0/2607238629 >> 172.21.15.137:0/2607238629 conn(0x7f81fc077830 msgr2=0x7f81fc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 -- 172.21.15.137:0/2607238629 shutdown_connections 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.683+0000 7f8203a00700 1 -- 172.21.15.137:0/2607238629 wait complete. 2024-03-26T13:50:25.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.685+0000 7f8203a00700 1 Processor -- start 2024-03-26T13:50:25.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.685+0000 7f8203a00700 1 -- start start 2024-03-26T13:50:25.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.685+0000 7f8203a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc102610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:25.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.685+0000 7f8203a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc10af30 con 0x7f81fc106a40 2024-03-26T13:50:25.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.686+0000 7f8201000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc102610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:25.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.686+0000 7f8201000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc102610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43012/0 (socket says 172.21.15.137:43012) 2024-03-26T13:50:25.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.686+0000 7f8201000700 1 -- 172.21.15.137:0/2754062772 learned_addr learned my addr 172.21.15.137:0/2754062772 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.686+0000 7f8201000700 1 -- 172.21.15.137:0/2754062772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc1044b0 con 0x7f81fc106a40 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.686+0000 7f8201000700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc102610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81f0006010 tx=0x7f81f0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.687+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81f0026070 con 0x7f81fc106a40 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.687+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81f000bc00 con 0x7f81fc106a40 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.687+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81f0003930 con 0x7f81fc106a40 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.687+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc102b50 con 0x7f81fc106a40 2024-03-26T13:50:25.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.687+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc103010 con 0x7f81fc106a40 2024-03-26T13:50:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.688+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f81f0029070 con 0x7f81fc106a40 2024-03-26T13:50:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.688+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc102180 con 0x7f81fc106a40 2024-03-26T13:50:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.688+0000 7f81f9600700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81e4039020 0x7f81e403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.689+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f81f001e070 con 0x7f81fc106a40 2024-03-26T13:50:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.689+0000 7f81fbe00700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81e4039020 0x7f81e403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.689+0000 7f81fbe00700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81e4039020 0x7f81e403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81e8006fd0 tx=0x7f81e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:25.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.694+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f81f0013070 con 0x7f81fc106a40 2024-03-26T13:50:25.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.922+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81fc103b40 con 0x7f81e4039020 2024-03-26T13:50:25.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.925+0000 7f81f9600700 1 -- 172.21.15.137:0/2754062772 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f81fc103b40 con 0x7f81e4039020 2024-03-26T13:50:25.917 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:25.917 INFO:teuthology.orchestra.run.smithi137.stdout:[{"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81e4039020 msgr2=0x7f81e403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81e4039020 0x7f81e403b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f81e8006fd0 tx=0x7f81e8006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 msgr2=0x7f81fc102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc102610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81f0006010 tx=0x7f81f0005780 comp rx=0 tx=0).stop 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 shutdown_connections 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81e4039020 0x7f81e403b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 --2- 172.21.15.137:0/2754062772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81fc106a40 0x7f81fc102610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 >> 172.21.15.137:0/2754062772 conn(0x7f81fc077830 msgr2=0x7f81fc069590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:25.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 shutdown_connections 2024-03-26T13:50:25.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:25.928+0000 7f8203a00700 1 -- 172.21.15.137:0/2754062772 wait complete. 2024-03-26T13:50:26.514 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:26 smithi137 ceph-mon[31539]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:50:26.566 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-03-26T13:50:26.567 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:50:26.567 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-03-26T13:50:26.624 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:50:26.624 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:50:26.719 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-03-26T13:50:26.719 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch host add smithi179 2024-03-26T13:50:27.149 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:27.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:27.550 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:27 smithi137 ceph-mon[31539]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:29.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.172+0000 7ff41da00700 1 -- 172.21.15.137:0/1859912170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 msgr2=0x7ff418104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.172+0000 7ff41da00700 1 --2- 172.21.15.137:0/1859912170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff418104380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff408009910 tx=0x7ff408009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.172+0000 7ff41da00700 1 -- 172.21.15.137:0/1859912170 shutdown_connections 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.172+0000 7ff41da00700 1 --2- 172.21.15.137:0/1859912170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff418104380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.172+0000 7ff41da00700 1 -- 172.21.15.137:0/1859912170 >> 172.21.15.137:0/1859912170 conn(0x7ff4180ff850 msgr2=0x7ff418101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.172+0000 7ff41da00700 1 -- 172.21.15.137:0/1859912170 shutdown_connections 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.173+0000 7ff41da00700 1 -- 172.21.15.137:0/1859912170 wait complete. 2024-03-26T13:50:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.174+0000 7ff41da00700 1 Processor -- start 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.174+0000 7ff41da00700 1 -- start start 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.174+0000 7ff41da00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff41819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.174+0000 7ff41da00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41810e220 con 0x7ff418103fa0 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.175+0000 7ff416a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff41819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.175+0000 7ff416a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff41819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40928/0 (socket says 172.21.15.137:40928) 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.175+0000 7ff416a00700 1 -- 172.21.15.137:0/1383933435 learned_addr learned my addr 172.21.15.137:0/1383933435 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.175+0000 7ff416a00700 1 -- 172.21.15.137:0/1383933435 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41819f940 con 0x7ff418103fa0 2024-03-26T13:50:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.175+0000 7ff416a00700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff41819f400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4080056d0 tx=0x7ff4080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:29.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.176+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff408026070 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.176+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff40800bba0 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.176+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41819fc30 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.176+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4080037f0 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.176+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4181995c0 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.177+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff418105480 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.177+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7ff408029070 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.177+0000 7ff40f400700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3fc039020 0x7ff3fc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.178+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff40801e070 con 0x7ff418103fa0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.178+0000 7ff416000700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3fc039020 0x7ff3fc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.178+0000 7ff416000700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3fc039020 0x7ff3fc03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff404006fd0 tx=0x7ff404006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:29.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.183+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff408013070 con 0x7ff418103fa0 2024-03-26T13:50:29.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:29.410+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) v1 -- 0x7ff41819aaf0 con 0x7ff3fc039020 2024-03-26T13:50:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:29 smithi137 ceph-mon[31539]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:30 smithi137 ceph-mon[31539]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: Deploying cephadm binary to smithi179 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:50:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:50:33.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:32 smithi137 ceph-mon[31539]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-26T13:50:34.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:34 smithi137 ceph-mon[31539]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:35.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.799+0000 7ff40f400700 1 -- 172.21.15.137:0/1383933435 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff41819aaf0 con 0x7ff3fc039020 2024-03-26T13:50:35.792 INFO:teuthology.orchestra.run.smithi137.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-03-26T13:50:35.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3fc039020 msgr2=0x7ff3fc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:35.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3fc039020 0x7ff3fc03b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff404006fd0 tx=0x7ff404006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:35.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 msgr2=0x7ff41819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:35.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff41819f400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4080056d0 tx=0x7ff4080057b0 comp rx=0 tx=0).stop 2024-03-26T13:50:35.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 shutdown_connections 2024-03-26T13:50:35.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3fc039020 0x7ff3fc03b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:35.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 --2- 172.21.15.137:0/1383933435 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff418103fa0 0x7ff41819f400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:35.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 >> 172.21.15.137:0/1383933435 conn(0x7ff4180ff850 msgr2=0x7ff418105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:35.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 shutdown_connections 2024-03-26T13:50:35.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:35.803+0000 7ff41da00700 1 -- 172.21.15.137:0/1383933435 wait complete. 2024-03-26T13:50:36.115 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:36 smithi137 ceph-mon[31539]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:50:36.115 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:36 smithi137 ceph-mon[31539]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:36.115 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:36.529 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch host ls --format=json 2024-03-26T13:50:36.965 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:37.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:37 smithi137 ceph-mon[31539]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:50:37.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:37 smithi137 ceph-mon[31539]: Added host smithi179 2024-03-26T13:50:37.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:37.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:37.254 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:38.134 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:38 smithi137 ceph-mon[31539]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T13:50:38.134 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:38 smithi137 ceph-mon[31539]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:38.134 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:38 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:50:38.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.910+0000 7fa446800700 1 -- 172.21.15.137:0/3577875303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 msgr2=0x7fa440102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:38.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.910+0000 7fa446800700 1 --2- 172.21.15.137:0/3577875303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440102190 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa430009910 tx=0x7fa430009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:38.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.910+0000 7fa446800700 1 -- 172.21.15.137:0/3577875303 shutdown_connections 2024-03-26T13:50:38.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.910+0000 7fa446800700 1 --2- 172.21.15.137:0/3577875303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440102190 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:38.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.910+0000 7fa446800700 1 -- 172.21.15.137:0/3577875303 >> 172.21.15.137:0/3577875303 conn(0x7fa4400fd660 msgr2=0x7fa4400ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:38.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.911+0000 7fa446800700 1 -- 172.21.15.137:0/3577875303 shutdown_connections 2024-03-26T13:50:38.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.911+0000 7fa446800700 1 -- 172.21.15.137:0/3577875303 wait complete. 2024-03-26T13:50:38.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.912+0000 7fa446800700 1 Processor -- start 2024-03-26T13:50:38.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.913+0000 7fa446800700 1 -- start start 2024-03-26T13:50:38.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.913+0000 7fa446800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:38.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.913+0000 7fa446800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44010c030 con 0x7fa440101db0 2024-03-26T13:50:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.913+0000 7fa43f400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.913+0000 7fa43f400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48212/0 (socket says 172.21.15.137:48212) 2024-03-26T13:50:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.913+0000 7fa43f400700 1 -- 172.21.15.137:0/3882070357 learned_addr learned my addr 172.21.15.137:0/3882070357 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.914+0000 7fa43f400700 1 -- 172.21.15.137:0/3882070357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa440197ad0 con 0x7fa440101db0 2024-03-26T13:50:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.914+0000 7fa43f400700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440197590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4300056d0 tx=0x7fa4300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.914+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa430026070 con 0x7fa440101db0 2024-03-26T13:50:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.914+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa43000bba0 con 0x7fa440101db0 2024-03-26T13:50:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.914+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4300037f0 con 0x7fa440101db0 2024-03-26T13:50:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.914+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa440197dc0 con 0x7fa440101db0 2024-03-26T13:50:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.915+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44019be60 con 0x7fa440101db0 2024-03-26T13:50:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.916+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa440103290 con 0x7fa440101db0 2024-03-26T13:50:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.916+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fa430029070 con 0x7fa440101db0 2024-03-26T13:50:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.916+0000 7fa437e00700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa424039020 0x7fa42403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.916+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa43001e070 con 0x7fa440101db0 2024-03-26T13:50:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.917+0000 7fa43ea00700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa424039020 0x7fa42403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.917+0000 7fa43ea00700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa424039020 0x7fa42403b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa42c006fd0 tx=0x7fa42c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:38.922+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa430013070 con 0x7fa440101db0 2024-03-26T13:50:39.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.150+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4401988f0 con 0x7fa424039020 2024-03-26T13:50:39.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.153+0000 7fa437e00700 1 -- 172.21.15.137:0/3882070357 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fa4401988f0 con 0x7fa424039020 2024-03-26T13:50:39.145 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:50:39.145 INFO:teuthology.orchestra.run.smithi137.stdout:[{"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa424039020 msgr2=0x7fa42403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa424039020 0x7fa42403b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa42c006fd0 tx=0x7fa42c006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 msgr2=0x7fa440197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440197590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4300056d0 tx=0x7fa4300057b0 comp rx=0 tx=0).stop 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 shutdown_connections 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa424039020 0x7fa42403b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 --2- 172.21.15.137:0/3882070357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa440101db0 0x7fa440197590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.155+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 >> 172.21.15.137:0/3882070357 conn(0x7fa4400fd660 msgr2=0x7fa440103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.156+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 shutdown_connections 2024-03-26T13:50:39.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:39.156+0000 7fa446800700 1 -- 172.21.15.137:0/3882070357 wait complete. 2024-03-26T13:50:39.515 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:39.515 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:39 smithi137 ceph-mon[31539]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:39.516 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:39.816 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-26T13:50:39.816 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd crush tunables default 2024-03-26T13:50:40.251 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:50:40.491 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:40 smithi137 ceph-mon[31539]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:50:42.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 -- 172.21.15.137:0/1934099882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 msgr2=0x7f6594102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:42.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 --2- 172.21.15.137:0/1934099882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594102190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6584009910 tx=0x7f6584009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:42.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 -- 172.21.15.137:0/1934099882 shutdown_connections 2024-03-26T13:50:42.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 --2- 172.21.15.137:0/1934099882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594102190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:42.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 -- 172.21.15.137:0/1934099882 >> 172.21.15.137:0/1934099882 conn(0x7f65940fd660 msgr2=0x7f65940ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:42.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 -- 172.21.15.137:0/1934099882 shutdown_connections 2024-03-26T13:50:42.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.205+0000 7f659ae00700 1 -- 172.21.15.137:0/1934099882 wait complete. 2024-03-26T13:50:42.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.206+0000 7f659ae00700 1 Processor -- start 2024-03-26T13:50:42.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.207+0000 7f659ae00700 1 -- start start 2024-03-26T13:50:42.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.207+0000 7f659ae00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594191590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:42.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.207+0000 7f659ae00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659410c030 con 0x7f6594101db0 2024-03-26T13:50:42.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.207+0000 7f6593e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594191590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:42.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.207+0000 7f6593e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594191590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48228/0 (socket says 172.21.15.137:48228) 2024-03-26T13:50:42.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.207+0000 7f6593e00700 1 -- 172.21.15.137:0/443989017 learned_addr learned my addr 172.21.15.137:0/443989017 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:50:42.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.208+0000 7f6593e00700 1 -- 172.21.15.137:0/443989017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65941954f0 con 0x7f6594101db0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.208+0000 7f6593e00700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594191590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6584006010 tx=0x7f6584005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.208+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6584026070 con 0x7f6594101db0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.208+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f658400bba0 con 0x7f6594101db0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.208+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65840038a0 con 0x7f6594101db0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.208+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6594191ad0 con 0x7f6594101db0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.209+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6594191f90 con 0x7f6594101db0 2024-03-26T13:50:42.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.210+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594103290 con 0x7f6594101db0 2024-03-26T13:50:42.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.210+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f6584029070 con 0x7f6594101db0 2024-03-26T13:50:42.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.210+0000 7f6590c00700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6578039020 0x7f657803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:42.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.210+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f658401e070 con 0x7f6594101db0 2024-03-26T13:50:42.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.211+0000 7f6593400700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6578039020 0x7f657803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:42.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.211+0000 7f6593400700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6578039020 0x7f657803b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6580006fd0 tx=0x7f6580006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:42.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.216+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6584013070 con 0x7f6594101db0 2024-03-26T13:50:42.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:42.443+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f65940637c0 con 0x7f6594101db0 2024-03-26T13:50:42.436 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:42 smithi137 ceph-mon[31539]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:42.436 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:43.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.063+0000 7f6590c00700 1 -- 172.21.15.137:0/443989017 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f658400fe20 con 0x7f6594101db0 2024-03-26T13:50:43.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6578039020 msgr2=0x7f657803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:43.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6578039020 0x7f657803b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6580006fd0 tx=0x7f6580006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 msgr2=0x7f6594191590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594191590 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6584006010 tx=0x7f6584005780 comp rx=0 tx=0).stop 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 shutdown_connections 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6578039020 0x7f657803b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 --2- 172.21.15.137:0/443989017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6594101db0 0x7f6594191590 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 >> 172.21.15.137:0/443989017 conn(0x7f65940fd660 msgr2=0x7f65940743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 shutdown_connections 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:50:43.067+0000 7f659ae00700 1 -- 172.21.15.137:0/443989017 wait complete. 2024-03-26T13:50:43.060 INFO:teuthology.orchestra.run.smithi137.stderr:adjusted tunables profile to default 2024-03-26T13:50:43.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:43 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/443989017' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T13:50:43.831 INFO:tasks.cephadm:Adding mon.a on smithi137 2024-03-26T13:50:43.832 INFO:tasks.cephadm:Adding mon.b on smithi159 2024-03-26T13:50:43.832 INFO:tasks.cephadm:Adding mon.c on smithi179 2024-03-26T13:50:43.832 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch apply mon '3;smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c' 2024-03-26T13:50:44.244 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-26T13:50:44.347 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-26T13:50:44.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:44 smithi137 ceph-mon[31539]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:44.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:44 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/443989017' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T13:50:44.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:44 smithi137 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:50:46.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:50:46.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 -- 172.21.15.179:0/2172451392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 msgr2=0x7f2af0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 --2- 172.21.15.179:0/2172451392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af0074620 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009910 tx=0x7f2ae0009c20 comp rx=0 tx=0).stop 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 -- 172.21.15.179:0/2172451392 shutdown_connections 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 --2- 172.21.15.179:0/2172451392 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af0074620 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 -- 172.21.15.179:0/2172451392 >> 172.21.15.179:0/2172451392 conn(0x7f2af00fd150 msgr2=0x7f2af00ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 -- 172.21.15.179:0/2172451392 shutdown_connections 2024-03-26T13:50:46.927 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.934+0000 7f2af5e00700 1 -- 172.21.15.179:0/2172451392 wait complete. 2024-03-26T13:50:46.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.935+0000 7f2af5e00700 1 Processor -- start 2024-03-26T13:50:46.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.936+0000 7f2af5e00700 1 -- start start 2024-03-26T13:50:46.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.936+0000 7f2af5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:46.928 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.936+0000 7f2af5e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af010e190 con 0x7f2af00761c0 2024-03-26T13:50:46.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.936+0000 7f2aeea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:46.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.937+0000 7f2aeea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.179:43028/0 (socket says 172.21.15.179:43028) 2024-03-26T13:50:46.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.937+0000 7f2aeea00700 1 -- 172.21.15.179:0/3112037117 learned_addr learned my addr 172.21.15.179:0/3112037117 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:50:46.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.937+0000 7f2aeea00700 1 -- 172.21.15.179:0/3112037117 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af019dd40 con 0x7f2af00761c0 2024-03-26T13:50:46.929 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.938+0000 7f2aeea00700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af019d800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005950 tx=0x7f2ae0004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.938+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ae0026070 con 0x7f2af00761c0 2024-03-26T13:50:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.938+0000 7f2af5e00700 1 -- 172.21.15.179:0/3112037117 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af019dfd0 con 0x7f2af00761c0 2024-03-26T13:50:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.938+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ae000bc00 con 0x7f2af00761c0 2024-03-26T13:50:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.938+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ae00039a0 con 0x7f2af00761c0 2024-03-26T13:50:46.930 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.938+0000 7f2af5e00700 1 -- 172.21.15.179:0/3112037117 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af01a0cc0 con 0x7f2af00761c0 2024-03-26T13:50:46.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.940+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f2ae0029070 con 0x7f2af00761c0 2024-03-26T13:50:46.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.940+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac8005330 con 0x7f2af00761c0 2024-03-26T13:50:46.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.940+0000 7f2af4a00700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2ad4038fc0 0x7f2ad403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:46.932 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.940+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2ae001e070 con 0x7f2af00761c0 2024-03-26T13:50:46.933 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.941+0000 7f2ae7e00700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2ad4038fc0 0x7f2ad403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:46.934 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.942+0000 7f2ae7e00700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2ad4038fc0 0x7f2ad403b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006fd0 tx=0x7f2ad8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:46.949 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:46.957+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ae0013070 con 0x7f2af00761c0 2024-03-26T13:50:47.185 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.193+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ac8000c80 con 0x7f2ad4038fc0 2024-03-26T13:50:47.201 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.209+0000 7f2af4a00700 1 -- 172.21.15.179:0/3112037117 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2ac8000c80 con 0x7f2ad4038fc0 2024-03-26T13:50:47.201 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-03-26T13:50:47.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.211+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2ad4038fc0 msgr2=0x7f2ad403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:47.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.211+0000 7f2ae5600700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2ad4038fc0 0x7f2ad403b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8006fd0 tx=0x7f2ad8006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:47.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 msgr2=0x7f2af019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:47.204 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af019d800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0005950 tx=0x7f2ae0004fa0 comp rx=0 tx=0).stop 2024-03-26T13:50:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 shutdown_connections 2024-03-26T13:50:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2ad4038fc0 0x7f2ad403b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 --2- 172.21.15.179:0/3112037117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2af00761c0 0x7f2af019d800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 >> 172.21.15.179:0/3112037117 conn(0x7f2af00fd150 msgr2=0x7f2af00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 shutdown_connections 2024-03-26T13:50:47.205 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:47.212+0000 7f2ae5600700 1 -- 172.21.15.179:0/3112037117 wait complete. 2024-03-26T13:50:47.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:47 smithi137 ceph-mon[31539]: Updating smithi179:/etc/ceph/ceph.conf 2024-03-26T13:50:47.945 DEBUG:teuthology.orchestra.run.smithi159:mon.b> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.b.service 2024-03-26T13:50:47.953 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.c.service 2024-03-26T13:50:47.957 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T13:50:47.957 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T13:50:48.381 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:50:48.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:48 smithi137 ceph-mon[31539]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:50:48.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:48 smithi137 ceph-mon[31539]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:48.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:49.429 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: Saving service mon spec with placement smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c;count:3 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:50:49.430 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:50:50.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:50 smithi137 ceph-mon[31539]: Deploying daemon mon.c on smithi179 2024-03-26T13:50:50.456 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:50 smithi137 ceph-mon[31539]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:50.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 -- 172.21.15.179:0/1777453452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 msgr2=0x7fb554074910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:50.606 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 --2- 172.21.15.179:0/1777453452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb554074910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb54400b0a0 tx=0x7fb54400b3b0 comp rx=0 tx=0).stop 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 -- 172.21.15.179:0/1777453452 shutdown_connections 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 --2- 172.21.15.179:0/1777453452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb554074910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 -- 172.21.15.179:0/1777453452 >> 172.21.15.179:0/1777453452 conn(0x7fb5540fd490 msgr2=0x7fb5540ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 -- 172.21.15.179:0/1777453452 shutdown_connections 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.455+0000 7fb553200700 1 -- 172.21.15.179:0/1777453452 wait complete. 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.456+0000 7fb553200700 1 Processor -- start 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.456+0000 7fb553200700 1 -- start start 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.457+0000 7fb553200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb5541a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.457+0000 7fb553200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541124d0 con 0x7fb554074530 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.457+0000 7fb551e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb5541a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.457+0000 7fb551e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb5541a36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.179:43052/0 (socket says 172.21.15.179:43052) 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.457+0000 7fb551e00700 1 -- 172.21.15.179:0/978948876 learned_addr learned my addr 172.21.15.179:0/978948876 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.458+0000 7fb551e00700 1 -- 172.21.15.179:0/978948876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5541a3bf0 con 0x7fb554074530 2024-03-26T13:50:50.607 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.458+0000 7fb551e00700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb5541a36b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5440076a0 tx=0x7fb544009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.458+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb544017040 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.459+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541a3e80 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.459+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb544004040 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.459+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb54401d450 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.459+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55419d830 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.460+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554075c70 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.460+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fb544024020 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.461+0000 7fb54b400700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb538038fc0 0x7fb53803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.461+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb544022020 con 0x7fb554074530 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.461+0000 7fb549400700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb538038fc0 0x7fb53803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.462+0000 7fb549400700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb538038fc0 0x7fb53803b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb53c006fd0 tx=0x7fb53c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:50:50.608 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.470+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb54402b020 con 0x7fb554074530 2024-03-26T13:50:50.772 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.779+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb55419ed60 con 0x7fb554074530 2024-03-26T13:50:50.773 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.781+0000 7fb54b400700 1 -- 172.21.15.179:0/978948876 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7fb54401cbf0 con 0x7fb554074530 2024-03-26T13:50:50.773 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:50:50.773 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","modified":"2024-03-26T13:48:42.954316Z","created":"2024-03-26T13:48:42.954316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-26T13:50:50.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb538038fc0 msgr2=0x7fb53803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:50.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb538038fc0 0x7fb53803b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb53c006fd0 tx=0x7fb53c006e40 comp rx=0 tx=0).stop 2024-03-26T13:50:50.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 msgr2=0x7fb5541a36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:50:50.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb5541a36b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5440076a0 tx=0x7fb544009800 comp rx=0 tx=0).stop 2024-03-26T13:50:50.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 shutdown_connections 2024-03-26T13:50:50.777 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb538038fc0 0x7fb53803b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:50.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 --2- 172.21.15.179:0/978948876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb554074530 0x7fb5541a36b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:50:50.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 >> 172.21.15.179:0/978948876 conn(0x7fb5540fd490 msgr2=0x7fb5540fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:50:50.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 shutdown_connections 2024-03-26T13:50:50.778 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:50:50.785+0000 7fb553200700 1 -- 172.21.15.179:0/978948876 wait complete. 2024-03-26T13:50:50.778 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-03-26T13:50:51.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:51 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/978948876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:50:52.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:52 smithi137 ceph-mon[31539]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:53.104 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T13:50:53.104 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T13:50:53.533 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:50:54.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:50:54 smithi137 ceph-mon[31539]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:50:56.042 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:50:55 smithi179 ceph-mon[35472]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-26T13:50:56.042 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:50:56 smithi179 bash[35429]: df3ce41e900ce8d2eb2b7d8b0b5ff2736e4c70a0c328b3f5b551725ba7ec6cf7 2024-03-26T13:50:56.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:50:56 smithi179 systemd[1]: Started Ceph mon.c for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: mon.a calling monitor election 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: mon.c calling monitor election 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: monmap e2: 2 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: fsmap 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: mgrmap e13: x(active, since 68s) 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: overall HEALTH_OK 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:01.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:01.454 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:51:01.454 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: mon.a calling monitor election 2024-03-26T13:51:01.454 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.454 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: mon.c calling monitor election 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: monmap e2: 2 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: fsmap 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: mgrmap e13: x(active, since 68s) 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: overall HEALTH_OK 2024-03-26T13:51:01.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:01.456 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.105+0000 7faebd400700 1 -- 172.21.15.179:0/1295893835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae9c005cc0 msgr2=0x7fae9c0080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.105+0000 7faebd400700 1 --2- 172.21.15.179:0/1295893835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae9c005cc0 0x7fae9c0080f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faeb80759c0 tx=0x7faea800ff20 comp rx=0 tx=0).stop 2024-03-26T13:51:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.105+0000 7faebd400700 1 -- 172.21.15.179:0/1295893835 shutdown_connections 2024-03-26T13:51:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.105+0000 7faebd400700 1 --2- 172.21.15.179:0/1295893835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae9c005cc0 0x7fae9c0080f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.105+0000 7faebd400700 1 -- 172.21.15.179:0/1295893835 >> 172.21.15.179:0/1295893835 conn(0x7faeb80fd720 msgr2=0x7faeb80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.106+0000 7faebd400700 1 -- 172.21.15.179:0/1295893835 shutdown_connections 2024-03-26T13:51:02.100 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.106+0000 7faebd400700 1 -- 172.21.15.179:0/1295893835 wait complete. 2024-03-26T13:51:02.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.109+0000 7faebd400700 1 Processor -- start 2024-03-26T13:51:02.101 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.109+0000 7faebd400700 1 -- start start 2024-03-26T13:51:02.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.110+0000 7faebd400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:02.102 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.110+0000 7faebd400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeb819e3d0 0x7faeb81a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:02.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.110+0000 7faebd400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8110170 con 0x7faeb819e3d0 2024-03-26T13:51:02.103 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.110+0000 7faebd400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb810fff0 con 0x7fae9c005cc0 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeafe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeb819e3d0 0x7faeb81a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:45906/0 (socket says 172.21.15.179:45906) 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 -- 172.21.15.179:0/1974920904 learned_addr learned my addr 172.21.15.179:0/1974920904 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 -- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 msgr2=0x7faeb819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 -- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 msgr2=0x7faeb819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.111+0000 7faeb6800700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-26T13:51:02.104 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.112+0000 7faeafe00700 1 -- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 msgr2=0x7faeb819de90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-26T13:51:02.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.112+0000 7faeafe00700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:02.105 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.112+0000 7faeafe00700 1 -- 172.21.15.179:0/1974920904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb819ea10 con 0x7faeb819e3d0 2024-03-26T13:51:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.112+0000 7faeafe00700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeb819e3d0 0x7faeb81a2710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faea000b4d0 tx=0x7faea000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.113+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7faea0011630 con 0x7faeb819e3d0 2024-03-26T13:51:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.113+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb81a2c50 con 0x7faeb819e3d0 2024-03-26T13:51:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.113+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faea0011c80 con 0x7faeb819e3d0 2024-03-26T13:51:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.113+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7faea0010da0 con 0x7faeb819e3d0 2024-03-26T13:51:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.113+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb81a31b0 con 0x7faeb819e3d0 2024-03-26T13:51:02.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.115+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb8075c70 con 0x7faeb819e3d0 2024-03-26T13:51:02.107 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.115+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7faea0018020 con 0x7faeb819e3d0 2024-03-26T13:51:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.115+0000 7faeaf400700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae94038dd0 0x7fae9403b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.115+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faea004b9b0 con 0x7faeb819e3d0 2024-03-26T13:51:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.116+0000 7faeb6800700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae94038dd0 0x7fae9403b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:02.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.117+0000 7faeb6800700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae94038dd0 0x7fae9403b290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faea800fad0 tx=0x7faea8014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:02.123 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.131+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faea001f020 con 0x7faeb819e3d0 2024-03-26T13:51:02.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:02 smithi179 ceph-mon[35472]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:02.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:02.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:02.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:02.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:02 smithi179 ceph-mon[35472]: Deploying daemon mon.b on smithi159 2024-03-26T13:51:02.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:02.435 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.443+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faeb81a34c0 con 0x7faeb819e3d0 2024-03-26T13:51:02.437 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.444+0000 7faeaf400700 1 -- 172.21.15.179:0/1974920904 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7faea0017dd0 con 0x7faeb819e3d0 2024-03-26T13:51:02.437 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:51:02.437 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","modified":"2024-03-26T13:50:55.977703Z","created":"2024-03-26T13:48:42.954316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-26T13:51:02.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae94038dd0 msgr2=0x7fae9403b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:02.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae94038dd0 0x7fae9403b290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7faea800fad0 tx=0x7faea8014040 comp rx=0 tx=0).stop 2024-03-26T13:51:02.440 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeb819e3d0 msgr2=0x7faeb81a2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeb819e3d0 0x7faeb81a2710 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faea000b4d0 tx=0x7faea000b890 comp rx=0 tx=0).stop 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 shutdown_connections 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae94038dd0 0x7fae9403b290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae9c005cc0 0x7faeb819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 --2- 172.21.15.179:0/1974920904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faeb819e3d0 0x7faeb81a2710 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.447+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 >> 172.21.15.179:0/1974920904 conn(0x7faeb80fd720 msgr2=0x7faeb81051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.448+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 shutdown_connections 2024-03-26T13:51:02.441 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:02.448+0000 7faebd400700 1 -- 172.21.15.179:0/1974920904 wait complete. 2024-03-26T13:51:02.442 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-03-26T13:51:02.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-mon[31539]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:02.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:02.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:02.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:02.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-mon[31539]: Deploying daemon mon.b on smithi159 2024-03-26T13:51:02.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:03.236 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:03 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/1974920904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:03.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:03 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/1974920904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:03.277 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:51:02 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:51:02.979+0000 7f8bf8c00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-26T13:51:04.306 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-26T13:51:04.306 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph mon dump -f json 2024-03-26T13:51:04.394 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:04 smithi179 ceph-mon[35472]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:04.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:04 smithi137 ceph-mon[31539]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:04.749 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:51:06.245 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:06 smithi179 ceph-mon[35472]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:06.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:06 smithi137 ceph-mon[31539]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:06.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 -- 172.21.15.179:0/1920983425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 msgr2=0x7f02281044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:06.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 --2- 172.21.15.179:0/1920983425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 0x7f02281044c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0218009910 tx=0x7f0218009c20 comp rx=0 tx=0).stop 2024-03-26T13:51:06.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 -- 172.21.15.179:0/1920983425 shutdown_connections 2024-03-26T13:51:06.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 --2- 172.21.15.179:0/1920983425 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 0x7f02281044c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:06.763 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 -- 172.21.15.179:0/1920983425 >> 172.21.15.179:0/1920983425 conn(0x7f02280ff990 msgr2=0x7f0228101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:06.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 -- 172.21.15.179:0/1920983425 shutdown_connections 2024-03-26T13:51:06.764 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.770+0000 7f022e800700 1 -- 172.21.15.179:0/1920983425 wait complete. 2024-03-26T13:51:06.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.772+0000 7f022e800700 1 Processor -- start 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.772+0000 7f022e800700 1 -- start start 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.772+0000 7f022e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 0x7f022819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.772+0000 7f022e800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 0x7f0228199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.772+0000 7f022e800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022810e360 con 0x7f02281040e0 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.772+0000 7f022e800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022810e1e0 con 0x7f022819fc00 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0226a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 0x7f0228199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0226a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 0x7f0228199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:45920/0 (socket says 172.21.15.179:45920) 2024-03-26T13:51:06.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0226a00700 1 -- 172.21.15.179:0/2625063033 learned_addr learned my addr 172.21.15.179:0/2625063033 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0227400700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 0x7f022819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0226a00700 1 -- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 msgr2=0x7f022819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0226a00700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 0x7f022819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.773+0000 7f0226a00700 1 -- 172.21.15.179:0/2625063033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02281a0240 con 0x7f022819fc00 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.774+0000 7f0226a00700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 0x7f0228199840 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f021400b700 tx=0x7f021400ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.774+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0214011630 con 0x7f022819fc00 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.774+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0214011c80 con 0x7f022819fc00 2024-03-26T13:51:06.768 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.774+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0214010c80 con 0x7f022819fc00 2024-03-26T13:51:06.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.774+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0228199de0 con 0x7f022819fc00 2024-03-26T13:51:06.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.774+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022819a340 con 0x7f022819fc00 2024-03-26T13:51:06.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.775+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f0214018020 con 0x7f022819fc00 2024-03-26T13:51:06.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.775+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02281055c0 con 0x7f022819fc00 2024-03-26T13:51:06.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.776+0000 7f021fe00700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f020c038db0 0x7f020c03b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:06.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.776+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f021404b4a0 con 0x7f022819fc00 2024-03-26T13:51:06.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.776+0000 7f0227400700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f020c038db0 0x7f020c03b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:06.771 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.777+0000 7f0227400700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f020c038db0 0x7f020c03b270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0218006010 tx=0x7f021801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:06.774 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.782+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f021401f070 con 0x7f022819fc00 2024-03-26T13:51:06.887 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 podman[35482]: 2024-03-26 13:51:06.845871392 +0000 UTC m=+0.365654944 container start a88340e27b397d82cf5b106a2419704e567b4a8c2bf15a3bcce42f64c79161ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T13:51:06.887 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: set uid:gid to 167:167 (ceph:ceph) 2024-03-26T13:51:06.887 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev), process ceph-mon, pid 2 2024-03-26T13:51:06.888 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pidfile_write: ignore empty --pid-file 2024-03-26T13:51:06.952 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:06.960+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140184f0 con 0x7f022819fc00 2024-03-26T13:51:07.083 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:07.091+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f022819a6b0 con 0x7f022819fc00 2024-03-26T13:51:07.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: load: jerasure load: lrc 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: RocksDB version: 7.9.2 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Git sha 0 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Compile date 2024-03-22 04:58:55 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: DB SUMMARY 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: DB Session ID: NA9LGCFP3R1M9HRZD3X6 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: CURRENT file: CURRENT 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: IDENTITY file: IDENTITY 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-26T13:51:07.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.error_if_exists: 0 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.create_if_missing: 0 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.paranoid_checks: 1 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.env: 0x5605b9e70b20 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.fs: PosixFileSystem 2024-03-26T13:51:07.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.info_log: 0x5605baae3960 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_file_opening_threads: 16 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.statistics: (nil) 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.use_fsync: 0 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_log_file_size: 0 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.keep_log_file_num: 1000 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.recycle_log_file_num: 0 2024-03-26T13:51:07.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.allow_fallocate: 1 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.allow_mmap_reads: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.allow_mmap_writes: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.use_direct_reads: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.create_missing_column_families: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.db_log_dir: 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.wal_dir: 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-26T13:51:07.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.advise_random_on_open: 1 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.db_write_buffer_size: 0 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.write_buffer_manager: 0x5605baab61e0 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-26T13:51:07.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.rate_limiter: (nil) 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.wal_recovery_mode: 2 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.enable_thread_tracking: 0 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.enable_pipelined_write: 0 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.unordered_write: 0 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-26T13:51:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.row_cache: None 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.wal_filter: None 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.allow_ingest_behind: 0 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.two_write_queues: 0 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.manual_wal_flush: 0 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.wal_compression: 0 2024-03-26T13:51:07.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.atomic_flush: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.log_readahead_size: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.best_efforts_recovery: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.allow_data_in_errors: 0 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.db_host_id: __hostname__ 2024-03-26T13:51:07.205 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_background_jobs: 2 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_background_compactions: -1 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_subcompactions: 1 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_total_wal_size: 0 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-26T13:51:07.206 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_open_files: -1 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bytes_per_sync: 0 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_readahead_size: 0 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_background_flushes: -1 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Compression algorithms supported: 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-26T13:51:07.207 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kZSTD supported: 0 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kXpressCompression supported: 0 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kLZ4HCCompression supported: 1 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kZlibCompression supported: 1 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kSnappyCompression supported: 1 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kLZ4Compression supported: 1 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: kBZip2Compression supported: 0 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-26T13:51:07.208 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.merge_operator: 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_filter: None 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_filter_factory: None 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.sst_partitioner_factory: None 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5605baae3a60) 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout: cache_index_and_filter_blocks: 1 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout: pin_top_level_index_and_filter: 1 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_type: 0 2024-03-26T13:51:07.209 INFO:journalctl@ceph.mon.b.smithi159.stdout: data_block_index_type: 0 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_shortening: 1 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: checksum: 4 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: no_block_cache: 0 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache: 0x5605baae9090 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_name: BinnedLRUCache 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_options: 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: capacity : 536870912 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: num_shard_bits : 4 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: strict_capacity_limit : 0 2024-03-26T13:51:07.210 INFO:journalctl@ceph.mon.b.smithi159.stdout: high_pri_pool_ratio: 0.000 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_compressed: (nil) 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: persistent_cache: (nil) 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_size: 4096 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_size_deviation: 10 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_restart_interval: 16 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_block_restart_interval: 1 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: metadata_block_size: 4096 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: partition_filters: 0 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: use_delta_encoding: 1 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: filter_policy: bloomfilter 2024-03-26T13:51:07.211 INFO:journalctl@ceph.mon.b.smithi159.stdout: whole_key_filtering: 1 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: verify_compression: 0 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: read_amp_bytes_per_bit: 0 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: format_version: 5 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: enable_index_compression: 1 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_align: 0 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: max_auto_readahead_size: 262144 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: prepopulate_block_cache: 0 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: initial_auto_readahead_size: 8192 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.write_buffer_size: 33554432 2024-03-26T13:51:07.212 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_write_buffer_number: 2 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression: NoCompression 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression: Disabled 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.prefix_extractor: nullptr 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.num_levels: 7 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-26T13:51:07.213 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.level: 32767 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.strategy: 0 2024-03-26T13:51:07.214 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.enabled: false 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.target_file_size_base: 67108864 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-26T13:51:07.215 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.arena_block_size: 1048576 2024-03-26T13:51:07.216 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.disable_auto_compactions: 0 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-26T13:51:07.217 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.table_properties_collectors: 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.inplace_update_support: 0 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-26T13:51:07.219 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.bloom_locality: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.max_successive_merges: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.paranoid_file_checks: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.force_consistency_checks: 1 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.report_bg_io_stats: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.ttl: 2592000 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-26T13:51:07.220 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.enable_blob_files: false 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.min_blob_size: 0 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.blob_file_size: 268435456 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.blob_file_starting_level: 0 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-26T13:51:07.221 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6e27233e-0764-42a3-bbeb-5ac09b442790 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711461066906458, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711461066907991, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711461066, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6e27233e-0764-42a3-bbeb-5ac09b442790", "db_session_id": "NA9LGCFP3R1M9HRZD3X6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711461066908320, "job": 1, "event": "recovery_finished"} 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5605babdc000 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: DB pointer 0x5605babc6000 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** DB Stats ** 2024-03-26T13:51:07.222 INFO:journalctl@ceph.mon.b.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** Compaction Stats [default] ** 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-26T13:51:07.223 INFO:journalctl@ceph.mon.b.smithi159.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** Compaction Stats [default] ** 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-26T13:51:07.224 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative compaction: 0.00 GB write, 0.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval compaction: 0.00 GB write, 0.20 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: Block cache BinnedLRUCache@0x5605baae9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-26T13:51:07.225 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: using public_addr v2:172.21.15.159:0/0 -> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: starting mon.b rank -1 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(???) e0 preinit fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 bash[35482]: a88340e27b397d82cf5b106a2419704e567b4a8c2bf15a3bcce42f64c79161ad 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 systemd[1]: Started Ceph mon.b for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).mds e1 new map 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout: e1 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T13:51:07.226 INFO:journalctl@ceph.mon.b.smithi159.stdout: legacy client fscid: -1 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout: No filesystems configured 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-26T13:51:07.227 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Deploying cephadm binary to smithi159 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Added host smithi159 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.228 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Deploying cephadm binary to smithi179 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-26T13:51:07.229 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Added host smithi179 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:07.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/443989017' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/443989017' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.231 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi179:/etc/ceph/ceph.conf 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Saving service mon spec with placement smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c;count:3 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:51:07.232 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.client.admin.keyring 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Deploying daemon mon.c on smithi179 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/978948876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:51:07.233 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.a calling monitor election 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.c calling monitor election 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: monmap e2: 2 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:51:07.234 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: fsmap 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mgrmap e13: x(active, since 68s) 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: overall HEALTH_OK 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: Deploying daemon mon.b on smithi159 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/1974920904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:07.235 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:07.236 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:06 smithi159 ceph-mon[35501]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-26T13:51:11.972 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.979+0000 7f021fe00700 1 -- 172.21.15.179:0/2625063033 <== mon.1 v2:172.21.15.179:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1289 (secure 0 0 0) 0x7f021400f540 con 0x7f022819fc00 2024-03-26T13:51:11.972 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-03-26T13:51:11.972 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","modified":"2024-03-26T13:51:06.957196Z","created":"2024-03-26T13:48:42.954316Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-26T13:51:11.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f020c038db0 msgr2=0x7f020c03b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:11.975 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f020c038db0 0x7f020c03b270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0218006010 tx=0x7f021801a040 comp rx=0 tx=0).stop 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 msgr2=0x7f0228199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 0x7f0228199840 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f021400b700 tx=0x7f021400ba10 comp rx=0 tx=0).stop 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 shutdown_connections 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f020c038db0 0x7f020c03b270 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02281040e0 0x7f022819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 --2- 172.21.15.179:0/2625063033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f022819fc00 0x7f0228199840 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 >> 172.21.15.179:0/2625063033 conn(0x7f02280ff990 msgr2=0x7f0228105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 shutdown_connections 2024-03-26T13:51:11.976 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:11.983+0000 7f022e800700 1 -- 172.21.15.179:0/2625063033 wait complete. 2024-03-26T13:51:11.977 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: mon.a calling monitor election 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: mon.c calling monitor election 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/2625063033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:12.345 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: mon.b calling monitor election 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: monmap e3: 3 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: fsmap 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: mgrmap e13: x(active, since 79s) 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: overall HEALTH_OK 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:12.346 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:12.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:51:12.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: mon.a calling monitor election 2024-03-26T13:51:12.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: mon.c calling monitor election 2024-03-26T13:51:12.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/2625063033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: mon.b calling monitor election 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: monmap e3: 3 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: fsmap 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: mgrmap e13: x(active, since 79s) 2024-03-26T13:51:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: overall HEALTH_OK 2024-03-26T13:51:12.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:12.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:12.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:51:12.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: mon.a calling monitor election 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: mon.c calling monitor election 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/2625063033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: mon.b calling monitor election 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: monmap e3: 3 mons at {a=[v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-26T13:51:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: fsmap 2024-03-26T13:51:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: osdmap e4: 0 total, 0 up, 0 in 2024-03-26T13:51:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: mgrmap e13: x(active, since 79s) 2024-03-26T13:51:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: overall HEALTH_OK 2024-03-26T13:51:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:12.715 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-26T13:51:12.715 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph config generate-minimal-conf 2024-03-26T13:51:13.129 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:51:13.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:13.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:13.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.423 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:13.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:13.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:13.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:13.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:51:13.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:14.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:14 smithi137 ceph-mon[31539]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:14.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:14 smithi179 ceph-mon[35472]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:14.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:14 smithi159 ceph-mon[35501]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.218+0000 7fbf00200700 1 -- 172.21.15.137:0/4187667613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 msgr2=0x7fbef8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.218+0000 7fbf00200700 1 --2- 172.21.15.137:0/4187667613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef8102190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009910 tx=0x7fbeec009c20 comp rx=0 tx=0).stop 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.218+0000 7fbf00200700 1 -- 172.21.15.137:0/4187667613 shutdown_connections 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.218+0000 7fbf00200700 1 --2- 172.21.15.137:0/4187667613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef8102190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.218+0000 7fbf00200700 1 -- 172.21.15.137:0/4187667613 >> 172.21.15.137:0/4187667613 conn(0x7fbef80fd660 msgr2=0x7fbef80ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.219+0000 7fbf00200700 1 -- 172.21.15.137:0/4187667613 shutdown_connections 2024-03-26T13:51:15.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.219+0000 7fbf00200700 1 -- 172.21.15.137:0/4187667613 wait complete. 2024-03-26T13:51:15.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 Processor -- start 2024-03-26T13:51:15.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 -- start start 2024-03-26T13:51:15.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:15.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef819fd20 0x7fbef81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbef8199ee0 0x7fbef819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef810c030 con 0x7fbef8101db0 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef810beb0 con 0x7fbef819fd20 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.220+0000 7fbf00200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef810c1b0 con 0x7fbef8199ee0 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52312/0 (socket says 172.21.15.137:52312) 2024-03-26T13:51:15.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 -- 172.21.15.137:0/2980913907 learned_addr learned my addr 172.21.15.137:0/2980913907 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:51:15.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefe200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbef8199ee0 0x7fbef819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:15.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefce00700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef819fd20 0x7fbef81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:15.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 -- 172.21.15.137:0/2980913907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbef8199ee0 msgr2=0x7fbef819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbef8199ee0 0x7fbef819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 -- 172.21.15.137:0/2980913907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef819fd20 msgr2=0x7fbef81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef819fd20 0x7fbef81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 -- 172.21.15.137:0/2980913907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef81a66b0 con 0x7fbef8101db0 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefe200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbef8199ee0 0x7fbef819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.221+0000 7fbefd800700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef819f7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbeec005b40 tx=0x7fbeec005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.222+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec026070 con 0x7fbef8101db0 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.222+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbeec00b6f0 con 0x7fbef8101db0 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.222+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0038e0 con 0x7fbef8101db0 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.222+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef81a68e0 con 0x7fbef8101db0 2024-03-26T13:51:15.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.222+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef81a6dd0 con 0x7fbef8101db0 2024-03-26T13:51:15.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.223+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef80637c0 con 0x7fbef8101db0 2024-03-26T13:51:15.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.223+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fbeec029070 con 0x7fbef8101db0 2024-03-26T13:51:15.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.224+0000 7fbef6000700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbee0039450 0x7fbee003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:15.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.224+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbeec04cdc0 con 0x7fbef8101db0 2024-03-26T13:51:15.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.224+0000 7fbefce00700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbee0039450 0x7fbee003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:15.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.225+0000 7fbefce00700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbee0039450 0x7fbee003b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009850 tx=0x7fbee4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:15.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.230+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbeec013070 con 0x7fbef8101db0 2024-03-26T13:51:15.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.454+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fbef81a7030 con 0x7fbef8101db0 2024-03-26T13:51:15.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:15.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:15.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-26T13:51:15.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-26T13:51:15.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: Updating smithi179:/etc/ceph/ceph.conf 2024-03-26T13:51:15.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:15.447 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:15 smithi137 ceph-mon[31539]: Updating smithi137:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:15.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.454+0000 7fbef6000700 1 -- 172.21.15.137:0/2980913907 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7fbeec030020 con 0x7fbef8101db0 2024-03-26T13:51:15.447 INFO:teuthology.orchestra.run.smithi137.stdout:# minimal ceph.conf for 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:51:15.447 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-03-26T13:51:15.447 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T13:51:15.447 INFO:teuthology.orchestra.run.smithi137.stdout: mon_host = [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-03-26T13:51:15.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbee0039450 msgr2=0x7fbee003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:15.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbee0039450 0x7fbee003b910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009850 tx=0x7fbee4008040 comp rx=0 tx=0).stop 2024-03-26T13:51:15.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 msgr2=0x7fbef819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:15.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef819f7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbeec005b40 tx=0x7fbeec005030 comp rx=0 tx=0).stop 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 shutdown_connections 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbee0039450 0x7fbee003b910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbef8199ee0 0x7fbef819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbef8101db0 0x7fbef819f7e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 --2- 172.21.15.137:0/2980913907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbef819fd20 0x7fbef81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.457+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 >> 172.21.15.137:0/2980913907 conn(0x7fbef80fd660 msgr2=0x7fbef8074500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.458+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 shutdown_connections 2024-03-26T13:51:15.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:15.458+0000 7fbf00200700 1 -- 172.21.15.137:0/2980913907 wait complete. 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: Updating smithi179:/etc/ceph/ceph.conf 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:15.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:15 smithi179 ceph-mon[35472]: Updating smithi137:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:15.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: Updating smithi137:/etc/ceph/ceph.conf 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: Updating smithi159:/etc/ceph/ceph.conf 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: Updating smithi179:/etc/ceph/ceph.conf 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:15.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:15 smithi159 ceph-mon[35501]: Updating smithi137:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.283 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-26T13:51:16.284 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:51:16.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T13:51:16.377 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:51:16.377 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:51:16.468 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:51:16.468 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2980913907' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.572 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:51:16.572 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:51:16.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2980913907' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:16.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: Updating smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: Updating smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/config/ceph.conf 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2980913907' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:16.709 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:51:16.709 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-03-26T13:51:16.812 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:51:16.812 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-26T13:51:16.956 INFO:tasks.cephadm:Adding mgr.x on smithi137 2024-03-26T13:51:16.956 INFO:tasks.cephadm:Adding mgr.y on smithi159 2024-03-26T13:51:16.956 INFO:tasks.cephadm:Adding mgr.z on smithi179 2024-03-26T13:51:16.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch apply mgr '3;smithi137=x;smithi159=y;smithi179=z' 2024-03-26T13:51:17.431 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:51:17.572 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.572 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.572 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.572 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.572 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T13:51:17.572 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:17.573 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:17.573 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:17.573 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: Reconfiguring daemon mon.a on smithi137 2024-03-26T13:51:17.573 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:17 smithi137 ceph-mon[31539]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: Reconfiguring daemon mon.a on smithi137 2024-03-26T13:51:17.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:17 smithi159 ceph-mon[35501]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:17.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:17.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: Reconfiguring mon.a (unknown last config time)... 2024-03-26T13:51:17.739 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:17.739 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:17.739 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:17.739 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: Reconfiguring daemon mon.a on smithi137 2024-03-26T13:51:17.739 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:17 smithi179 ceph-mon[35472]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 -- 172.21.15.179:0/33070778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c102590 msgr2=0x7f6a7c104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 --2- 172.21.15.179:0/33070778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c102590 0x7f6a7c104980 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009910 tx=0x7f6a70009c20 comp rx=0 tx=0).stop 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 -- 172.21.15.179:0/33070778 shutdown_connections 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 --2- 172.21.15.179:0/33070778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c102590 0x7f6a7c104980 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 -- 172.21.15.179:0/33070778 >> 172.21.15.179:0/33070778 conn(0x7f6a7c0fbfc0 msgr2=0x7f6a7c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 -- 172.21.15.179:0/33070778 shutdown_connections 2024-03-26T13:51:19.379 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.385+0000 7f6a82200700 1 -- 172.21.15.179:0/33070778 wait complete. 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 Processor -- start 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 -- start start 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a7c102590 0x7f6a7c1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 0x7f6a7c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c197b60 0x7f6a7c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1079e0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c107860 con 0x7f6a7c197b60 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.387+0000 7f6a82200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c107b60 con 0x7f6a7c102590 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.388+0000 7f6a7be00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 0x7f6a7c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.388+0000 7f6a7be00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 0x7f6a7c19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.179:52936/0 (socket says 172.21.15.179:52936) 2024-03-26T13:51:19.381 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.388+0000 7f6a7be00700 1 -- 172.21.15.179:0/2384404985 learned_addr learned my addr 172.21.15.179:0/2384404985 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.388+0000 7f6a80e00700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a7c102590 0x7f6a7c1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.388+0000 7f6a81800700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c197b60 0x7f6a7c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a7be00700 1 -- 172.21.15.179:0/2384404985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a7c102590 msgr2=0x7f6a7c1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a7be00700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a7c102590 0x7f6a7c1971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a7be00700 1 -- 172.21.15.179:0/2384404985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c197b60 msgr2=0x7f6a7c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a7be00700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c197b60 0x7f6a7c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a7be00700 1 -- 172.21.15.179:0/2384404985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c19f5b0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a7be00700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 0x7f6a7c19c750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b780 tx=0x7f6a6800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68004230 con 0x7f6a7c1976e0 2024-03-26T13:51:19.383 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c19f8a0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.389+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6a680043d0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.390+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68011b60 con 0x7f6a7c1976e0 2024-03-26T13:51:19.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.390+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c19feb0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.384 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.391+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c0637c0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.391+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f6a680036a0 con 0x7f6a7c1976e0 2024-03-26T13:51:19.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.392+0000 7f6a79600700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a64039450 0x7f6a6403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:19.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.392+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a6804bc20 con 0x7f6a7c1976e0 2024-03-26T13:51:19.385 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.392+0000 7f6a80e00700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a64039450 0x7f6a6403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:19.386 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.393+0000 7f6a80e00700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a64039450 0x7f6a6403b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a70005190 tx=0x7f6a7001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:19.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.408+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6a68020710 con 0x7f6a7c1976e0 2024-03-26T13:51:19.644 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.651+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi159=y;smithi179=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a7c0625d0 con 0x7f6a64039450 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: Reconfiguring mon.b (monmap changed)... 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: Reconfiguring daemon mon.b on smithi159 2024-03-26T13:51:19.653 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:19 smithi159 ceph-mon[35501]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:19.658 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.665+0000 7f6a79600700 1 -- 172.21.15.179:0/2384404985 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6a7c0625d0 con 0x7f6a64039450 2024-03-26T13:51:19.658 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.667+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a64039450 msgr2=0x7f6a6403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.667+0000 7f6a82200700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a64039450 0x7f6a6403b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6a70005190 tx=0x7f6a7001a040 comp rx=0 tx=0).stop 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.667+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 msgr2=0x7f6a7c19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.667+0000 7f6a82200700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 0x7f6a7c19c750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800b780 tx=0x7f6a6800ba90 comp rx=0 tx=0).stop 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 shutdown_connections 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a64039450 0x7f6a6403b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c197b60 0x7f6a7c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a7c102590 0x7f6a7c1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.661 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 --2- 172.21.15.179:0/2384404985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6a7c1976e0 0x7f6a7c19c750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 >> 172.21.15.179:0/2384404985 conn(0x7f6a7c0fbfc0 msgr2=0x7f6a7c0fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 shutdown_connections 2024-03-26T13:51:19.662 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:51:19.668+0000 7f6a82200700 1 -- 172.21.15.179:0/2384404985 wait complete. 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: Reconfiguring mon.b (monmap changed)... 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: Reconfiguring daemon mon.b on smithi159 2024-03-26T13:51:19.738 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:19 smithi179 ceph-mon[35472]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: Reconfiguring mon.b (monmap changed)... 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: Reconfiguring daemon mon.b on smithi159 2024-03-26T13:51:19.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:19 smithi137 ceph-mon[31539]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:20.344 DEBUG:teuthology.orchestra.run.smithi159:mgr.y> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.y.service 2024-03-26T13:51:20.352 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.z.service 2024-03-26T13:51:20.358 INFO:tasks.cephadm:Deploying OSDs... 2024-03-26T13:51:20.358 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:51:20.359 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T13:51:20.415 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T13:51:20.415 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-03-26T13:51:20.463 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T13:51:20.463 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:20.463 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-26T13:51:20.464 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:20.464 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:20.464 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-26 13:50:08.599735230 +0000 2024-03-26T13:51:20.464 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-26 13:44:15.594402567 +0000 2024-03-26T13:51:20.464 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-26 13:44:15.594402567 +0000 2024-03-26T13:51:20.464 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-26 13:44:15.594402567 +0000 2024-03-26T13:51:20.464 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T13:51:20.555 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-26T13:51:20.555 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-26T13:51:20.555 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000329005 s, 1.6 MB/s 2024-03-26T13:51:20.559 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T13:51:20.612 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-26 13:50:08.664734064 +0000 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-26 13:44:16.175392268 +0000 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-26 13:44:16.175392268 +0000 2024-03-26T13:51:20.659 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-26 13:44:16.175392268 +0000 2024-03-26T13:51:20.659 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T13:51:20.737 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-26T13:51:20.737 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-26T13:51:20.737 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000368691 s, 1.4 MB/s 2024-03-26T13:51:20.741 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T13:51:20.794 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-03-26T13:51:20.839 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T13:51:20.839 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:20.839 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-26T13:51:20.840 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:20.840 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:20.840 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-26 13:50:08.732732845 +0000 2024-03-26T13:51:20.840 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-26 13:44:16.741382235 +0000 2024-03-26T13:51:20.840 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-26 13:44:16.741382235 +0000 2024-03-26T13:51:20.840 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-26 13:44:16.741382235 +0000 2024-03-26T13:51:20.840 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi159=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: Saving service mgr spec with placement smithi137=x;smithi159=y;smithi179=z;count:3 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:20.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi159=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: Saving service mgr spec with placement smithi137=x;smithi159=y;smithi179=z;count:3 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:20.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi137=x;smithi159=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: Saving service mgr spec with placement smithi137=x;smithi159=y;smithi179=z;count:3 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-26T13:51:20.964 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-26T13:51:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-26T13:51:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000386837 s, 1.3 MB/s 2024-03-26T13:51:20.978 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T13:51:21.032 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-03-26T13:51:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T13:51:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-26 13:50:08.802731589 +0000 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-26 13:44:17.309372166 +0000 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-26 13:44:17.309372166 +0000 2024-03-26T13:51:21.081 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-03-26 13:44:17.309372166 +0000 2024-03-26T13:51:21.081 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T13:51:21.171 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-03-26T13:51:21.171 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-03-26T13:51:21.171 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000357282 s, 1.4 MB/s 2024-03-26T13:51:21.175 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T13:51:21.225 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-03-26T13:51:21.225 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T13:51:21.283 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T13:51:21.283 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-26 13:51:06.102495618 +0000 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-26 13:44:15.848479444 +0000 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-26 13:44:15.848479444 +0000 2024-03-26T13:51:21.335 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-26 13:44:15.848479444 +0000 2024-03-26T13:51:21.336 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T13:51:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-26T13:51:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-26T13:51:21.427 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000405052 s, 1.3 MB/s 2024-03-26T13:51:21.431 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T13:51:21.485 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-26 13:51:06.103495598 +0000 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-26 13:44:16.423468422 +0000 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-26 13:44:16.423468422 +0000 2024-03-26T13:51:21.533 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-26 13:44:16.423468422 +0000 2024-03-26T13:51:21.533 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T13:51:21.609 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-26T13:51:21.609 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-26T13:51:21.609 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000347761 s, 1.5 MB/s 2024-03-26T13:51:21.612 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T13:51:21.662 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-03-26T13:51:21.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:21 smithi179 ceph-mon[35472]: Reconfiguring mon.c (monmap changed)... 2024-03-26T13:51:21.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:21 smithi179 ceph-mon[35472]: Reconfiguring daemon mon.c on smithi179 2024-03-26T13:51:21.685 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:21 smithi179 ceph-mon[35472]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T13:51:21.708 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-26 13:51:06.104495579 +0000 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-26 13:44:16.992457514 +0000 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-26 13:44:16.992457514 +0000 2024-03-26T13:51:21.709 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-26 13:44:16.992457514 +0000 2024-03-26T13:51:21.709 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T13:51:21.787 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-26T13:51:21.787 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-26T13:51:21.787 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000323793 s, 1.6 MB/s 2024-03-26T13:51:21.790 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T13:51:21.841 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-03-26 13:51:06.105495560 +0000 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-03-26 13:44:17.557446683 +0000 2024-03-26T13:51:21.890 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-03-26 13:44:17.557446683 +0000 2024-03-26T13:51:21.891 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-03-26 13:44:17.557446683 +0000 2024-03-26T13:51:21.891 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T13:51:21.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:21 smithi159 ceph-mon[35501]: Reconfiguring mon.c (monmap changed)... 2024-03-26T13:51:21.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:21 smithi159 ceph-mon[35501]: Reconfiguring daemon mon.c on smithi179 2024-03-26T13:51:21.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:21 smithi159 ceph-mon[35501]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:21.979 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-03-26T13:51:21.979 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-03-26T13:51:21.979 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000341637 s, 1.5 MB/s 2024-03-26T13:51:21.983 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T13:51:22.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:21 smithi137 ceph-mon[31539]: Reconfiguring mon.c (monmap changed)... 2024-03-26T13:51:22.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:21 smithi137 ceph-mon[31539]: Reconfiguring daemon mon.c on smithi179 2024-03-26T13:51:22.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:21 smithi137 ceph-mon[31539]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:22.039 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-03-26T13:51:22.039 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-03-26T13:51:22.096 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-26T13:51:22.096 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-03-26 13:50:54.411498618 +0000 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-03-26 13:44:18.043395111 +0000 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-03-26 13:44:18.043395111 +0000 2024-03-26T13:51:22.140 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-03-26 13:44:18.043395111 +0000 2024-03-26T13:51:22.141 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-26T13:51:22.265 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-03-26T13:51:22.265 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-03-26T13:51:22.265 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000350024 s, 1.5 MB/s 2024-03-26T13:51:22.268 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-26T13:51:22.315 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-03-26T13:51:22.360 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-26T13:51:22.360 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:22.360 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-26T13:51:22.360 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:22.361 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:22.361 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-03-26 13:50:54.411498618 +0000 2024-03-26T13:51:22.361 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-03-26 13:44:18.611382564 +0000 2024-03-26T13:51:22.361 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-03-26 13:44:18.611382564 +0000 2024-03-26T13:51:22.361 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-03-26 13:44:18.611382564 +0000 2024-03-26T13:51:22.361 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-26T13:51:22.488 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-03-26T13:51:22.488 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-03-26T13:51:22.488 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.0003713 s, 1.4 MB/s 2024-03-26T13:51:22.492 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-26T13:51:22.549 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-03-26T13:51:22.592 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-26T13:51:22.592 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:22.592 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-26T13:51:22.593 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:22.593 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:22.593 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-03-26 13:50:54.412498596 +0000 2024-03-26T13:51:22.593 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-03-26 13:44:19.154370570 +0000 2024-03-26T13:51:22.593 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-03-26 13:44:19.154370570 +0000 2024-03-26T13:51:22.593 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-03-26 13:44:19.154370570 +0000 2024-03-26T13:51:22.593 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-26T13:51:22.719 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-03-26T13:51:22.720 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-03-26T13:51:22.720 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000292269 s, 1.8 MB/s 2024-03-26T13:51:22.724 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-26T13:51:22.769 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-03-26 13:50:54.413498574 +0000 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-03-26 13:44:19.723358002 +0000 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-03-26 13:44:19.723358002 +0000 2024-03-26T13:51:22.853 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-03-26 13:44:19.723358002 +0000 2024-03-26T13:51:22.853 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-26T13:51:22.964 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-03-26T13:51:22.964 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-03-26T13:51:22.964 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000325689 s, 1.6 MB/s 2024-03-26T13:51:22.969 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-26T13:51:23.016 INFO:tasks.cephadm:Deploying osd.0 on smithi137 with /dev/vg_nvme/lv_4... 2024-03-26T13:51:23.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T13:51:23.448 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.908 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:23.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T13:51:23.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:23.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:23.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: Deploying daemon mgr.z on smithi179 2024-03-26T13:51:23.909 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:23 smithi179 ceph-mon[35472]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:23.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: Deploying daemon mgr.z on smithi179 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:23 smithi159 ceph-mon[35501]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:23.946 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T13:51:23.947 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:23.947 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:23.947 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: Deploying daemon mgr.z on smithi179 2024-03-26T13:51:23.947 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:23 smithi137 ceph-mon[31539]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:25.667 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:25 smithi179 systemd[1]: Starting Ceph mgr.z for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:51:25.983 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:25 smithi179 podman[37189]: 2024-03-26 13:51:25.616318767 +0000 UTC m=+0.034258400 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:51:25.984 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:25 smithi179 podman[37189]: 2024-03-26 13:51:25.711923967 +0000 UTC m=+0.129863639 container create c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T13:51:25.984 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:25 smithi179 podman[37189]: 2024-03-26 13:51:25.932343029 +0000 UTC m=+0.350282698 container init c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T13:51:25.984 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:25 smithi179 podman[37189]: 2024-03-26 13:51:25.941717061 +0000 UTC m=+0.359656711 container start c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T13:51:25.984 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:25 smithi179 ceph-mgr[37208]: -- 172.21.15.179:0/1099841604 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5622061065a0 con 0x5622060db000 2024-03-26T13:51:26.253 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:25 smithi137 ceph-mon[31539]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:26.253 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T13:51:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T13:51:26.254 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-26T13:51:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-26T13:51:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036271 s, 289 MB/s 2024-03-26T13:51:26.254 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-26T13:51:26.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:25 smithi179 ceph-mon[35472]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:26.416 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:26 smithi179 bash[37189]: c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 2024-03-26T13:51:26.416 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:26 smithi179 systemd[1]: Started Ceph mgr.z for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:51:26.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:25 smithi159 ceph-mon[35501]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:27.073 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-03-26T13:51:27.166 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:26.688+0000 7f51f5f13200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T13:51:27.166 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:26.805+0000 7f51f5f13200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: Deploying daemon mgr.y on smithi159 2024-03-26T13:51:27.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:27 smithi137 ceph-mon[31539]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:27.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:27.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T13:51:27.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:27.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:27.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: Deploying daemon mgr.y on smithi159 2024-03-26T13:51:27.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:27 smithi159 ceph-mon[35501]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:27.508 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: Deploying daemon mgr.y on smithi159 2024-03-26T13:51:27.583 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-mon[35472]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:27.880 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:27.590+0000 7f51f5f13200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T13:51:27.880 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:27.711+0000 7f51f5f13200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T13:51:27.880 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:27.887+0000 7f51f5f13200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T13:51:28.166 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:28.009+0000 7f51f5f13200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T13:51:28.166 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:28.117+0000 7f51f5f13200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T13:51:28.694 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:28.448+0000 7f51f5f13200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T13:51:28.695 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:28.589+0000 7f51f5f13200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T13:51:28.695 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:28.701+0000 7f51f5f13200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T13:51:29.034 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:28.824+0000 7f51f5f13200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T13:51:29.034 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:29.041+0000 7f51f5f13200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T13:51:29.445 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 systemd[1]: Starting Ceph mgr.y for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.461+0000 7f0f18e00700 1 -- 172.21.15.137:0/1363453008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14104410 msgr2=0x7f0f14104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.461+0000 7f0f18e00700 1 --2- 172.21.15.137:0/1363453008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14104410 0x7f0f14104830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f040099c0 tx=0x7f0f04009cd0 comp rx=0 tx=0).stop 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 -- 172.21.15.137:0/1363453008 shutdown_connections 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 --2- 172.21.15.137:0/1363453008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f14105fb0 0x7f0f1410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 --2- 172.21.15.137:0/1363453008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14104410 0x7f0f14104830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 --2- 172.21.15.137:0/1363453008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f14105610 0x7f0f14105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 -- 172.21.15.137:0/1363453008 >> 172.21.15.137:0/1363453008 conn(0x7f0f140ff9b0 msgr2=0x7f0f14101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 -- 172.21.15.137:0/1363453008 shutdown_connections 2024-03-26T13:51:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.462+0000 7f0f18e00700 1 -- 172.21.15.137:0/1363453008 wait complete. 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.463+0000 7f0f18e00700 1 Processor -- start 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.463+0000 7f0f18e00700 1 -- start start 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f18e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f14104410 0x7f0f1419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f18e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f14105610 0x7f0f1419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f18e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 0x7f0f141a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f18e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1410fee0 con 0x7f0f14105fb0 2024-03-26T13:51:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f18e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1410fd60 con 0x7f0f14104410 2024-03-26T13:51:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f18e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14110060 con 0x7f0f14105610 2024-03-26T13:51:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f12800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 0x7f0f141a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f12800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 0x7f0f141a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58020/0 (socket says 172.21.15.137:58020) 2024-03-26T13:51:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f12800700 1 -- 172.21.15.137:0/747190717 learned_addr learned my addr 172.21.15.137:0/747190717 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:51:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.464+0000 7f0f0be00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f14105610 0x7f0f1419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f12800700 1 -- 172.21.15.137:0/747190717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f14105610 msgr2=0x7f0f1419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f12800700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f14105610 0x7f0f1419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f12800700 1 -- 172.21.15.137:0/747190717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f14104410 msgr2=0x7f0f1419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f12800700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f14104410 0x7f0f1419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f12800700 1 -- 172.21.15.137:0/747190717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f141a21b0 con 0x7f0f14105fb0 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f12800700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 0x7f0f141a1b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c007f10 tx=0x7f0f0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c004bb0 con 0x7f0f14105fb0 2024-03-26T13:51:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f0c004d50 con 0x7f0f14105fb0 2024-03-26T13:51:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f141a24a0 con 0x7f0f14105fb0 2024-03-26T13:51:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c005990 con 0x7f0f14105fb0 2024-03-26T13:51:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.465+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f141a29e0 con 0x7f0f14105fb0 2024-03-26T13:51:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.466+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f14068e50 con 0x7f0f14105fb0 2024-03-26T13:51:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.466+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f0f0c01a020 con 0x7f0f14105fb0 2024-03-26T13:51:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.467+0000 7f0f0aa00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ef8039200 0x7f0ef803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:51:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.467+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0f0c04b9b0 con 0x7f0f14105fb0 2024-03-26T13:51:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.467+0000 7f0f11e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ef8039200 0x7f0ef803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:51:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.468+0000 7f0f11e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ef8039200 0x7f0ef803b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f040036f0 tx=0x7f0f04000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:51:29.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.473+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f0c010070 con 0x7f0f14105fb0 2024-03-26T13:51:29.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:29.702+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f140625d0 con 0x7f0ef8039200 2024-03-26T13:51:29.833 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 podman[36866]: 2024-03-26 13:51:29.550345515 +0000 UTC m=+0.126837678 container create 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-26T13:51:29.834 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 podman[36866]: 2024-03-26 13:51:29.459421417 +0000 UTC m=+0.035913556 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:51:29.834 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 podman[36866]: 2024-03-26 13:51:29.782076872 +0000 UTC m=+0.358569081 container init 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-26T13:51:29.834 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 podman[36866]: 2024-03-26 13:51:29.792761507 +0000 UTC m=+0.369253643 container start 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-26T13:51:30.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:29 smithi159 ceph-mon[35501]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:30.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:51:30.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:51:30.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:30.195 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 ceph-mgr[36886]: -- 172.21.15.159:0/3415636721 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5589737a05a0 con 0x558973774c00 2024-03-26T13:51:30.196 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 bash[36866]: 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 2024-03-26T13:51:30.196 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:29 smithi159 systemd[1]: Started Ceph mgr.y for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:51:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:29 smithi137 ceph-mon[31539]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:51:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:51:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:30.338 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:29 smithi179 ceph-mon[35472]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:30.338 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:51:30.338 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:51:30.338 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:30.666 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:30.346+0000 7f51f5f13200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T13:51:30.666 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:30.524+0000 7f51f5f13200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T13:51:30.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:30.537+0000 7f1e30f9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-26T13:51:30.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:30.662+0000 7f1e30f9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-26T13:51:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:31 smithi137 ceph-mon[31539]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:31.367 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:30.961+0000 7f51f5f13200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T13:51:31.367 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:31.096+0000 7f51f5f13200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T13:51:31.367 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-mon[35472]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:31.368 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.368 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.368 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.368 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.368 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:31.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-mon[35501]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:51:31.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:31.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:31.665 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:31.374+0000 7f51f5f13200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:51:31.665 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:31.619+0000 7f51f5f13200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T13:51:31.780 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:31.506+0000 7f1e30f9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-26T13:51:31.780 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:31.631+0000 7f1e30f9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-26T13:51:32.041 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:31.819+0000 7f1e30f9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-26T13:51:32.041 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:31.940+0000 7f1e30f9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-26T13:51:32.042 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:32.048+0000 7f1e30f9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-26T13:51:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:32 smithi137 ceph-mon[31539]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:32.344 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:32 smithi179 ceph-mon[35472]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:32.345 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:32.003+0000 7f51f5f13200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:51:32.403 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:32 smithi159 ceph-mon[35501]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:32.673 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:32.410+0000 7f1e30f9d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-26T13:51:32.673 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:32.560+0000 7f1e30f9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-26T13:51:32.916 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:32.352+0000 7f51f5f13200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T13:51:32.916 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:32.653+0000 7f51f5f13200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T13:51:32.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:32.680+0000 7f1e30f9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-26T13:51:32.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:32.812+0000 7f1e30f9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-mon[35501]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.445 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:33.047+0000 7f1e30f9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-26T13:51:33.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:33 smithi137 ceph-mon[31539]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:33.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.665 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.665 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.665 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:33.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-mon[35472]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:33.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:34.166 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:33.677+0000 7f51f5f13200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T13:51:34.508 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:34.382+0000 7f51f5f13200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T13:51:34.508 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:34.515+0000 7f51f5f13200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T13:51:34.508 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:34.508 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2857055638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6b26e4-ccfd-4457-91b0-240679242c9e"}]: dispatch 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2857055638' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f6b26e4-ccfd-4457-91b0-240679242c9e"}]': finished 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: Reconfiguring daemon mgr.x on smithi137 2024-03-26T13:51:34.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2840232242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:51:34.565 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:34.404+0000 7f1e30f9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2857055638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6b26e4-ccfd-4457-91b0-240679242c9e"}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2857055638' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f6b26e4-ccfd-4457-91b0-240679242c9e"}]': finished 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: Reconfiguring daemon mgr.x on smithi137 2024-03-26T13:51:34.566 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2840232242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:51:34.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:34.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:34.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:34.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2857055638' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f6b26e4-ccfd-4457-91b0-240679242c9e"}]: dispatch 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2857055638' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f6b26e4-ccfd-4457-91b0-240679242c9e"}]': finished 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: osdmap e5: 1 total, 0 up, 1 in 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: Reconfiguring mgr.x (unknown last config time)... 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: Reconfiguring daemon mgr.x on smithi137 2024-03-26T13:51:34.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2840232242' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:51:34.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:34.572+0000 7f1e30f9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-26T13:51:35.166 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 13:51:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z[37204]: 2024-03-26T13:51:34.806+0000 7f51f5f13200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T13:51:35.296 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:34.982+0000 7f1e30f9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-26T13:51:35.296 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:35.099+0000 7f1e30f9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-26T13:51:35.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:35.312+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 14) v1 ==== 91699+0+0 (secure 0 0 0) 0x7f0f0c00fd40 con 0x7f0f14105fb0 2024-03-26T13:51:35.585 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:35.352+0000 7f1e30f9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:51:35.586 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T13:51:35.586 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-mon[35501]: Standby manager daemon z started 2024-03-26T13:51:35.586 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T13:51:35.586 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T13:51:35.586 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T13:51:35.586 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-mon[35501]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:35.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:35 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T13:51:35.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:35 smithi179 ceph-mon[35472]: Standby manager daemon z started 2024-03-26T13:51:35.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:35 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T13:51:35.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:35 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T13:51:35.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:35 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T13:51:35.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:35 smithi179 ceph-mon[35472]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:35 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-26T13:51:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:35 smithi137 ceph-mon[31539]: Standby manager daemon z started 2024-03-26T13:51:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:35 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T13:51:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:35 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-26T13:51:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:35 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.179:0/3765557280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T13:51:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:35 smithi137 ceph-mon[31539]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:35.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:35.592+0000 7f1e30f9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-26T13:51:36.271 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:35.961+0000 7f1e30f9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-26T13:51:36.572 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:36.278+0000 7f1e30f9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-26T13:51:36.573 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:36 smithi159 ceph-mon[35501]: mgrmap e14: x(active, since 102s), standbys: z 2024-03-26T13:51:36.573 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T13:51:36.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:36 smithi179 ceph-mon[35472]: mgrmap e14: x(active, since 102s), standbys: z 2024-03-26T13:51:36.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T13:51:36.776 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:36 smithi137 ceph-mon[31539]: mgrmap e14: x(active, since 102s), standbys: z 2024-03-26T13:51:36.776 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-26T13:51:36.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:36.579+0000 7f1e30f9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-26T13:51:37.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:37 smithi159 ceph-mon[35501]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:37.603 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:37 smithi137 ceph-mon[31539]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:37.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:37 smithi179 ceph-mon[35472]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:37.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:37.588+0000 7f1e30f9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-26T13:51:38.673 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:38.275+0000 7f1e30f9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-26T13:51:38.673 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:38.401+0000 7f1e30f9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-26T13:51:38.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:38.758+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mon.0 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f0f0c01c410 con 0x7f0f14105fb0 2024-03-26T13:51:38.944 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y[36882]: 2024-03-26T13:51:38.680+0000 7f1e30f9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-26T13:51:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-mon[35501]: Standby manager daemon y started 2024-03-26T13:51:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T13:51:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T13:51:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T13:51:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:38 smithi159 ceph-mon[35501]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T13:51:38.969 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:38 smithi137 ceph-mon[31539]: Standby manager daemon y started 2024-03-26T13:51:38.969 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:38 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T13:51:38.969 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:38 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T13:51:38.969 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:38 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T13:51:38.969 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:38 smithi137 ceph-mon[31539]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T13:51:39.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:38 smithi179 ceph-mon[35472]: Standby manager daemon y started 2024-03-26T13:51:39.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:38 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-26T13:51:39.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:38 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-26T13:51:39.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:38 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-26T13:51:39.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:38 smithi179 ceph-mon[35472]: from='mgr.? 172.21.15.159:0/3829295267' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-26T13:51:39.880 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:39 smithi137 ceph-mon[31539]: mgrmap e15: x(active, since 105s), standbys: y, z 2024-03-26T13:51:39.881 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T13:51:39.881 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:39 smithi137 ceph-mon[31539]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:40.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:39 smithi179 ceph-mon[35472]: mgrmap e15: x(active, since 105s), standbys: y, z 2024-03-26T13:51:40.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T13:51:40.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:39 smithi179 ceph-mon[35472]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:40.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:39 smithi159 ceph-mon[35501]: mgrmap e15: x(active, since 105s), standbys: y, z 2024-03-26T13:51:40.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-26T13:51:40.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:39 smithi159 ceph-mon[35501]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:40.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:40.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:40.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:40.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:40.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:40.870 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:41.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:41.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:41.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:41.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:41.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:41.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:41.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:41.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:41.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:41.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:41.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:41.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:42.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:41 smithi179 ceph-mon[35472]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:42.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:41 smithi159 ceph-mon[35501]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:42.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:41 smithi137 ceph-mon[31539]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:43.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T13:51:43.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:43.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T13:51:43.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:43.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-26T13:51:43.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:43.979 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:43 smithi137 ceph-mon[31539]: Deploying daemon osd.0 on smithi137 2024-03-26T13:51:43.979 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:43 smithi137 ceph-mon[31539]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:44.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:43 smithi179 ceph-mon[35472]: Deploying daemon osd.0 on smithi137 2024-03-26T13:51:44.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:43 smithi179 ceph-mon[35472]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:44.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:43 smithi159 ceph-mon[35501]: Deploying daemon osd.0 on smithi137 2024-03-26T13:51:44.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:43 smithi159 ceph-mon[35501]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:46.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:46 smithi137 ceph-mon[31539]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:46.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:46 smithi179 ceph-mon[35472]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:46.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:46 smithi159 ceph-mon[35501]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:48.118 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:48 smithi137 ceph-mon[31539]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:48.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:48 smithi179 ceph-mon[35472]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:48.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:48 smithi159 ceph-mon[35501]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:50.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:50 smithi137 ceph-mon[31539]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:50.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:50 smithi179 ceph-mon[35472]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:50.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:50 smithi159 ceph-mon[35501]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:51.276 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:51 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:51.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:51 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:51.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:51 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:51.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:51 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:51.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:51 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:51.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:51 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:51.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:51 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:51.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:51 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:51.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:51 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:52.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:52 smithi137 ceph-mon[31539]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:52.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:52 smithi179 ceph-mon[35472]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:52.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:52 smithi159 ceph-mon[35501]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:54.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:54 smithi137 ceph-mon[31539]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:54.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:54 smithi179 ceph-mon[35472]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:54.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:54 smithi159 ceph-mon[35501]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:55.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:55.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:55.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:55 smithi137 ceph-mon[31539]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T13:51:55.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:55.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:55.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:55 smithi179 ceph-mon[35472]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T13:51:55.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.450+0000 7f0f0aa00700 1 -- 172.21.15.137:0/747190717 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0f140625d0 con 0x7f0ef8039200 2024-03-26T13:51:55.443 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 0 on host 'smithi137' 2024-03-26T13:51:55.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:51:55.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:51:55.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:55.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:55 smithi159 ceph-mon[35501]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-26T13:51:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ef8039200 msgr2=0x7f0ef803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ef8039200 0x7f0ef803b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f040036f0 tx=0x7f0f04000bc0 comp rx=0 tx=0).stop 2024-03-26T13:51:55.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 msgr2=0x7f0f141a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 0x7f0f141a1b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c007f10 tx=0x7f0f0c00d3b0 comp rx=0 tx=0).stop 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 shutdown_connections 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ef8039200 0x7f0ef803b6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0f14105fb0 0x7f0f141a1b10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f14104410 0x7f0f1419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 --2- 172.21.15.137:0/747190717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0f14105610 0x7f0f1419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 >> 172.21.15.137:0/747190717 conn(0x7f0f140ff9b0 msgr2=0x7f0f14108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 shutdown_connections 2024-03-26T13:51:55.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:51:55.454+0000 7f0f18e00700 1 -- 172.21.15.137:0/747190717 wait complete. 2024-03-26T13:51:56.210 DEBUG:teuthology.orchestra.run.smithi137:osd.0> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.0.service 2024-03-26T13:51:56.215 INFO:tasks.cephadm:Deploying osd.1 on smithi159 with /dev/vg_nvme/lv_4... 2024-03-26T13:51:56.215 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T13:51:56.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:56.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T13:51:56.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T13:51:56.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:51:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:56.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:56.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: osdmap e6: 1 total, 0 up, 1 in 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:51:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:51:56.657 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:51:57.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:57 smithi179 ceph-mon[35472]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:51:57.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:57 smithi179 ceph-mon[35472]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T13:51:57.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:57.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:57 smithi159 ceph-mon[35501]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:51:57.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:57 smithi159 ceph-mon[35501]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T13:51:57.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:57.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:57.512 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:57 smithi137 ceph-mon[31539]: from='osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:51:57.513 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:57 smithi137 ceph-mon[31539]: osdmap e7: 1 total, 0 up, 1 in 2024-03-26T13:51:57.513 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:57.513 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:58.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:58 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:51:58.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:58 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:51:58.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:58 smithi137 ceph-mon[31539]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:58.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:58 smithi137 ceph-mon[31539]: osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] boot 2024-03-26T13:51:58.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:58 smithi137 ceph-mon[31539]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T13:51:58.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:58 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:58 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:51:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:58 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:51:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:58 smithi179 ceph-mon[35472]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:58 smithi179 ceph-mon[35472]: osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] boot 2024-03-26T13:51:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:58 smithi179 ceph-mon[35472]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T13:51:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:58 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:58.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:58 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:51:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:58 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:51:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:58 smithi159 ceph-mon[35501]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-26T13:51:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:58 smithi159 ceph-mon[35501]: osd.0 [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] boot 2024-03-26T13:51:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:58 smithi159 ceph-mon[35501]: osdmap e8: 1 total, 1 up, 1 in 2024-03-26T13:51:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:58 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-26T13:51:59.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:59 smithi179 ceph-mon[35472]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T13:51:59.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:51:59 smithi179 ceph-mon[35472]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:51:59.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:59 smithi159 ceph-mon[35501]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T13:51:59.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:51:59 smithi159 ceph-mon[35501]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:51:59.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:59 smithi137 ceph-mon[31539]: osdmap e9: 1 total, 1 up, 1 in 2024-03-26T13:51:59.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:51:59 smithi137 ceph-mon[31539]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:51:59.819 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T13:51:59.819 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T13:51:59.819 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-26T13:51:59.819 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-26T13:51:59.819 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363011 s, 289 MB/s 2024-03-26T13:51:59.819 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-26T13:52:00.608 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-03-26T13:52:01.023 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:52:02.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi137 2024-03-26T13:52:02.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:52:02.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi137 to 17091M 2024-03-26T13:52:02.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:02.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:02.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:01 smithi137 ceph-mon[31539]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi137 2024-03-26T13:52:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:52:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi137 to 17091M 2024-03-26T13:52:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:01 smithi179 ceph-mon[35472]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:02.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi137 2024-03-26T13:52:02.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:52:02.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi137 to 17091M 2024-03-26T13:52:02.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:02.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:02.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:02.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:01 smithi159 ceph-mon[35501]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 -- 172.21.15.159:0/3367595941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68104410 msgr2=0x7f5a68104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/3367595941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68104410 0x7f5a68104830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a580099c0 tx=0x7f5a58009cd0 comp rx=0 tx=0).stop 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 -- 172.21.15.159:0/3367595941 shutdown_connections 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/3367595941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 0x7f5a6810c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/3367595941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68104410 0x7f5a68104830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/3367595941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a68105610 0x7f5a68105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 -- 172.21.15.159:0/3367595941 >> 172.21.15.159:0/3367595941 conn(0x7f5a680ff9b0 msgr2=0x7f5a68101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:52:03.083 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 -- 172.21.15.159:0/3367595941 shutdown_connections 2024-03-26T13:52:03.084 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.091+0000 7f5a6ce00700 1 -- 172.21.15.159:0/3367595941 wait complete. 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.092+0000 7f5a6ce00700 1 Processor -- start 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.092+0000 7f5a6ce00700 1 -- start start 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a6ce00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a68104410 0x7f5a6819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a6ce00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 0x7f5a6819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a6ce00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 0x7f5a681a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a6ce00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6810fea0 con 0x7f5a68105fb0 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a6ce00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6810fd20 con 0x7f5a68105610 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a6ce00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a68110020 con 0x7f5a68104410 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 0x7f5a6819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 0x7f5a6819aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:54732/0 (socket says 172.21.15.159:54732) 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65400700 1 -- 172.21.15.159:0/1525427567 learned_addr learned my addr 172.21.15.159:0/1525427567 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65e00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a68104410 0x7f5a6819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:03.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a66800700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 0x7f5a681a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:03.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65400700 1 -- 172.21.15.159:0/1525427567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a68104410 msgr2=0x7f5a6819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:03.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65400700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a68104410 0x7f5a6819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:03.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.093+0000 7f5a65400700 1 -- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 msgr2=0x7f5a681a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:03.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a65400700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 0x7f5a681a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:03.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a65400700 1 -- 172.21.15.159:0/1525427567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a681a21a0 con 0x7f5a68105610 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a66800700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 0x7f5a681a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a65400700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 0x7f5a6819aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400d980 tx=0x7f5a5400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54017070 con 0x7f5a68105610 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a681a2490 con 0x7f5a68105610 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a54005020 con 0x7f5a68105610 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54009b50 con 0x7f5a68105610 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.094+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a681a29d0 con 0x7f5a68105610 2024-03-26T13:52:03.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.095+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a68068e50 con 0x7f5a68105610 2024-03-26T13:52:03.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.098+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f5a54009cf0 con 0x7f5a68105610 2024-03-26T13:52:03.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.099+0000 7f5a5ea00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5a4c0a2800 0x7f5a4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:03.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.099+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f5a540cd1b0 con 0x7f5a68105610 2024-03-26T13:52:03.091 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.100+0000 7f5a65e00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5a4c0a2800 0x7f5a4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:03.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.101+0000 7f5a65e00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5a4c0a2800 0x7f5a4c0a4cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a58000c00 tx=0x7f5a58005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:52:03.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.103+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5a54020030 con 0x7f5a68105610 2024-03-26T13:52:03.325 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:03.333+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a68062690 con 0x7f5a4c0a2800 2024-03-26T13:52:04.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:04 smithi137 ceph-mon[31539]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:04.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:52:04.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:52:04.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:04.284 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:04 smithi159 ceph-mon[35501]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:04.284 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:52:04.284 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:52:04.284 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:04 smithi179 ceph-mon[35472]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:52:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:52:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:05.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:05 smithi137 ceph-mon[31539]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:52:05.303 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:05 smithi159 ceph-mon[35501]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:52:05.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:05 smithi179 ceph-mon[35472]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:52:06.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:06 smithi159 ceph-mon[35501]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:06.276 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:06 smithi137 ceph-mon[31539]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:06.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:06 smithi179 ceph-mon[35472]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:08.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:08.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/3722437319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]: dispatch 2024-03-26T13:52:08.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]: dispatch 2024-03-26T13:52:08.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]': finished 2024-03-26T13:52:08.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T13:52:08.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:08.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/1544079352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/3722437319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]: dispatch 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]: dispatch 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]': finished 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/1544079352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/3722437319' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]: dispatch 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]: dispatch 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40fd05e9-e41a-4c27-9f67-43693dae21d8"}]': finished 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: osdmap e10: 2 total, 1 up, 2 in 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:08.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/1544079352' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:52:10.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:10 smithi137 ceph-mon[31539]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:10.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:10 smithi179 ceph-mon[35472]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:10.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:10 smithi159 ceph-mon[35501]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:12.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:12 smithi179 ceph-mon[35472]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:12.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:12 smithi159 ceph-mon[35501]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:12.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:12 smithi137 ceph-mon[31539]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:14.373 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:14 smithi159 ceph-mon[35501]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:14.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:14 smithi179 ceph-mon[35472]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:14 smithi137 ceph-mon[31539]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:16.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:16 smithi179 ceph-mon[35472]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:16.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T13:52:16.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:16.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:16 smithi159 ceph-mon[35501]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:16.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T13:52:16.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:16 smithi137 ceph-mon[31539]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-26T13:52:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:17.314 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:17 smithi159 ceph-mon[35501]: Deploying daemon osd.1 on smithi159 2024-03-26T13:52:17.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:17 smithi179 ceph-mon[35472]: Deploying daemon osd.1 on smithi159 2024-03-26T13:52:17.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:17 smithi137 ceph-mon[31539]: Deploying daemon osd.1 on smithi159 2024-03-26T13:52:18.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:18 smithi159 ceph-mon[35501]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:18.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:18 smithi179 ceph-mon[35472]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:18.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:18 smithi137 ceph-mon[31539]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:20.329 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:20 smithi137 ceph-mon[31539]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:20.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:20 smithi179 ceph-mon[35472]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:20.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:20 smithi159 ceph-mon[35501]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:22.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:22 smithi137 ceph-mon[31539]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:22.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:22 smithi179 ceph-mon[35472]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:22.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:22 smithi159 ceph-mon[35501]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:24.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:24 smithi179 ceph-mon[35472]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:24.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:24 smithi159 ceph-mon[35501]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:24.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:24 smithi137 ceph-mon[31539]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:25.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:25.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:25.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:25.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:25 smithi159 ceph-mon[35501]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:25.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:25.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:25.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:25.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:25 smithi137 ceph-mon[31539]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:25.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:25.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:25.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:25.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:25 smithi179 ceph-mon[35472]: pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:28.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:28 smithi137 ceph-mon[31539]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:28.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:28.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:28.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:28 smithi179 ceph-mon[35472]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:28 smithi159 ceph-mon[35501]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.038 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.044+0000 7f5a5ea00700 1 -- 172.21.15.159:0/1525427567 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a68062690 con 0x7f5a4c0a2800 2024-03-26T13:52:29.038 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 1 on host 'smithi159' 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.047+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5a4c0a2800 msgr2=0x7f5a4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.047+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5a4c0a2800 0x7f5a4c0a4cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5a58000c00 tx=0x7f5a58005fb0 comp rx=0 tx=0).stop 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.047+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 msgr2=0x7f5a6819aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.047+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 0x7f5a6819aa80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400d980 tx=0x7f5a5400dc90 comp rx=0 tx=0).stop 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 shutdown_connections 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5a4c0a2800 0x7f5a4c0a4cc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5a68105fb0 0x7f5a681a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a68104410 0x7f5a6819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:29.041 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 --2- 172.21.15.159:0/1525427567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a68105610 0x7f5a6819aa80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:29.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 >> 172.21.15.159:0/1525427567 conn(0x7f5a680ff9b0 msgr2=0x7f5a681089b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:52:29.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 shutdown_connections 2024-03-26T13:52:29.042 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:52:29.049+0000 7f5a6ce00700 1 -- 172.21.15.159:0/1525427567 wait complete. 2024-03-26T13:52:29.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:29 smithi159 ceph-mon[35501]: from='osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T13:52:29.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:29 smithi159 ceph-mon[35501]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T13:52:29.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:29.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:29 smithi137 ceph-mon[31539]: from='osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T13:52:29.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:29 smithi137 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T13:52:29.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:29.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:29 smithi179 ceph-mon[35472]: from='osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T13:52:29.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:29 smithi179 ceph-mon[35472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-26T13:52:29.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:29.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:29.841 DEBUG:teuthology.orchestra.run.smithi159:osd.1> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.1.service 2024-03-26T13:52:29.847 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2024-03-26T13:52:29.847 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-26T13:52:30.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-mon[35501]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:30.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-mon[35501]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T13:52:30.046 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-mon[35501]: from='osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:52:30.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-mon[35501]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T13:52:30.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:30.047 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-mon[35501]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:52:30.047 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1[40085]: 2024-03-26T13:52:30.047+0000 7fcb96400700 -1 osd.1 0 waiting for initial osdmap 2024-03-26T13:52:30.047 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:52:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1[40085]: 2024-03-26T13:52:30.053+0000 7fcb91200700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:52:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:30 smithi137 ceph-mon[31539]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:30 smithi137 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T13:52:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:30 smithi137 ceph-mon[31539]: from='osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:52:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:30 smithi137 ceph-mon[31539]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T13:52:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:30.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:30 smithi137 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:52:30.282 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:52:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:30 smithi179 ceph-mon[35472]: pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:30 smithi179 ceph-mon[35472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-26T13:52:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:30 smithi179 ceph-mon[35472]: from='osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:52:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:30 smithi179 ceph-mon[35472]: osdmap e11: 2 total, 1 up, 2 in 2024-03-26T13:52:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:30 smithi179 ceph-mon[35472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:52:31.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:31 smithi159 ceph-mon[35501]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:52:31.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:31 smithi159 ceph-mon[35501]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T13:52:31.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:31.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:31.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:31 smithi137 ceph-mon[31539]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:52:31.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:31 smithi137 ceph-mon[31539]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T13:52:31.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:31.353 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:31.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:31 smithi179 ceph-mon[35472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:52:31.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:31 smithi179 ceph-mon[35472]: osdmap e12: 2 total, 1 up, 2 in 2024-03-26T13:52:31.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:31.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:32.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:32 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:52:32.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:32 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:52:32.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:32 smithi159 ceph-mon[35501]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:32.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:32 smithi159 ceph-mon[35501]: osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] boot 2024-03-26T13:52:32.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:32 smithi159 ceph-mon[35501]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T13:52:32.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:32.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:32 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:52:32.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:32 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:52:32.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:32 smithi137 ceph-mon[31539]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:32.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:32 smithi137 ceph-mon[31539]: osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] boot 2024-03-26T13:52:32.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:32 smithi137 ceph-mon[31539]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T13:52:32.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:32 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:52:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:32 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:52:32.355 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:32 smithi179 ceph-mon[35472]: pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-26T13:52:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:32 smithi179 ceph-mon[35472]: osd.1 [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] boot 2024-03-26T13:52:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:32 smithi179 ceph-mon[35472]: osdmap e13: 2 total, 2 up, 2 in 2024-03-26T13:52:32.356 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-26T13:52:33.165 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:33 smithi179 ceph-mon[35472]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T13:52:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-26T13:52:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-26T13:52:33.182 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-03-26T13:52:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-03-26T13:52:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0389322 s, 269 MB/s 2024-03-26T13:52:33.182 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-03-26T13:52:33.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:33 smithi159 ceph-mon[35501]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T13:52:33.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:33 smithi137 ceph-mon[31539]: osdmap e14: 2 total, 2 up, 2 in 2024-03-26T13:52:33.982 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-03-26T13:52:34.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:34 smithi159 ceph-mon[35501]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:34.406 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:52:34.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:34 smithi179 ceph-mon[35472]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:34.526 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:34 smithi137 ceph-mon[31539]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi159 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi159 to 17091M 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:35.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:35.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:35 smithi179 ceph-mon[35472]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi159 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi159 to 17091M 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:35 smithi159 ceph-mon[35501]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi159 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi159 to 17091M 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:35 smithi137 ceph-mon[31539]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 -- 172.21.15.179:0/3704132386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 msgr2=0x7f8c48076c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 --2- 172.21.15.179:0/3704132386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c48076c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009990 tx=0x7f8c34009ca0 comp rx=0 tx=0).stop 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 -- 172.21.15.179:0/3704132386 shutdown_connections 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 --2- 172.21.15.179:0/3704132386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48077170 0x7f8c48078fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 --2- 172.21.15.179:0/3704132386 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48078220 0x7f8c48078640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 --2- 172.21.15.179:0/3704132386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c48076c30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 -- 172.21.15.179:0/3704132386 >> 172.21.15.179:0/3704132386 conn(0x7f8c480ff9b0 msgr2=0x7f8c48101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 -- 172.21.15.179:0/3704132386 shutdown_connections 2024-03-26T13:52:36.315 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.322+0000 7f8c4f400700 1 -- 172.21.15.179:0/3704132386 wait complete. 2024-03-26T13:52:36.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.323+0000 7f8c4f400700 1 Processor -- start 2024-03-26T13:52:36.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 -- start start 2024-03-26T13:52:36.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c4819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:36.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48077170 0x7f8c4819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:36.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48078220 0x7f8c481a6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:36.316 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48114430 con 0x7f8c48077170 2024-03-26T13:52:36.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481142b0 con 0x7f8c48078220 2024-03-26T13:52:36.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4f400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481145b0 con 0x7f8c480767b0 2024-03-26T13:52:36.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4ca00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c4819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:36.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4ca00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c4819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:46878/0 (socket says 172.21.15.179:46878) 2024-03-26T13:52:36.317 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.324+0000 7f8c4ca00700 1 -- 172.21.15.179:0/77371572 learned_addr learned my addr 172.21.15.179:0/77371572 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:52:36.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c3fe00700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48077170 0x7f8c4819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:36.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4d400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48078220 0x7f8c481a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:36.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4ca00700 1 -- 172.21.15.179:0/77371572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48078220 msgr2=0x7f8c481a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:36.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4ca00700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48078220 0x7f8c481a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:36.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4ca00700 1 -- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48077170 msgr2=0x7f8c4819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:52:36.318 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4ca00700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48077170 0x7f8c4819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4ca00700 1 -- 172.21.15.179:0/77371572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c481a66e0 con 0x7f8c480767b0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4d400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48078220 0x7f8c481a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4ca00700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c4819ea80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c38007df0 tx=0x7f8c3800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38016020 con 0x7f8c480767b0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8c3800ad60 con 0x7f8c480767b0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380117e0 con 0x7f8c480767b0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.325+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c481a69d0 con 0x7f8c480767b0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.326+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c481a6f60 con 0x7f8c480767b0 2024-03-26T13:52:36.319 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.327+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48068e50 con 0x7f8c480767b0 2024-03-26T13:52:36.321 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.329+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f8c38019020 con 0x7f8c480767b0 2024-03-26T13:52:36.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.330+0000 7f8c46000700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8c2c0a2900 0x7f8c2c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:52:36.322 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.330+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f8c380cd580 con 0x7f8c480767b0 2024-03-26T13:52:36.323 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.331+0000 7f8c3fe00700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8c2c0a2900 0x7f8c2c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:52:36.324 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.332+0000 7f8c3fe00700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8c2c0a2900 0x7f8c2c0a4dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c340036e0 tx=0x7f8c34003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:52:36.326 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.334+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8c38020070 con 0x7f8c480767b0 2024-03-26T13:52:36.555 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:52:36.563+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c480625d0 con 0x7f8c2c0a2900 2024-03-26T13:52:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:52:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:52:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:36.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:52:36.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:52:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:37.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:52:37.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:52:37.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:37.754 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:37 smithi179 ceph-mon[35472]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:52:37.754 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:37 smithi179 ceph-mon[35472]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:37.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:37 smithi159 ceph-mon[35501]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:52:37.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:37 smithi159 ceph-mon[35501]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:38.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:37 smithi137 ceph-mon[31539]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:52:38.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:37 smithi137 ceph-mon[31539]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:40.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:40 smithi137 ceph-mon[31539]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:40.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:40 smithi179 ceph-mon[35472]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:40.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:40 smithi159 ceph-mon[35501]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:41.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:41 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/4283282604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]: dispatch 2024-03-26T13:52:41.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:41 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]: dispatch 2024-03-26T13:52:41.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:41 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]': finished 2024-03-26T13:52:41.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:41 smithi179 ceph-mon[35472]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T13:52:41.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:52:41.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/4283282604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]: dispatch 2024-03-26T13:52:41.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:41 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]: dispatch 2024-03-26T13:52:41.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:41 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]': finished 2024-03-26T13:52:41.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:41 smithi137 ceph-mon[31539]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T13:52:41.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:52:41.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:41 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/4283282604' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]: dispatch 2024-03-26T13:52:41.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:41 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]: dispatch 2024-03-26T13:52:41.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:41 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "76a74368-6c04-4a19-b56d-432b5145e385"}]': finished 2024-03-26T13:52:41.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:41 smithi159 ceph-mon[35501]: osdmap e15: 3 total, 2 up, 3 in 2024-03-26T13:52:41.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:52:42.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:42 smithi137 ceph-mon[31539]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:42.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:42 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/3132421206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:52:42.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:42 smithi179 ceph-mon[35472]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:42.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:42 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/3132421206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:52:42.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:42 smithi159 ceph-mon[35501]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:42.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:42 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/3132421206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:52:44.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:44 smithi137 ceph-mon[31539]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:44.381 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:44 smithi179 ceph-mon[35472]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:44 smithi159 ceph-mon[35501]: pgmap v90: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:46.187 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:46 smithi179 ceph-mon[35472]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:46.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:46 smithi137 ceph-mon[31539]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:46.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:46 smithi159 ceph-mon[35501]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:48.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:48 smithi179 ceph-mon[35472]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:48.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:48 smithi159 ceph-mon[35501]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:48.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:48 smithi137 ceph-mon[31539]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:50.329 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:50 smithi137 ceph-mon[31539]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:50.330 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T13:52:50.330 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:50.330 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:50 smithi137 ceph-mon[31539]: Deploying daemon osd.2 on smithi179 2024-03-26T13:52:50.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:50 smithi179 ceph-mon[35472]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:50.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T13:52:50.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:50.357 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:50 smithi179 ceph-mon[35472]: Deploying daemon osd.2 on smithi179 2024-03-26T13:52:50.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:50 smithi159 ceph-mon[35501]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:50.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-26T13:52:50.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:52:50.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:50 smithi159 ceph-mon[35501]: Deploying daemon osd.2 on smithi179 2024-03-26T13:52:52.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:52 smithi179 ceph-mon[35472]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:52.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:52 smithi159 ceph-mon[35501]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:52.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:52 smithi137 ceph-mon[31539]: pgmap v94: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:54.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:54 smithi179 ceph-mon[35472]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:54 smithi159 ceph-mon[35501]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:54.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:54 smithi137 ceph-mon[31539]: pgmap v95: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:56.394 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:56 smithi179 ceph-mon[35472]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:56.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:56 smithi159 ceph-mon[35501]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:56.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:56 smithi137 ceph-mon[31539]: pgmap v96: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:58.070 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:58.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:58.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:58.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:52:57 smithi179 ceph-mon[35472]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:58.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:58.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:58.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:58.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:52:57 smithi159 ceph-mon[35501]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:52:58.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:58.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:52:58.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:52:58.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:52:57 smithi137 ceph-mon[31539]: pgmap v97: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:00.165 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:00 smithi179 ceph-mon[35472]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:00.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:00.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:00 smithi137 ceph-mon[31539]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:00.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:00.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:00 smithi159 ceph-mon[35501]: pgmap v98: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:00.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:01.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:01.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:01 smithi137 ceph-mon[31539]: from='osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T13:53:01.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:01 smithi137 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T13:53:01.407 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:01.407 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:01 smithi179 ceph-mon[35472]: from='osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T13:53:01.407 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:01 smithi179 ceph-mon[35472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T13:53:01.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:01.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:01 smithi159 ceph-mon[35501]: from='osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T13:53:01.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:01 smithi159 ceph-mon[35501]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-26T13:53:01.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.531+0000 7f8c46000700 1 -- 172.21.15.179:0/77371572 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c480625d0 con 0x7f8c2c0a2900 2024-03-26T13:53:01.524 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.535+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8c2c0a2900 msgr2=0x7f8c2c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.535+0000 7f8c4f400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8c2c0a2900 0x7f8c2c0a4dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8c340036e0 tx=0x7f8c34003870 comp rx=0 tx=0).stop 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.535+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 msgr2=0x7f8c4819ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.535+0000 7f8c4f400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c4819ea80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8c38007df0 tx=0x7f8c3800a340 comp rx=0 tx=0).stop 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.535+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 shutdown_connections 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8c2c0a2900 0x7f8c2c0a4dc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c48078220 0x7f8c481a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c480767b0 0x7f8c4819ea80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 --2- 172.21.15.179:0/77371572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c48077170 0x7f8c4819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 >> 172.21.15.179:0/77371572 conn(0x7f8c480ff9b0 msgr2=0x7f8c481014b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 shutdown_connections 2024-03-26T13:53:01.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:53:01.536+0000 7f8c4f400700 1 -- 172.21.15.179:0/77371572 wait complete. 2024-03-26T13:53:02.117 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:02.117 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.118 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T13:53:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:53:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T13:53:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:02.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:53:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:02.297 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.2.service 2024-03-26T13:53:02.303 INFO:tasks.cephadm:Deploying osd.3 on smithi137 with /dev/vg_nvme/lv_3... 2024-03-26T13:53:02.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T13:53:02.392 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: pgmap v99: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:02.392 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: osdmap e16: 3 total, 2 up, 3 in 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:02.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:02.738 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:53:02.986 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2[40490]: 2024-03-26T13:53:02.724+0000 7f63ce000700 -1 osd.2 0 waiting for initial osdmap 2024-03-26T13:53:02.986 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2[40490]: 2024-03-26T13:53:02.748+0000 7f63c3e00700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:53:03.173 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:03 smithi137 ceph-mon[31539]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:53:03.173 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:03 smithi137 ceph-mon[31539]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T13:53:03.173 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:03 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:03.173 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:03 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:03.173 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:03 smithi137 ceph-mon[31539]: from='osd.2 ' entity='osd.2' 2024-03-26T13:53:03.258 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:03 smithi179 ceph-mon[35472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:53:03.258 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:03 smithi179 ceph-mon[35472]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T13:53:03.258 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:03 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:03.258 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:03 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:03.258 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:03 smithi179 ceph-mon[35472]: from='osd.2 ' entity='osd.2' 2024-03-26T13:53:03.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:03 smithi159 ceph-mon[35501]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:53:03.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:03 smithi159 ceph-mon[35501]: osdmap e17: 3 total, 2 up, 3 in 2024-03-26T13:53:03.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:03 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:03.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:03 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:03.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:03 smithi159 ceph-mon[35501]: from='osd.2 ' entity='osd.2' 2024-03-26T13:53:04.404 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:53:04.405 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:53:04.405 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: pgmap v102: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:04.405 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:04.405 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] boot 2024-03-26T13:53:04.405 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T13:53:04.405 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: pgmap v102: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] boot 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T13:53:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: pgmap v102: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: osd.2 [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] boot 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: osdmap e18: 3 total, 3 up, 3 in 2024-03-26T13:53:04.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-26T13:53:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T13:53:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T13:53:05.544 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-26T13:53:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-26T13:53:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0352861 s, 297 MB/s 2024-03-26T13:53:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-26T13:53:05.912 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:05 smithi137 ceph-mon[31539]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T13:53:05.912 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:05 smithi137 ceph-mon[31539]: pgmap v105: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:05.913 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T13:53:05.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:05 smithi179 ceph-mon[35472]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T13:53:05.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:05 smithi179 ceph-mon[35472]: pgmap v105: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:05.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T13:53:06.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:05 smithi159 ceph-mon[35501]: osdmap e19: 3 total, 3 up, 3 in 2024-03-26T13:53:06.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:05 smithi159 ceph-mon[35501]: pgmap v105: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:06.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-26T13:53:06.323 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-03-26T13:53:06.759 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:53:07.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T13:53:07.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:06 smithi137 ceph-mon[31539]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T13:53:07.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T13:53:07.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T13:53:07.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:06 smithi179 ceph-mon[35472]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T13:53:07.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T13:53:07.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-26T13:53:07.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:06 smithi159 ceph-mon[35501]: osdmap e20: 3 total, 3 up, 3 in 2024-03-26T13:53:07.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T13:53:07.620 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:07 smithi137 sudo[50581]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T13:53:07.686 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:07 smithi137 sudo[50581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:07.686 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:07 smithi137 sudo[50581]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:07.686 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:07 smithi137 sudo[50581]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:07.941 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:07 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:08.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:08.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:07 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: osdmap e21: 3 total, 3 up, 3 in 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: pgmap v108: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:08.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:07 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:08.456 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:08 smithi137 sudo[50590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T13:53:08.457 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:08 smithi137 sudo[50590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:08.457 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:08 smithi137 sudo[50590]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:08.457 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 13:53:08 smithi137 sudo[50590]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 -- 172.21.15.137:0/3379882238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf0103d60 msgr2=0x7fbcf010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 --2- 172.21.15.137:0/3379882238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf0103d60 0x7fbcf010a610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b0d0 tx=0x7fbce800b3e0 comp rx=0 tx=0).stop 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 -- 172.21.15.137:0/3379882238 shutdown_connections 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 --2- 172.21.15.137:0/3379882238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf0103d60 0x7fbcf010a610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 --2- 172.21.15.137:0/3379882238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf01021c0 0x7fbcf01025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 --2- 172.21.15.137:0/3379882238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcf01033c0 0x7fbcf0103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 -- 172.21.15.137:0/3379882238 >> 172.21.15.137:0/3379882238 conn(0x7fbcf00fd7a0 msgr2=0x7fbcf00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.963+0000 7fbcf6400700 1 -- 172.21.15.137:0/3379882238 shutdown_connections 2024-03-26T13:53:08.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.964+0000 7fbcf6400700 1 -- 172.21.15.137:0/3379882238 wait complete. 2024-03-26T13:53:08.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.965+0000 7fbcf6400700 1 Processor -- start 2024-03-26T13:53:08.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.965+0000 7fbcf6400700 1 -- start start 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.965+0000 7fbcf6400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 0x7fbcf01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcf6400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf01033c0 0x7fbcf0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcf6400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcf0103d60 0x7fbcf019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcf6400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf010dcc0 con 0x7fbcf01021c0 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcf6400700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf010db40 con 0x7fbcf01033c0 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcf6400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf010de40 con 0x7fbcf0103d60 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 0x7fbcf01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 0x7fbcf01982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57436/0 (socket says 172.21.15.137:57436) 2024-03-26T13:53:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 -- 172.21.15.137:0/243957232 learned_addr learned my addr 172.21.15.137:0/243957232 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbceea00700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf01033c0 0x7fbcf0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcefe00700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcf0103d60 0x7fbcf019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 -- 172.21.15.137:0/243957232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcf0103d60 msgr2=0x7fbcf019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcf0103d60 0x7fbcf019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 -- 172.21.15.137:0/243957232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf01033c0 msgr2=0x7fbcf0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf01033c0 0x7fbcf0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:08.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.966+0000 7fbcef400700 1 -- 172.21.15.137:0/243957232 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf019ff30 con 0x7fbcf01021c0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.967+0000 7fbcef400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 0x7fbcf01982d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbce000d9b0 tx=0x7fbce000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.967+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0017030 con 0x7fbcf01021c0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.967+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbce00044d0 con 0x7fbcf01021c0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.967+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf01a0220 con 0x7fbcf01021c0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.967+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00039e0 con 0x7fbcf01021c0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.967+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf01a07b0 con 0x7fbcf01021c0 2024-03-26T13:53:08.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.968+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf0068e50 con 0x7fbcf01021c0 2024-03-26T13:53:08.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.970+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbce001a070 con 0x7fbcf01021c0 2024-03-26T13:53:08.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.971+0000 7fbce7e00700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbcd40a2990 0x7fbcd40a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:08.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.972+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3237+0+0 (secure 0 0 0) 0x7fbce00cd0e0 con 0x7fbcf01021c0 2024-03-26T13:53:08.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.972+0000 7fbceea00700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbcd40a2990 0x7fbcd40a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:08.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.972+0000 7fbceea00700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbcd40a2990 0x7fbcd40a4e50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc005950 tx=0x7fbcdc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:53:08.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:08.977+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbce009b4b0 con 0x7fbcf01021c0 2024-03-26T13:53:09.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:08 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi179 2024-03-26T13:53:09.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:08 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi179 to 17091M 2024-03-26T13:53:09.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:08 smithi179 ceph-mon[35472]: mgrmap e16: x(active, since 3m), standbys: y, z 2024-03-26T13:53:09.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:08 smithi179 ceph-mon[35472]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T13:53:09.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:08 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:09.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:08 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi179 2024-03-26T13:53:09.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:08 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi179 to 17091M 2024-03-26T13:53:09.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:08 smithi159 ceph-mon[35501]: mgrmap e16: x(active, since 3m), standbys: y, z 2024-03-26T13:53:09.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:08 smithi159 ceph-mon[35501]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T13:53:09.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:08 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:09.195 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:08 smithi159 sudo[43888]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T13:53:09.195 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:08 smithi159 sudo[43888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:09.195 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:08 smithi159 sudo[43888]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:09.195 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:08 smithi159 sudo[43888]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:09.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:09.209+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcf00625d0 con 0x7fbcd40a2990 2024-03-26T13:53:09.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:08 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi179 2024-03-26T13:53:09.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:08 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi179 to 17091M 2024-03-26T13:53:09.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:08 smithi137 ceph-mon[31539]: mgrmap e16: x(active, since 3m), standbys: y, z 2024-03-26T13:53:09.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:08 smithi137 ceph-mon[31539]: osdmap e22: 3 total, 3 up, 3 in 2024-03-26T13:53:09.201 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:08 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:09.695 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:09 smithi159 sudo[43891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T13:53:09.695 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:09 smithi159 sudo[43891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:09.695 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:09 smithi159 sudo[43891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:09.695 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 13:53:09 smithi159 sudo[43891]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:10.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:09 smithi137 ceph-mon[31539]: pgmap v110: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:10.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:53:10.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:53:10.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:10.131 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:10.166 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44661]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T13:53:10.166 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44661]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:10.166 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44661]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:10.166 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44661]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:10.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:09 smithi179 ceph-mon[35472]: pgmap v110: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:10.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:53:10.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:53:10.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:10.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:10.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:09 smithi159 ceph-mon[35501]: pgmap v110: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:10.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:53:10.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:53:10.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:10.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:10.916 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T13:53:10.916 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:10.916 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44664]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:10.916 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 13:53:10 smithi179 sudo[44664]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:11.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T13:53:11.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:11.224 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50779]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:11.224 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50779]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:11.898 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T13:53:11.898 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:11.898 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50807]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:11.898 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 sudo[50807]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:11.899 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 ceph-mon[31539]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:53:11.899 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:11.899 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:11 smithi137 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:12.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:11 smithi179 ceph-mon[35472]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:53:12.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:12.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:11 smithi179 ceph-mon[35472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:12.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:11 smithi159 ceph-mon[35501]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:53:12.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:12.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:11 smithi159 ceph-mon[35501]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:12.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44667]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T13:53:12.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44667]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:12.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44667]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:12.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44667]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:13.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44670]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44670]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44670]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 sudo[44670]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: pgmap v111: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:13.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:13.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:13.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:13.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: pgmap v111: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:13.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:13.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:13.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:13.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:13.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:13.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:13.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:13.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:13 smithi159 sudo[43894]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-26T13:53:13.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:13 smithi159 sudo[43894]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:13.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:13 smithi159 sudo[43894]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:13.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:13 smithi159 sudo[43894]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:14.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:14.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:14.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:14.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:14.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:14.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 304 KiB/s wr, 19 op/s 2024-03-26T13:53:14.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:14 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/602768414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba6d4b45-933b-4346-879f-ead75f3faf3b"}]: dispatch 2024-03-26T13:53:14.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 sudo[43897]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-26T13:53:14.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 sudo[43897]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-26T13:53:14.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 sudo[43897]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-26T13:53:14.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 sudo[43897]: pam_unix(sudo:session): session closed for user root 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: pgmap v111: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:14.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 304 KiB/s wr, 19 op/s 2024-03-26T13:53:14.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:14 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/602768414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba6d4b45-933b-4346-879f-ead75f3faf3b"}]: dispatch 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 304 KiB/s wr, 19 op/s 2024-03-26T13:53:14.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:14 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/602768414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba6d4b45-933b-4346-879f-ead75f3faf3b"}]: dispatch 2024-03-26T13:53:15.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:15 smithi179 ceph-mon[35472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:15.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:15 smithi179 ceph-mon[35472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:15.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:15 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/602768414' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba6d4b45-933b-4346-879f-ead75f3faf3b"}]': finished 2024-03-26T13:53:15.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:15 smithi179 ceph-mon[35472]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T13:53:15.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:15.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:15 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3696762084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:53:15.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:15 smithi159 ceph-mon[35501]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:15.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:15 smithi159 ceph-mon[35501]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:15.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:15 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/602768414' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba6d4b45-933b-4346-879f-ead75f3faf3b"}]': finished 2024-03-26T13:53:15.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:15 smithi159 ceph-mon[35501]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T13:53:15.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:15.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:15 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3696762084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:53:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:15 smithi137 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-26T13:53:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:15 smithi137 ceph-mon[31539]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-26T13:53:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:15 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/602768414' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba6d4b45-933b-4346-879f-ead75f3faf3b"}]': finished 2024-03-26T13:53:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:15 smithi137 ceph-mon[31539]: osdmap e23: 4 total, 3 up, 4 in 2024-03-26T13:53:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:15.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:15 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3696762084' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:53:16.415 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:16 smithi179 ceph-mon[35472]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 219 KiB/s wr, 13 op/s 2024-03-26T13:53:16.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:16 smithi159 ceph-mon[35501]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 219 KiB/s wr, 13 op/s 2024-03-26T13:53:16.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:16 smithi137 ceph-mon[31539]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 219 KiB/s wr, 13 op/s 2024-03-26T13:53:18.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:18 smithi179 ceph-mon[35472]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-03-26T13:53:18.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:18 smithi159 ceph-mon[35501]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-03-26T13:53:18.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:18 smithi137 ceph-mon[31539]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 171 KiB/s wr, 10 op/s 2024-03-26T13:53:20.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:20 smithi137 ceph-mon[31539]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-26T13:53:20.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:20 smithi179 ceph-mon[35472]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-26T13:53:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:20 smithi159 ceph-mon[35501]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-26T13:53:22.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:22 smithi137 ceph-mon[31539]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-26T13:53:22.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:22 smithi179 ceph-mon[35472]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-26T13:53:22.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:22 smithi159 ceph-mon[35501]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-03-26T13:53:24.403 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:24 smithi137 ceph-mon[31539]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-26T13:53:24.404 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:24 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T13:53:24.404 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:24 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:24.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:24 smithi179 ceph-mon[35472]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-26T13:53:24.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:24 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T13:53:24.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:24 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:24.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:24 smithi159 ceph-mon[35501]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-26T13:53:24.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:24 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-26T13:53:24.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:24 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:25.192 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:25 smithi137 ceph-mon[31539]: Deploying daemon osd.3 on smithi137 2024-03-26T13:53:25.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:25 smithi179 ceph-mon[35472]: Deploying daemon osd.3 on smithi137 2024-03-26T13:53:25.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:25 smithi159 ceph-mon[35501]: Deploying daemon osd.3 on smithi137 2024-03-26T13:53:26.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:26 smithi137 ceph-mon[31539]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 842 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-26T13:53:26.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:26 smithi179 ceph-mon[35472]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 842 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-26T13:53:26.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:26 smithi159 ceph-mon[35501]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 842 B/s rd, 24 KiB/s wr, 2 op/s 2024-03-26T13:53:28.202 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:28 smithi137 ceph-mon[31539]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T13:53:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:28 smithi179 ceph-mon[35472]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T13:53:28.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:28 smithi159 ceph-mon[35501]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T13:53:30.395 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:30 smithi137 ceph-mon[31539]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T13:53:30.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:30 smithi179 ceph-mon[35472]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T13:53:30.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:30 smithi159 ceph-mon[35501]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-03-26T13:53:32.386 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:32 smithi137 ceph-mon[31539]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:32.386 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:32.386 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:32.386 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:32.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:32 smithi179 ceph-mon[35472]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:32.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:32.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:32.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:32.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:32 smithi159 ceph-mon[35501]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:32.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:32.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:32.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:34.243 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:34 smithi137 ceph-mon[31539]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:34.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:34 smithi179 ceph-mon[35472]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:34.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:34 smithi159 ceph-mon[35501]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:35.773 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:35.773 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:35.773 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:35 smithi179 ceph-mon[35472]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:35.773 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:35 smithi179 ceph-mon[35472]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T13:53:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:35 smithi137 ceph-mon[31539]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:35.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:35 smithi137 ceph-mon[31539]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T13:53:35.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:35.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:35.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:35 smithi159 ceph-mon[35501]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:35.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:35 smithi159 ceph-mon[35501]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-26T13:53:36.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.149+0000 7fbce7e00700 1 -- 172.21.15.137:0/243957232 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbcf00625d0 con 0x7fbcd40a2990 2024-03-26T13:53:36.142 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 3 on host 'smithi137' 2024-03-26T13:53:36.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbcd40a2990 msgr2=0x7fbcd40a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:36.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbcd40a2990 0x7fbcd40a4e50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc005950 tx=0x7fbcdc0058e0 comp rx=0 tx=0).stop 2024-03-26T13:53:36.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 msgr2=0x7fbcf01982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:36.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 0x7fbcf01982d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbce000d9b0 tx=0x7fbce000dd70 comp rx=0 tx=0).stop 2024-03-26T13:53:36.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 shutdown_connections 2024-03-26T13:53:36.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbcd40a2990 0x7fbcd40a4e50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:36.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbcf0103d60 0x7fbcf019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:36.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbcf01021c0 0x7fbcf01982d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:36.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 --2- 172.21.15.137:0/243957232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf01033c0 0x7fbcf0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:36.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.153+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 >> 172.21.15.137:0/243957232 conn(0x7fbcf00fd7a0 msgr2=0x7fbcf01066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:53:36.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.154+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 shutdown_connections 2024-03-26T13:53:36.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:53:36.154+0000 7fbcf6400700 1 -- 172.21.15.137:0/243957232 wait complete. 2024-03-26T13:53:36.903 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:36.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: osdmap e24: 4 total, 3 up, 4 in 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:36.958 DEBUG:teuthology.orchestra.run.smithi137:osd.3> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.3.service 2024-03-26T13:53:36.963 INFO:tasks.cephadm:Deploying osd.4 on smithi159 with /dev/vg_nvme/lv_3... 2024-03-26T13:53:36.963 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T13:53:37.418 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:53:37.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:37 smithi179 ceph-mon[35472]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:53:37.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:37 smithi179 ceph-mon[35472]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T13:53:37.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:37.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:37.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:37 smithi179 ceph-mon[35472]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:37.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:37 smithi179 ceph-mon[35472]: osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] boot 2024-03-26T13:53:37.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:37 smithi159 ceph-mon[35501]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:53:37.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:37 smithi159 ceph-mon[35501]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T13:53:37.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:37.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:37.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:37 smithi159 ceph-mon[35501]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:37.927 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:37 smithi159 ceph-mon[35501]: osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] boot 2024-03-26T13:53:38.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:37 smithi137 ceph-mon[31539]: from='osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:53:38.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:37 smithi137 ceph-mon[31539]: osdmap e25: 4 total, 3 up, 4 in 2024-03-26T13:53:38.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:38.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:38.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:37 smithi137 ceph-mon[31539]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-26T13:53:38.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:37 smithi137 ceph-mon[31539]: osd.3 [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] boot 2024-03-26T13:53:38.700 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:38 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:53:38.701 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:38 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:53:38.701 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:38 smithi137 ceph-mon[31539]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T13:53:38.701 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:38 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:38.701 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:38 smithi137 ceph-mon[31539]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T13:53:38.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:38 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:53:38.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:38 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:53:38.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:38 smithi179 ceph-mon[35472]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T13:53:38.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:38 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:38.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:38 smithi179 ceph-mon[35472]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T13:53:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:38 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:53:38.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:38 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:53:38.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:38 smithi159 ceph-mon[35501]: osdmap e26: 4 total, 4 up, 4 in 2024-03-26T13:53:38.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:38 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-26T13:53:38.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:38 smithi159 ceph-mon[35501]: osdmap e27: 4 total, 4 up, 4 in 2024-03-26T13:53:39.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:39 smithi179 ceph-mon[35472]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:39.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:39 smithi179 ceph-mon[35472]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T13:53:39.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:39 smithi159 ceph-mon[35501]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:39.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:39 smithi159 ceph-mon[35501]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T13:53:40.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:39 smithi137 ceph-mon[31539]: pgmap v130: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:40.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:39 smithi137 ceph-mon[31539]: osdmap e28: 4 total, 4 up, 4 in 2024-03-26T13:53:40.490 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T13:53:40.490 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T13:53:40.490 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-26T13:53:40.490 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-26T13:53:40.490 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0326755 s, 321 MB/s 2024-03-26T13:53:40.490 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-26T13:53:41.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-03-26T13:53:41.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:41 smithi137 ceph-mon[31539]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T13:53:41.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:41 smithi137 ceph-mon[31539]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:41.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:41 smithi159 ceph-mon[35501]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T13:53:41.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:41 smithi159 ceph-mon[35501]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:41.833 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:53:41.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:41 smithi179 ceph-mon[35472]: osdmap e29: 4 total, 4 up, 4 in 2024-03-26T13:53:41.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:41 smithi179 ceph-mon[35472]: pgmap v133: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi137 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi137 to 8545M 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:42.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi137 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi137 to 8545M 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:42.778 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: osdmap e30: 4 total, 4 up, 4 in 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi137 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:53:42.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi137 to 8545M 2024-03-26T13:53:42.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:42.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:42.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:53:42.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:53:43.683 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:43 smithi159 ceph-mon[35501]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-26T13:53:43.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:43 smithi137 ceph-mon[31539]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-26T13:53:43.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:43 smithi179 ceph-mon[35472]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 -- 172.21.15.159:0/669210303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca54103d60 msgr2=0x7fca5410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 --2- 172.21.15.159:0/669210303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca54103d60 0x7fca5410a610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00b0d0 tx=0x7fca4c00b3e0 comp rx=0 tx=0).stop 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 -- 172.21.15.159:0/669210303 shutdown_connections 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 --2- 172.21.15.159:0/669210303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca54103d60 0x7fca5410a610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 --2- 172.21.15.159:0/669210303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca541021c0 0x7fca541025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 --2- 172.21.15.159:0/669210303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca541033c0 0x7fca54103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 -- 172.21.15.159:0/669210303 >> 172.21.15.159:0/669210303 conn(0x7fca540fd7a0 msgr2=0x7fca540ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 -- 172.21.15.159:0/669210303 shutdown_connections 2024-03-26T13:53:43.991 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:43.999+0000 7fca59c00700 1 -- 172.21.15.159:0/669210303 wait complete. 2024-03-26T13:53:43.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 Processor -- start 2024-03-26T13:53:43.992 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 -- start start 2024-03-26T13:53:43.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca541021c0 0x7fca5419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:43.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 0x7fca5419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:43.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca54103d60 0x7fca541a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:43.993 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5410dc50 con 0x7fca54103d60 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5410dad0 con 0x7fca541033c0 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.000+0000 7fca59c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5410ddd0 con 0x7fca541021c0 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 0x7fca5419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 0x7fca5419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:45162/0 (socket says 172.21.15.159:45162) 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 -- 172.21.15.159:0/1844571398 learned_addr learned my addr 172.21.15.159:0/1844571398 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52a00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca541021c0 0x7fca5419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca53400700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca54103d60 0x7fca541a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 -- 172.21.15.159:0/1844571398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca541021c0 msgr2=0x7fca5419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca541021c0 0x7fca5419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 -- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca54103d60 msgr2=0x7fca541a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca54103d60 0x7fca541a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.001+0000 7fca52000700 1 -- 172.21.15.159:0/1844571398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca541a2070 con 0x7fca541033c0 2024-03-26T13:53:43.994 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.002+0000 7fca52000700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 0x7fca5419a950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c9b0 tx=0x7fca4000cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:53:43.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.002+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40007b00 con 0x7fca541033c0 2024-03-26T13:53:43.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.002+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca40007ca0 con 0x7fca541033c0 2024-03-26T13:53:43.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.002+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca541a2360 con 0x7fca541033c0 2024-03-26T13:53:43.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.002+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca400186f0 con 0x7fca541033c0 2024-03-26T13:53:43.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.002+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca541a28f0 con 0x7fca541033c0 2024-03-26T13:53:43.995 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.003+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca54068e50 con 0x7fca541033c0 2024-03-26T13:53:43.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.005+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fca4000f450 con 0x7fca541033c0 2024-03-26T13:53:43.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.007+0000 7fca4b400700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca380a2910 0x7fca380a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:53:43.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.007+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3669+0+0 (secure 0 0 0) 0x7fca400cdb70 con 0x7fca541033c0 2024-03-26T13:53:43.999 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.008+0000 7fca52a00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca380a2910 0x7fca380a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:53:44.000 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.009+0000 7fca52a00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca380a2910 0x7fca380a4dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca44005950 tx=0x7fca440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:53:44.002 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.010+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca40014030 con 0x7fca541033c0 2024-03-26T13:53:44.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:53:44.245+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fca54075ae0 con 0x7fca380a2910 2024-03-26T13:53:44.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:53:44.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:53:44.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:44.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:53:44.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:53:44.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:44.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:53:44.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:53:44.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:45.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:45 smithi179 ceph-mon[35472]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:53:45.666 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:45 smithi179 ceph-mon[35472]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T13:53:45.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:45 smithi137 ceph-mon[31539]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:53:45.777 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:45 smithi137 ceph-mon[31539]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T13:53:45.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:45 smithi159 ceph-mon[35501]: from='client.34202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:53:45.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:45 smithi159 ceph-mon[35501]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-26T13:53:48.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:48 smithi137 ceph-mon[31539]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T13:53:48.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:48 smithi179 ceph-mon[35472]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T13:53:48.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:48 smithi159 ceph-mon[35501]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T13:53:49.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:49 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/1609556282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]: dispatch 2024-03-26T13:53:49.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:49 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]: dispatch 2024-03-26T13:53:49.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:49 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]': finished 2024-03-26T13:53:49.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:49 smithi159 ceph-mon[35501]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T13:53:49.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:53:49.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:49 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/1609556282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]: dispatch 2024-03-26T13:53:49.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:49 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]: dispatch 2024-03-26T13:53:49.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:49 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]': finished 2024-03-26T13:53:49.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:49 smithi137 ceph-mon[31539]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T13:53:49.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:53:49.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:49 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/1609556282' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]: dispatch 2024-03-26T13:53:49.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:49 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]: dispatch 2024-03-26T13:53:49.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:49 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5"}]': finished 2024-03-26T13:53:49.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:49 smithi179 ceph-mon[35472]: osdmap e31: 5 total, 4 up, 5 in 2024-03-26T13:53:49.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:53:50.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:50 smithi179 ceph-mon[35472]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T13:53:50.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:50 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/3232696756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:53:50.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:50 smithi159 ceph-mon[35501]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T13:53:50.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:50 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/3232696756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:53:50.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:50 smithi137 ceph-mon[31539]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-26T13:53:50.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:50 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/3232696756' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:53:52.277 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:52 smithi137 ceph-mon[31539]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T13:53:52.392 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:52 smithi159 ceph-mon[35501]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T13:53:52.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:52 smithi179 ceph-mon[35472]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-26T13:53:54.376 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:54 smithi159 ceph-mon[35501]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:54.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:54 smithi179 ceph-mon[35472]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:54.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:54 smithi137 ceph-mon[31539]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:56.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:56 smithi159 ceph-mon[35501]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:56.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:56 smithi179 ceph-mon[35472]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:56.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:56 smithi137 ceph-mon[31539]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:58 smithi179 ceph-mon[35472]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:58.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:58 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T13:53:58.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:58 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:58 smithi159 ceph-mon[35501]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:58 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T13:53:58.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:58 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:58.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:58 smithi137 ceph-mon[31539]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:53:58.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:58 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-26T13:53:58.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:58 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:53:59.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:53:59 smithi159 ceph-mon[35501]: Deploying daemon osd.4 on smithi159 2024-03-26T13:53:59.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:53:59 smithi179 ceph-mon[35472]: Deploying daemon osd.4 on smithi159 2024-03-26T13:53:59.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:53:59 smithi137 ceph-mon[31539]: Deploying daemon osd.4 on smithi159 2024-03-26T13:54:00.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:00 smithi179 ceph-mon[35472]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:00.417 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:00 smithi159 ceph-mon[35501]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:00.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:00 smithi137 ceph-mon[31539]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:02.358 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:02 smithi159 ceph-mon[35501]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:02.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:02 smithi179 ceph-mon[35472]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:02.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:02 smithi137 ceph-mon[31539]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:04.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:04 smithi179 ceph-mon[35472]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:04.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:04 smithi159 ceph-mon[35501]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:04.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:04 smithi137 ceph-mon[31539]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:06.166 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:06 smithi179 ceph-mon[35472]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:06.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:06.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:06.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:06 smithi159 ceph-mon[35501]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:06.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:06 smithi137 ceph-mon[31539]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:06.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:06.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:06.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:08 smithi159 ceph-mon[35501]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:08.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:08 smithi179 ceph-mon[35472]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:08.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:08 smithi137 ceph-mon[31539]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:10.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:10 smithi179 ceph-mon[35472]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:10.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:10 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:10.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:10 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:10.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:10 smithi179 ceph-mon[35472]: from='osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T13:54:10.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:10 smithi179 ceph-mon[35472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T13:54:10.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:10 smithi159 ceph-mon[35501]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:10.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:10 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:10.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:10 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:10.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:10 smithi159 ceph-mon[35501]: from='osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T13:54:10.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:10 smithi159 ceph-mon[35501]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T13:54:10.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:10 smithi137 ceph-mon[31539]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:10.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:10.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:10.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:10 smithi137 ceph-mon[31539]: from='osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T13:54:10.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:10 smithi137 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-26T13:54:10.917 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.924+0000 7fca4b400700 1 -- 172.21.15.159:0/1844571398 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fca54075ae0 con 0x7fca380a2910 2024-03-26T13:54:10.917 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 4 on host 'smithi159' 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca380a2910 msgr2=0x7fca380a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca380a2910 0x7fca380a4dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca44005950 tx=0x7fca440058e0 comp rx=0 tx=0).stop 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 msgr2=0x7fca5419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 0x7fca5419a950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fca4000c9b0 tx=0x7fca4000cd70 comp rx=0 tx=0).stop 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 shutdown_connections 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca380a2910 0x7fca380a4dd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca54103d60 0x7fca541a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:10.922 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca541021c0 0x7fca5419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:10.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 --2- 172.21.15.159:0/1844571398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca541033c0 0x7fca5419a950 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:10.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 >> 172.21.15.159:0/1844571398 conn(0x7fca540fd7a0 msgr2=0x7fca540743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:54:10.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.928+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 shutdown_connections 2024-03-26T13:54:10.923 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:54:10.929+0000 7fca59c00700 1 -- 172.21.15.159:0/1844571398 wait complete. 2024-03-26T13:54:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T13:54:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T13:54:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:11.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: from='osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:54:11.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:54:11.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:11.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:11.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:11 smithi159 ceph-mon[35501]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:11.784 DEBUG:teuthology.orchestra.run.smithi159:osd.4> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.4.service 2024-03-26T13:54:11.788 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-03-26T13:54:11.789 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-26T13:54:11.875 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T13:54:11.875 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T13:54:11.875 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:11.875 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: from='osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:54:11.876 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:54:11.876 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:11.876 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:11.876 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:11 smithi179 ceph-mon[35472]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: osdmap e32: 5 total, 4 up, 5 in 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: from='osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:12.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:11 smithi137 ceph-mon[31539]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:12.233 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:54:12.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:12 smithi179 ceph-mon[35472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:54:12.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:12 smithi179 ceph-mon[35472]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T13:54:12.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:12.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:12.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:12 smithi159 ceph-mon[35501]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:54:12.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:12 smithi159 ceph-mon[35501]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T13:54:12.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:12.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:13.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:12 smithi137 ceph-mon[31539]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:54:13.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:12 smithi137 ceph-mon[31539]: osdmap e33: 5 total, 4 up, 5 in 2024-03-26T13:54:13.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:13.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:13.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:13 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:54:13.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:13 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:54:13.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:13 smithi179 ceph-mon[35472]: osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] boot 2024-03-26T13:54:13.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:13 smithi179 ceph-mon[35472]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T13:54:13.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:13.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:13 smithi179 ceph-mon[35472]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:13.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:13 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:54:13.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:13 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:54:13.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:13 smithi159 ceph-mon[35501]: osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] boot 2024-03-26T13:54:13.903 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:13 smithi159 ceph-mon[35501]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T13:54:13.904 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:13.904 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:13 smithi159 ceph-mon[35501]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:14.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:13 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:54:14.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:13 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:54:14.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:13 smithi137 ceph-mon[31539]: osd.4 [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] boot 2024-03-26T13:54:14.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:13 smithi137 ceph-mon[31539]: osdmap e34: 5 total, 5 up, 5 in 2024-03-26T13:54:14.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-26T13:54:14.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:13 smithi137 ceph-mon[31539]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-26T13:54:14.916 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:14 smithi179 ceph-mon[35472]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T13:54:14.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:14 smithi159 ceph-mon[35501]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T13:54:15.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:14 smithi137 ceph-mon[31539]: osdmap e35: 5 total, 5 up, 5 in 2024-03-26T13:54:15.034 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-26T13:54:15.034 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-26T13:54:15.034 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-03-26T13:54:15.034 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-03-26T13:54:15.034 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0368379 s, 285 MB/s 2024-03-26T13:54:15.035 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-03-26T13:54:15.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:15 smithi179 ceph-mon[35472]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:15.758 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-03-26T13:54:15.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:15 smithi159 ceph-mon[35501]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:16.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:15 smithi137 ceph-mon[31539]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:16.208 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi159 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi159 to 8545M 2024-03-26T13:54:17.760 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:17.761 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:17 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi159 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi159 to 8545M 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:17.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:17.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:17.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:17.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:17.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:17.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:17.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:17 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi159 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi159 to 8545M 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:18.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:18.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:18.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:18.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:18.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:18.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:17 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:18.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 -- 172.21.15.179:0/4025463916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 msgr2=0x7f2418103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:18.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 --2- 172.21.15.179:0/4025463916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f2418103740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2404009990 tx=0x7f2404009ca0 comp rx=0 tx=0).stop 2024-03-26T13:54:18.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 -- 172.21.15.179:0/4025463916 shutdown_connections 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 --2- 172.21.15.179:0/4025463916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2418103c80 0x7f241810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 --2- 172.21.15.179:0/4025463916 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24181020e0 0x7f2418102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 --2- 172.21.15.179:0/4025463916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f2418103740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 -- 172.21.15.179:0/4025463916 >> 172.21.15.179:0/4025463916 conn(0x7f24180fd6a0 msgr2=0x7f24180ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 -- 172.21.15.179:0/4025463916 shutdown_connections 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.298+0000 7f241d600700 1 -- 172.21.15.179:0/4025463916 wait complete. 2024-03-26T13:54:18.292 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 Processor -- start 2024-03-26T13:54:18.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 -- start start 2024-03-26T13:54:18.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24181020e0 0x7f241819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:18.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f241819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:18.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2418103c80 0x7f24181a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:18.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241810dbb0 con 0x7f24181020e0 2024-03-26T13:54:18.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.300+0000 7f241d600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241810da30 con 0x7f2418103c80 2024-03-26T13:54:18.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f241d600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241810dd30 con 0x7f24181032e0 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f241819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f241819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:36340/0 (socket says 172.21.15.179:36340) 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 -- 172.21.15.179:0/4175616421 learned_addr learned my addr 172.21.15.179:0/4175616421 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417e00700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24181020e0 0x7f241819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f241cc00700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2418103c80 0x7f24181a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 -- 172.21.15.179:0/4175616421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2418103c80 msgr2=0x7f24181a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2418103c80 0x7f24181a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 -- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24181020e0 msgr2=0x7f241819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24181020e0 0x7f241819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.301+0000 7f2417400700 1 -- 172.21.15.179:0/4175616421 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24181a20c0 con 0x7f24181032e0 2024-03-26T13:54:18.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.302+0000 7f2417400700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f241819a9f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24040057c0 tx=0x7f2404003710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:54:18.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.302+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404027070 con 0x7f24181032e0 2024-03-26T13:54:18.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.302+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2404004080 con 0x7f24181032e0 2024-03-26T13:54:18.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.302+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24181a2350 con 0x7f24181032e0 2024-03-26T13:54:18.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.302+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404022df0 con 0x7f24181032e0 2024-03-26T13:54:18.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.302+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24181a27a0 con 0x7f24181032e0 2024-03-26T13:54:18.296 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.303+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2418068e50 con 0x7f24181032e0 2024-03-26T13:54:18.299 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.306+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f240402a070 con 0x7f24181032e0 2024-03-26T13:54:18.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.308+0000 7f2414c00700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23fc0a2890 0x7f23fc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:18.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.308+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f24040cf1d0 con 0x7f24181032e0 2024-03-26T13:54:18.301 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.309+0000 7f2417e00700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23fc0a2890 0x7f23fc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:18.302 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.310+0000 7f2417e00700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23fc0a2890 0x7f23fc0a4d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2418103140 tx=0x7f240800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:54:18.305 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.312+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2404031020 con 0x7f24181032e0 2024-03-26T13:54:18.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:18.542+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f24181a2ac0 con 0x7f23fc0a2890 2024-03-26T13:54:18.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:54:18.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:54:18.917 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:19.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:54:19.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:54:19.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:19.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:54:19.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:54:19.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:19.794 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:19 smithi179 ceph-mon[35472]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:54:19.794 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:19 smithi179 ceph-mon[35472]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:20.027 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:19 smithi137 ceph-mon[31539]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:54:20.028 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:19 smithi137 ceph-mon[31539]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:20.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:19 smithi159 ceph-mon[35501]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:54:20.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:19 smithi159 ceph-mon[35501]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:22.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:22 smithi179 ceph-mon[35472]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:22.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:22 smithi159 ceph-mon[35501]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:22.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:22 smithi137 ceph-mon[31539]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:23.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:23 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/241015960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]: dispatch 2024-03-26T13:54:23.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:23 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]: dispatch 2024-03-26T13:54:23.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:23 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]': finished 2024-03-26T13:54:23.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:23 smithi179 ceph-mon[35472]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T13:54:23.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:23.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:23 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/241015960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]: dispatch 2024-03-26T13:54:23.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:23 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]: dispatch 2024-03-26T13:54:23.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:23 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]': finished 2024-03-26T13:54:23.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:23 smithi159 ceph-mon[35501]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T13:54:23.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:23.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:23 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/241015960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]: dispatch 2024-03-26T13:54:23.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:23 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]: dispatch 2024-03-26T13:54:23.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:23 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bdbb0679-03d9-4423-873d-2c9c968896ac"}]': finished 2024-03-26T13:54:23.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:23 smithi137 ceph-mon[31539]: osdmap e36: 6 total, 5 up, 6 in 2024-03-26T13:54:23.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:24.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:24 smithi137 ceph-mon[31539]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:24.223 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:24 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/2837162747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:54:24.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:24 smithi179 ceph-mon[35472]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:24.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:24 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/2837162747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:54:24.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:24 smithi159 ceph-mon[35501]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:24.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:24 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/2837162747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:54:26.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:26 smithi179 ceph-mon[35472]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:26.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:26 smithi159 ceph-mon[35501]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:26.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:26 smithi137 ceph-mon[31539]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:28.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:28 smithi179 ceph-mon[35472]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:28.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:28 smithi159 ceph-mon[35501]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:28.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:28 smithi137 ceph-mon[31539]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:30.312 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:30 smithi179 ceph-mon[35472]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:30.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:30 smithi159 ceph-mon[35501]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:30.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:30 smithi137 ceph-mon[31539]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:32.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:32 smithi179 ceph-mon[35472]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:32.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T13:54:32.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:32.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:32 smithi159 ceph-mon[35501]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:32.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T13:54:32.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:32.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:32 smithi137 ceph-mon[31539]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:32.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-26T13:54:32.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:33.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:33 smithi179 ceph-mon[35472]: Deploying daemon osd.5 on smithi179 2024-03-26T13:54:33.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:33 smithi159 ceph-mon[35501]: Deploying daemon osd.5 on smithi179 2024-03-26T13:54:33.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:33 smithi137 ceph-mon[31539]: Deploying daemon osd.5 on smithi179 2024-03-26T13:54:34.338 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:34 smithi179 ceph-mon[35472]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:34.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:34 smithi159 ceph-mon[35501]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:34.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:34 smithi137 ceph-mon[31539]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:36.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:36 smithi179 ceph-mon[35472]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:36.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:36 smithi159 ceph-mon[35501]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:36.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:36 smithi137 ceph-mon[31539]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:38.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:38 smithi179 ceph-mon[35472]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:38.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:38 smithi159 ceph-mon[35501]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:38.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:38 smithi137 ceph-mon[31539]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:40.416 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:40 smithi179 ceph-mon[35472]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:40.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:40.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:40.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:40.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:40 smithi159 ceph-mon[35501]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:40.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:40.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:40.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:40.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:40 smithi137 ceph-mon[31539]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:40.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:40.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:40.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:42.255 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:42 smithi179 ceph-mon[35472]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:42.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:42 smithi159 ceph-mon[35501]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:42.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:42 smithi137 ceph-mon[31539]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:44.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:44.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:44.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:44.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T13:54:44.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:44 smithi159 ceph-mon[35501]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T13:54:44.403 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.410+0000 7f2414c00700 1 -- 172.21.15.179:0/4175616421 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f24181a2ac0 con 0x7f23fc0a2890 2024-03-26T13:54:44.403 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T13:54:44.404 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:44 smithi179 ceph-mon[35472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T13:54:44.406 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.413+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23fc0a2890 msgr2=0x7f23fc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:44.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.413+0000 7f241d600700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23fc0a2890 0x7f23fc0a4d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2418103140 tx=0x7f240800a380 comp rx=0 tx=0).stop 2024-03-26T13:54:44.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.413+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 msgr2=0x7f241819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:44.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.413+0000 7f241d600700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f241819a9f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24040057c0 tx=0x7f2404003710 comp rx=0 tx=0).stop 2024-03-26T13:54:44.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 shutdown_connections 2024-03-26T13:54:44.407 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23fc0a2890 0x7f23fc0a4d50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:44.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2418103c80 0x7f24181a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:44.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24181020e0 0x7f241819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:44.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 --2- 172.21.15.179:0/4175616421 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24181032e0 0x7f241819a9f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:44.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 >> 172.21.15.179:0/4175616421 conn(0x7f24180fd6a0 msgr2=0x7f24180743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:54:44.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 shutdown_connections 2024-03-26T13:54:44.408 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:54:44.414+0000 7f241d600700 1 -- 172.21.15.179:0/4175616421 wait complete. 2024-03-26T13:54:44.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:44.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:44.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:44.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:44.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T13:54:44.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:44 smithi137 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-26T13:54:45.187 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.5.service 2024-03-26T13:54:45.193 INFO:tasks.cephadm:Deploying osd.6 on smithi137 with /dev/vg_nvme/lv_2... 2024-03-26T13:54:45.193 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:45.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-mon[35472]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:45.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:45 smithi137 ceph-mon[31539]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:45.622 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:54:45.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-26T13:54:45.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: osdmap e37: 6 total, 5 up, 6 in 2024-03-26T13:54:45.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:45.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:54:45.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:54:45.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:45.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:45.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:54:45.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:45 smithi159 ceph-mon[35501]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:46.169 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T13:54:45.796+0000 7f8167c00700 -1 osd.5 0 waiting for initial osdmap 2024-03-26T13:54:46.169 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 13:54:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T13:54:45.819+0000 7f815d000700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:54:46.496 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:46 smithi137 ceph-mon[31539]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:54:46.496 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:46 smithi137 ceph-mon[31539]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T13:54:46.496 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:46.496 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:46.496 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:46 smithi137 ceph-mon[31539]: from='osd.5 ' entity='osd.5' 2024-03-26T13:54:46.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:46 smithi179 ceph-mon[35472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:54:46.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:46 smithi179 ceph-mon[35472]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T13:54:46.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:46 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:46.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:46 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:46.588 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:46 smithi179 ceph-mon[35472]: from='osd.5 ' entity='osd.5' 2024-03-26T13:54:46.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:46 smithi159 ceph-mon[35501]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:54:46.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:46 smithi159 ceph-mon[35501]: osdmap e38: 6 total, 5 up, 6 in 2024-03-26T13:54:46.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:46 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:46.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:46 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:46.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:46 smithi159 ceph-mon[35501]: from='osd.5 ' entity='osd.5' 2024-03-26T13:54:47.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:54:47.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:54:47.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:47.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] boot 2024-03-26T13:54:47.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T13:54:47.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:47.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:47.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:47 smithi137 ceph-mon[31539]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T13:54:47.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] boot 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:47.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:47 smithi179 ceph-mon[35472]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: osd.5 [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] boot 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: osdmap e39: 6 total, 6 up, 6 in 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-03-26T13:54:47.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:47 smithi159 ceph-mon[35501]: osdmap e40: 6 total, 6 up, 6 in 2024-03-26T13:54:48.439 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T13:54:48.439 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T13:54:48.439 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-26T13:54:48.439 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-26T13:54:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0374091 s, 280 MB/s 2024-03-26T13:54:48.440 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-26T13:54:49.279 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-03-26T13:54:49.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:49 smithi137 ceph-mon[31539]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T13:54:49.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:49 smithi137 ceph-mon[31539]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:54:49.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:49 smithi179 ceph-mon[35472]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T13:54:49.609 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:49 smithi179 ceph-mon[35472]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:54:49.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:49 smithi159 ceph-mon[35501]: osdmap e41: 6 total, 6 up, 6 in 2024-03-26T13:54:49.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:49 smithi159 ceph-mon[35501]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:54:49.724 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:54:50.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi179 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi179 to 8545M 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T13:54:50.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi179 2024-03-26T13:54:50.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi179 to 8545M 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:50.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: osdmap e42: 6 total, 6 up, 6 in 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi179 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:54:50.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi179 to 8545M 2024-03-26T13:54:50.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:50.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:50.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:54:50.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:54:51.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:51 smithi137 ceph-mon[31539]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T13:54:51.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:51 smithi137 ceph-mon[31539]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:54:51.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:51 smithi179 ceph-mon[35472]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T13:54:51.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:51 smithi179 ceph-mon[35472]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:54:51.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:51 smithi159 ceph-mon[35501]: osdmap e43: 6 total, 6 up, 6 in 2024-03-26T13:54:51.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:51 smithi159 ceph-mon[35501]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 -- 172.21.15.137:0/3577457436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 msgr2=0x7fa724103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 --2- 172.21.15.137:0/3577457436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa724103830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0099c0 tx=0x7fa70c009cd0 comp rx=0 tx=0).stop 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 -- 172.21.15.137:0/3577457436 shutdown_connections 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 --2- 172.21.15.137:0/3577457436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa724103d70 0x7fa72410a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 --2- 172.21.15.137:0/3577457436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7241021d0 0x7fa7241025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 --2- 172.21.15.137:0/3577457436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa724103830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 -- 172.21.15.137:0/3577457436 >> 172.21.15.137:0/3577457436 conn(0x7fa7240fd730 msgr2=0x7fa7240ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 -- 172.21.15.137:0/3577457436 shutdown_connections 2024-03-26T13:54:51.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.849+0000 7fa72ae00700 1 -- 172.21.15.137:0/3577457436 wait complete. 2024-03-26T13:54:51.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 Processor -- start 2024-03-26T13:54:51.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 -- start start 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7241021d0 0x7fa72419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa72419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa724103d70 0x7fa7241a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72410dcb0 con 0x7fa7241033d0 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72410db30 con 0x7fa7241021d0 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.851+0000 7fa72ae00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72410de30 con 0x7fa724103d70 2024-03-26T13:54:51.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa72419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa72419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48918/0 (socket says 172.21.15.137:48918) 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 -- 172.21.15.137:0/2793836494 learned_addr learned my addr 172.21.15.137:0/2793836494 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa723e00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7241021d0 0x7fa72419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa728e00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa724103d70 0x7fa7241a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 -- 172.21.15.137:0/2793836494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa724103d70 msgr2=0x7fa7241a1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa724103d70 0x7fa7241a1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 -- 172.21.15.137:0/2793836494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7241021d0 msgr2=0x7fa72419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7241021d0 0x7fa72419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.852+0000 7fa71b400700 1 -- 172.21.15.137:0/2793836494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7241a2140 con 0x7fa7241033d0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.853+0000 7fa71b400700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa72419aa20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009990 tx=0x7fa70c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.853+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c027070 con 0x7fa7241033d0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.853+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7241a23d0 con 0x7fa7241033d0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.853+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa70c022470 con 0x7fa7241033d0 2024-03-26T13:54:51.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.853+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00f6c0 con 0x7fa7241033d0 2024-03-26T13:54:51.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.853+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7241a2880 con 0x7fa7241033d0 2024-03-26T13:54:51.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.854+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa724068e50 con 0x7fa7241033d0 2024-03-26T13:54:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.856+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa70c02a070 con 0x7fa7241033d0 2024-03-26T13:54:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.858+0000 7fa721600700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7080a2910 0x7fa7080a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:54:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.858+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fa70c0cf320 con 0x7fa7241033d0 2024-03-26T13:54:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.858+0000 7fa723e00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7080a2910 0x7fa7080a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:54:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.859+0000 7fa723e00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7080a2910 0x7fa7080a4dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7140060b0 tx=0x7fa714007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:54:51.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:51.861+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa70c031070 con 0x7fa7241033d0 2024-03-26T13:54:52.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:54:52.094+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7241a2ba0 con 0x7fa7080a2910 2024-03-26T13:54:52.414 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:52 smithi137 ceph-mon[31539]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:54:52.414 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:54:52.414 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:54:52.414 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:52 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:52.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:52 smithi179 ceph-mon[35472]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:54:52.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:52 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:54:52.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:52 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:54:52.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:52 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:52.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:52 smithi159 ceph-mon[35501]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:54:52.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:52 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:54:52.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:52 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:54:52.696 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:52 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:54:53.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:53 smithi137 ceph-mon[31539]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-26T13:54:53.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:53 smithi179 ceph-mon[35472]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-26T13:54:53.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:53 smithi159 ceph-mon[35501]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-03-26T13:54:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:56 smithi179 ceph-mon[35472]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T13:54:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1332517099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9"}]: dispatch 2024-03-26T13:54:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1332517099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9"}]': finished 2024-03-26T13:54:56.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:56 smithi179 ceph-mon[35472]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T13:54:56.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:54:56.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:56 smithi159 ceph-mon[35501]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T13:54:56.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1332517099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9"}]: dispatch 2024-03-26T13:54:56.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1332517099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9"}]': finished 2024-03-26T13:54:56.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:56 smithi159 ceph-mon[35501]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T13:54:56.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:54:56.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:56 smithi137 ceph-mon[31539]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T13:54:56.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1332517099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9"}]: dispatch 2024-03-26T13:54:56.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1332517099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9"}]': finished 2024-03-26T13:54:56.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:56 smithi137 ceph-mon[31539]: osdmap e44: 7 total, 6 up, 7 in 2024-03-26T13:54:56.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:54:57.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:57 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2141225908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:54:57.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:57 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2141225908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:54:57.527 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:57 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2141225908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:54:58.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:54:58 smithi179 ceph-mon[35472]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-26T13:54:58.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:54:58 smithi159 ceph-mon[35501]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-26T13:54:58.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:54:58 smithi137 ceph-mon[31539]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-26T13:55:00.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:00 smithi179 ceph-mon[35472]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-26T13:55:00.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:00 smithi159 ceph-mon[35501]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-26T13:55:00.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:00 smithi137 ceph-mon[31539]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-26T13:55:02.320 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:02 smithi137 ceph-mon[31539]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:55:02.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:02 smithi179 ceph-mon[35472]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:55:02.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:02 smithi159 ceph-mon[35501]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:55:04.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:04 smithi137 ceph-mon[31539]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:04.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:04 smithi179 ceph-mon[35472]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:04.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:04 smithi159 ceph-mon[35501]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:06.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:06 smithi179 ceph-mon[35472]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:06.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T13:55:06.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:06.440 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:06 smithi137 ceph-mon[31539]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:06.440 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T13:55:06.440 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:06.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:06 smithi159 ceph-mon[35501]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:06.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-26T13:55:06.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:07.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:07 smithi179 ceph-mon[35472]: Deploying daemon osd.6 on smithi137 2024-03-26T13:55:07.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:07 smithi159 ceph-mon[35501]: Deploying daemon osd.6 on smithi137 2024-03-26T13:55:07.479 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:07 smithi137 ceph-mon[31539]: Deploying daemon osd.6 on smithi137 2024-03-26T13:55:08.228 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:08 smithi137 ceph-mon[31539]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:08.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:08 smithi179 ceph-mon[35472]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:08.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:08 smithi159 ceph-mon[35501]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:10.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:10 smithi179 ceph-mon[35472]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:10.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:10 smithi159 ceph-mon[35501]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:10.463 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:10 smithi137 ceph-mon[31539]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:12.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:12 smithi137 ceph-mon[31539]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:12.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:12 smithi179 ceph-mon[35472]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:12.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:12 smithi159 ceph-mon[35501]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:14.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:14 smithi179 ceph-mon[35472]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:14.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:14.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:14.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:14 smithi159 ceph-mon[35501]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:14.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:14.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:14 smithi137 ceph-mon[31539]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:14.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:14.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:14.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:16.350 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:16 smithi137 ceph-mon[31539]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:16.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:16 smithi179 ceph-mon[35472]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:16.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:16 smithi159 ceph-mon[35501]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:18.278 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:18.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:18.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:18.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:18 smithi137 ceph-mon[31539]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:18 smithi179 ceph-mon[35472]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T13:55:18.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:18.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:18.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:18.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:18.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:18 smithi159 ceph-mon[35501]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-26T13:55:18.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.883+0000 7fa721600700 1 -- 172.21.15.137:0/2793836494 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa7241a2ba0 con 0x7fa7080a2910 2024-03-26T13:55:18.877 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 6 on host 'smithi137' 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7080a2910 msgr2=0x7fa7080a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7080a2910 0x7fa7080a4dd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7140060b0 tx=0x7fa714007b80 comp rx=0 tx=0).stop 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 msgr2=0x7fa72419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa72419aa20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009990 tx=0x7fa70c0047b0 comp rx=0 tx=0).stop 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 shutdown_connections 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7080a2910 0x7fa7080a4dd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa724103d70 0x7fa7241a1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7241021d0 0x7fa72419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 --2- 172.21.15.137:0/2793836494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7241033d0 0x7fa72419aa20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.886+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 >> 172.21.15.137:0/2793836494 conn(0x7fa7240fd730 msgr2=0x7fa7240743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:55:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.887+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 shutdown_connections 2024-03-26T13:55:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:55:18.887+0000 7fa72ae00700 1 -- 172.21.15.137:0/2793836494 wait complete. 2024-03-26T13:55:19.709 DEBUG:teuthology.orchestra.run.smithi137:osd.6> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.6.service 2024-03-26T13:55:19.714 INFO:tasks.cephadm:Deploying osd.7 on smithi159 with /dev/vg_nvme/lv_2... 2024-03-26T13:55:19.714 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T13:55:19.930 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:19.931 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:19 smithi137 ceph-mon[31539]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:20.140 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:19 smithi179 ceph-mon[35472]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: osdmap e45: 7 total, 6 up, 7 in 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:20.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:19 smithi159 ceph-mon[35501]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:20.530 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T13:55:20.351+0000 7f153a800700 -1 osd.6 0 waiting for initial osdmap 2024-03-26T13:55:20.530 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T13:55:20.375+0000 7f1530600700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:55:20.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:20 smithi159 ceph-mon[35501]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:55:20.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:20 smithi159 ceph-mon[35501]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T13:55:20.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:20.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:20.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:20 smithi159 ceph-mon[35501]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' 2024-03-26T13:55:21.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-mon[31539]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:55:21.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-mon[31539]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T13:55:21.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:21.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:21.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:20 smithi137 ceph-mon[31539]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' 2024-03-26T13:55:21.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:20 smithi179 ceph-mon[35472]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:55:21.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:20 smithi179 ceph-mon[35472]: osdmap e46: 7 total, 6 up, 7 in 2024-03-26T13:55:21.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:21.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:21.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:20 smithi179 ceph-mon[35472]: from='osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149]' entity='osd.6' 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] boot 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T13:55:22.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] boot 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T13:55:22.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:22.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:55:22.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:55:22.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:22.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-03-26T13:55:22.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: osd.6 [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] boot 2024-03-26T13:55:22.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: osdmap e47: 7 total, 7 up, 7 in 2024-03-26T13:55:22.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0319423 s, 328 MB/s 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 2024-03-26T13:55:23.141 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-26T13:55:23.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:23 smithi159 ceph-mon[35501]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T13:55:23.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:23 smithi159 ceph-mon[35501]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:24.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:23 smithi137 ceph-mon[31539]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T13:55:24.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:23 smithi137 ceph-mon[31539]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:24.082 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-03-26T13:55:24.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:23 smithi179 ceph-mon[35472]: osdmap e48: 7 total, 7 up, 7 in 2024-03-26T13:55:24.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:23 smithi179 ceph-mon[35472]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:24.511 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:55:25.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi137 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi137 to 5697M 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:25 smithi179 ceph-mon[35472]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi137 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi137 to 5697M 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:25.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:25.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:25 smithi159 ceph-mon[35501]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi137 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi137 to 5697M 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:25.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:25 smithi137 ceph-mon[31539]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 -- 172.21.15.159:0/532474135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105610 msgr2=0x7f6074105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 --2- 172.21.15.159:0/532474135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105610 0x7f6074105a70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f60600099c0 tx=0x7f6060009cd0 comp rx=0 tx=0).stop 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 -- 172.21.15.159:0/532474135 shutdown_connections 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 --2- 172.21.15.159:0/532474135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6074105fb0 0x7f607410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 --2- 172.21.15.159:0/532474135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074104410 0x7f6074104830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 --2- 172.21.15.159:0/532474135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105610 0x7f6074105a70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:26.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.793+0000 7f607ae00700 1 -- 172.21.15.159:0/532474135 >> 172.21.15.159:0/532474135 conn(0x7f60740ff9b0 msgr2=0x7f6074101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:55:26.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.794+0000 7f607ae00700 1 -- 172.21.15.159:0/532474135 shutdown_connections 2024-03-26T13:55:26.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.794+0000 7f607ae00700 1 -- 172.21.15.159:0/532474135 wait complete. 2024-03-26T13:55:26.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 Processor -- start 2024-03-26T13:55:26.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 -- start start 2024-03-26T13:55:26.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6074104410 0x7f607419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:55:26.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074105610 0x7f607419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:55:26.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 0x7f60741a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:55:26.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607410fee0 con 0x7f6074105610 2024-03-26T13:55:26.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607410fd60 con 0x7f6074105fb0 2024-03-26T13:55:26.789 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f607ae00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6074110060 con 0x7f6074104410 2024-03-26T13:55:26.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.795+0000 7f6078e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 0x7f60741a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:55:26.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 0x7f60741a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:39730/0 (socket says 172.21.15.159:39730) 2024-03-26T13:55:26.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 -- 172.21.15.159:0/3873125680 learned_addr learned my addr 172.21.15.159:0/3873125680 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-26T13:55:26.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6073e00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6074104410 0x7f607419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:55:26.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6073400700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074105610 0x7f607419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:55:26.790 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 -- 172.21.15.159:0/3873125680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6074104410 msgr2=0x7f607419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6074104410 0x7f607419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 -- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074105610 msgr2=0x7f607419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074105610 0x7f607419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 -- 172.21.15.159:0/3873125680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60741a21b0 con 0x7f6074105fb0 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6073400700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074105610 0x7f607419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.796+0000 7f6078e00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 0x7f60741a1b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f606c00ada0 tx=0x7f606c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.797+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c017020 con 0x7f6074105fb0 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.797+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60741a24a0 con 0x7f6074105fb0 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.797+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f606c00ce30 con 0x7f6074105fb0 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.797+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0053a0 con 0x7f6074105fb0 2024-03-26T13:55:26.791 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.797+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60741a2990 con 0x7f6074105fb0 2024-03-26T13:55:26.792 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.798+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6074068e50 con 0x7f6074105fb0 2024-03-26T13:55:26.794 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.800+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f606c00c980 con 0x7f6074105fb0 2024-03-26T13:55:26.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.802+0000 7f6070c00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60580a2890 0x7f60580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:55:26.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.802+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f606c0cdab0 con 0x7f6074105fb0 2024-03-26T13:55:26.795 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.802+0000 7f6073e00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60580a2890 0x7f60580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:55:26.796 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.803+0000 7f6073e00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60580a2890 0x7f60580a4d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6074105470 tx=0x7f606400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:55:26.798 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:26.805+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f606c021020 con 0x7f6074105fb0 2024-03-26T13:55:27.030 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:27.037+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6074062690 con 0x7f60580a2890 2024-03-26T13:55:27.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:55:27.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:55:27.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:27.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:55:27.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:55:27.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:27.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:55:27.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:55:27.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:28.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:28 smithi159 ceph-mon[35501]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:28.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:28 smithi159 ceph-mon[35501]: from='client.34253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:55:28.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:28 smithi179 ceph-mon[35472]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:28.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:28 smithi179 ceph-mon[35472]: from='client.34253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:55:28.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:28 smithi137 ceph-mon[31539]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:28.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:28 smithi137 ceph-mon[31539]: from='client.34253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:55:30.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:30 smithi179 ceph-mon[35472]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:30.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:30 smithi159 ceph-mon[35501]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:30.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:30 smithi137 ceph-mon[31539]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/2392564886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]: dispatch 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]: dispatch 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]': finished 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:32.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/2272892953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:55:32.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:32.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/2392564886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]: dispatch 2024-03-26T13:55:32.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]: dispatch 2024-03-26T13:55:32.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]': finished 2024-03-26T13:55:32.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T13:55:32.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:32.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/2272892953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/2392564886' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]: dispatch 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]: dispatch 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54ae5a2e-89d5-4a31-801f-0d04c514f63c"}]': finished 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: osdmap e49: 8 total, 7 up, 8 in 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:32.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/2272892953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:55:34.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:34 smithi179 ceph-mon[35472]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:34.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:34 smithi159 ceph-mon[35501]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:34.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:34 smithi137 ceph-mon[31539]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:36.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:36 smithi179 ceph-mon[35472]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:36.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:36 smithi159 ceph-mon[35501]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:36.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:36 smithi137 ceph-mon[31539]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:38.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:38 smithi179 ceph-mon[35472]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:38.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:38 smithi159 ceph-mon[35501]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:38.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:38 smithi137 ceph-mon[31539]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:40.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:40 smithi179 ceph-mon[35472]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:40.423 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:40 smithi159 ceph-mon[35501]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:40.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:40 smithi137 ceph-mon[31539]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:41.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T13:55:41.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:41.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T13:55:41.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:41.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-26T13:55:41.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:42.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:42 smithi179 ceph-mon[35472]: Deploying daemon osd.7 on smithi159 2024-03-26T13:55:42.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:42 smithi179 ceph-mon[35472]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:42.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:42 smithi159 ceph-mon[35501]: Deploying daemon osd.7 on smithi159 2024-03-26T13:55:42.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:42 smithi159 ceph-mon[35501]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:42.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:42 smithi137 ceph-mon[31539]: Deploying daemon osd.7 on smithi159 2024-03-26T13:55:42.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:42 smithi137 ceph-mon[31539]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:44.416 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:44 smithi159 ceph-mon[35501]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:44.417 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:44 smithi179 ceph-mon[35472]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:44.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:44 smithi137 ceph-mon[31539]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:46.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:46 smithi179 ceph-mon[35472]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:46.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:46 smithi159 ceph-mon[35501]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:46.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:46 smithi137 ceph-mon[31539]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:48.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:48 smithi179 ceph-mon[35472]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:48.446 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:48 smithi159 ceph-mon[35501]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:48.528 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:48 smithi137 ceph-mon[31539]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:50.367 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:50 smithi159 ceph-mon[35501]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:50.367 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:50.367 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:50.367 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:50.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:50 smithi179 ceph-mon[35472]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:50.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:50.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:50.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:50 smithi137 ceph-mon[31539]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:50.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:52.158 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:52 smithi159 ceph-mon[35501]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:52.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:52 smithi179 ceph-mon[35472]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:52.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:52 smithi137 ceph-mon[31539]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:53.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:53.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:53.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:53.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:53.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:53.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:53.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T13:55:53.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:53 smithi159 ceph-mon[35501]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T13:55:54.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:54.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:54.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:54.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:54.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:54.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:54.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T13:55:54.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:53 smithi137 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T13:55:54.090 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.095+0000 7f6070c00700 1 -- 172.21.15.159:0/3873125680 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6074062690 con 0x7f60580a2890 2024-03-26T13:55:54.091 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 7 on host 'smithi159' 2024-03-26T13:55:54.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60580a2890 msgr2=0x7f60580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:54.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60580a2890 0x7f60580a4d50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6074105470 tx=0x7f606400b410 comp rx=0 tx=0).stop 2024-03-26T13:55:54.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 msgr2=0x7f60741a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:55:54.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 0x7f60741a1b10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f606c00ada0 tx=0x7f606c00c330 comp rx=0 tx=0).stop 2024-03-26T13:55:54.094 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 shutdown_connections 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60580a2890 0x7f60580a4d50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6074105fb0 0x7f60741a1b10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6074104410 0x7f607419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 --2- 172.21.15.159:0/3873125680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6074105610 0x7f607419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.099+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 >> 172.21.15.159:0/3873125680 conn(0x7f60740ff9b0 msgr2=0x7f60741089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.100+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 shutdown_connections 2024-03-26T13:55:54.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:55:54.100+0000 7f607ae00700 1 -- 172.21.15.159:0/3873125680 wait complete. 2024-03-26T13:55:54.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:54.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:54.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:55:54.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:55:54.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:54.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:54.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T13:55:54.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:53 smithi179 ceph-mon[35472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-26T13:55:54.986 DEBUG:teuthology.orchestra.run.smithi159:osd.7> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.7.service 2024-03-26T13:55:54.990 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2024-03-26T13:55:54.991 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-26T13:55:55.059 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T13:55:55.059 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T13:55:55.060 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:55.060 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:55:55.060 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:55:55.060 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:55.060 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:55.060 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:55.080 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T13:55:55.080 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T13:55:55.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:55.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:55:55.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:55:55.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:55.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:55.081 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:55.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-26T13:55:55.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: osdmap e50: 8 total, 7 up, 8 in 2024-03-26T13:55:55.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:55.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:55:55.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:55:55.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:55.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:55:55.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:55:55.424 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:55:55.697 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T13:55:55.439+0000 7f0148c00700 -1 osd.7 0 waiting for initial osdmap 2024-03-26T13:55:55.698 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T13:55:55.474+0000 7f013e000700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:55:55.882 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:55 smithi179 ceph-mon[35472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:55:55.882 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:55 smithi179 ceph-mon[35472]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T13:55:55.882 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:55.883 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:55.883 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:55 smithi179 ceph-mon[35472]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:55.883 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:55 smithi179 ceph-mon[35472]: from='osd.7 ' entity='osd.7' 2024-03-26T13:55:56.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-mon[35501]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:55:56.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-mon[35501]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T13:55:56.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:56.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:56.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-mon[35501]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:56.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:55 smithi159 ceph-mon[35501]: from='osd.7 ' entity='osd.7' 2024-03-26T13:55:56.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:55 smithi137 ceph-mon[31539]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:55:56.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:55 smithi137 ceph-mon[31539]: osdmap e51: 8 total, 7 up, 8 in 2024-03-26T13:55:56.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:56.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:56.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:55 smithi137 ceph-mon[31539]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-03-26T13:55:56.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:55 smithi137 ceph-mon[31539]: from='osd.7 ' entity='osd.7' 2024-03-26T13:55:57.033 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:56 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:55:57.033 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:56 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:55:57.033 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:57.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:56 smithi159 ceph-mon[35501]: osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] boot 2024-03-26T13:55:57.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:56 smithi159 ceph-mon[35501]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T13:55:57.034 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:56 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:57.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:56 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:55:57.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:56 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:55:57.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:57.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:56 smithi179 ceph-mon[35472]: osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] boot 2024-03-26T13:55:57.167 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:56 smithi179 ceph-mon[35472]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T13:55:57.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:56 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:57.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:56 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:55:57.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:56 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:55:57.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:57.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:56 smithi137 ceph-mon[31539]: osd.7 [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] boot 2024-03-26T13:55:57.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:56 smithi137 ceph-mon[31539]: osdmap e52: 8 total, 8 up, 8 in 2024-03-26T13:55:57.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:56 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-26T13:55:58.036 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:57 smithi179 ceph-mon[35472]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T13:55:58.036 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:57 smithi179 ceph-mon[35472]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:55:58.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:57 smithi159 ceph-mon[35501]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T13:55:58.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:57 smithi159 ceph-mon[35501]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:55:58.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:57 smithi137 ceph-mon[31539]: osdmap e53: 8 total, 8 up, 8 in 2024-03-26T13:55:58.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:57 smithi137 ceph-mon[31539]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:55:58.308 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-26T13:55:58.308 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-26T13:55:58.308 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-03-26T13:55:58.308 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-03-26T13:55:58.308 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0315963 s, 332 MB/s 2024-03-26T13:55:58.308 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-03-26T13:55:58.980 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:58 smithi179 ceph-mon[35472]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T13:55:58.981 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:58 smithi179 ceph-mon[35472]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T13:55:59.119 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:58 smithi159 ceph-mon[35501]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T13:55:59.119 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:58 smithi159 ceph-mon[35501]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T13:55:59.134 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-03-26T13:55:59.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:58 smithi137 ceph-mon[31539]: osdmap e54: 8 total, 8 up, 8 in 2024-03-26T13:55:59.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:58 smithi137 ceph-mon[31539]: osdmap e55: 8 total, 8 up, 8 in 2024-03-26T13:55:59.575 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:56:00.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:59 smithi179 ceph-mon[35472]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:00.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:55:59 smithi179 ceph-mon[35472]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T13:56:00.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:59 smithi159 ceph-mon[35501]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:00.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:55:59 smithi159 ceph-mon[35501]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T13:56:00.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:59 smithi137 ceph-mon[31539]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:00.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:55:59 smithi137 ceph-mon[31539]: osdmap e56: 8 total, 8 up, 8 in 2024-03-26T13:56:00.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T13:56:00.904 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi159 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi159 to 5697M 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:00.905 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:00 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T13:56:01.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi159 2024-03-26T13:56:01.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi159 to 5697M 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:01.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:00 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T13:56:01.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi159 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi159 to 5697M 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:01.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:00 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:01.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 -- 172.21.15.179:0/353441797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 msgr2=0x7f0b24106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:01.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 --2- 172.21.15.179:0/353441797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b24106450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009990 tx=0x7f0b1c009ca0 comp rx=0 tx=0).stop 2024-03-26T13:56:01.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 -- 172.21.15.179:0/353441797 shutdown_connections 2024-03-26T13:56:01.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 --2- 172.21.15.179:0/353441797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b24106450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:01.695 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 --2- 172.21.15.179:0/353441797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b24100480 0x7f0b241008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:01.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 --2- 172.21.15.179:0/353441797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 0x7f0b24101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:01.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.700+0000 7f0b2b400700 1 -- 172.21.15.179:0/353441797 >> 172.21.15.179:0/353441797 conn(0x7f0b240fbe00 msgr2=0x7f0b240fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:56:01.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.701+0000 7f0b2b400700 1 -- 172.21.15.179:0/353441797 shutdown_connections 2024-03-26T13:56:01.696 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.701+0000 7f0b2b400700 1 -- 172.21.15.179:0/353441797 wait complete. 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 Processor -- start 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 -- start start 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b24100480 0x7f0b2419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 0x7f0b2419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b241a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b24114320 con 0x7f0b241011f0 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241141a0 con 0x7f0b24100480 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.702+0000 7f0b2b400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b241144a0 con 0x7f0b24101bb0 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b241a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b241a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:44016/0 (socket says 172.21.15.179:44016) 2024-03-26T13:56:01.697 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 -- 172.21.15.179:0/833579889 learned_addr learned my addr 172.21.15.179:0/833579889 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:56:01.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b23e00700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 0x7f0b2419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:01.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b28a00700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b24100480 0x7f0b2419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:01.698 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 -- 172.21.15.179:0/833579889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b24100480 msgr2=0x7f0b2419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b24100480 0x7f0b2419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 -- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 msgr2=0x7f0b2419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 0x7f0b2419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b29400700 1 -- 172.21.15.179:0/833579889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b241a64d0 con 0x7f0b24101bb0 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.703+0000 7f0b23e00700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 0x7f0b2419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.704+0000 7f0b29400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b241a5e30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c006010 tx=0x7f0b1c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.704+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c003d20 con 0x7f0b24101bb0 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.704+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0b1c004830 con 0x7f0b24101bb0 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.704+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b241a6760 con 0x7f0b24101bb0 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.704+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0055b0 con 0x7f0b24101bb0 2024-03-26T13:56:01.699 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.704+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b241a6b50 con 0x7f0b24101bb0 2024-03-26T13:56:01.700 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.705+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b24068e50 con 0x7f0b24101bb0 2024-03-26T13:56:01.701 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.708+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0b1c003ec0 con 0x7f0b24101bb0 2024-03-26T13:56:01.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.709+0000 7f0b21600700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0b080a2890 0x7f0b080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:01.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.709+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f0b1c0cef90 con 0x7f0b24101bb0 2024-03-26T13:56:01.703 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.710+0000 7f0b28a00700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0b080a2890 0x7f0b080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:01.704 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.710+0000 7f0b28a00700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0b080a2890 0x7f0b080a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009d10 tx=0x7f0b14009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:56:01.707 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.714+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b1c013070 con 0x7f0b24101bb0 2024-03-26T13:56:01.940 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:01.945+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b240625d0 con 0x7f0b080a2890 2024-03-26T13:56:02.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:01 smithi179 ceph-mon[35472]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:02.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:01 smithi159 ceph-mon[35501]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:02.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:01 smithi137 ceph-mon[31539]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:03.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:02 smithi159 ceph-mon[35501]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:56:03.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:56:03.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:56:03.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:03.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:02 smithi179 ceph-mon[35472]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:56:03.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:56:03.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:56:03.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:03.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:02 smithi137 ceph-mon[31539]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:56:03.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:56:03.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:56:03.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:03.911 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:03 smithi179 ceph-mon[35472]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:04.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:03 smithi159 ceph-mon[35501]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:04.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:03 smithi137 ceph-mon[31539]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:05.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:04 smithi179 ceph-mon[35472]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T13:56:05.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:04 smithi179 ceph-mon[35472]: Cluster is now healthy 2024-03-26T13:56:05.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:04 smithi159 ceph-mon[35501]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T13:56:05.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:04 smithi159 ceph-mon[35501]: Cluster is now healthy 2024-03-26T13:56:05.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:04 smithi137 ceph-mon[31539]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-03-26T13:56:05.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:04 smithi137 ceph-mon[31539]: Cluster is now healthy 2024-03-26T13:56:06.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:05 smithi179 ceph-mon[35472]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:06.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:05 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/2178515368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]: dispatch 2024-03-26T13:56:06.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:05 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]: dispatch 2024-03-26T13:56:06.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:05 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]': finished 2024-03-26T13:56:06.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:05 smithi179 ceph-mon[35472]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T13:56:06.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:06.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:05 smithi159 ceph-mon[35501]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:06.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:05 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/2178515368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]: dispatch 2024-03-26T13:56:06.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:05 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]: dispatch 2024-03-26T13:56:06.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:05 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]': finished 2024-03-26T13:56:06.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:05 smithi159 ceph-mon[35501]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T13:56:06.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:06.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:05 smithi137 ceph-mon[31539]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:06.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:05 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/2178515368' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]: dispatch 2024-03-26T13:56:06.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:05 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]: dispatch 2024-03-26T13:56:06.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:05 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "54d0b6d1-2751-465f-89fb-d866d56a94e1"}]': finished 2024-03-26T13:56:06.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:05 smithi137 ceph-mon[31539]: osdmap e57: 9 total, 8 up, 9 in 2024-03-26T13:56:06.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:07.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:06 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/2656385877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:56:07.196 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/2656385877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:56:07.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:06 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/2656385877' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:56:08.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:07 smithi179 ceph-mon[35472]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:08.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:07 smithi159 ceph-mon[35501]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:08.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:07 smithi137 ceph-mon[31539]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:10.364 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:10 smithi179 ceph-mon[35472]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:10.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:10 smithi159 ceph-mon[35501]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:10.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:10 smithi137 ceph-mon[31539]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:12.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:12 smithi179 ceph-mon[35472]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:12.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:12 smithi159 ceph-mon[35501]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:12.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:12 smithi137 ceph-mon[31539]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:14.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:14 smithi179 ceph-mon[35472]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:14.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:14 smithi159 ceph-mon[35501]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:14.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:14 smithi137 ceph-mon[31539]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:15.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T13:56:15.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:15.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T13:56:15.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:15.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-26T13:56:15.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:16.292 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:16 smithi179 ceph-mon[35472]: Deploying daemon osd.8 on smithi179 2024-03-26T13:56:16.292 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:16 smithi179 ceph-mon[35472]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:16 smithi159 ceph-mon[35501]: Deploying daemon osd.8 on smithi179 2024-03-26T13:56:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:16 smithi159 ceph-mon[35501]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:16.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:16 smithi137 ceph-mon[31539]: Deploying daemon osd.8 on smithi179 2024-03-26T13:56:16.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:16 smithi137 ceph-mon[31539]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:18.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:18 smithi179 ceph-mon[35472]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:18.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:18 smithi159 ceph-mon[35501]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:18.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:18 smithi137 ceph-mon[31539]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:20.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:20 smithi179 ceph-mon[35472]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:20.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:20 smithi159 ceph-mon[35501]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:20.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:20 smithi137 ceph-mon[31539]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:22.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:22 smithi179 ceph-mon[35472]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:22.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:22 smithi159 ceph-mon[35501]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:22.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:22 smithi137 ceph-mon[31539]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:24.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:24 smithi179 ceph-mon[35472]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:24.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:24 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:24.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:24 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:24.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:24 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:24.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:24 smithi159 ceph-mon[35501]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:24.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:24 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:24.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:24 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:24.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:24 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:24.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:24 smithi137 ceph-mon[31539]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:24.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:24 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:24.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:24 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:24.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:24 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:26.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:26 smithi179 ceph-mon[35472]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:26.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:26 smithi159 ceph-mon[35501]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:26.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:26 smithi137 ceph-mon[31539]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:28.261 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:28 smithi179 ceph-mon[35472]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:28.261 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:28.261 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:28.262 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:28 smithi179 ceph-mon[35472]: from='osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T13:56:28.262 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:28 smithi179 ceph-mon[35472]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T13:56:28.410 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2024-03-26T13:56:28.410 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.414+0000 7f0b21600700 1 -- 172.21.15.179:0/833579889 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0b240625d0 con 0x7f0b080a2890 2024-03-26T13:56:28.411 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0b080a2890 msgr2=0x7f0b080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0b080a2890 0x7f0b080a4d50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009d10 tx=0x7f0b14009040 comp rx=0 tx=0).stop 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 msgr2=0x7f0b241a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b241a5e30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c006010 tx=0x7f0b1c0047b0 comp rx=0 tx=0).stop 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 shutdown_connections 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0b080a2890 0x7f0b080a4d50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b24101bb0 0x7f0b241a5e30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b24100480 0x7f0b2419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 --2- 172.21.15.179:0/833579889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b241011f0 0x7f0b2419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 >> 172.21.15.179:0/833579889 conn(0x7f0b240fbe00 msgr2=0x7f0b24102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 shutdown_connections 2024-03-26T13:56:28.412 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:56:28.417+0000 7f0b2b400700 1 -- 172.21.15.179:0/833579889 wait complete. 2024-03-26T13:56:28.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:28 smithi159 ceph-mon[35501]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:28.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:28.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:28.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:28 smithi159 ceph-mon[35501]: from='osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T13:56:28.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:28 smithi159 ceph-mon[35501]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T13:56:28.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:28 smithi137 ceph-mon[31539]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:28.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:28.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:28.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:28 smithi137 ceph-mon[31539]: from='osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T13:56:28.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:28 smithi137 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-26T13:56:29.241 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.8.service 2024-03-26T13:56:29.249 INFO:tasks.cephadm:Deploying osd.9 on smithi137 with /dev/vg_nvme/lv_1... 2024-03-26T13:56:29.249 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T13:56:29.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:29.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:29.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T13:56:29.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T13:56:29.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: from='osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:56:29.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:29.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:56:29.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:29 smithi179 ceph-mon[35472]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:29.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: from='osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:56:29.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:29 smithi159 ceph-mon[35501]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:29.701 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: osdmap e58: 9 total, 8 up, 9 in 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: from='osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:56:29.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:29 smithi137 ceph-mon[31539]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:30.398 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T13:56:30.038+0000 7f46d0c00700 -1 osd.8 0 waiting for initial osdmap 2024-03-26T13:56:30.398 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T13:56:30.070+0000 7f46c6a00700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:56:30.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:30 smithi137 ceph-mon[31539]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:56:30.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:30 smithi137 ceph-mon[31539]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T13:56:30.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:30.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:30.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:30 smithi137 ceph-mon[31539]: from='osd.8 ' entity='osd.8' 2024-03-26T13:56:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-mon[35472]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:56:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-mon[35472]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T13:56:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:30 smithi179 ceph-mon[35472]: from='osd.8 ' entity='osd.8' 2024-03-26T13:56:30.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:30 smithi159 ceph-mon[35501]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:56:30.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:30 smithi159 ceph-mon[35501]: osdmap e59: 9 total, 8 up, 9 in 2024-03-26T13:56:30.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:30.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:30.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:30 smithi159 ceph-mon[35501]: from='osd.8 ' entity='osd.8' 2024-03-26T13:56:31.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:56:31.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:56:31.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:31.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] boot 2024-03-26T13:56:31.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T13:56:31.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:31.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:31 smithi179 ceph-mon[35472]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:31.693 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:56:31.693 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:56:31.694 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:31.694 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] boot 2024-03-26T13:56:31.694 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T13:56:31.694 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:31.694 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:31 smithi137 ceph-mon[31539]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:31.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:56:31.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:56:31.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:31.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: osd.8 [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] boot 2024-03-26T13:56:31.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: osdmap e60: 9 total, 9 up, 9 in 2024-03-26T13:56:31.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-26T13:56:31.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:31 smithi159 ceph-mon[35501]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-26T13:56:32.697 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:32 smithi137 ceph-mon[31539]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T13:56:32.697 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T13:56:32.697 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T13:56:32.697 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-03-26T13:56:32.697 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-03-26T13:56:32.697 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0372502 s, 281 MB/s 2024-03-26T13:56:32.697 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-03-26T13:56:32.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:32 smithi159 ceph-mon[35501]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T13:56:32.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:32 smithi179 ceph-mon[35472]: osdmap e61: 9 total, 9 up, 9 in 2024-03-26T13:56:33.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-03-26T13:56:33.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:33 smithi137 ceph-mon[31539]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T13:56:33.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:33 smithi137 ceph-mon[31539]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:33.869 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:33 smithi179 ceph-mon[35472]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T13:56:33.869 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:33 smithi179 ceph-mon[35472]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:33.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:33 smithi159 ceph-mon[35501]: osdmap e62: 9 total, 9 up, 9 in 2024-03-26T13:56:33.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:33 smithi159 ceph-mon[35501]: pgmap v252: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:34.018 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:56:34.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T13:56:34.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.779 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:34.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:34.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: osdmap e63: 9 total, 9 up, 9 in 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:56:34.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:34.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:34.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:34.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi179 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi179 to 5697M 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:35.699 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:35 smithi137 ceph-mon[31539]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:35.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi179 2024-03-26T13:56:35.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi179 to 5697M 2024-03-26T13:56:35.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T13:56:35.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:35.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:35.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:35.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:35.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:35 smithi179 ceph-mon[35472]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi179 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi179 to 5697M 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: osdmap e64: 9 total, 9 up, 9 in 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:56:35.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:35.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:35 smithi159 ceph-mon[35501]: pgmap v255: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:36.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.156+0000 7ff284e00700 1 -- 172.21.15.137:0/3822739719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff280074bf0 msgr2=0x7ff280075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:36.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.156+0000 7ff284e00700 1 --2- 172.21.15.137:0/3822739719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff280074bf0 0x7ff280075070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009990 tx=0x7ff26c009ca0 comp rx=0 tx=0).stop 2024-03-26T13:56:36.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.156+0000 7ff284e00700 1 -- 172.21.15.137:0/3822739719 shutdown_connections 2024-03-26T13:56:36.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.156+0000 7ff284e00700 1 --2- 172.21.15.137:0/3822739719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff28010a0a0 0x7ff28010c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:36.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.156+0000 7ff284e00700 1 --2- 172.21.15.137:0/3822739719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2800761c0 0x7ff280074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:36.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.157+0000 7ff284e00700 1 --2- 172.21.15.137:0/3822739719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff280074bf0 0x7ff280075070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:36.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.157+0000 7ff284e00700 1 -- 172.21.15.137:0/3822739719 >> 172.21.15.137:0/3822739719 conn(0x7ff2800fd730 msgr2=0x7ff2800ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:56:36.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.157+0000 7ff284e00700 1 -- 172.21.15.137:0/3822739719 shutdown_connections 2024-03-26T13:56:36.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.157+0000 7ff284e00700 1 -- 172.21.15.137:0/3822739719 wait complete. 2024-03-26T13:56:36.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 Processor -- start 2024-03-26T13:56:36.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 -- start start 2024-03-26T13:56:36.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff280074bf0 0x7ff28019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:36.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 0x7ff28019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:36.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff28010a0a0 0x7ff2801a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28010fe80 con 0x7ff2800761c0 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28010fd00 con 0x7ff28010a0a0 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.158+0000 7ff284e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280110000 con 0x7ff280074bf0 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 0x7ff28019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 0x7ff28019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50214/0 (socket says 172.21.15.137:50214) 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 -- 172.21.15.137:0/2142980823 learned_addr learned my addr 172.21.15.137:0/2142980823 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27de00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff280074bf0 0x7ff28019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:36.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27e800700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff28010a0a0 0x7ff2801a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:36.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 -- 172.21.15.137:0/2142980823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff280074bf0 msgr2=0x7ff28019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:36.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff280074bf0 0x7ff28019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:36.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 -- 172.21.15.137:0/2142980823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff28010a0a0 msgr2=0x7ff2801a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:56:36.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff28010a0a0 0x7ff2801a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.159+0000 7ff27d400700 1 -- 172.21.15.137:0/2142980823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2801a2180 con 0x7ff2800761c0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.160+0000 7ff27d400700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 0x7ff28019aa60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000c00 tx=0x7ff26c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.160+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c027070 con 0x7ff2800761c0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.160+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2801a2410 con 0x7ff2800761c0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.160+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff26c004010 con 0x7ff2800761c0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.160+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c022cc0 con 0x7ff2800761c0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.160+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2801a2940 con 0x7ff2800761c0 2024-03-26T13:56:36.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.161+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff280068e50 con 0x7ff2800761c0 2024-03-26T13:56:36.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.163+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff26c02a070 con 0x7ff2800761c0 2024-03-26T13:56:36.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.165+0000 7ff276a00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2640a2910 0x7ff2640a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:56:36.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.165+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7ff26c0cf530 con 0x7ff2800761c0 2024-03-26T13:56:36.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.165+0000 7ff27de00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2640a2910 0x7ff2640a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:56:36.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.166+0000 7ff27de00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2640a2910 0x7ff2640a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2800765a0 tx=0x7ff27000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:56:36.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.168+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff26c031070 con 0x7ff2800761c0 2024-03-26T13:56:36.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:56:36.400+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2800625d0 con 0x7ff2640a2910 2024-03-26T13:56:37.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:56:37.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:56:37.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:36 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:37.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:56:37.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:56:37.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:36 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:37.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:56:37.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:56:37.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:36 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:38.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:37 smithi137 ceph-mon[31539]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:56:38.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:37 smithi137 ceph-mon[31539]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T13:56:38.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:37 smithi179 ceph-mon[35472]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:56:38.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:37 smithi179 ceph-mon[35472]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T13:56:38.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:37 smithi159 ceph-mon[35501]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:56:38.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:37 smithi159 ceph-mon[35501]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-26T13:56:40.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:40 smithi179 ceph-mon[35472]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T13:56:40.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:40 smithi159 ceph-mon[35501]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T13:56:40.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:40 smithi137 ceph-mon[31539]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-26T13:56:41.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:41 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1974375464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86070f8d-a3f6-4a0f-9936-8c52710b2e85"}]: dispatch 2024-03-26T13:56:41.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:41 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1974375464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86070f8d-a3f6-4a0f-9936-8c52710b2e85"}]': finished 2024-03-26T13:56:41.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:41 smithi179 ceph-mon[35472]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T13:56:41.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:56:41.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:41 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1974375464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86070f8d-a3f6-4a0f-9936-8c52710b2e85"}]: dispatch 2024-03-26T13:56:41.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:41 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1974375464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86070f8d-a3f6-4a0f-9936-8c52710b2e85"}]': finished 2024-03-26T13:56:41.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:41 smithi159 ceph-mon[35501]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T13:56:41.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:56:41.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1974375464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86070f8d-a3f6-4a0f-9936-8c52710b2e85"}]: dispatch 2024-03-26T13:56:41.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1974375464' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86070f8d-a3f6-4a0f-9936-8c52710b2e85"}]': finished 2024-03-26T13:56:41.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:41 smithi137 ceph-mon[31539]: osdmap e65: 10 total, 9 up, 10 in 2024-03-26T13:56:41.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:56:42.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:42 smithi179 ceph-mon[35472]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T13:56:42.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:42 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4277620929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:56:42.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:42 smithi159 ceph-mon[35501]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T13:56:42.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:42 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4277620929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:56:42.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:42 smithi137 ceph-mon[31539]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-26T13:56:42.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:42 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4277620929' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:56:44.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:44 smithi179 ceph-mon[35472]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-26T13:56:44.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:44 smithi159 ceph-mon[35501]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-26T13:56:44.467 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:44 smithi137 ceph-mon[31539]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-26T13:56:46.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:46 smithi179 ceph-mon[35472]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:56:46.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:46 smithi159 ceph-mon[35501]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:56:46.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:46 smithi137 ceph-mon[31539]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:56:48.347 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:48 smithi137 ceph-mon[31539]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:48.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:48 smithi179 ceph-mon[35472]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:48.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:48 smithi159 ceph-mon[35501]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:49.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:49 smithi137 ceph-mon[31539]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:49.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:49 smithi179 ceph-mon[35472]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:49.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:49 smithi159 ceph-mon[35501]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:50.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T13:56:50.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:50 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:50.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:50 smithi137 ceph-mon[31539]: Deploying daemon osd.9 on smithi137 2024-03-26T13:56:50.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T13:56:50.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:50 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:50.919 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:50 smithi179 ceph-mon[35472]: Deploying daemon osd.9 on smithi137 2024-03-26T13:56:50.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-26T13:56:50.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:50 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:56:50.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:50 smithi159 ceph-mon[35501]: Deploying daemon osd.9 on smithi137 2024-03-26T13:56:51.869 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:51 smithi137 ceph-mon[31539]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:51.918 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:51 smithi179 ceph-mon[35472]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:51.947 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:51 smithi159 ceph-mon[35501]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:54.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:54 smithi137 ceph-mon[31539]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:54.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:54 smithi179 ceph-mon[35472]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:54.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:54 smithi159 ceph-mon[35501]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:56.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:56 smithi179 ceph-mon[35472]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:56.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:56 smithi159 ceph-mon[35501]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:56.460 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:56 smithi137 ceph-mon[31539]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:58.279 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:58 smithi137 ceph-mon[31539]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:58.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:58 smithi179 ceph-mon[35472]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:58.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:58 smithi159 ceph-mon[35501]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:59.397 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:59 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:59.397 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:59 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:59.397 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:59 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:59.397 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:56:59 smithi137 ceph-mon[31539]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:59.668 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:59 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:59.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:59 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:59.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:59 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:59.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:56:59 smithi179 ceph-mon[35472]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:56:59.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:59 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:59.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:59 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:56:59.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:59 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:56:59.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:56:59 smithi159 ceph-mon[35501]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:02.401 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:02 smithi159 ceph-mon[35501]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:02.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:02 smithi179 ceph-mon[35472]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:02.446 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:02 smithi137 ceph-mon[31539]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:03.192 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:03 smithi137 ceph-mon[31539]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T13:57:03.192 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:03 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:03.192 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:03 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:03.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.338+0000 7ff276a00700 1 -- 172.21.15.137:0/2142980823 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff2800625d0 con 0x7ff2640a2910 2024-03-26T13:57:03.333 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 9 on host 'smithi137' 2024-03-26T13:57:03.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2640a2910 msgr2=0x7ff2640a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:03.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2640a2910 0x7ff2640a4dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2800765a0 tx=0x7ff27000b3c0 comp rx=0 tx=0).stop 2024-03-26T13:57:03.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 msgr2=0x7ff28019aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:03.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 0x7ff28019aa60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000c00 tx=0x7ff26c009580 comp rx=0 tx=0).stop 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 shutdown_connections 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2640a2910 0x7ff2640a4dd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.341+0000 7ff284e00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff28010a0a0 0x7ff2801a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.342+0000 7ff284e00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff280074bf0 0x7ff28019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.342+0000 7ff284e00700 1 --2- 172.21.15.137:0/2142980823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2800761c0 0x7ff28019aa60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.342+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 >> 172.21.15.137:0/2142980823 conn(0x7ff2800fd730 msgr2=0x7ff280105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.342+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 shutdown_connections 2024-03-26T13:57:03.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:57:03.342+0000 7ff284e00700 1 -- 172.21.15.137:0/2142980823 wait complete. 2024-03-26T13:57:03.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:03 smithi179 ceph-mon[35472]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T13:57:03.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:03 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:03.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:03 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:03.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:03 smithi159 ceph-mon[35501]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-26T13:57:03.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:03 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:03.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:03 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.071 DEBUG:teuthology.orchestra.run.smithi137:osd.9> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.9.service 2024-03-26T13:57:04.075 INFO:tasks.cephadm:Deploying osd.10 on smithi159 with /dev/vg_nvme/lv_1... 2024-03-26T13:57:04.075 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:04 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:04 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:04.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-26T13:57:04.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: osdmap e66: 10 total, 9 up, 10 in 2024-03-26T13:57:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-03-26T13:57:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:04 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:04.521 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:57:05.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:05 smithi179 ceph-mon[35472]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:57:05.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:05 smithi179 ceph-mon[35472]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T13:57:05.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:05.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:05.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:05 smithi159 ceph-mon[35501]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:57:05.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:05 smithi159 ceph-mon[35501]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T13:57:05.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:05.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:05.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:05 smithi137 ceph-mon[31539]: from='osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-03-26T13:57:05.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:05 smithi137 ceph-mon[31539]: osdmap e67: 10 total, 9 up, 10 in 2024-03-26T13:57:05.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:05.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] boot 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T13:57:06.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:06 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] boot 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T13:57:06.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:06 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: osd.9 [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] boot 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: osdmap e68: 10 total, 10 up, 10 in 2024-03-26T13:57:06.424 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:06 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-26T13:57:07.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:07 smithi179 ceph-mon[35472]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T13:57:07.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:07 smithi159 ceph-mon[35501]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T13:57:07.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:07 smithi137 ceph-mon[31539]: osdmap e69: 10 total, 10 up, 10 in 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0348623 s, 301 MB/s 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 2024-03-26T13:57:07.531 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-03-26T13:57:08.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:08 smithi159 ceph-mon[35501]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:08.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:08 smithi137 ceph-mon[31539]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:08.416 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-03-26T13:57:08.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:08 smithi179 ceph-mon[35472]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:08.834 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T13:57:10.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi137 2024-03-26T13:57:10.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi137 to 4272M 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi137 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi137 to 4272M 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi137 2024-03-26T13:57:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi137 to 4272M 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 -- 172.21.15.159:0/2179970213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a0103d60 msgr2=0x7f40a010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 --2- 172.21.15.159:0/2179970213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a0103d60 0x7f40a010a610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f409800b0d0 tx=0x7f409800b3e0 comp rx=0 tx=0).stop 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 -- 172.21.15.159:0/2179970213 shutdown_connections 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 --2- 172.21.15.159:0/2179970213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a0103d60 0x7f40a010a610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 --2- 172.21.15.159:0/2179970213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40a01021c0 0x7f40a01025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 --2- 172.21.15.159:0/2179970213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40a01033c0 0x7f40a0103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:11.021 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 -- 172.21.15.159:0/2179970213 >> 172.21.15.159:0/2179970213 conn(0x7f40a00fd7a0 msgr2=0x7f40a00ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:57:11.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 -- 172.21.15.159:0/2179970213 shutdown_connections 2024-03-26T13:57:11.022 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.026+0000 7f40a4a00700 1 -- 172.21.15.159:0/2179970213 wait complete. 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.027+0000 7f40a4a00700 1 Processor -- start 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.027+0000 7f40a4a00700 1 -- start start 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f40a4a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40a01021c0 0x7f40a019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f40a4a00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 0x7f40a019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f40a4a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40a0103d60 0x7f40a01a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f40a4a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a010dca0 con 0x7f40a0103d60 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f40a4a00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a010db20 con 0x7f40a01033c0 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f40a4a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a010de20 con 0x7f40a01021c0 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 0x7f40a019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:11.024 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 0x7f40a019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:55174/0 (socket says 172.21.15.159:55174) 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 -- 172.21.15.159:0/171272685 learned_addr learned my addr 172.21.15.159:0/171272685 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409de00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40a01021c0 0x7f40a019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409e800700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40a0103d60 0x7f40a01a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 -- 172.21.15.159:0/171272685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40a01021c0 msgr2=0x7f40a019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40a01021c0 0x7f40a019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 -- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40a0103d60 msgr2=0x7f40a01a1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.028+0000 7f409d400700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40a0103d60 0x7f40a01a1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f409d400700 1 -- 172.21.15.159:0/171272685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a01a20c0 con 0x7f40a01033c0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f409d400700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 0x7f40a019a9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c9b0 tx=0x7f408c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c007b00 con 0x7f40a01033c0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f408c007ca0 con 0x7f40a01033c0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c0186f0 con 0x7f40a01033c0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a01a23b0 con 0x7f40a01033c0 2024-03-26T13:57:11.025 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.029+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a01a28f0 con 0x7f40a01033c0 2024-03-26T13:57:11.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.030+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0068e50 con 0x7f40a01033c0 2024-03-26T13:57:11.026 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.032+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f408c00f450 con 0x7f40a01033c0 2024-03-26T13:57:11.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.034+0000 7f4096a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40840a2890 0x7f40840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:11.028 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.034+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f408c0ce380 con 0x7f40a01033c0 2024-03-26T13:57:11.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.034+0000 7f409de00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40840a2890 0x7f40840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:11.029 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.035+0000 7f409de00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40840a2890 0x7f40840a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4090005950 tx=0x7f40900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:57:11.033 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.038+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f408c014030 con 0x7f40a01033c0 2024-03-26T13:57:11.266 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:11.271+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f40a0075ae0 con 0x7f40840a2890 2024-03-26T13:57:12.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:12 smithi179 ceph-mon[35472]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:12.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:57:12.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:57:12.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:12.427 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:12 smithi159 ceph-mon[35501]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:12.427 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:57:12.427 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:57:12.427 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:12 smithi137 ceph-mon[31539]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:57:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:57:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:13.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:13 smithi179 ceph-mon[35472]: from='client.34331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:57:13.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:13 smithi159 ceph-mon[35501]: from='client.34331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:57:13.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:13 smithi137 ceph-mon[31539]: from='client.34331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:57:14.348 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:14 smithi159 ceph-mon[35501]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:14.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:14 smithi179 ceph-mon[35472]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:14.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:14 smithi137 ceph-mon[31539]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:16 smithi179 ceph-mon[35472]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:16 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/1661005370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]: dispatch 2024-03-26T13:57:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:16 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]: dispatch 2024-03-26T13:57:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:16 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]': finished 2024-03-26T13:57:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:16 smithi179 ceph-mon[35472]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T13:57:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:16 smithi159 ceph-mon[35501]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:16 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/1661005370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]: dispatch 2024-03-26T13:57:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:16 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]: dispatch 2024-03-26T13:57:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:16 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]': finished 2024-03-26T13:57:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:16 smithi159 ceph-mon[35501]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T13:57:16.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:16.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:16 smithi137 ceph-mon[31539]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:16.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:16 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/1661005370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]: dispatch 2024-03-26T13:57:16.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:16 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]: dispatch 2024-03-26T13:57:16.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:16 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95b5ff77-4547-400f-96ef-142faa248b14"}]': finished 2024-03-26T13:57:16.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:16 smithi137 ceph-mon[31539]: osdmap e70: 11 total, 10 up, 11 in 2024-03-26T13:57:16.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:17.418 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:17 smithi179 ceph-mon[35472]: from='client.? 172.21.15.159:0/3165669039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:57:17.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:17 smithi159 ceph-mon[35501]: from='client.? 172.21.15.159:0/3165669039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:57:17.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:17 smithi137 ceph-mon[31539]: from='client.? 172.21.15.159:0/3165669039' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:57:18.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:18 smithi179 ceph-mon[35472]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:18.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:18 smithi159 ceph-mon[35501]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:18.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:18 smithi137 ceph-mon[31539]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:20.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:20 smithi179 ceph-mon[35472]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:20.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:20 smithi159 ceph-mon[35501]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:20.529 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:20 smithi137 ceph-mon[31539]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:22.197 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:22 smithi159 ceph-mon[35501]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:22.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:22 smithi179 ceph-mon[35472]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:22.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:22 smithi137 ceph-mon[31539]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:24.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:24 smithi137 ceph-mon[31539]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:24.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:24 smithi179 ceph-mon[35472]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:24.423 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:24 smithi159 ceph-mon[35501]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:25.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T13:57:25.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:25.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T13:57:25.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:25.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-26T13:57:25.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:26.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:26 smithi159 ceph-mon[35501]: Deploying daemon osd.10 on smithi159 2024-03-26T13:57:26.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:26 smithi159 ceph-mon[35501]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:26.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:26 smithi179 ceph-mon[35472]: Deploying daemon osd.10 on smithi159 2024-03-26T13:57:26.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:26 smithi179 ceph-mon[35472]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:26.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:26 smithi137 ceph-mon[31539]: Deploying daemon osd.10 on smithi159 2024-03-26T13:57:26.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:26 smithi137 ceph-mon[31539]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:28.392 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:28 smithi159 ceph-mon[35501]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:28.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:28 smithi179 ceph-mon[35472]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:28.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:28 smithi137 ceph-mon[31539]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:30.340 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:30 smithi159 ceph-mon[35501]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:30.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:30 smithi179 ceph-mon[35472]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:30.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:30 smithi137 ceph-mon[31539]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:32.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:32 smithi179 ceph-mon[35472]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:32.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:32 smithi159 ceph-mon[35501]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:32.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:32 smithi137 ceph-mon[31539]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:34.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:34 smithi159 ceph-mon[35501]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:34.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:34.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:34.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:34.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:34 smithi137 ceph-mon[31539]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:34.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:34.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:34.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:34 smithi179 ceph-mon[35472]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:34.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:36.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:36 smithi159 ceph-mon[35501]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:36.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:36 smithi137 ceph-mon[31539]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:36.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:36 smithi179 ceph-mon[35472]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:38.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:38 smithi159 ceph-mon[35501]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:38.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:38 smithi159 ceph-mon[35501]: from='osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T13:57:38.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:38 smithi159 ceph-mon[35501]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T13:57:38.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:38 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:38.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:38 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:38.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:38 smithi137 ceph-mon[31539]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:38.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:38 smithi137 ceph-mon[31539]: from='osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T13:57:38.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:38 smithi137 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T13:57:38.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:38 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:38.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:38 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:38.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:38 smithi179 ceph-mon[35472]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:38.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:38 smithi179 ceph-mon[35472]: from='osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T13:57:38.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:38 smithi179 ceph-mon[35472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-26T13:57:38.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:38 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:38.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:38 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.095+0000 7f4096a00700 1 -- 172.21.15.159:0/171272685 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f40a0075ae0 con 0x7f40840a2890 2024-03-26T13:57:39.092 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 10 on host 'smithi159' 2024-03-26T13:57:39.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40840a2890 msgr2=0x7f40840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:39.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40840a2890 0x7f40840a4d50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4090005950 tx=0x7f40900058e0 comp rx=0 tx=0).stop 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 msgr2=0x7f40a019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 0x7f40a019a9a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c9b0 tx=0x7f408c00cd70 comp rx=0 tx=0).stop 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 shutdown_connections 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40840a2890 0x7f40840a4d50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40a0103d60 0x7f40a01a1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40a01021c0 0x7f40a019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 --2- 172.21.15.159:0/171272685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40a01033c0 0x7f40a019a9a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.099+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 >> 172.21.15.159:0/171272685 conn(0x7f40a00fd7a0 msgr2=0x7f40a00743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.100+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 shutdown_connections 2024-03-26T13:57:39.098 INFO:teuthology.orchestra.run.smithi159.stderr:2024-03-26T13:57:39.100+0000 7f40a4a00700 1 -- 172.21.15.159:0/171272685 wait complete. 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:39.239 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:39.240 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.240 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.240 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T13:57:39.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-26T13:57:39.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: osdmap e71: 11 total, 10 up, 11 in 2024-03-26T13:57:39.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:57:39.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:39.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-03-26T13:57:39.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:39.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:39.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:39.915 DEBUG:teuthology.orchestra.run.smithi159:osd.10> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.10.service 2024-03-26T13:57:39.922 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2024-03-26T13:57:39.922 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-26T13:57:40.377 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:57:40.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:40 smithi179 ceph-mon[35472]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:40.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:40 smithi179 ceph-mon[35472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:57:40.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:40 smithi179 ceph-mon[35472]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T13:57:40.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:40.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:40.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:40.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:40 smithi137 ceph-mon[31539]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:40.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:40 smithi137 ceph-mon[31539]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:57:40.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:40 smithi137 ceph-mon[31539]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T13:57:40.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:40.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:40.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:40.595 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:40 smithi159 ceph-mon[35501]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:40.595 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:40 smithi159 ceph-mon[35501]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-03-26T13:57:40.596 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:40 smithi159 ceph-mon[35501]: osdmap e72: 11 total, 10 up, 11 in 2024-03-26T13:57:40.596 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:40.596 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:40.596 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:57:41.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:57:41.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:57:41.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:41.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] boot 2024-03-26T13:57:41.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:41 smithi137 ceph-mon[31539]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] boot 2024-03-26T13:57:41.531 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T13:57:41.532 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:41.532 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:41 smithi179 ceph-mon[35472]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: osd.10 [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] boot 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: osdmap e73: 11 total, 11 up, 11 in 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-26T13:57:41.594 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:41 smithi159 ceph-mon[35501]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-26T13:57:42.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:42 smithi137 ceph-mon[31539]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T13:57:42.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:42 smithi179 ceph-mon[35472]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T13:57:42.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:42 smithi159 ceph-mon[35501]: osdmap e74: 11 total, 11 up, 11 in 2024-03-26T13:57:43.157 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-26T13:57:43.158 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-26T13:57:43.158 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-03-26T13:57:43.158 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-03-26T13:57:43.158 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0354667 s, 296 MB/s 2024-03-26T13:57:43.158 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-03-26T13:57:43.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:43 smithi179 ceph-mon[35472]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T13:57:43.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:43 smithi179 ceph-mon[35472]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:43.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:43 smithi137 ceph-mon[31539]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T13:57:43.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:43 smithi137 ceph-mon[31539]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:43.697 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:43 smithi159 ceph-mon[35501]: osdmap e75: 11 total, 11 up, 11 in 2024-03-26T13:57:43.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:43 smithi159 ceph-mon[35501]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:44.001 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-03-26T13:57:44.443 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:57:44.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:44 smithi137 ceph-mon[31539]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T13:57:44.581 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:44 smithi179 ceph-mon[35472]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T13:57:44.669 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:44 smithi159 ceph-mon[35501]: osdmap e76: 11 total, 11 up, 11 in 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi159 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.593 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.594 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi159 to 4272M 2024-03-26T13:57:45.594 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.594 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:45.594 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:45.594 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.594 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:45 smithi179 ceph-mon[35472]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:45.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T13:57:45.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi159 2024-03-26T13:57:45.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi159 to 4272M 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:45 smithi159 ceph-mon[35501]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:45.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: osdmap e77: 11 total, 11 up, 11 in 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi159 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi159 to 4272M 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:57:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:45 smithi137 ceph-mon[31539]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 -- 172.21.15.179:0/3026186531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084105510 msgr2=0x7f9084105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 --2- 172.21.15.179:0/3026186531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084105510 0x7f9084105970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f906c009990 tx=0x7f906c009ca0 comp rx=0 tx=0).stop 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 -- 172.21.15.179:0/3026186531 shutdown_connections 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 --2- 172.21.15.179:0/3026186531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084105eb0 0x7f908410c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 --2- 172.21.15.179:0/3026186531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084104310 0x7f9084104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 --2- 172.21.15.179:0/3026186531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084105510 0x7f9084105970 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 -- 172.21.15.179:0/3026186531 >> 172.21.15.179:0/3026186531 conn(0x7f90840ff8b0 msgr2=0x7f9084101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 -- 172.21.15.179:0/3026186531 shutdown_connections 2024-03-26T13:57:46.481 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.485+0000 7f9083e00700 1 -- 172.21.15.179:0/3026186531 wait complete. 2024-03-26T13:57:46.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 Processor -- start 2024-03-26T13:57:46.482 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 -- start start 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 0x7f908419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084105510 0x7f908419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084105eb0 0x7f90841a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908410fdb0 con 0x7f9084105510 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908410fc30 con 0x7f9084105eb0 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.486+0000 7f9083e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908410ff30 con 0x7f9084104310 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 0x7f908419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 0x7f908419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:50578/0 (socket says 172.21.15.179:50578) 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 -- 172.21.15.179:0/525903671 learned_addr learned my addr 172.21.15.179:0/525903671 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f907be00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084105510 0x7f908419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9083400700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084105eb0 0x7f90841a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 -- 172.21.15.179:0/525903671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084105eb0 msgr2=0x7f90841a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084105eb0 0x7f90841a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:46.483 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 -- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084105510 msgr2=0x7f908419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:57:46.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084105510 0x7f908419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:57:46.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9082a00700 1 -- 172.21.15.179:0/525903671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90841a2080 con 0x7f9084104310 2024-03-26T13:57:46.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.487+0000 7f9083400700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084105eb0 0x7f90841a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:57:46.484 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.488+0000 7f9082a00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 0x7f908419a420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f907400c920 tx=0x7f907400cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:57:46.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.488+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074004220 con 0x7f9084104310 2024-03-26T13:57:46.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.488+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9074007d70 con 0x7f9084104310 2024-03-26T13:57:46.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.488+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074003d40 con 0x7f9084104310 2024-03-26T13:57:46.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.488+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90841a2370 con 0x7f9084104310 2024-03-26T13:57:46.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.488+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90841a2860 con 0x7f9084104310 2024-03-26T13:57:46.485 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.489+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9084068e50 con 0x7f9084104310 2024-03-26T13:57:46.488 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.493+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f90740078c0 con 0x7f9084104310 2024-03-26T13:57:46.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.494+0000 7f907b400700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90680a2910 0x7f90680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:57:46.489 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.494+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f90740ce470 con 0x7f9084104310 2024-03-26T13:57:46.490 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.495+0000 7f907be00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90680a2910 0x7f90680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:57:46.491 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.496+0000 7f907be00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90680a2910 0x7f90680a4dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f906c00b5c0 tx=0x7f906c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:57:46.492 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.497+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9074014030 con 0x7f9084104310 2024-03-26T13:57:46.723 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:57:46.727+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f90840625d0 con 0x7f90680a2910 2024-03-26T13:57:47.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:46 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:57:47.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:46 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:57:47.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:46 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:47.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:46 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:57:47.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:46 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:57:47.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:46 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:47.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-26T13:57:47.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-26T13:57:47.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:57:48.057 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:47 smithi179 ceph-mon[35472]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:57:48.057 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:47 smithi179 ceph-mon[35472]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-26T13:57:48.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:47 smithi159 ceph-mon[35501]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:57:48.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:47 smithi159 ceph-mon[35501]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-26T13:57:48.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:47 smithi137 ceph-mon[31539]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:57:48.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:47 smithi137 ceph-mon[31539]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-03-26T13:57:50.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:50 smithi179 ceph-mon[35472]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T13:57:50.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:50 smithi159 ceph-mon[35501]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T13:57:50.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:50 smithi137 ceph-mon[31539]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-26T13:57:51.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:51 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/2540839367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]: dispatch 2024-03-26T13:57:51.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:51 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]: dispatch 2024-03-26T13:57:51.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:51 smithi179 ceph-mon[35472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]': finished 2024-03-26T13:57:51.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:51 smithi179 ceph-mon[35472]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T13:57:51.397 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:51 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:57:51.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:51 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/2540839367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]: dispatch 2024-03-26T13:57:51.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:51 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]: dispatch 2024-03-26T13:57:51.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:51 smithi159 ceph-mon[35501]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]': finished 2024-03-26T13:57:51.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:51 smithi159 ceph-mon[35501]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T13:57:51.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:51 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:57:51.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:51 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/2540839367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]: dispatch 2024-03-26T13:57:51.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:51 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]: dispatch 2024-03-26T13:57:51.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:51 smithi137 ceph-mon[31539]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cefee215-43f2-4ed3-adbc-b2784f7cbddc"}]': finished 2024-03-26T13:57:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:51 smithi137 ceph-mon[31539]: osdmap e78: 12 total, 11 up, 12 in 2024-03-26T13:57:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:51 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:57:52.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:52 smithi179 ceph-mon[35472]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-26T13:57:52.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:52 smithi179 ceph-mon[35472]: from='client.? 172.21.15.179:0/3930395195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:57:52.432 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:52 smithi137 ceph-mon[31539]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-26T13:57:52.432 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:52 smithi137 ceph-mon[31539]: from='client.? 172.21.15.179:0/3930395195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:57:52.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:52 smithi159 ceph-mon[35501]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-26T13:57:52.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:52 smithi159 ceph-mon[35501]: from='client.? 172.21.15.179:0/3930395195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-26T13:57:54.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:54 smithi179 ceph-mon[35472]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-26T13:57:54.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:54 smithi159 ceph-mon[35501]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-26T13:57:54.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:54 smithi137 ceph-mon[31539]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-03-26T13:57:56.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:56 smithi179 ceph-mon[35472]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:57:56.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:56 smithi159 ceph-mon[35501]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:57:56.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:56 smithi137 ceph-mon[31539]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-26T13:57:58.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:57:58 smithi179 ceph-mon[35472]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:58.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:57:58 smithi159 ceph-mon[35501]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:57:58.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:57:58 smithi137 ceph-mon[31539]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:00.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:00 smithi179 ceph-mon[35472]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:00.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:00 smithi159 ceph-mon[35501]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:00.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:00 smithi137 ceph-mon[31539]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:01.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T13:58:01.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:01 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:01.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:01 smithi179 ceph-mon[35472]: Deploying daemon osd.11 on smithi179 2024-03-26T13:58:01.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T13:58:01.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:01 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:01.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:01 smithi159 ceph-mon[35501]: Deploying daemon osd.11 on smithi179 2024-03-26T13:58:01.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-26T13:58:01.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:01 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:01.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:01 smithi137 ceph-mon[31539]: Deploying daemon osd.11 on smithi179 2024-03-26T13:58:02.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:02 smithi179 ceph-mon[35472]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:02.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:02 smithi159 ceph-mon[35501]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:02.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:02 smithi137 ceph-mon[31539]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:04.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:04 smithi179 ceph-mon[35472]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:04 smithi159 ceph-mon[35501]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:04.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:04 smithi137 ceph-mon[31539]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:06.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:06 smithi179 ceph-mon[35472]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:06 smithi159 ceph-mon[35501]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:06.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:06 smithi137 ceph-mon[31539]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:08.321 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:08 smithi179 ceph-mon[35472]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:08.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:08 smithi159 ceph-mon[35501]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:08.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:08 smithi137 ceph-mon[31539]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:09.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:09.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:09.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:09.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:09.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:09.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:09.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:09.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:09.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:10.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:10 smithi179 ceph-mon[35472]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:10.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:10 smithi159 ceph-mon[35501]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:10.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:10 smithi137 ceph-mon[31539]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:12.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:12 smithi179 ceph-mon[35472]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:12.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:12 smithi159 ceph-mon[35501]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:12.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:12 smithi137 ceph-mon[31539]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:13.291 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.294+0000 7f907b400700 1 -- 172.21.15.179:0/525903671 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f90840625d0 con 0x7f90680a2910 2024-03-26T13:58:13.292 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2024-03-26T13:58:13.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90680a2910 msgr2=0x7f90680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:13.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90680a2910 0x7f90680a4dd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f906c00b5c0 tx=0x7f906c005fb0 comp rx=0 tx=0).stop 2024-03-26T13:58:13.293 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 msgr2=0x7f908419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:13.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 0x7f908419a420 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f907400c920 tx=0x7f907400cce0 comp rx=0 tx=0).stop 2024-03-26T13:58:13.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 shutdown_connections 2024-03-26T13:58:13.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90680a2910 0x7f90680a4dd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:13.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9084105eb0 0x7f90841a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:13.294 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9084104310 0x7f908419a420 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:13.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 --2- 172.21.15.179:0/525903671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9084105510 0x7f908419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:13.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 >> 172.21.15.179:0/525903671 conn(0x7f90840ff8b0 msgr2=0x7f9084108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:13.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 shutdown_connections 2024-03-26T13:58:13.295 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:13.298+0000 7f9083e00700 1 -- 172.21.15.179:0/525903671 wait complete. 2024-03-26T13:58:13.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T13:58:13.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:13 smithi137 ceph-mon[31539]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:13.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T13:58:13.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:13 smithi179 ceph-mon[35472]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:13.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:13.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:13.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:13.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T13:58:13.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-26T13:58:13.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:13 smithi159 ceph-mon[35501]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:14.084 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.11.service 2024-03-26T13:58:14.090 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-26T13:58:14.090 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd stat -f json 2024-03-26T13:58:14.279 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:14.279 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:14.279 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:14.279 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T13:58:14.279 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T13:58:14.280 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:14.280 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:58:14.280 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-mon[35472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:58:14.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:14.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:14.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:14.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T13:58:14.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T13:58:14.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:14.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:58:14.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:14 smithi137 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:58:14.552 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:58:14.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:14.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:14.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:14.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-26T13:58:14.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: osdmap e79: 12 total, 11 up, 12 in 2024-03-26T13:58:14.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:14.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:58:14.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:14 smithi159 ceph-mon[35501]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-03-26T13:58:15.294 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T13:58:14.925+0000 7f3385800700 -1 osd.11 0 waiting for initial osdmap 2024-03-26T13:58:15.294 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 13:58:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T13:58:14.978+0000 7f337b600700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-26T13:58:15.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:15 smithi159 ceph-mon[35501]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:58:15.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:15 smithi159 ceph-mon[35501]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T13:58:15.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:15.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:15.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:15 smithi159 ceph-mon[35501]: from='osd.11 ' entity='osd.11' 2024-03-26T13:58:15.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:15 smithi159 ceph-mon[35501]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:15.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:15 smithi137 ceph-mon[31539]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:58:15.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:15 smithi137 ceph-mon[31539]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T13:58:15.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:15.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:15.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:15 smithi137 ceph-mon[31539]: from='osd.11 ' entity='osd.11' 2024-03-26T13:58:15.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:15 smithi137 ceph-mon[31539]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:15.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:15 smithi179 ceph-mon[35472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-03-26T13:58:15.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:15 smithi179 ceph-mon[35472]: osdmap e80: 12 total, 11 up, 12 in 2024-03-26T13:58:15.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:15.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:15.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:15 smithi179 ceph-mon[35472]: from='osd.11 ' entity='osd.11' 2024-03-26T13:58:15.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:15 smithi179 ceph-mon[35472]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-26T13:58:16.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:16 smithi179 ceph-mon[35472]: purged_snaps scrub starts 2024-03-26T13:58:16.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:16 smithi179 ceph-mon[35472]: purged_snaps scrub ok 2024-03-26T13:58:16.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:16.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:16 smithi179 ceph-mon[35472]: osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] boot 2024-03-26T13:58:16.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:16 smithi179 ceph-mon[35472]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T13:58:16.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:16.565 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:16 smithi137 ceph-mon[31539]: purged_snaps scrub starts 2024-03-26T13:58:16.565 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:16 smithi137 ceph-mon[31539]: purged_snaps scrub ok 2024-03-26T13:58:16.565 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:16.565 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:16 smithi137 ceph-mon[31539]: osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] boot 2024-03-26T13:58:16.565 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:16 smithi137 ceph-mon[31539]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T13:58:16.565 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:16.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:16 smithi159 ceph-mon[35501]: purged_snaps scrub starts 2024-03-26T13:58:16.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:16 smithi159 ceph-mon[35501]: purged_snaps scrub ok 2024-03-26T13:58:16.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:16.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:16 smithi159 ceph-mon[35501]: osd.11 [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] boot 2024-03-26T13:58:16.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:16 smithi159 ceph-mon[35501]: osdmap e81: 12 total, 12 up, 12 in 2024-03-26T13:58:16.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 -- 172.21.15.137:0/2150601737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff834074bf0 msgr2=0x7ff834075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 --2- 172.21.15.137:0/2150601737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff834074bf0 0x7ff834075070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009990 tx=0x7ff81c009ca0 comp rx=0 tx=0).stop 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 -- 172.21.15.137:0/2150601737 shutdown_connections 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 --2- 172.21.15.137:0/2150601737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff83410e7b0 0x7ff834110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 --2- 172.21.15.137:0/2150601737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8340761c0 0x7ff834074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 --2- 172.21.15.137:0/2150601737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff834074bf0 0x7ff834075070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 -- 172.21.15.137:0/2150601737 >> 172.21.15.137:0/2150601737 conn(0x7ff8340fd730 msgr2=0x7ff8340ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:16.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 -- 172.21.15.137:0/2150601737 shutdown_connections 2024-03-26T13:58:16.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.865+0000 7ff83a000700 1 -- 172.21.15.137:0/2150601737 wait complete. 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.866+0000 7ff83a000700 1 Processor -- start 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.866+0000 7ff83a000700 1 -- start start 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff83a000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff834074bf0 0x7ff83419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff83a000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8340761c0 0x7ff83419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff83a000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 0x7ff8341a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff83a000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8341142b0 con 0x7ff83410e7b0 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff83a000700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834114130 con 0x7ff8340761c0 2024-03-26T13:58:16.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff83a000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834114430 con 0x7ff834074bf0 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff833e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 0x7ff8341a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff833e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 0x7ff8341a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57988/0 (socket says 172.21.15.137:57988) 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff833e00700 1 -- 172.21.15.137:0/139593490 learned_addr learned my addr 172.21.15.137:0/139593490 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff833400700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff834074bf0 0x7ff83419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.867+0000 7ff82aa00700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8340761c0 0x7ff83419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff833e00700 1 -- 172.21.15.137:0/139593490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff834074bf0 msgr2=0x7ff83419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff833e00700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff834074bf0 0x7ff83419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff833e00700 1 -- 172.21.15.137:0/139593490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8340761c0 msgr2=0x7ff83419ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff833e00700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8340761c0 0x7ff83419ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:16.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff833e00700 1 -- 172.21.15.137:0/139593490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8341a6570 con 0x7ff83410e7b0 2024-03-26T13:58:16.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff833e00700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 0x7ff8341a5ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff82c007f10 tx=0x7ff82c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:16.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c004bb0 con 0x7ff83410e7b0 2024-03-26T13:58:16.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.868+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff82c004d50 con 0x7ff83410e7b0 2024-03-26T13:58:16.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.869+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c005990 con 0x7ff83410e7b0 2024-03-26T13:58:16.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.869+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8341a6860 con 0x7ff83410e7b0 2024-03-26T13:58:16.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.869+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8341a6d50 con 0x7ff83410e7b0 2024-03-26T13:58:16.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.870+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff834068e50 con 0x7ff83410e7b0 2024-03-26T13:58:16.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.872+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff82c01a020 con 0x7ff83410e7b0 2024-03-26T13:58:16.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.874+0000 7ff830c00700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff8140a2890 0x7ff8140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:16.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.874+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff82c0cdb60 con 0x7ff83410e7b0 2024-03-26T13:58:16.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.874+0000 7ff833400700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff8140a2890 0x7ff8140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:16.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.875+0000 7ff833400700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff8140a2890 0x7ff8140a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff824005ea0 tx=0x7ff824005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:16.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:16.878+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff82c010070 con 0x7ff83410e7b0 2024-03-26T13:58:17.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.101+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7ff834062da0 con 0x7ff83410e7b0 2024-03-26T13:58:17.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.102+0000 7ff830c00700 1 -- 172.21.15.137:0/139593490 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7ff82c09b000 con 0x7ff83410e7b0 2024-03-26T13:58:17.099 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:17.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff8140a2890 msgr2=0x7ff8140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:17.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff8140a2890 0x7ff8140a4d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff824005ea0 tx=0x7ff824005e30 comp rx=0 tx=0).stop 2024-03-26T13:58:17.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 msgr2=0x7ff8341a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:17.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 0x7ff8341a5ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff82c007f10 tx=0x7ff82c00d3b0 comp rx=0 tx=0).stop 2024-03-26T13:58:17.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 shutdown_connections 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff8140a2890 0x7ff8140a4d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff83410e7b0 0x7ff8341a5ed0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff834074bf0 0x7ff83419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.105+0000 7ff83a000700 1 --2- 172.21.15.137:0/139593490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8340761c0 0x7ff83419ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.106+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 >> 172.21.15.137:0/139593490 conn(0x7ff8340fd730 msgr2=0x7ff834101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.106+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 shutdown_connections 2024-03-26T13:58:17.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:17.106+0000 7ff83a000700 1 -- 172.21.15.137:0/139593490 wait complete. 2024-03-26T13:58:17.477 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:17 smithi137 ceph-mon[31539]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:17.477 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:17 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/139593490' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T13:58:17.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:17 smithi179 ceph-mon[35472]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:17.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:17 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/139593490' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T13:58:17.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:17 smithi159 ceph-mon[35501]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:17.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:17 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/139593490' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-26T13:58:17.914 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711461495,"num_in_osds":12,"osd_in_since":1711461470,"num_remapped_pgs":0} 2024-03-26T13:58:17.915 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T13:58:18.333 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:58:18.571 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:18 smithi179 ceph-mon[35472]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T13:58:18.660 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:18 smithi137 ceph-mon[31539]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T13:58:18.698 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:18 smithi159 ceph-mon[35501]: osdmap e82: 12 total, 12 up, 12 in 2024-03-26T13:58:19.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: Detected new or changed devices on smithi179 2024-03-26T13:58:19.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi179 to 4272M 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:19 smithi179 ceph-mon[35472]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: Detected new or changed devices on smithi179 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi179 to 4272M 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:19.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:19 smithi159 ceph-mon[35501]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: Detected new or changed devices on smithi179 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi179 to 4272M 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:20.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:19 smithi137 ceph-mon[31539]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 -- 172.21.15.137:0/2671193928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc810112ff0 msgr2=0x7fc8101153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 --2- 172.21.15.137:0/2671193928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc810112ff0 0x7fc8101153e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b0a0 tx=0x7fc80800b3b0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 -- 172.21.15.137:0/2671193928 shutdown_connections 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 --2- 172.21.15.137:0/2671193928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc810112ff0 0x7fc8101153e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 --2- 172.21.15.137:0/2671193928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8100766c0 0x7fc810076aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 --2- 172.21.15.137:0/2671193928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810076fe0 0x7fc810112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 -- 172.21.15.137:0/2671193928 >> 172.21.15.137:0/2671193928 conn(0x7fc8100ffe60 msgr2=0x7fc810102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 -- 172.21.15.137:0/2671193928 shutdown_connections 2024-03-26T13:58:20.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.584+0000 7fc817200700 1 -- 172.21.15.137:0/2671193928 wait complete. 2024-03-26T13:58:20.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 Processor -- start 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 -- start start 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 0x7fc81019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc810076fe0 0x7fc81019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810112ff0 0x7fc8101a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810118970 con 0x7fc8100766c0 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8101187f0 con 0x7fc810112ff0 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc817200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810118af0 con 0x7fc810076fe0 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.586+0000 7fc80fe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 0x7fc81019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 0x7fc81019eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36370/0 (socket says 172.21.15.137:36370) 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 -- 172.21.15.137:0/2840302966 learned_addr learned my addr 172.21.15.137:0/2840302966 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80f400700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc810076fe0 0x7fc81019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:20.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc815200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810112ff0 0x7fc8101a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 -- 172.21.15.137:0/2840302966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc810076fe0 msgr2=0x7fc81019f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc810076fe0 0x7fc81019f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 -- 172.21.15.137:0/2840302966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810112ff0 msgr2=0x7fc8101a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810112ff0 0x7fc8101a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 -- 172.21.15.137:0/2840302966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8101a3ea0 con 0x7fc8100766c0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc815200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810112ff0 0x7fc8101a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.587+0000 7fc80fe00700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 0x7fc81019eee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc80000d950 tx=0x7fc80000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.588+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800016070 con 0x7fc8100766c0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.588+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8101a4190 con 0x7fc8100766c0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.588+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc800010460 con 0x7fc8100766c0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.588+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000f620 con 0x7fc8100766c0 2024-03-26T13:58:20.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.588+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8101aba10 con 0x7fc8100766c0 2024-03-26T13:58:20.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.589+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc810068e50 con 0x7fc8100766c0 2024-03-26T13:58:20.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.591+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc800019070 con 0x7fc8100766c0 2024-03-26T13:58:20.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.592+0000 7fc80cc00700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7f40a2910 0x7fc7f40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:20.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.592+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc8000ce250 con 0x7fc8100766c0 2024-03-26T13:58:20.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.592+0000 7fc80f400700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7f40a2910 0x7fc7f40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:20.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.593+0000 7fc80f400700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7f40a2910 0x7fc7f40a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005950 tx=0x7fc7fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:20.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.598+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc80009b6f0 con 0x7fc8100766c0 2024-03-26T13:58:20.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.822+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc81002da30 con 0x7fc8100766c0 2024-03-26T13:58:20.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.825+0000 7fc80cc00700 1 -- 172.21.15.137:0/2840302966 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fc80009aff0 con 0x7fc8100766c0 2024-03-26T13:58:20.822 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:20.822 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:58:17.300101+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:58:20.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.828+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7f40a2910 msgr2=0x7fc7f40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:20.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.828+0000 7fc817200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7f40a2910 0x7fc7f40a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc005950 tx=0x7fc7fc0058e0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.828+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 msgr2=0x7fc81019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.828+0000 7fc817200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 0x7fc81019eee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc80000d950 tx=0x7fc80000dd10 comp rx=0 tx=0).stop 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 shutdown_connections 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7f40a2910 0x7fc7f40a4dd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc810112ff0 0x7fc8101a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8100766c0 0x7fc81019eee0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 --2- 172.21.15.137:0/2840302966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc810076fe0 0x7fc81019f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 >> 172.21.15.137:0/2840302966 conn(0x7fc8100ffe60 msgr2=0x7fc810102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 shutdown_connections 2024-03-26T13:58:20.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:20.829+0000 7fc817200700 1 -- 172.21.15.137:0/2840302966 wait complete. 2024-03-26T13:58:21.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:20 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2840302966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:58:21.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:20 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2840302966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:58:21.280 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:20 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2840302966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:58:21.812 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T13:58:21.813 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-26T13:58:22.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:21 smithi137 ceph-mon[31539]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:22.169 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:21 smithi179 ceph-mon[35472]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:22.198 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:21 smithi159 ceph-mon[35501]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:22.253 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:58:24.389 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:24 smithi137 ceph-mon[31539]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:24.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:24 smithi179 ceph-mon[35472]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:24.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:24 smithi159 ceph-mon[35501]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:24.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 -- 172.21.15.137:0/1684092661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 msgr2=0x7fb52c108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 --2- 172.21.15.137:0/1684092661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c108c20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb518009990 tx=0x7fb518009ca0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 -- 172.21.15.137:0/1684092661 shutdown_connections 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 --2- 172.21.15.137:0/1684092661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c109160 0x7fb52c115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 --2- 172.21.15.137:0/1684092661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c10e7a0 0x7fb52c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 --2- 172.21.15.137:0/1684092661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c108c20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 -- 172.21.15.137:0/1684092661 >> 172.21.15.137:0/1684092661 conn(0x7fb52c075960 msgr2=0x7fb52c077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 -- 172.21.15.137:0/1684092661 shutdown_connections 2024-03-26T13:58:24.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.565+0000 7fb533600700 1 -- 172.21.15.137:0/1684092661 wait complete. 2024-03-26T13:58:24.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.566+0000 7fb533600700 1 Processor -- start 2024-03-26T13:58:24.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 -- start start 2024-03-26T13:58:24.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:24.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c109160 0x7fb52c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:24.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c10e7a0 0x7fb52c1a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1189e0 con 0x7fb52c1087a0 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c118860 con 0x7fb52c109160 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb533600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c118b60 con 0x7fb52c10e7a0 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb530c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb530c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c19ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36394/0 (socket says 172.21.15.137:36394) 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.567+0000 7fb530c00700 1 -- 172.21.15.137:0/402790490 learned_addr learned my addr 172.21.15.137:0/402790490 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb52be00700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c109160 0x7fb52c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb531600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c10e7a0 0x7fb52c1a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb530c00700 1 -- 172.21.15.137:0/402790490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c10e7a0 msgr2=0x7fb52c1a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb530c00700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c10e7a0 0x7fb52c1a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb530c00700 1 -- 172.21.15.137:0/402790490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c109160 msgr2=0x7fb52c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb530c00700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c109160 0x7fb52c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb530c00700 1 -- 172.21.15.137:0/402790490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c1a3e10 con 0x7fb52c1087a0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb530c00700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c19ee50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d980 tx=0x7fb52000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.568+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520017070 con 0x7fb52c1087a0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.569+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb520004830 con 0x7fb52c1087a0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.569+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c1a4100 con 0x7fb52c1087a0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.569+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb520005440 con 0x7fb52c1087a0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.569+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c1ab980 con 0x7fb52c1087a0 2024-03-26T13:58:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.570+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c110d30 con 0x7fb52c1087a0 2024-03-26T13:58:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.572+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb5200055e0 con 0x7fb52c1087a0 2024-03-26T13:58:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.573+0000 7fb529600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5140a2890 0x7fb5140a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.573+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb5200ce520 con 0x7fb52c1087a0 2024-03-26T13:58:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.573+0000 7fb52be00700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5140a2890 0x7fb5140a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.574+0000 7fb52be00700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5140a2890 0x7fb5140a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb51801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:24.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.578+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb520020020 con 0x7fb52c1087a0 2024-03-26T13:58:24.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.795+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb52c1a0160 con 0x7fb52c1087a0 2024-03-26T13:58:24.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.796+0000 7fb529600700 1 -- 172.21.15.137:0/402790490 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb52009b9c0 con 0x7fb52c1087a0 2024-03-26T13:58:24.792 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T13:58:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5140a2890 msgr2=0x7fb5140a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5140a2890 0x7fb5140a4d50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb518006010 tx=0x7fb51801a040 comp rx=0 tx=0).stop 2024-03-26T13:58:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 msgr2=0x7fb52c19ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c19ee50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb52000d980 tx=0x7fb52000dd40 comp rx=0 tx=0).stop 2024-03-26T13:58:24.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 shutdown_connections 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5140a2890 0x7fb5140a4d50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb52c10e7a0 0x7fb52c1a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb52c1087a0 0x7fb52c19ee50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 --2- 172.21.15.137:0/402790490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb52c109160 0x7fb52c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.799+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 >> 172.21.15.137:0/402790490 conn(0x7fb52c075960 msgr2=0x7fb52c0761d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.800+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 shutdown_connections 2024-03-26T13:58:24.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:24.800+0000 7fb533600700 1 -- 172.21.15.137:0/402790490 wait complete. 2024-03-26T13:58:25.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:25 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/402790490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:58:25.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:25 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/402790490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:58:25.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:25 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/402790490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:58:25.665 INFO:tasks.cephadm:Adding mds.a on smithi137 2024-03-26T13:58:25.665 INFO:tasks.cephadm:Adding mds.d on smithi137 2024-03-26T13:58:25.665 INFO:tasks.cephadm:Adding mds.g on smithi137 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.j on smithi137 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.b on smithi159 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.e on smithi159 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.h on smithi159 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.k on smithi159 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.c on smithi179 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.f on smithi179 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.i on smithi179 2024-03-26T13:58:25.666 INFO:tasks.cephadm:Adding mds.l on smithi179 2024-03-26T13:58:25.666 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph orch apply mds all '12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2024-03-26T13:58:26.122 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T13:58:26.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:26 smithi179 ceph-mon[35472]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:26.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:26 smithi159 ceph-mon[35501]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:26.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:26 smithi137 ceph-mon[31539]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 -- 172.21.15.179:0/3669014632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 msgr2=0x7fa150107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 --2- 172.21.15.179:0/3669014632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa150107980 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa138009990 tx=0x7fa138009ca0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 -- 172.21.15.179:0/3669014632 shutdown_connections 2024-03-26T13:58:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 --2- 172.21.15.179:0/3669014632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150107ec0 0x7fa15010a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.257 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 --2- 172.21.15.179:0/3669014632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa150102c60 0x7fa150105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 --2- 172.21.15.179:0/3669014632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa150107980 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 -- 172.21.15.179:0/3669014632 >> 172.21.15.179:0/3669014632 conn(0x7fa1500fc690 msgr2=0x7fa1500feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:28.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 -- 172.21.15.179:0/3669014632 shutdown_connections 2024-03-26T13:58:28.258 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.261+0000 7fa156200700 1 -- 172.21.15.179:0/3669014632 wait complete. 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.262+0000 7fa156200700 1 Processor -- start 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.262+0000 7fa156200700 1 -- start start 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.262+0000 7fa156200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150102c60 0x7fa15019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.262+0000 7fa156200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa15019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa156200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa150107ec0 0x7fa1501a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa156200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15010dd20 con 0x7fa150102c60 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa156200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15010dba0 con 0x7fa150107ec0 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa156200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15010dea0 con 0x7fa150105590 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa15019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa15019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:45982/0 (socket says 172.21.15.179:45982) 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 -- 172.21.15.179:0/3424373243 learned_addr learned my addr 172.21.15.179:0/3424373243 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14f400700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150102c60 0x7fa15019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:28.260 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 -- 172.21.15.179:0/3424373243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa150107ec0 msgr2=0x7fa1501a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa150107ec0 0x7fa1501a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 -- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150102c60 msgr2=0x7fa15019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150102c60 0x7fa15019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.263+0000 7fa14ea00700 1 -- 172.21.15.179:0/3424373243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1501a2270 con 0x7fa150105590 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.264+0000 7fa14ea00700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa15019ab50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa138000c00 tx=0x7fa138009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.264+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138027070 con 0x7fa150105590 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.264+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1501a2500 con 0x7fa150105590 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.264+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa138004190 con 0x7fa150105590 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.264+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800f460 con 0x7fa150105590 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.264+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1501a29b0 con 0x7fa150105590 2024-03-26T13:58:28.261 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.265+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150068e50 con 0x7fa150105590 2024-03-26T13:58:28.262 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.267+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa13802a070 con 0x7fa150105590 2024-03-26T13:58:28.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.268+0000 7fa147e00700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1340a2910 0x7fa1340a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:28.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.269+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa1380cf8e0 con 0x7fa150105590 2024-03-26T13:58:28.264 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.269+0000 7fa14f400700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1340a2910 0x7fa1340a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:28.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.270+0000 7fa14f400700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1340a2910 0x7fa1340a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa140006fd0 tx=0x7fa140008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:28.269 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.273+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa138013070 con 0x7fa150105590 2024-03-26T13:58:28.419 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:28 smithi179 ceph-mon[35472]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:28.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:28 smithi159 ceph-mon[35501]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:28.500 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.504+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fa150062890 con 0x7fa1340a2910 2024-03-26T13:58:28.518 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.522+0000 7fa147e00700 1 -- 172.21.15.179:0/3424373243 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fa150062890 con 0x7fa1340a2910 2024-03-26T13:58:28.518 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2024-03-26T13:58:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1340a2910 msgr2=0x7fa1340a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:28.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1340a2910 0x7fa1340a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa140006fd0 tx=0x7fa140008040 comp rx=0 tx=0).stop 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 msgr2=0x7fa15019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa15019ab50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa138000c00 tx=0x7fa138009f70 comp rx=0 tx=0).stop 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 shutdown_connections 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1340a2910 0x7fa1340a4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa150107ec0 0x7fa1501a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa150102c60 0x7fa15019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 --2- 172.21.15.179:0/3424373243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa150105590 0x7fa15019ab50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 >> 172.21.15.179:0/3424373243 conn(0x7fa1500fc690 msgr2=0x7fa1501023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.525+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 shutdown_connections 2024-03-26T13:58:28.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-03-26T13:58:28.526+0000 7fa156200700 1 -- 172.21.15.179:0/3424373243 wait complete. 2024-03-26T13:58:28.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:28 smithi137 ceph-mon[31539]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:29.245 DEBUG:teuthology.orchestra.run.smithi137:mds.a> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.a.service 2024-03-26T13:58:29.254 DEBUG:teuthology.orchestra.run.smithi137:mds.d> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.d.service 2024-03-26T13:58:29.299 DEBUG:teuthology.orchestra.run.smithi137:mds.g> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.g.service 2024-03-26T13:58:29.344 DEBUG:teuthology.orchestra.run.smithi137:mds.j> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.j.service 2024-03-26T13:58:29.389 DEBUG:teuthology.orchestra.run.smithi159:mds.b> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.b.service 2024-03-26T13:58:29.396 DEBUG:teuthology.orchestra.run.smithi159:mds.e> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.e.service 2024-03-26T13:58:29.442 DEBUG:teuthology.orchestra.run.smithi159:mds.h> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.h.service 2024-03-26T13:58:29.489 DEBUG:teuthology.orchestra.run.smithi159:mds.k> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.k.service 2024-03-26T13:58:29.535 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.c.service 2024-03-26T13:58:29.541 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.f.service 2024-03-26T13:58:29.586 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.i.service 2024-03-26T13:58:29.632 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.l.service 2024-03-26T13:58:29.677 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-26T13:58:29.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: Saving service mds.all spec with placement smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:29.764 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:29.765 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:29.765 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:29.765 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:29.765 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: Deploying daemon mds.l on smithi179 2024-03-26T13:58:29.765 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:29 smithi137 ceph-mon[31539]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: Saving service mds.all spec with placement smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:29.864 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:29.865 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:29.865 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:29.865 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:29.865 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: Deploying daemon mds.l on smithi179 2024-03-26T13:58:29.865 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:29 smithi179 ceph-mon[35472]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 -- 172.21.15.137:0/870453089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe318104ad0 msgr2=0x7fe318104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 --2- 172.21.15.137:0/870453089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe318104ad0 0x7fe318104f50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe300009900 tx=0x7fe30002bc40 comp rx=0 tx=0).stop 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 -- 172.21.15.137:0/870453089 shutdown_connections 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 --2- 172.21.15.137:0/870453089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 0x7fe31810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 --2- 172.21.15.137:0/870453089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe318104ad0 0x7fe318104f50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 --2- 172.21.15.137:0/870453089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe3181038e0 0x7fe318103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 -- 172.21.15.137:0/870453089 >> 172.21.15.137:0/870453089 conn(0x7fe3180fee50 msgr2=0x7fe318101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 -- 172.21.15.137:0/870453089 shutdown_connections 2024-03-26T13:58:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.912+0000 7fe31d000640 1 -- 172.21.15.137:0/870453089 wait complete. 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.913+0000 7fe31d000640 1 Processor -- start 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 -- start start 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 0x7fe318199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe318104ad0 0x7fe31819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 0x7fe3181a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31810f4c0 con 0x7fe3181038e0 2024-03-26T13:58:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31810f340 con 0x7fe318105490 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe31d000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31810f640 con 0x7fe318104ad0 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 0x7fe318199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 0x7fe318199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36658/0 (socket says 172.21.15.137:36658) 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 -- 172.21.15.137:0/554687057 learned_addr learned my addr 172.21.15.137:0/554687057 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe30fe00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe318104ad0 0x7fe31819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe316800640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 0x7fe3181a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 -- 172.21.15.137:0/554687057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe318104ad0 msgr2=0x7fe31819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe318104ad0 0x7fe31819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 -- 172.21.15.137:0/554687057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 msgr2=0x7fe3181a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.914+0000 7fe315e00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 0x7fe3181a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe315e00640 1 -- 172.21.15.137:0/554687057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3181a1750 con 0x7fe3181038e0 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe316800640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 0x7fe3181a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe315e00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 0x7fe318199b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe30800c970 tx=0x7fe30800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:29.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308007c10 con 0x7fe3181038e0 2024-03-26T13:58:29.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3181a1a40 con 0x7fe3181038e0 2024-03-26T13:58:29.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe308007db0 con 0x7fe3181038e0 2024-03-26T13:58:29.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30800f7d0 con 0x7fe3181038e0 2024-03-26T13:58:29.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.915+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3181a1fe0 con 0x7fe3181038e0 2024-03-26T13:58:29.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.916+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe318068490 con 0x7fe3181038e0 2024-03-26T13:58:29.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.918+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe308018480 con 0x7fe3181038e0 2024-03-26T13:58:29.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.920+0000 7fe30ea00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:29.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.921+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3080ce9f0 con 0x7fe3181038e0 2024-03-26T13:58:29.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.921+0000 7fe30fe00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:29.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.921+0000 7fe30fe00640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe300006010 tx=0x7fe300000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:29.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:29.925+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe30809bf20 con 0x7fe3181038e0 2024-03-26T13:58:29.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: Saving service mds.all spec with placement smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: Deploying daemon mds.l on smithi179 2024-03-26T13:58:29.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:29 smithi159 ceph-mon[35501]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:30.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.128+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe31810ac00 con 0x7fe3181038e0 2024-03-26T13:58:30.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.131+0000 7fe30ea00640 1 -- 172.21.15.137:0/554687057 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fe30809b820 con 0x7fe3181038e0 2024-03-26T13:58:30.127 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:30.127 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:58:17.300101+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:58:30.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.133+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2fc0a2800 msgr2=0x7fe2fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.133+0000 7fe31d000640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe300006010 tx=0x7fe300000bc0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.133+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 msgr2=0x7fe318199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.133+0000 7fe31d000640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 0x7fe318199b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe30800c970 tx=0x7fe30800ce30 comp rx=0 tx=0).stop 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 shutdown_connections 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe318105490 0x7fe3181a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe318104ad0 0x7fe31819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 --2- 172.21.15.137:0/554687057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3181038e0 0x7fe318199b30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 >> 172.21.15.137:0/554687057 conn(0x7fe3180fee50 msgr2=0x7fe318107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 shutdown_connections 2024-03-26T13:58:30.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.134+0000 7fe31d000640 1 -- 172.21.15.137:0/554687057 wait complete. 2024-03-26T13:58:30.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T13:58:30.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T13:58:30.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.373+0000 7f6713a00640 1 -- 172.21.15.137:0/4185366563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 msgr2=0x7f670c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.373+0000 7f6713a00640 1 --2- 172.21.15.137:0/4185366563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c074580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009930 tx=0x7f66f802bc20 comp rx=0 tx=0).stop 2024-03-26T13:58:30.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 -- 172.21.15.137:0/4185366563 shutdown_connections 2024-03-26T13:58:30.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 --2- 172.21.15.137:0/4185366563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f670c10dc80 0x7f670c110070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 --2- 172.21.15.137:0/4185366563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c074580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 --2- 172.21.15.137:0/4185366563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f670c0756e0 0x7f670c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 -- 172.21.15.137:0/4185366563 >> 172.21.15.137:0/4185366563 conn(0x7f670c0fcc40 msgr2=0x7f670c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:30.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 -- 172.21.15.137:0/4185366563 shutdown_connections 2024-03-26T13:58:30.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.374+0000 7f6713a00640 1 -- 172.21.15.137:0/4185366563 wait complete. 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 Processor -- start 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 -- start start 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f670c0756e0 0x7f670c19e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f670c10dc80 0x7f670c1a5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c113820 con 0x7f670c074100 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1136a0 con 0x7f670c0756e0 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6713a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c1139a0 con 0x7f670c10dc80 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6711000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6711000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36666/0 (socket says 172.21.15.137:36666) 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.375+0000 7f6711000640 1 -- 172.21.15.137:0/12571373 learned_addr learned my addr 172.21.15.137:0/12571373 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6711000640 1 -- 172.21.15.137:0/12571373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f670c10dc80 msgr2=0x7f670c1a5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6711000640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f670c10dc80 0x7f670c1a5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6711000640 1 -- 172.21.15.137:0/12571373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f670c0756e0 msgr2=0x7f670c19e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6711000640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f670c0756e0 0x7f670c19e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6711000640 1 -- 172.21.15.137:0/12571373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c1a5ba0 con 0x7f670c074100 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6711000640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c19df80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f670000c890 tx=0x7f670000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700004220 con 0x7f670c074100 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f67000043c0 con 0x7f670c074100 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700004a00 con 0x7f670c074100 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670c1a5e90 con 0x7f670c074100 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.376+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c1a6300 con 0x7f670c074100 2024-03-26T13:58:30.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.378+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c103e50 con 0x7f670c074100 2024-03-26T13:58:30.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.379+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6700004ba0 con 0x7f670c074100 2024-03-26T13:58:30.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.380+0000 7f6709600640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66f40a2880 0x7f66f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.381+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67000ce360 con 0x7f670c074100 2024-03-26T13:58:30.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.381+0000 7f670be00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66f40a2880 0x7f66f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:30.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.382+0000 7f670be00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66f40a2880 0x7f66f40a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f66f8036040 tx=0x7f66f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:30.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.385+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6700014030 con 0x7f670c074100 2024-03-26T13:58:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.579+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f670c0623a0 con 0x7f670c074100 2024-03-26T13:58:30.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.580+0000 7f6709600640 1 -- 172.21.15.137:0/12571373 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f670009b800 con 0x7f670c074100 2024-03-26T13:58:30.575 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.582+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66f40a2880 msgr2=0x7f66f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.582+0000 7f6713a00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66f40a2880 0x7f66f40a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f66f8036040 tx=0x7f66f8000bc0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.582+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 msgr2=0x7f670c19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.582+0000 7f6713a00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c19df80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f670000c890 tx=0x7f670000cd50 comp rx=0 tx=0).stop 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 shutdown_connections 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66f40a2880 0x7f66f40a4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f670c0756e0 0x7f670c19e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f670c10dc80 0x7f670c1a5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 --2- 172.21.15.137:0/12571373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f670c074100 0x7f670c19df80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 >> 172.21.15.137:0/12571373 conn(0x7f670c0fcc40 msgr2=0x7f670c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:30.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 shutdown_connections 2024-03-26T13:58:30.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.583+0000 7f6713a00640 1 -- 172.21.15.137:0/12571373 wait complete. 2024-03-26T13:58:30.598 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/554687057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:58:30.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/12571373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:58:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.810+0000 7f13ef800640 1 -- 172.21.15.137:0/2449288875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8104af0 msgr2=0x7f13e8106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.810+0000 7f13ef800640 1 --2- 172.21.15.137:0/2449288875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8104af0 0x7f13e8106ee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009900 tx=0x7f13d402bc40 comp rx=0 tx=0).stop 2024-03-26T13:58:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.810+0000 7f13ef800640 1 -- 172.21.15.137:0/2449288875 shutdown_connections 2024-03-26T13:58:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.810+0000 7f13ef800640 1 --2- 172.21.15.137:0/2449288875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13e8107420 0x7f13e8109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.810+0000 7f13ef800640 1 --2- 172.21.15.137:0/2449288875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8104af0 0x7f13e8106ee0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.810+0000 7f13ef800640 1 --2- 172.21.15.137:0/2449288875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e81021c0 0x7f13e81045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.811+0000 7f13ef800640 1 -- 172.21.15.137:0/2449288875 >> 172.21.15.137:0/2449288875 conn(0x7f13e80fbbd0 msgr2=0x7f13e80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.811+0000 7f13ef800640 1 -- 172.21.15.137:0/2449288875 shutdown_connections 2024-03-26T13:58:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.811+0000 7f13ef800640 1 -- 172.21.15.137:0/2449288875 wait complete. 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.811+0000 7f13ef800640 1 Processor -- start 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 -- start start 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13e81021c0 0x7f13e8101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e8104af0 0x7f13e8100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 0x7f13e81006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e810d240 con 0x7f13e8107420 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e810d0c0 con 0x7f13e81021c0 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ef800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e810d3c0 con 0x7f13e8104af0 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ed800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 0x7f13e81006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ed800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 0x7f13e81006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36680/0 (socket says 172.21.15.137:36680) 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ed800640 1 -- 172.21.15.137:0/3927187530 learned_addr learned my addr 172.21.15.137:0/3927187530 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13ece00640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13e81021c0 0x7f13e8101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.812+0000 7f13e7e00640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e8104af0 0x7f13e8100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ed800640 1 -- 172.21.15.137:0/3927187530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e8104af0 msgr2=0x7f13e8100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ed800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e8104af0 0x7f13e8100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ed800640 1 -- 172.21.15.137:0/3927187530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13e81021c0 msgr2=0x7f13e8101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ed800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13e81021c0 0x7f13e8101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ed800640 1 -- 172.21.15.137:0/3927187530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e8100f40 con 0x7f13e8107420 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13e7e00640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e8104af0 0x7f13e8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ed800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 0x7f13e81006b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13e000cce0 tx=0x7f13e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000f450 con 0x7f13e8107420 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f13e000faa0 con 0x7f13e8107420 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e81a1940 con 0x7f13e8107420 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0018b70 con 0x7f13e8107420 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.813+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e81a1e40 con 0x7f13e8107420 2024-03-26T13:58:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.815+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80fd6c0 con 0x7f13e8107420 2024-03-26T13:58:30.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.816+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f13e0018d10 con 0x7f13e8107420 2024-03-26T13:58:30.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.819+0000 7f13e5600640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13d00a2800 0x7f13d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:30.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.819+0000 7f13ece00640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13d00a2800 0x7f13d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:30.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.819+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13e00cec10 con 0x7f13e8107420 2024-03-26T13:58:30.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.819+0000 7f13ece00640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13d00a2800 0x7f13d00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f13dc005e10 tx=0x7f13dc005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:30.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:30.823+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f13e0014030 con 0x7f13e8107420 2024-03-26T13:58:30.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/554687057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:58:30.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/12571373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:58:31.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/554687057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:58:31.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/12571373' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:58:31.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.062+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e802d070 con 0x7f13e8107420 2024-03-26T13:58:31.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.063+0000 7f13e5600640 1 -- 172.21.15.137:0/3927187530 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f13e009c0b0 con 0x7f13e8107420 2024-03-26T13:58:31.059 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:31.059 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-26T13:58:31.059 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 1 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13d00a2800 msgr2=0x7f13d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13d00a2800 0x7f13d00a4cc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f13dc005e10 tx=0x7f13dc005da0 comp rx=0 tx=0).stop 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 msgr2=0x7f13e81006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 0x7f13e81006b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13e000cce0 tx=0x7f13e000a620 comp rx=0 tx=0).stop 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 shutdown_connections 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13d00a2800 0x7f13d00a4cc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13e8107420 0x7f13e81006b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13e8104af0 0x7f13e8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 --2- 172.21.15.137:0/3927187530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13e81021c0 0x7f13e8101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 >> 172.21.15.137:0/3927187530 conn(0x7f13e80fbbd0 msgr2=0x7f13e8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 shutdown_connections 2024-03-26T13:58:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:31.066+0000 7f13ef800640 1 -- 172.21.15.137:0/3927187530 wait complete. 2024-03-26T13:58:31.849 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:31 smithi179 ceph-mon[35472]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:31.849 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:31 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3927187530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:31.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:31 smithi159 ceph-mon[35501]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:31.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:31 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3927187530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:32.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:31 smithi137 ceph-mon[31539]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:32.030 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:31 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3927187530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:32.143 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:31 smithi179 podman[65076]: 2024-03-26 13:58:31.875250129 +0000 UTC m=+0.150053348 container create c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T13:58:32.143 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:32 smithi179 podman[65076]: 2024-03-26 13:58:32.146452407 +0000 UTC m=+0.421255629 container init c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T13:58:32.420 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:32 smithi179 podman[65076]: 2024-03-26 13:58:32.156668505 +0000 UTC m=+0.431471694 container start c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T13:58:32.420 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:32 smithi179 ceph-mds[65094]: -- 172.21.15.179:0/1638956510 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a9ef507a40 con 0x55a9ef4da800 2024-03-26T13:58:32.420 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l[65090]: starting mds.l at 2024-03-26T13:58:32.420 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:32 smithi179 bash[65076]: c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b 2024-03-26T13:58:32.420 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 13:58:32 smithi179 systemd[1]: Started Ceph mds.l for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:33.081 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:33.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 -- 172.21.15.137:0/3935624888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8104ad0 msgr2=0x7f7bb8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:33.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 --2- 172.21.15.137:0/3935624888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8104ad0 0x7f7bb8104f50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009900 tx=0x7f7ba002bc30 comp rx=0 tx=0).stop 2024-03-26T13:58:33.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 -- 172.21.15.137:0/3935624888 shutdown_connections 2024-03-26T13:58:33.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 --2- 172.21.15.137:0/3935624888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb8105490 0x7f7bb810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 --2- 172.21.15.137:0/3935624888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8104ad0 0x7f7bb8104f50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 --2- 172.21.15.137:0/3935624888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb81038e0 0x7f7bb8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.303+0000 7f7bbd600640 1 -- 172.21.15.137:0/3935624888 >> 172.21.15.137:0/3935624888 conn(0x7f7bb80fee50 msgr2=0x7f7bb8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:33.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.304+0000 7f7bbd600640 1 -- 172.21.15.137:0/3935624888 shutdown_connections 2024-03-26T13:58:33.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.304+0000 7f7bbd600640 1 -- 172.21.15.137:0/3935624888 wait complete. 2024-03-26T13:58:33.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 Processor -- start 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 -- start start 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb81038e0 0x7f7bb8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb8104ad0 0x7f7bb819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 0x7f7bb81a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb810f3f0 con 0x7f7bb8105490 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb810f270 con 0x7f7bb8104ad0 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bbd600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb810f570 con 0x7f7bb81038e0 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bb7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 0x7f7bb81a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bb7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 0x7f7bb81a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36696/0 (socket says 172.21.15.137:36696) 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.305+0000 7f7bb7400640 1 -- 172.21.15.137:0/578136092 learned_addr learned my addr 172.21.15.137:0/578136092 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb6a00640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb81038e0 0x7f7bb8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb7400640 1 -- 172.21.15.137:0/578136092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb81038e0 msgr2=0x7f7bb8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb7400640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb81038e0 0x7f7bb8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb7400640 1 -- 172.21.15.137:0/578136092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb8104ad0 msgr2=0x7f7bb819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb7400640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb8104ad0 0x7f7bb819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb7400640 1 -- 172.21.15.137:0/578136092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb81a1740 con 0x7f7bb8105490 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bb7400640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 0x7f7bb81a10d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009fd0 tx=0x7f7bb000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0017070 con 0x7f7bb8105490 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7bb0003e20 con 0x7f7bb8105490 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0010460 con 0x7f7bb8105490 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.306+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb81a1a30 con 0x7f7bb8105490 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.307+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb81a1f30 con 0x7f7bb8105490 2024-03-26T13:58:33.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.308+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb810acf0 con 0x7f7bb8105490 2024-03-26T13:58:33.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.309+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7bb0003970 con 0x7f7bb8105490 2024-03-26T13:58:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.311+0000 7f7bafe00640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b9c0a2800 0x7f7b9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.311+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7bb00ce500 con 0x7f7bb8105490 2024-03-26T13:58:33.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.311+0000 7f7bb6a00640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b9c0a2800 0x7f7b9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:33.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.312+0000 7f7bb6a00640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b9c0a2800 0x7f7b9c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8104930 tx=0x7f7ba8005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:33.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.315+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7bb009b9a0 con 0x7f7bb8105490 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.179:6832/2724223163,v1:172.21.15.179:6833/2724223163] up:boot 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: fsmap 1 up:standby 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:33.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:33.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.179:6832/2724223163,v1:172.21.15.179:6833/2724223163] up:boot 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: fsmap 1 up:standby 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:33.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:33.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.556+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb802d070 con 0x7f7bb8105490 2024-03-26T13:58:33.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.557+0000 7f7bafe00640 1 -- 172.21.15.137:0/578136092 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f7bb009b2a0 con 0x7f7bb8105490 2024-03-26T13:58:33.553 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:33.553 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T13:58:33.553 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 2 2024-03-26T13:58:33.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.560+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b9c0a2800 msgr2=0x7f7b9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:33.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.560+0000 7f7bbd600640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b9c0a2800 0x7f7b9c0a4cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8104930 tx=0x7f7ba8005c60 comp rx=0 tx=0).stop 2024-03-26T13:58:33.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.560+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 msgr2=0x7f7bb81a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:33.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.560+0000 7f7bbd600640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 0x7f7bb81a10d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009fd0 tx=0x7f7bb000b260 comp rx=0 tx=0).stop 2024-03-26T13:58:33.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 shutdown_connections 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b9c0a2800 0x7f7b9c0a4cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb8104ad0 0x7f7bb819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb8105490 0x7f7bb81a10d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 --2- 172.21.15.137:0/578136092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb81038e0 0x7f7bb8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 >> 172.21.15.137:0/578136092 conn(0x7f7bb80fee50 msgr2=0x7f7bb8107ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 shutdown_connections 2024-03-26T13:58:33.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:33.561+0000 7f7bbd600640 1 -- 172.21.15.137:0/578136092 wait complete. 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.179:6832/2724223163,v1:172.21.15.179:6833/2724223163] up:boot 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: fsmap 1 up:standby 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:33.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:34.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:34 smithi159 ceph-mon[35501]: Deploying daemon mds.k on smithi159 2024-03-26T13:58:34.481 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:34 smithi159 ceph-mon[35501]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:34.481 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/578136092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:34.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:34 smithi137 ceph-mon[31539]: Deploying daemon mds.k on smithi159 2024-03-26T13:58:34.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:34 smithi137 ceph-mon[31539]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:34.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/578136092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:34.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:34 smithi179 ceph-mon[35472]: Deploying daemon mds.k on smithi159 2024-03-26T13:58:34.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:34 smithi179 ceph-mon[35472]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:34.572 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/578136092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:35.577 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:35.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.800+0000 7f2f52000640 1 -- 172.21.15.137:0/3197535271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1048e0 msgr2=0x7f2f4c104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.800+0000 7f2f52000640 1 --2- 172.21.15.137:0/3197535271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c104d60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009900 tx=0x7f2f3802bc30 comp rx=0 tx=0).stop 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 -- 172.21.15.137:0/3197535271 shutdown_connections 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 --2- 172.21.15.137:0/3197535271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1052a0 0x7f2f4c10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 --2- 172.21.15.137:0/3197535271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c104d60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 --2- 172.21.15.137:0/3197535271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 -- 172.21.15.137:0/3197535271 >> 172.21.15.137:0/3197535271 conn(0x7f2f4c0fec80 msgr2=0x7f2f4c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 -- 172.21.15.137:0/3197535271 shutdown_connections 2024-03-26T13:58:35.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.801+0000 7f2f52000640 1 -- 172.21.15.137:0/3197535271 wait complete. 2024-03-26T13:58:35.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 Processor -- start 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 -- start start 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f4c1052a0 0x7f2f4c1a1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c10f360 con 0x7f2f4c1036f0 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c10f1e0 con 0x7f2f4c1048e0 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.802+0000 7f2f52000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c10f4e0 con 0x7f2f4c1052a0 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36714/0 (socket says 172.21.15.137:36714) 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 -- 172.21.15.137:0/2877323263 learned_addr learned my addr 172.21.15.137:0/2877323263 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f4be00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f51600640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f4c1052a0 0x7f2f4c1a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:35.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 -- 172.21.15.137:0/2877323263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f4c1052a0 msgr2=0x7f2f4c1a1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f4c1052a0 0x7f2f4c1a1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 -- 172.21.15.137:0/2877323263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1048e0 msgr2=0x7f2f4c199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f50c00640 1 -- 172.21.15.137:0/2877323263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c1a1680 con 0x7f2f4c1036f0 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.803+0000 7f2f4be00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.804+0000 7f2f50c00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c199a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000b450 tx=0x7f2f4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.804+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40015020 con 0x7f2f4c1036f0 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.804+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f40011d40 con 0x7f2f4c1036f0 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.804+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000f340 con 0x7f2f4c1036f0 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.804+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c1a1970 con 0x7f2f4c1036f0 2024-03-26T13:58:35.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.804+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c1a1ef0 con 0x7f2f4c1036f0 2024-03-26T13:58:35.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.805+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c10aa60 con 0x7f2f4c1036f0 2024-03-26T13:58:35.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.807+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2f40018020 con 0x7f2f4c1036f0 2024-03-26T13:58:35.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.808+0000 7f2f49600640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f340a2880 0x7f2f340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:35.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.808+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2f400ce160 con 0x7f2f4c1036f0 2024-03-26T13:58:35.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.809+0000 7f2f4be00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f340a2880 0x7f2f340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:35.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.810+0000 7f2f4be00640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f340a2880 0x7f2f340a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c19af70 tx=0x7f2f38005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:35.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:35.814+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2f4001f020 con 0x7f2f4c1036f0 2024-03-26T13:58:35.810 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:35 smithi159 systemd[1]: Starting Ceph mds.k for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:58:36.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.048+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f4c02d070 con 0x7f2f4c1036f0 2024-03-26T13:58:36.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.049+0000 7f2f49600640 1 -- 172.21.15.137:0/2877323263 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f2f4000d040 con 0x7f2f4c1036f0 2024-03-26T13:58:36.045 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:36.045 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-26T13:58:36.045 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 2 2024-03-26T13:58:36.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.051+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f340a2880 msgr2=0x7f2f340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.051+0000 7f2f52000640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f340a2880 0x7f2f340a4d40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c19af70 tx=0x7f2f38005fd0 comp rx=0 tx=0).stop 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.051+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 msgr2=0x7f2f4c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.051+0000 7f2f52000640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c199a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000b450 tx=0x7f2f4000b910 comp rx=0 tx=0).stop 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 shutdown_connections 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f340a2880 0x7f2f340a4d40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f4c1048e0 0x7f2f4c199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f4c1052a0 0x7f2f4c1a1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 --2- 172.21.15.137:0/2877323263 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f4c1036f0 0x7f2f4c199a60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 >> 172.21.15.137:0/2877323263 conn(0x7f2f4c0fec80 msgr2=0x7f2f4c107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 shutdown_connections 2024-03-26T13:58:36.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:36.052+0000 7f2f52000640 1 -- 172.21.15.137:0/2877323263 wait complete. 2024-03-26T13:58:36.167 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:35 smithi159 podman[64873]: 2024-03-26 13:58:35.718254002 +0000 UTC m=+0.034916023 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:58:36.167 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:35 smithi159 podman[64873]: 2024-03-26 13:58:35.882688377 +0000 UTC m=+0.199350425 container create 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T13:58:36.168 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:36 smithi159 podman[64873]: 2024-03-26 13:58:36.169127839 +0000 UTC m=+0.485789894 container init 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T13:58:36.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:36 smithi159 ceph-mon[35501]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:36.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:36 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2877323263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:36.449 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:36 smithi159 podman[64873]: 2024-03-26 13:58:36.181057021 +0000 UTC m=+0.497719049 container start 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T13:58:36.449 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:36 smithi159 ceph-mds[64892]: -- 172.21.15.159:0/2891440216 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bc549ffa40 con 0x55bc549d3000 2024-03-26T13:58:36.449 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k[64888]: starting mds.k at 2024-03-26T13:58:36.449 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:36 smithi159 bash[64873]: 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd 2024-03-26T13:58:36.449 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 13:58:36 smithi159 systemd[1]: Started Ceph mds.k for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:36.530 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:36 smithi137 ceph-mon[31539]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:36.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:36 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2877323263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:36.669 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:36 smithi179 ceph-mon[35472]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:36.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:36 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2877323263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: Deploying daemon mds.j on smithi137 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6832/1469472888,v1:172.21.15.159:6833/1469472888] up:boot 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: fsmap 2 up:standby 2024-03-26T13:58:37.675 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: Deploying daemon mds.j on smithi137 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6832/1469472888,v1:172.21.15.159:6833/1469472888] up:boot 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: fsmap 2 up:standby 2024-03-26T13:58:37.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: Deploying daemon mds.j on smithi137 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:37.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6832/1469472888,v1:172.21.15.159:6833/1469472888] up:boot 2024-03-26T13:58:37.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: fsmap 2 up:standby 2024-03-26T13:58:37.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T13:58:38.067 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2631023319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 msgr2=0x7f4c4810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2631023319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c4810bd20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009ca0 tx=0x7f4c4002c120 comp rx=0 tx=0).stop 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2631023319 shutdown_connections 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2631023319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 0x7f4c48104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2631023319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c4810bd20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2631023319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c481038e0 0x7f4c48103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2631023319 >> 172.21.15.137:0/2631023319 conn(0x7f4c480fee90 msgr2=0x7f4c481012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:38.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2631023319 shutdown_connections 2024-03-26T13:58:38.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.303+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2631023319 wait complete. 2024-03-26T13:58:38.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 Processor -- start 2024-03-26T13:58:38.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 -- start start 2024-03-26T13:58:38.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c481038e0 0x7f4c48199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:38.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 0x7f4c4819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:38.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c481a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4810f500 con 0x7f4c48105490 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4810f380 con 0x7f4c48104ad0 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4810f680 con 0x7f4c481038e0 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.304+0000 7f4c4ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c481a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c481a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43876/0 (socket says 172.21.15.137:43876) 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 -- 172.21.15.137:0/2313196755 learned_addr learned my addr 172.21.15.137:0/2313196755 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 -- 172.21.15.137:0/2313196755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c481038e0 msgr2=0x7f4c48199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c47e00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c481038e0 0x7f4c48199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c481038e0 0x7f4c48199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 -- 172.21.15.137:0/2313196755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 msgr2=0x7f4c4819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c47400640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 0x7f4c4819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:38.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 0x7f4c4819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c4ce00640 1 -- 172.21.15.137:0/2313196755 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c481a17f0 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.305+0000 7f4c47400640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 0x7f4c4819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.306+0000 7f4c4ce00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c481a1180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c40032380 tx=0x7f4c40035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.306+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40039040 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.306+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c40044070 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.306+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c481a1a80 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.306+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400043a0 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.306+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c481a1ff0 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.308+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4810ac50 con 0x7f4c48105490 2024-03-26T13:58:38.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.309+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4c40004540 con 0x7f4c48105490 2024-03-26T13:58:38.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.311+0000 7f4c44c00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c2c0a2800 0x7f4c2c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:38.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.311+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c400f0cd0 con 0x7f4c48105490 2024-03-26T13:58:38.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.311+0000 7f4c47e00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c2c0a2800 0x7f4c2c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:38.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.312+0000 7f4c47e00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c2c0a2800 0x7f4c2c0a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c48104930 tx=0x7f4c38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:38.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.315+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c400098b0 con 0x7f4c48105490 2024-03-26T13:58:38.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.560+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c4802d070 con 0x7f4c48105490 2024-03-26T13:58:38.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.561+0000 7f4c44c00640 1 -- 172.21.15.137:0/2313196755 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f4c400be170 con 0x7f4c48105490 2024-03-26T13:58:38.557 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:38.557 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T13:58:38.557 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 3 2024-03-26T13:58:38.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c2c0a2800 msgr2=0x7f4c2c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:38.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c2c0a2800 0x7f4c2c0a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c48104930 tx=0x7f4c38008040 comp rx=0 tx=0).stop 2024-03-26T13:58:38.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 msgr2=0x7f4c481a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:38.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c481a1180 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c40032380 tx=0x7f4c40035ce0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 shutdown_connections 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c2c0a2800 0x7f4c2c0a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c48104ad0 0x7f4c4819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c48105490 0x7f4c481a1180 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 --2- 172.21.15.137:0/2313196755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c481038e0 0x7f4c48199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 >> 172.21.15.137:0/2313196755 conn(0x7f4c480fee90 msgr2=0x7f4c48107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 shutdown_connections 2024-03-26T13:58:38.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:38.564+0000 7f4c4ee00640 1 -- 172.21.15.137:0/2313196755 wait complete. 2024-03-26T13:58:38.751 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:38 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2313196755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:38.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:38 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2313196755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:38.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:38 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2313196755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:39.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:39 smithi137 ceph-mon[31539]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:39.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:39 smithi179 ceph-mon[35472]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:39.948 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:39 smithi159 ceph-mon[35501]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:40.177 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:39 smithi137 podman[71880]: 2024-03-26 13:58:39.909393308 +0000 UTC m=+0.164321108 container create 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T13:58:40.177 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:40 smithi137 podman[71880]: 2024-03-26 13:58:40.180062681 +0000 UTC m=+0.434990520 container init 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-26T13:58:40.531 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:40 smithi137 podman[71880]: 2024-03-26 13:58:40.191446955 +0000 UTC m=+0.446374742 container start 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:58:40.531 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:40 smithi137 ceph-mds[71898]: -- 172.21.15.137:0/625507249 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f1e7309a40 con 0x55f1e72dd000 2024-03-26T13:58:40.531 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:40 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j[71894]: starting mds.j at 2024-03-26T13:58:40.531 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:40 smithi137 bash[71880]: 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 2024-03-26T13:58:40.531 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 13:58:40 smithi137 systemd[1]: Started Ceph mds.j for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:40.581 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:40.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 -- 172.21.15.137:0/3274224295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c1038e0 msgr2=0x7f384c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 --2- 172.21.15.137:0/3274224295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c1038e0 0x7f384c103d00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f383c009900 tx=0x7f383c02bc30 comp rx=0 tx=0).stop 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 -- 172.21.15.137:0/3274224295 shutdown_connections 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 --2- 172.21.15.137:0/3274224295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c105490 0x7f384c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 --2- 172.21.15.137:0/3274224295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c104ad0 0x7f384c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 --2- 172.21.15.137:0/3274224295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c1038e0 0x7f384c103d00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.791+0000 7f3852400640 1 -- 172.21.15.137:0/3274224295 >> 172.21.15.137:0/3274224295 conn(0x7f384c0fee90 msgr2=0x7f384c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.792+0000 7f3852400640 1 -- 172.21.15.137:0/3274224295 shutdown_connections 2024-03-26T13:58:40.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.792+0000 7f3852400640 1 -- 172.21.15.137:0/3274224295 wait complete. 2024-03-26T13:58:40.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 Processor -- start 2024-03-26T13:58:40.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 -- start start 2024-03-26T13:58:40.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c1038e0 0x7f384c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:40.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 0x7f384c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:40.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c105490 0x7f384c1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:40.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c10f450 con 0x7f384c104ad0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c10f2d0 con 0x7f384c105490 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3852400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c10f5d0 con 0x7f384c1038e0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3843e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 0x7f384c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3843e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 0x7f384c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43892/0 (socket says 172.21.15.137:43892) 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.793+0000 7f3843e00640 1 -- 172.21.15.137:0/2256193529 learned_addr learned my addr 172.21.15.137:0/2256193529 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f384b400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c1038e0 0x7f384c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f384be00640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c105490 0x7f384c1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3843e00640 1 -- 172.21.15.137:0/2256193529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c1038e0 msgr2=0x7f384c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3843e00640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c1038e0 0x7f384c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3843e00640 1 -- 172.21.15.137:0/2256193529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c105490 msgr2=0x7f384c1a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3843e00640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c105490 0x7f384c1a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3843e00640 1 -- 172.21.15.137:0/2256193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c1a1770 con 0x7f384c104ad0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3843e00640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 0x7f384c19a090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f383400d770 tx=0x7f383400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.794+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834017070 con 0x7f384c104ad0 2024-03-26T13:58:40.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.795+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384c1a1a60 con 0x7f384c104ad0 2024-03-26T13:58:40.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.795+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f38340054d0 con 0x7f384c104ad0 2024-03-26T13:58:40.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.795+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834004cf0 con 0x7f384c104ad0 2024-03-26T13:58:40.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.795+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384c1a1f60 con 0x7f384c104ad0 2024-03-26T13:58:40.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.796+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384c10acd0 con 0x7f384c104ad0 2024-03-26T13:58:40.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.798+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3834004e90 con 0x7f384c104ad0 2024-03-26T13:58:40.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.799+0000 7f3848c00640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38300a2800 0x7f38300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:40.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.800+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f38340ce0d0 con 0x7f384c104ad0 2024-03-26T13:58:40.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.800+0000 7f384b400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38300a2800 0x7f38300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:40.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.800+0000 7f384b400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38300a2800 0x7f38300a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f383c02c240 tx=0x7f383c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:40.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:40.804+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f383409b610 con 0x7f384c104ad0 2024-03-26T13:58:41.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.046+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f384c02d070 con 0x7f384c104ad0 2024-03-26T13:58:41.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.047+0000 7f3848c00640 1 -- 172.21.15.137:0/2256193529 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2916 (secure 0 0 0) 0x7f383409af10 con 0x7f384c104ad0 2024-03-26T13:58:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:41.042 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T13:58:41.043 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 4 2024-03-26T13:58:41.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.049+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38300a2800 msgr2=0x7f38300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:41.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.049+0000 7f3852400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38300a2800 0x7f38300a4cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f383c02c240 tx=0x7f383c005fd0 comp rx=0 tx=0).stop 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.049+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 msgr2=0x7f384c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.049+0000 7f3852400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 0x7f384c19a090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f383400d770 tx=0x7f383400dc30 comp rx=0 tx=0).stop 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 shutdown_connections 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38300a2800 0x7f38300a4cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c105490 0x7f384c1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f384c104ad0 0x7f384c19a090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 --2- 172.21.15.137:0/2256193529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f384c1038e0 0x7f384c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 >> 172.21.15.137:0/2256193529 conn(0x7f384c0fee90 msgr2=0x7f384c107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 shutdown_connections 2024-03-26T13:58:41.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:41.050+0000 7f3852400640 1 -- 172.21.15.137:0/2256193529 wait complete. 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: Deploying daemon mds.d on smithi137 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.137:6834/4090447897,v1:172.21.15.137:6835/4090447897] up:boot 2024-03-26T13:58:41.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: fsmap 3 up:standby 2024-03-26T13:58:41.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T13:58:41.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2256193529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:41.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:41 smithi159 ceph-mon[35501]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: Deploying daemon mds.d on smithi137 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.137:6834/4090447897,v1:172.21.15.137:6835/4090447897] up:boot 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: fsmap 3 up:standby 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2256193529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:41.730 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:41 smithi137 ceph-mon[31539]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: Deploying daemon mds.d on smithi137 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.137:6834/4090447897,v1:172.21.15.137:6835/4090447897] up:boot 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: fsmap 3 up:standby 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2256193529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:41.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:41 smithi179 ceph-mon[35472]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:43.065 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:43.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 -- 172.21.15.137:0/4618229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006b660 msgr2=0x7ff2a006ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:43.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4618229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006b660 0x7ff2a006ba80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009900 tx=0x7ff28c02bc50 comp rx=0 tx=0).stop 2024-03-26T13:58:43.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 -- 172.21.15.137:0/4618229 shutdown_connections 2024-03-26T13:58:43.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4618229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a006c8a0 0x7ff2a006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4618229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2a006d240 0x7ff2a0071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4618229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006b660 0x7ff2a006ba80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 -- 172.21.15.137:0/4618229 >> 172.21.15.137:0/4618229 conn(0x7ff2a00fbb30 msgr2=0x7ff2a00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 -- 172.21.15.137:0/4618229 shutdown_connections 2024-03-26T13:58:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.410+0000 7ff2a5600640 1 -- 172.21.15.137:0/4618229 wait complete. 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.411+0000 7ff2a5600640 1 Processor -- start 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.411+0000 7ff2a5600640 1 -- start start 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff2a5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a006b660 0x7ff2a01a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff2a5600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 0x7ff2a01a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff2a5600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2a006d240 0x7ff2a01a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff2a5600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a0114970 con 0x7ff2a006c8a0 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff2a5600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a01147f0 con 0x7ff2a006b660 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff2a5600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a0114af0 con 0x7ff2a006d240 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff29e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 0x7ff2a01a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff29e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 0x7ff2a01a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43922/0 (socket says 172.21.15.137:43922) 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff29e000640 1 -- 172.21.15.137:0/4032724147 learned_addr learned my addr 172.21.15.137:0/4032724147 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.412+0000 7ff29ea00640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a006b660 0x7ff2a01a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff29e000640 1 -- 172.21.15.137:0/4032724147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2a006d240 msgr2=0x7ff2a01a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff29e000640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2a006d240 0x7ff2a01a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff29e000640 1 -- 172.21.15.137:0/4032724147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a006b660 msgr2=0x7ff2a01a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff29e000640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a006b660 0x7ff2a01a2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff29e000640 1 -- 172.21.15.137:0/4032724147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a01a9f40 con 0x7ff2a006c8a0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff29e000640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 0x7ff2a01a28b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff29800ebc0 tx=0x7ff29800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298010630 con 0x7ff2a006c8a0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff298010c80 con 0x7ff2a006c8a0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a01aa230 con 0x7ff2a006c8a0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298018b30 con 0x7ff2a006c8a0 2024-03-26T13:58:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.413+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a01aa7d0 con 0x7ff2a006c8a0 2024-03-26T13:58:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.414+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a01aaa30 con 0x7ff2a006c8a0 2024-03-26T13:58:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.417+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff298018cd0 con 0x7ff2a006c8a0 2024-03-26T13:58:43.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.418+0000 7ff297400640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2840a2880 0x7ff2840a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:43.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.418+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff2980cecd0 con 0x7ff2a006c8a0 2024-03-26T13:58:43.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.418+0000 7ff29ea00640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2840a2880 0x7ff2840a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:43.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.419+0000 7ff29ea00640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2840a2880 0x7ff2840a4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff28c02c190 tx=0x7ff28c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:43.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.422+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff29809c170 con 0x7ff2a006c8a0 2024-03-26T13:58:43.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.670+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2a01aaca0 con 0x7ff2a006c8a0 2024-03-26T13:58:43.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.671+0000 7ff297400640 1 -- 172.21.15.137:0/4032724147 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2916 (secure 0 0 0) 0x7ff29809ba70 con 0x7ff2a006c8a0 2024-03-26T13:58:43.667 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:43.667 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T13:58:43.667 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 4 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2840a2880 msgr2=0x7ff2840a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2840a2880 0x7ff2840a4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff28c02c190 tx=0x7ff28c005fd0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 msgr2=0x7ff2a01a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 0x7ff2a01a28b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff29800ebc0 tx=0x7ff29800c6a0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 shutdown_connections 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2840a2880 0x7ff2840a4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2a006d240 0x7ff2a01a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2a006c8a0 0x7ff2a01a28b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 --2- 172.21.15.137:0/4032724147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2a006b660 0x7ff2a01a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 >> 172.21.15.137:0/4032724147 conn(0x7ff2a00fbb30 msgr2=0x7ff2a00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 shutdown_connections 2024-03-26T13:58:43.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:43.674+0000 7ff2a5600640 1 -- 172.21.15.137:0/4032724147 wait complete. 2024-03-26T13:58:44.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:44 smithi137 ceph-mon[31539]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:44.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:44 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4032724147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:44.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:44 smithi179 ceph-mon[35472]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:44.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:44 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4032724147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:44.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:44 smithi159 ceph-mon[35501]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:44.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:44 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4032724147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:44.781 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 13:58:44 smithi137 systemd[1]: Started Ceph mds.d for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:45.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: Deploying daemon mds.e on smithi159 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.137:6836/4057650912,v1:172.21.15.137:6837/4057650912] up:boot 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: fsmap 4 up:standby 2024-03-26T13:58:45.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T13:58:45.780 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: Deploying daemon mds.e on smithi159 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.137:6836/4057650912,v1:172.21.15.137:6837/4057650912] up:boot 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: fsmap 4 up:standby 2024-03-26T13:58:45.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T13:58:45.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 -- 172.21.15.137:0/2219703928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a80105490 msgr2=0x7f2a8010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:45.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 --2- 172.21.15.137:0/2219703928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a80105490 0x7f2a8010bd20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009900 tx=0x7f2a7802bc30 comp rx=0 tx=0).stop 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 -- 172.21.15.137:0/2219703928 shutdown_connections 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 --2- 172.21.15.137:0/2219703928 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a80105490 0x7f2a8010bd20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 --2- 172.21.15.137:0/2219703928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a80104ad0 0x7f2a80104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 --2- 172.21.15.137:0/2219703928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a801038e0 0x7f2a80103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 -- 172.21.15.137:0/2219703928 >> 172.21.15.137:0/2219703928 conn(0x7f2a800fee90 msgr2=0x7f2a801012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 -- 172.21.15.137:0/2219703928 shutdown_connections 2024-03-26T13:58:45.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.920+0000 7f2a85c00640 1 -- 172.21.15.137:0/2219703928 wait complete. 2024-03-26T13:58:45.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.921+0000 7f2a85c00640 1 Processor -- start 2024-03-26T13:58:45.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.921+0000 7f2a85c00640 1 -- start start 2024-03-26T13:58:45.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a85c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 0x7f2a80199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:45.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a85c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a80104ad0 0x7f2a8019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:45.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a85c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a80105490 0x7f2a801a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:45.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a85c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8010f460 con 0x7f2a801038e0 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a85c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8010f2e0 con 0x7f2a80105490 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a85c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8010f5e0 con 0x7f2a80104ad0 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 0x7f2a80199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 0x7f2a80199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43972/0 (socket says 172.21.15.137:43972) 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 -- 172.21.15.137:0/3064899122 learned_addr learned my addr 172.21.15.137:0/3064899122 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7f400640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a80105490 0x7f2a801a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7e000640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a80104ad0 0x7f2a8019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 -- 172.21.15.137:0/3064899122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a80104ad0 msgr2=0x7f2a8019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:45.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a80104ad0 0x7f2a8019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 -- 172.21.15.137:0/3064899122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a80105490 msgr2=0x7f2a801a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a80105490 0x7f2a801a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.922+0000 7f2a7ea00640 1 -- 172.21.15.137:0/3064899122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a801a16f0 con 0x7f2a801038e0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.923+0000 7f2a7ea00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 0x7f2a80199ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009fd0 tx=0x7f2a7000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.923+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70011470 con 0x7f2a801038e0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.923+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a801a19e0 con 0x7f2a801038e0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.923+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a70011ac0 con 0x7f2a801038e0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.923+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7001aa50 con 0x7f2a801038e0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.923+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a801a1f80 con 0x7f2a801038e0 2024-03-26T13:58:45.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.924+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8010ac00 con 0x7f2a801038e0 2024-03-26T13:58:45.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: Deploying daemon mds.e on smithi159 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.137:6836/4057650912,v1:172.21.15.137:6837/4057650912] up:boot 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: fsmap 4 up:standby 2024-03-26T13:58:45.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-26T13:58:45.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.926+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2a70011c70 con 0x7f2a801038e0 2024-03-26T13:58:45.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.927+0000 7f2a77400640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a640a2800 0x7f2a640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:45.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.928+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a700cec90 con 0x7f2a801038e0 2024-03-26T13:58:45.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.928+0000 7f2a7e000640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a640a2800 0x7f2a640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:45.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.928+0000 7f2a7e000640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a640a2800 0x7f2a640a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a68005fd0 tx=0x7f2a6800b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:45.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:45.932+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a70016030 con 0x7f2a801038e0 2024-03-26T13:58:46.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.171+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a8002d070 con 0x7f2a801038e0 2024-03-26T13:58:46.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.172+0000 7f2a77400640 1 -- 172.21.15.137:0/3064899122 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3710 (secure 0 0 0) 0x7f2a7009c130 con 0x7f2a801038e0 2024-03-26T13:58:46.169 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:46.169 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T13:58:46.169 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 5 2024-03-26T13:58:46.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a640a2800 msgr2=0x7f2a640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:46.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a640a2800 0x7f2a640a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a68005fd0 tx=0x7f2a6800b510 comp rx=0 tx=0).stop 2024-03-26T13:58:46.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 msgr2=0x7f2a80199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:46.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 0x7f2a80199ad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2a70009fd0 tx=0x7f2a7000c270 comp rx=0 tx=0).stop 2024-03-26T13:58:46.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 shutdown_connections 2024-03-26T13:58:46.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a640a2800 0x7f2a640a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a80105490 0x7f2a801a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a80104ad0 0x7f2a8019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 --2- 172.21.15.137:0/3064899122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a801038e0 0x7f2a80199ad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.175+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 >> 172.21.15.137:0/3064899122 conn(0x7f2a800fee90 msgr2=0x7f2a80107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.176+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 shutdown_connections 2024-03-26T13:58:46.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:46.176+0000 7f2a85c00640 1 -- 172.21.15.137:0/3064899122 wait complete. 2024-03-26T13:58:46.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:46 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3064899122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:46.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:46 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3064899122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:47.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:46 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3064899122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:47.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:47 smithi179 ceph-mon[35472]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:47.941 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:47 smithi159 ceph-mon[35501]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:48.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:47 smithi137 ceph-mon[31539]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:48.191 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:48.199 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:47 smithi159 systemd[1]: Starting Ceph mds.e for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.413+0000 7f7199e00640 1 -- 172.21.15.137:0/592269590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 msgr2=0x7f7194104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.413+0000 7f7199e00640 1 --2- 172.21.15.137:0/592269590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f7194104b40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f717c009900 tx=0x7f717c02bc50 comp rx=0 tx=0).stop 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 -- 172.21.15.137:0/592269590 shutdown_connections 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 --2- 172.21.15.137:0/592269590 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7194105080 0x7f7194105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 --2- 172.21.15.137:0/592269590 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f7194104b40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 --2- 172.21.15.137:0/592269590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71940ffd60 0x7f7194102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 -- 172.21.15.137:0/592269590 >> 172.21.15.137:0/592269590 conn(0x7f71940fb960 msgr2=0x7f71940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:48.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 -- 172.21.15.137:0/592269590 shutdown_connections 2024-03-26T13:58:48.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.414+0000 7f7199e00640 1 -- 172.21.15.137:0/592269590 wait complete. 2024-03-26T13:58:48.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 Processor -- start 2024-03-26T13:58:48.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 -- start start 2024-03-26T13:58:48.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71940ffd60 0x7f719419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:48.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f719419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:48.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7194105080 0x7f71941a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:48.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7194113640 con 0x7f71941026d0 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71941134c0 con 0x7f7194105080 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f7199e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71941137c0 con 0x7f71940ffd60 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f718be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f719419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f718be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f719419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38866/0 (socket says 172.21.15.137:38866) 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.415+0000 7f718be00640 1 -- 172.21.15.137:0/4009479943 learned_addr learned my addr 172.21.15.137:0/4009479943 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7198a00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71940ffd60 0x7f719419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7199400640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7194105080 0x7f71941a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f718be00640 1 -- 172.21.15.137:0/4009479943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71940ffd60 msgr2=0x7f719419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:48.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f718be00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71940ffd60 0x7f719419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f718be00640 1 -- 172.21.15.137:0/4009479943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7194105080 msgr2=0x7f71941a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f718be00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7194105080 0x7f71941a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f718be00640 1 -- 172.21.15.137:0/4009479943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71941a5940 con 0x7f71941026d0 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7199400640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7194105080 0x7f71941a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f718be00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f719419e270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f717c000c00 tx=0x7f717c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c043020 con 0x7f71941026d0 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f717c0054c0 con 0x7f71941026d0 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c003d80 con 0x7f71941026d0 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71941a5bd0 con 0x7f71941026d0 2024-03-26T13:58:48.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.416+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71941a6170 con 0x7f71941026d0 2024-03-26T13:58:48.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.418+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71940fd450 con 0x7f71941026d0 2024-03-26T13:58:48.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.419+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f717c046020 con 0x7f71941026d0 2024-03-26T13:58:48.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.421+0000 7f7192000640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71780a2880 0x7f71780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:48.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.421+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f717c0f0b50 con 0x7f71941026d0 2024-03-26T13:58:48.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.421+0000 7f7198a00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71780a2880 0x7f71780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:48.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.422+0000 7f7198a00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71780a2880 0x7f71780a4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7184005c40 tx=0x7f718400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:48.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.425+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f717c04d020 con 0x7f71941026d0 2024-03-26T13:58:48.633 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 podman[65176]: 2024-03-26 13:58:48.209246413 +0000 UTC m=+0.036549713 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:58:48.633 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 podman[65176]: 2024-03-26 13:58:48.3386188 +0000 UTC m=+0.165922133 container create d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319) 2024-03-26T13:58:48.633 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 podman[65176]: 2024-03-26 13:58:48.63704521 +0000 UTC m=+0.464348549 container init d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T13:58:48.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.664+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f719402d070 con 0x7f71941026d0 2024-03-26T13:58:48.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.666+0000 7f7192000640 1 -- 172.21.15.137:0/4009479943 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3710 (secure 0 0 0) 0x7f717c0be110 con 0x7f71941026d0 2024-03-26T13:58:48.662 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:48.662 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-26T13:58:48.662 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 5 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71780a2880 msgr2=0x7f71780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71780a2880 0x7f71780a4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7184005c40 tx=0x7f718400a3b0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 msgr2=0x7f719419e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f719419e270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f717c000c00 tx=0x7f717c0042f0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 shutdown_connections 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71780a2880 0x7f71780a4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7194105080 0x7f71941a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71941026d0 0x7f719419e270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 --2- 172.21.15.137:0/4009479943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71940ffd60 0x7f719419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:48.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.668+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 >> 172.21.15.137:0/4009479943 conn(0x7f71940fb960 msgr2=0x7f71941006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:48.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.669+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 shutdown_connections 2024-03-26T13:58:48.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:48.669+0000 7f7199e00640 1 -- 172.21.15.137:0/4009479943 wait complete. 2024-03-26T13:58:48.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:48 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4009479943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:48.949 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 podman[65176]: 2024-03-26 13:58:48.64794105 +0000 UTC m=+0.475244356 container start d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-26T13:58:48.949 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 ceph-mds[65195]: -- 172.21.15.159:0/4215004093 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55723b7bda40 con 0x55723b790800 2024-03-26T13:58:48.949 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e[65191]: starting mds.e at 2024-03-26T13:58:48.949 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 bash[65176]: d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 2024-03-26T13:58:48.950 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 13:58:48 smithi159 systemd[1]: Started Ceph mds.e for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:49.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:48 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4009479943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:49.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:48 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4009479943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:50.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] up:boot 2024-03-26T13:58:50.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: fsmap 5 up:standby 2024-03-26T13:58:50.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T13:58:50.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: Deploying daemon mds.f on smithi179 2024-03-26T13:58:50.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:49 smithi137 ceph-mon[31539]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:50.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] up:boot 2024-03-26T13:58:50.107 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: fsmap 5 up:standby 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: Deploying daemon mds.f on smithi179 2024-03-26T13:58:50.108 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:49 smithi179 ceph-mon[35472]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:50.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] up:boot 2024-03-26T13:58:50.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: fsmap 5 up:standby 2024-03-26T13:58:50.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T13:58:50.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:50.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:50.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:50.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:50.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: Deploying daemon mds.f on smithi179 2024-03-26T13:58:50.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:49 smithi159 ceph-mon[35501]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:50.684 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:50.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 -- 172.21.15.137:0/1181076864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c0772a0 msgr2=0x7f900c077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:50.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 --2- 172.21.15.137:0/1181076864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c0772a0 0x7f900c077700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009900 tx=0x7f8ff402bc30 comp rx=0 tx=0).stop 2024-03-26T13:58:50.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 -- 172.21.15.137:0/1181076864 shutdown_connections 2024-03-26T13:58:50.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 --2- 172.21.15.137:0/1181076864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c077c40 0x7f900c0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:50.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 --2- 172.21.15.137:0/1181076864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c0772a0 0x7f900c077700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:50.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 --2- 172.21.15.137:0/1181076864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c076060 0x7f900c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:50.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 -- 172.21.15.137:0/1181076864 >> 172.21.15.137:0/1181076864 conn(0x7f900c0fee50 msgr2=0x7f900c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:50.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.911+0000 7f9012000640 1 -- 172.21.15.137:0/1181076864 shutdown_connections 2024-03-26T13:58:50.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.912+0000 7f9012000640 1 -- 172.21.15.137:0/1181076864 wait complete. 2024-03-26T13:58:50.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.912+0000 7f9012000640 1 Processor -- start 2024-03-26T13:58:50.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 -- start start 2024-03-26T13:58:50.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c076060 0x7f900c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:50.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c0772a0 0x7f900c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:50.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 0x7f900c1a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:50.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c113860 con 0x7f900c077c40 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1136e0 con 0x7f900c0772a0 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9012000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1139e0 con 0x7f900c076060 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f900be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 0x7f900c1a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f900be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 0x7f900c1a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38882/0 (socket says 172.21.15.137:38882) 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f900be00640 1 -- 172.21.15.137:0/3825409113 learned_addr learned my addr 172.21.15.137:0/3825409113 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f900b400640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c076060 0x7f900c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.913+0000 7f9002a00640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c0772a0 0x7f900c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f900be00640 1 -- 172.21.15.137:0/3825409113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c076060 msgr2=0x7f900c19def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f900be00640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c076060 0x7f900c19def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:50.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f900be00640 1 -- 172.21.15.137:0/3825409113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c0772a0 msgr2=0x7f900c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f900be00640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c0772a0 0x7f900c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f900be00640 1 -- 172.21.15.137:0/3825409113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c1a5b10 con 0x7f900c077c40 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f9002a00640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c0772a0 0x7f900c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f900be00640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 0x7f900c1a54a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f900400eaa0 tx=0x7f900400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900400cdb0 con 0x7f900c077c40 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9004005020 con 0x7f900c077c40 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040116d0 con 0x7f900c077c40 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c1a5e00 con 0x7f900c077c40 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.914+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1a6270 con 0x7f900c077c40 2024-03-26T13:58:50.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.915+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c110430 con 0x7f900c077c40 2024-03-26T13:58:50.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.917+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f90040051d0 con 0x7f900c077c40 2024-03-26T13:58:50.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.919+0000 7f9008c00640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8ff00a2880 0x7f8ff00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:50.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.919+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f90040ce780 con 0x7f900c077c40 2024-03-26T13:58:50.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.919+0000 7f900b400640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8ff00a2880 0x7f8ff00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:50.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.920+0000 7f900b400640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8ff00a2880 0x7f8ff00a4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00bbe0 tx=0x7f8ffc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:50.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:50.924+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9004016030 con 0x7f900c077c40 2024-03-26T13:58:51.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.163+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f900c062340 con 0x7f900c077c40 2024-03-26T13:58:51.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.165+0000 7f9008c00640 1 -- 172.21.15.137:0/3825409113 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4504 (secure 0 0 0) 0x7f900409bc20 con 0x7f900c077c40 2024-03-26T13:58:51.161 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:51.161 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T13:58:51.161 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 6 2024-03-26T13:58:51.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.167+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8ff00a2880 msgr2=0x7f8ff00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.167+0000 7f9012000640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8ff00a2880 0x7f8ff00a4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00bbe0 tx=0x7f8ffc00b390 comp rx=0 tx=0).stop 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.167+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 msgr2=0x7f900c1a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.167+0000 7f9012000640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 0x7f900c1a54a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f900400eaa0 tx=0x7f900400ef60 comp rx=0 tx=0).stop 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 shutdown_connections 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8ff00a2880 0x7f8ff00a4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f900c0772a0 0x7f900c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f900c077c40 0x7f900c1a54a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 --2- 172.21.15.137:0/3825409113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c076060 0x7f900c19def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 >> 172.21.15.137:0/3825409113 conn(0x7f900c0fee50 msgr2=0x7f900c1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:51.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 shutdown_connections 2024-03-26T13:58:51.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:51.168+0000 7f9012000640 1 -- 172.21.15.137:0/3825409113 wait complete. 2024-03-26T13:58:52.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:52 smithi179 ceph-mon[35472]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:52.318 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:52 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3825409113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:52.433 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:52 smithi137 ceph-mon[31539]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:52.433 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:52 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3825409113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:52.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:52 smithi159 ceph-mon[35501]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:52.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:52 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3825409113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:52.633 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 podman[65394]: 2024-03-26 13:58:52.37497755 +0000 UTC m=+0.185329725 container create a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-26T13:58:52.634 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 podman[65394]: 2024-03-26 13:58:52.637438616 +0000 UTC m=+0.447790814 container init a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T13:58:52.920 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 podman[65394]: 2024-03-26 13:58:52.647501753 +0000 UTC m=+0.457853900 container start a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:58:52.920 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 ceph-mds[65413]: -- 172.21.15.179:0/1152813252 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ddc08da40 con 0x563ddc060c00 2024-03-26T13:58:52.920 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f[65409]: starting mds.f at 2024-03-26T13:58:52.921 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 bash[65394]: a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e 2024-03-26T13:58:52.921 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 13:58:52 smithi179 systemd[1]: Started Ceph mds.f for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:53.184 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 -- 172.21.15.137:0/3680310925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec1038e0 msgr2=0x7fdaec103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 --2- 172.21.15.137:0/3680310925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec1038e0 0x7fdaec103d00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009900 tx=0x7fdae002bc30 comp rx=0 tx=0).stop 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 -- 172.21.15.137:0/3680310925 shutdown_connections 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 --2- 172.21.15.137:0/3680310925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdaec105490 0x7fdaec10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 --2- 172.21.15.137:0/3680310925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104ad0 0x7fdaec104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 --2- 172.21.15.137:0/3680310925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec1038e0 0x7fdaec103d00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 -- 172.21.15.137:0/3680310925 >> 172.21.15.137:0/3680310925 conn(0x7fdaec0fee50 msgr2=0x7fdaec101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:53.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 -- 172.21.15.137:0/3680310925 shutdown_connections 2024-03-26T13:58:53.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.393+0000 7fdaf4400640 1 -- 172.21.15.137:0/3680310925 wait complete. 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 Processor -- start 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 -- start start 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdaec1038e0 0x7fdaec199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104ad0 0x7fdaec19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 0x7fdaec1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec10f450 con 0x7fdaec105490 2024-03-26T13:58:53.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec10f2d0 con 0x7fdaec1038e0 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.394+0000 7fdaf4400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec10f5d0 con 0x7fdaec104ad0 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 0x7fdaec1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 0x7fdaec1a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38900/0 (socket says 172.21.15.137:38900) 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 -- 172.21.15.137:0/4189455474 learned_addr learned my addr 172.21.15.137:0/4189455474 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf1a00640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdaec1038e0 0x7fdaec199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf1000640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104ad0 0x7fdaec19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 -- 172.21.15.137:0/4189455474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104ad0 msgr2=0x7fdaec19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104ad0 0x7fdaec19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 -- 172.21.15.137:0/4189455474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdaec1038e0 msgr2=0x7fdaec199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdaec1038e0 0x7fdaec199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 -- 172.21.15.137:0/4189455474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec1a1770 con 0x7fdaec105490 2024-03-26T13:58:53.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.395+0000 7fdaf2400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 0x7fdaec1a1100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdae400eaa0 tx=0x7fdae400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:53.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.396+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae400cdb0 con 0x7fdaec105490 2024-03-26T13:58:53.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.396+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdae4005020 con 0x7fdaec105490 2024-03-26T13:58:53.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.396+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae40116d0 con 0x7fdaec105490 2024-03-26T13:58:53.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.396+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec1a1a60 con 0x7fdaec105490 2024-03-26T13:58:53.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.396+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec1a1f60 con 0x7fdaec105490 2024-03-26T13:58:53.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.397+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaec10ac20 con 0x7fdaec105490 2024-03-26T13:58:53.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.399+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdae40051d0 con 0x7fdaec105490 2024-03-26T13:58:53.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.400+0000 7fdaea000640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdad40a2800 0x7fdad40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:53.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.401+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdae40ce840 con 0x7fdaec105490 2024-03-26T13:58:53.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.401+0000 7fdaf1a00640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdad40a2800 0x7fdad40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:53.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.401+0000 7fdaf1a00640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdad40a2800 0x7fdad40a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdae002c240 tx=0x7fdae002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:53.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.405+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdae4016030 con 0x7fdaec105490 2024-03-26T13:58:53.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.652+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaec02d070 con 0x7fdaec105490 2024-03-26T13:58:53.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.653+0000 7fdaea000640 1 -- 172.21.15.137:0/4189455474 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5298 (secure 0 0 0) 0x7fdae409bce0 con 0x7fdaec105490 2024-03-26T13:58:53.649 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:53.649 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T13:58:53.649 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 7 2024-03-26T13:58:53.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdad40a2800 msgr2=0x7fdad40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdad40a2800 0x7fdad40a4cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdae002c240 tx=0x7fdae002c7e0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 msgr2=0x7fdaec1a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 0x7fdaec1a1100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdae400eaa0 tx=0x7fdae400ef60 comp rx=0 tx=0).stop 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 shutdown_connections 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdad40a2800 0x7fdad40a4cc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdaec105490 0x7fdaec1a1100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdaec104ad0 0x7fdaec19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 --2- 172.21.15.137:0/4189455474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdaec1038e0 0x7fdaec199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.655+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 >> 172.21.15.137:0/4189455474 conn(0x7fdaec0fee50 msgr2=0x7fdaec107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.656+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 shutdown_connections 2024-03-26T13:58:53.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:53.656+0000 7fdaf4400640 1 -- 172.21.15.137:0/4189455474 wait complete. 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: Deploying daemon mds.c on smithi179 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.179:6834/2045604256,v1:172.21.15.179:6835/2045604256] up:boot 2024-03-26T13:58:54.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: fsmap 6 up:standby 2024-03-26T13:58:54.096 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T13:58:54.096 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:53 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4189455474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:54.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: Deploying daemon mds.c on smithi179 2024-03-26T13:58:54.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:54.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.179:6834/2045604256,v1:172.21.15.179:6835/2045604256] up:boot 2024-03-26T13:58:54.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: fsmap 6 up:standby 2024-03-26T13:58:54.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T13:58:54.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:53 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4189455474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:54.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: Deploying daemon mds.c on smithi179 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.179:6834/2045604256,v1:172.21.15.179:6835/2045604256] up:boot 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: fsmap 6 up:standby 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-26T13:58:54.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:53 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4189455474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:55.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 -- 172.21.15.137:0/334077271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec109890 msgr2=0x7f81ec110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 --2- 172.21.15.137:0/334077271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec109890 0x7f81ec110120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009ca0 tx=0x7f81e402c0f0 comp rx=0 tx=0).stop 2024-03-26T13:58:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 -- 172.21.15.137:0/334077271 shutdown_connections 2024-03-26T13:58:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 --2- 172.21.15.137:0/334077271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec108ed0 0x7f81ec109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:55.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 --2- 172.21.15.137:0/334077271 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec109890 0x7f81ec110120 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 --2- 172.21.15.137:0/334077271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec107ce0 0x7f81ec108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 -- 172.21.15.137:0/334077271 >> 172.21.15.137:0/334077271 conn(0x7f81ec076cb0 msgr2=0x7f81ec0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.898+0000 7f81f1400640 1 -- 172.21.15.137:0/334077271 shutdown_connections 2024-03-26T13:58:55.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.899+0000 7f81f1400640 1 -- 172.21.15.137:0/334077271 wait complete. 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 Processor -- start 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 -- start start 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 0x7f81ec19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec108ed0 0x7f81ec19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec109890 0x7f81ec1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec113810 con 0x7f81ec107ce0 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec113690 con 0x7f81ec109890 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81f1400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec113990 con 0x7f81ec108ed0 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.900+0000 7f81ea800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 0x7f81ec19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:55.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 0x7f81ec19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38920/0 (socket says 172.21.15.137:38920) 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 -- 172.21.15.137:0/1945810551 learned_addr learned my addr 172.21.15.137:0/1945810551 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81e1e00640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec108ed0 0x7f81ec19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81eb200640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec109890 0x7f81ec1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 -- 172.21.15.137:0/1945810551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec108ed0 msgr2=0x7f81ec19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec108ed0 0x7f81ec19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 -- 172.21.15.137:0/1945810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec109890 msgr2=0x7f81ec1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec109890 0x7f81ec1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 -- 172.21.15.137:0/1945810551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec1a5a50 con 0x7f81ec107ce0 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81eb200640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec109890 0x7f81ec1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81ea800640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 0x7f81ec19de30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9c0 tx=0x7f81dc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:55.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.901+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00ccc0 con 0x7f81ec107ce0 2024-03-26T13:58:55.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.902+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81dc0035f0 con 0x7f81ec107ce0 2024-03-26T13:58:55.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.902+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00ce60 con 0x7f81ec107ce0 2024-03-26T13:58:55.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.902+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec1a5d40 con 0x7f81ec107ce0 2024-03-26T13:58:55.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.902+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec1a6180 con 0x7f81ec107ce0 2024-03-26T13:58:55.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.903+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec10f000 con 0x7f81ec107ce0 2024-03-26T13:58:55.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.905+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f81dc0037a0 con 0x7f81ec107ce0 2024-03-26T13:58:55.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.906+0000 7f81e3e00640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81d00a2800 0x7f81d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:55.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.906+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81dc0cebc0 con 0x7f81ec107ce0 2024-03-26T13:58:55.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.906+0000 7f81e1e00640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81d00a2800 0x7f81d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:55.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.907+0000 7f81e1e00640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81d00a2800 0x7f81d00a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81ec19f340 tx=0x7f81d400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:55.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:55.910+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f81dc09c060 con 0x7f81ec107ce0 2024-03-26T13:58:56.116 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:55 smithi179 systemd[1]: Starting Ceph mds.c for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:58:56.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.150+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81ec02d070 con 0x7f81ec107ce0 2024-03-26T13:58:56.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.152+0000 7f81e3e00640 1 -- 172.21.15.137:0/1945810551 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5298 (secure 0 0 0) 0x7f81ec02d070 con 0x7f81ec107ce0 2024-03-26T13:58:56.148 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:56.148 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T13:58:56.148 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 7 2024-03-26T13:58:56.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81d00a2800 msgr2=0x7f81d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:56.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81d00a2800 0x7f81d00a4cc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81ec19f340 tx=0x7f81d400a2d0 comp rx=0 tx=0).stop 2024-03-26T13:58:56.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 msgr2=0x7f81ec19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:56.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 0x7f81ec19de30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9c0 tx=0x7f81dc00edc0 comp rx=0 tx=0).stop 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 shutdown_connections 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81d00a2800 0x7f81d00a4cc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81ec109890 0x7f81ec1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81ec108ed0 0x7f81ec19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 --2- 172.21.15.137:0/1945810551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81ec107ce0 0x7f81ec19de30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 >> 172.21.15.137:0/1945810551 conn(0x7f81ec076cb0 msgr2=0x7f81ec10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 shutdown_connections 2024-03-26T13:58:56.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:56.155+0000 7f81f1400640 1 -- 172.21.15.137:0/1945810551 wait complete. 2024-03-26T13:58:56.384 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:56 smithi179 ceph-mon[35472]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:56.384 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 podman[65694]: 2024-03-26 13:58:56.10612048 +0000 UTC m=+0.036406021 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:58:56.385 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 podman[65694]: 2024-03-26 13:58:56.232333904 +0000 UTC m=+0.162619472 container create 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:58:56.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:56 smithi159 ceph-mon[35501]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:56.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:56 smithi137 ceph-mon[31539]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:56.921 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 podman[65694]: 2024-03-26 13:58:56.503378274 +0000 UTC m=+0.433663846 container init 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T13:58:56.921 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 podman[65694]: 2024-03-26 13:58:56.513707538 +0000 UTC m=+0.443993086 container start 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T13:58:56.921 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 ceph-mds[65713]: -- 172.21.15.179:0/4041498603 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f791039a40 con 0x55f79100cc00 2024-03-26T13:58:56.921 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c[65709]: starting mds.c at 2024-03-26T13:58:56.921 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 bash[65694]: 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 2024-03-26T13:58:56.921 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 13:58:56 smithi179 systemd[1]: Started Ceph mds.c for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:58:57.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1945810551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:57.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:57.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:57.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:57 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:57.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1945810551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:57.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:57.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:57.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:57 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:57.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1945810551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:57.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:58:57.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:58:57.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:58:57.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:57 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:58:58.172 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:58:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 -- 172.21.15.137:0/554783103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30105490 msgr2=0x7f4f3010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 --2- 172.21.15.137:0/554783103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30105490 0x7f4f3010bd20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009ca0 tx=0x7f4f2c02c100 comp rx=0 tx=0).stop 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 -- 172.21.15.137:0/554783103 shutdown_connections 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 --2- 172.21.15.137:0/554783103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30104ad0 0x7f4f30104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 --2- 172.21.15.137:0/554783103 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30105490 0x7f4f3010bd20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 --2- 172.21.15.137:0/554783103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f301038e0 0x7f4f30103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 -- 172.21.15.137:0/554783103 >> 172.21.15.137:0/554783103 conn(0x7f4f300fee50 msgr2=0x7f4f30101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 -- 172.21.15.137:0/554783103 shutdown_connections 2024-03-26T13:58:58.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.394+0000 7f4f38800640 1 -- 172.21.15.137:0/554783103 wait complete. 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.395+0000 7f4f38800640 1 Processor -- start 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.395+0000 7f4f38800640 1 -- start start 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f38800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f301038e0 0x7f4f30199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f38800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 0x7f4f30199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f38800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30105490 0x7f4f301a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f38800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3010f410 con 0x7f4f30104ad0 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f38800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3010f290 con 0x7f4f30105490 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f38800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3010f590 con 0x7f4f301038e0 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 0x7f4f30199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 0x7f4f30199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53860/0 (socket says 172.21.15.137:53860) 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 -- 172.21.15.137:0/3187777871 learned_addr learned my addr 172.21.15.137:0/3187777871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:58:58.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35e00640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f301038e0 0x7f4f30199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f36800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30105490 0x7f4f301a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 -- 172.21.15.137:0/3187777871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f301038e0 msgr2=0x7f4f30199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f301038e0 0x7f4f30199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 -- 172.21.15.137:0/3187777871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30105490 msgr2=0x7f4f301a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30105490 0x7f4f301a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f35400640 1 -- 172.21.15.137:0/3187777871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f301a1650 con 0x7f4f30104ad0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.396+0000 7f4f36800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30105490 0x7f4f301a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.397+0000 7f4f35400640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 0x7f4f30199f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c920 tx=0x7f4f1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.397+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c007c70 con 0x7f4f30104ad0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.397+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f1c007e10 con 0x7f4f30104ad0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.397+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f301a1940 con 0x7f4f30104ad0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.397+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c018820 con 0x7f4f30104ad0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.397+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f301a1ee0 con 0x7f4f30104ad0 2024-03-26T13:58:58.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.398+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3010ac00 con 0x7f4f30104ad0 2024-03-26T13:58:58.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.400+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4f1c0189c0 con 0x7f4f30104ad0 2024-03-26T13:58:58.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.401+0000 7f4f2aa00640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4f180a2880 0x7f4f180a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:58:58.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.402+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f1c0ce9c0 con 0x7f4f30104ad0 2024-03-26T13:58:58.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.402+0000 7f4f35e00640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4f180a2880 0x7f4f180a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:58:58.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.402+0000 7f4f35e00640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4f180a2880 0x7f4f180a4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005fd0 tx=0x7f4f24005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:58:58.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.404+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4f1c014030 con 0x7f4f30104ad0 2024-03-26T13:58:58.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:58 smithi179 ceph-mon[35472]: Deploying daemon mds.h on smithi159 2024-03-26T13:58:58.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:58 smithi179 ceph-mon[35472]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:58.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:58 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.179:6836/7146430,v1:172.21.15.179:6837/7146430] up:boot 2024-03-26T13:58:58.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:58 smithi179 ceph-mon[35472]: fsmap 7 up:standby 2024-03-26T13:58:58.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:58 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T13:58:58.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:58 smithi159 ceph-mon[35501]: Deploying daemon mds.h on smithi159 2024-03-26T13:58:58.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:58 smithi159 ceph-mon[35501]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:58.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:58 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.179:6836/7146430,v1:172.21.15.179:6837/7146430] up:boot 2024-03-26T13:58:58.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:58 smithi159 ceph-mon[35501]: fsmap 7 up:standby 2024-03-26T13:58:58.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:58 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T13:58:58.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:58 smithi137 ceph-mon[31539]: Deploying daemon mds.h on smithi159 2024-03-26T13:58:58.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:58 smithi137 ceph-mon[31539]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:58:58.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:58 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.179:6836/7146430,v1:172.21.15.179:6837/7146430] up:boot 2024-03-26T13:58:58.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:58 smithi137 ceph-mon[31539]: fsmap 7 up:standby 2024-03-26T13:58:58.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:58 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T13:58:58.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.643+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f3002d070 con 0x7f4f30104ad0 2024-03-26T13:58:58.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.645+0000 7f4f2aa00640 1 -- 172.21.15.137:0/3187777871 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f4f1c09c010 con 0x7f4f30104ad0 2024-03-26T13:58:58.641 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:58:58.641 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T13:58:58.641 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 8 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4f180a2880 msgr2=0x7f4f180a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4f180a2880 0x7f4f180a4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4f24005fd0 tx=0x7f4f24005850 comp rx=0 tx=0).stop 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 msgr2=0x7f4f30199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 0x7f4f30199f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c920 tx=0x7f4f1c00cde0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 shutdown_connections 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4f180a2880 0x7f4f180a4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f30105490 0x7f4f301a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f30104ad0 0x7f4f30199f70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 --2- 172.21.15.137:0/3187777871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f301038e0 0x7f4f30199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:58:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 >> 172.21.15.137:0/3187777871 conn(0x7f4f300fee50 msgr2=0x7f4f30107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:58:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 shutdown_connections 2024-03-26T13:58:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:58:58.648+0000 7f4f38800640 1 -- 172.21.15.137:0/3187777871 wait complete. 2024-03-26T13:58:59.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:58:59 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3187777871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:59.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:58:59 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3187777871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:58:59.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:58:59 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3187777871' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:00.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:00 smithi179 ceph-mon[35472]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:00.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:00 smithi159 ceph-mon[35501]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:00.449 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 systemd[1]: Starting Ceph mds.h for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:59:00.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:00 smithi137 ceph-mon[31539]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:00.665 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:00.836 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 podman[65478]: 2024-03-26 13:59:00.438239405 +0000 UTC m=+0.033159018 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:59:00.836 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 podman[65478]: 2024-03-26 13:59:00.544060555 +0000 UTC m=+0.138980208 container create 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-26T13:59:00.836 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 podman[65478]: 2024-03-26 13:59:00.839738966 +0000 UTC m=+0.434658622 container init 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 -- 172.21.15.137:0/2015802151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c1026d0 msgr2=0x7fb53c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 --2- 172.21.15.137:0/2015802151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c1026d0 0x7fb53c102b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb528009900 tx=0x7fb52802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 -- 172.21.15.137:0/2015802151 shutdown_connections 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 --2- 172.21.15.137:0/2015802151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb53c103090 0x7fb53c109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 --2- 172.21.15.137:0/2015802151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c1026d0 0x7fb53c102b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 --2- 172.21.15.137:0/2015802151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 0x7fb53c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 -- 172.21.15.137:0/2015802151 >> 172.21.15.137:0/2015802151 conn(0x7fb53c0fca30 msgr2=0x7fb53c0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 -- 172.21.15.137:0/2015802151 shutdown_connections 2024-03-26T13:59:00.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.891+0000 7fb540e00640 1 -- 172.21.15.137:0/2015802151 wait complete. 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.892+0000 7fb540e00640 1 Processor -- start 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.892+0000 7fb540e00640 1 -- start start 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.892+0000 7fb540e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 0x7fb53c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb540e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb53c1026d0 0x7fb53c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb540e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 0x7fb53c1a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb540e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c10d060 con 0x7fb53c103090 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb540e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c10cee0 con 0x7fb53c1014e0 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb540e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c10d1e0 con 0x7fb53c1026d0 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 0x7fb53c1a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 0x7fb53c1a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53890/0 (socket says 172.21.15.137:53890) 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 -- 172.21.15.137:0/2693296880 learned_addr learned my addr 172.21.15.137:0/2693296880 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:00.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53b400640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 0x7fb53c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53aa00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb53c1026d0 0x7fb53c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 -- 172.21.15.137:0/2693296880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb53c1026d0 msgr2=0x7fb53c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb53c1026d0 0x7fb53c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 -- 172.21.15.137:0/2693296880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 msgr2=0x7fb53c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 0x7fb53c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53be00640 1 -- 172.21.15.137:0/2693296880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c1a15e0 con 0x7fb53c103090 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.893+0000 7fb53b400640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 0x7fb53c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.894+0000 7fb53be00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 0x7fb53c1a0f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b970 tx=0x7fb53400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.894+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534019070 con 0x7fb53c103090 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.894+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb534014440 con 0x7fb53c103090 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.894+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340075d0 con 0x7fb53c103090 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.894+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c1a18d0 con 0x7fb53c103090 2024-03-26T13:59:00.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.894+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1a1e50 con 0x7fb53c103090 2024-03-26T13:59:00.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.895+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c1a2080 con 0x7fb53c103090 2024-03-26T13:59:00.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.897+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb53401c070 con 0x7fb53c103090 2024-03-26T13:59:00.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.898+0000 7fb533e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5200a2800 0x7fb5200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:00.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.899+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb5340cdca0 con 0x7fb53c103090 2024-03-26T13:59:00.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.899+0000 7fb53b400640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5200a2800 0x7fb5200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:00.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.899+0000 7fb53b400640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5200a2800 0x7fb5200a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0058c0 tx=0x7fb52c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:00.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:00.904+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb53409b140 con 0x7fb53c103090 2024-03-26T13:59:01.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.141+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb53c1a22f0 con 0x7fb53c103090 2024-03-26T13:59:01.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.143+0000 7fb533e00640 1 -- 172.21.15.137:0/2693296880 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7fb53409aa40 con 0x7fb53c103090 2024-03-26T13:59:01.139 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:01.139 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-26T13:59:01.139 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 8 2024-03-26T13:59:01.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5200a2800 msgr2=0x7fb5200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:01.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5200a2800 0x7fb5200a4cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0058c0 tx=0x7fb52c005850 comp rx=0 tx=0).stop 2024-03-26T13:59:01.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 msgr2=0x7fb53c1a0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 0x7fb53c1a0f70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb53400b970 tx=0x7fb53400be30 comp rx=0 tx=0).stop 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 shutdown_connections 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb5200a2800 0x7fb5200a4cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb53c103090 0x7fb53c1a0f70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb53c1026d0 0x7fb53c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 --2- 172.21.15.137:0/2693296880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c1014e0 0x7fb53c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 >> 172.21.15.137:0/2693296880 conn(0x7fb53c0fca30 msgr2=0x7fb53c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.145+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 shutdown_connections 2024-03-26T13:59:01.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:01.146+0000 7fb540e00640 1 -- 172.21.15.137:0/2693296880 wait complete. 2024-03-26T13:59:01.199 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 podman[65478]: 2024-03-26 13:59:00.850128534 +0000 UTC m=+0.445048213 container start 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T13:59:01.199 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 ceph-mds[65497]: -- 172.21.15.159:0/2819681554 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5641229e3a40 con 0x5641229b6c00 2024-03-26T13:59:01.199 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h[65493]: starting mds.h at 2024-03-26T13:59:01.199 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 bash[65478]: 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b 2024-03-26T13:59:01.199 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 13:59:00 smithi159 systemd[1]: Started Ceph mds.h for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2693296880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: Deploying daemon mds.i on smithi179 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] up:boot 2024-03-26T13:59:02.334 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: fsmap 8 up:standby 2024-03-26T13:59:02.335 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:02 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:02.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2693296880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:02.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: Deploying daemon mds.i on smithi179 2024-03-26T13:59:02.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] up:boot 2024-03-26T13:59:02.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: fsmap 8 up:standby 2024-03-26T13:59:02.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:02 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T13:59:02.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2693296880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: Deploying daemon mds.i on smithi179 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] up:boot 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: fsmap 8 up:standby 2024-03-26T13:59:02.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:02 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T13:59:03.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 -- 172.21.15.137:0/3800998063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 msgr2=0x7f4eb8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 --2- 172.21.15.137:0/3800998063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb8076480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009900 tx=0x7f4eac02bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 -- 172.21.15.137:0/3800998063 shutdown_connections 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 --2- 172.21.15.137:0/3800998063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eb8077c40 0x7f4eb80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 --2- 172.21.15.137:0/3800998063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb80772a0 0x7f4eb8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 --2- 172.21.15.137:0/3800998063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb8076480 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 -- 172.21.15.137:0/3800998063 >> 172.21.15.137:0/3800998063 conn(0x7f4eb80fee50 msgr2=0x7f4eb8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:03.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 -- 172.21.15.137:0/3800998063 shutdown_connections 2024-03-26T13:59:03.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.379+0000 7f4ec0000640 1 -- 172.21.15.137:0/3800998063 wait complete. 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.380+0000 7f4ec0000640 1 Processor -- start 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.380+0000 7f4ec0000640 1 -- start start 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ec0000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ec0000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eb80772a0 0x7f4eb819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ec0000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8077c40 0x7f4eb81a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ec0000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb8113860 con 0x7f4eb8076060 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ec0000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81136e0 con 0x7f4eb80772a0 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ec0000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb81139e0 con 0x7f4eb8077c40 2024-03-26T13:59:03.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb819df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53910/0 (socket says 172.21.15.137:53910) 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 -- 172.21.15.137:0/1694356525 learned_addr learned my addr 172.21.15.137:0/1694356525 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebe000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8077c40 0x7f4eb81a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 -- 172.21.15.137:0/1694356525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8077c40 msgr2=0x7f4eb81a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8077c40 0x7f4eb81a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 -- 172.21.15.137:0/1694356525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eb80772a0 msgr2=0x7f4eb819e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebcc00640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eb80772a0 0x7f4eb819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eb80772a0 0x7f4eb819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebd600640 1 -- 172.21.15.137:0/1694356525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb81a5b40 con 0x7f4eb8076060 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.381+0000 7f4ebe000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8077c40 0x7f4eb81a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.382+0000 7f4ebd600640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb819df20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4eac000c00 tx=0x7f4eac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.382+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac043020 con 0x7f4eb8076060 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.382+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4eac0054c0 con 0x7f4eb8076060 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.382+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb81a5dd0 con 0x7f4eb8076060 2024-03-26T13:59:03.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.382+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac003d70 con 0x7f4eb8076060 2024-03-26T13:59:03.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.382+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb81a6230 con 0x7f4eb8076060 2024-03-26T13:59:03.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.383+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb8110420 con 0x7f4eb8076060 2024-03-26T13:59:03.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.385+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4eac046020 con 0x7f4eb8076060 2024-03-26T13:59:03.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.387+0000 7f4eb6000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ea00a2800 0x7f4ea00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:03.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.387+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4eac0f0c70 con 0x7f4eb8076060 2024-03-26T13:59:03.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.387+0000 7f4ebcc00640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ea00a2800 0x7f4ea00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:03.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.388+0000 7f4ebcc00640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ea00a2800 0x7f4ea00a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005fd0 tx=0x7f4ea4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:03.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.392+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4eac04d020 con 0x7f4eb8076060 2024-03-26T13:59:03.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.630+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4eb802d070 con 0x7f4eb8076060 2024-03-26T13:59:03.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.631+0000 7f4eb6000640 1 -- 172.21.15.137:0/1694356525 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f4eac0be230 con 0x7f4eb8076060 2024-03-26T13:59:03.627 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:03.628 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T13:59:03.628 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 9 2024-03-26T13:59:03.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ea00a2800 msgr2=0x7f4ea00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:03.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ea00a2800 0x7f4ea00a4cc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005fd0 tx=0x7f4ea4005850 comp rx=0 tx=0).stop 2024-03-26T13:59:03.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 msgr2=0x7f4eb819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:03.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb819df20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4eac000c00 tx=0x7f4eac0042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 shutdown_connections 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ea00a2800 0x7f4ea00a4cc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4eb80772a0 0x7f4eb819e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4eb8077c40 0x7f4eb81a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 --2- 172.21.15.137:0/1694356525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4eb8076060 0x7f4eb819df20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.634+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 >> 172.21.15.137:0/1694356525 conn(0x7f4eb80fee50 msgr2=0x7f4eb81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.635+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 shutdown_connections 2024-03-26T13:59:03.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:03.635+0000 7f4ec0000640 1 -- 172.21.15.137:0/1694356525 wait complete. 2024-03-26T13:59:04.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:04 smithi179 ceph-mon[35472]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:04.389 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:04 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1694356525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:04 smithi159 ceph-mon[35501]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:04.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:04 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1694356525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:04.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:04 smithi137 ceph-mon[31539]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:04.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:04 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1694356525' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:04.671 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 podman[65998]: 2024-03-26 13:59:04.424280062 +0000 UTC m=+0.158718085 container create 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 podman[65998]: 2024-03-26 13:59:04.71979349 +0000 UTC m=+0.454231556 container init 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 podman[65998]: 2024-03-26 13:59:04.729632669 +0000 UTC m=+0.464070665 container start 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 ceph-mds[66017]: -- 172.21.15.179:0/4153563862 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558416ffa4e0 con 0x558416fd0800 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 ceph-mds[66017]: -- 172.21.15.179:0/4153563862 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558416ffda40 con 0x558416fd0800 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i[66013]: starting mds.i at 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 bash[65998]: 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 2024-03-26T13:59:05.171 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 13:59:04 smithi179 systemd[1]: Started Ceph mds.i for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:59:05.651 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 -- 172.21.15.137:0/2490478835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 msgr2=0x7f6130074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 --2- 172.21.15.137:0/2490478835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130074580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f611c009900 tx=0x7f611c02bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 -- 172.21.15.137:0/2490478835 shutdown_connections 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 --2- 172.21.15.137:0/2490478835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61301095a0 0x7f613010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 --2- 172.21.15.137:0/2490478835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130074580 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 --2- 172.21.15.137:0/2490478835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300756e0 0x7f6130073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 -- 172.21.15.137:0/2490478835 >> 172.21.15.137:0/2490478835 conn(0x7f61300fcc50 msgr2=0x7f61300ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 -- 172.21.15.137:0/2490478835 shutdown_connections 2024-03-26T13:59:05.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.857+0000 7f6138200640 1 -- 172.21.15.137:0/2490478835 wait complete. 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.858+0000 7f6138200640 1 Processor -- start 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.858+0000 7f6138200640 1 -- start start 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6138200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6138200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300756e0 0x7f6130199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6138200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61301095a0 0x7f61301a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6138200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613010f420 con 0x7f6130074100 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6138200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613010f2a0 con 0x7f61301095a0 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6138200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613010f5a0 con 0x7f61300756e0 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6135800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:05.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6135800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53932/0 (socket says 172.21.15.137:53932) 2024-03-26T13:59:05.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6135800640 1 -- 172.21.15.137:0/2589878513 learned_addr learned my addr 172.21.15.137:0/2589878513 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:05.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6136200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61301095a0 0x7f61301a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:05.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.859+0000 7f6134e00640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300756e0 0x7f6130199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6135800640 1 -- 172.21.15.137:0/2589878513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300756e0 msgr2=0x7f6130199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6135800640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300756e0 0x7f6130199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6135800640 1 -- 172.21.15.137:0/2589878513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61301095a0 msgr2=0x7f61301a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6135800640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61301095a0 0x7f61301a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6135800640 1 -- 172.21.15.137:0/2589878513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61301a1690 con 0x7f6130074100 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6135800640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130199a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f612400c920 tx=0x7f612400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124007c70 con 0x7f6130074100 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61301a1980 con 0x7f6130074100 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6124007e10 con 0x7f6130074100 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.860+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124018820 con 0x7f6130074100 2024-03-26T13:59:05.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.861+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61301a1ef0 con 0x7f6130074100 2024-03-26T13:59:05.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.862+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61301082d0 con 0x7f6130074100 2024-03-26T13:59:05.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.864+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f61240189c0 con 0x7f6130074100 2024-03-26T13:59:05.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.866+0000 7f612e000640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61180a2800 0x7f61180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:05.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.866+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f61240ce700 con 0x7f6130074100 2024-03-26T13:59:05.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.866+0000 7f6134e00640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61180a2800 0x7f61180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:05.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.867+0000 7f6134e00640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61180a2800 0x7f61180a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f611c000c00 tx=0x7f611c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:05.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:05.870+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6124014030 con 0x7f6130074100 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: Deploying daemon mds.g on smithi137 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.179:6838/2441458416,v1:172.21.15.179:6839/2441458416] up:boot 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: fsmap 9 up:standby 2024-03-26T13:59:06.009 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:05 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T13:59:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.122+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f613002d070 con 0x7f6130074100 2024-03-26T13:59:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.124+0000 7f612e000640 1 -- 172.21.15.137:0/2589878513 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f612409bba0 con 0x7f6130074100 2024-03-26T13:59:06.120 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:06.120 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T13:59:06.120 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 10 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61180a2800 msgr2=0x7f61180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61180a2800 0x7f61180a4cc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f611c000c00 tx=0x7f611c02c7e0 comp rx=0 tx=0).stop 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 msgr2=0x7f6130199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130199a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f612400c920 tx=0x7f612400cde0 comp rx=0 tx=0).stop 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 shutdown_connections 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61180a2800 0x7f61180a4cc0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61301095a0 0x7f61301a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61300756e0 0x7f6130199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:06.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 --2- 172.21.15.137:0/2589878513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6130074100 0x7f6130199a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.126+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 >> 172.21.15.137:0/2589878513 conn(0x7f61300fcc50 msgr2=0x7f61301054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.127+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 shutdown_connections 2024-03-26T13:59:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:06.127+0000 7f6138200640 1 -- 172.21.15.137:0/2589878513 wait complete. 2024-03-26T13:59:06.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: Deploying daemon mds.g on smithi137 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.179:6838/2441458416,v1:172.21.15.179:6839/2441458416] up:boot 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: fsmap 9 up:standby 2024-03-26T13:59:06.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:05 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: Deploying daemon mds.g on smithi137 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.179:6838/2441458416,v1:172.21.15.179:6839/2441458416] up:boot 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: fsmap 9 up:standby 2024-03-26T13:59:06.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:05 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T13:59:07.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:06 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2589878513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:07.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:06 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2589878513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:07.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2589878513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:08.147 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:08.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:08 smithi137 ceph-mon[31539]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:08.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.359+0000 7fa51d400640 1 -- 172.21.15.137:0/1947104883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5181038e0 msgr2=0x7fa518103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:08.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.359+0000 7fa51d400640 1 --2- 172.21.15.137:0/1947104883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5181038e0 0x7fa518103d00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa508009900 tx=0x7fa50802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:08.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 -- 172.21.15.137:0/1947104883 shutdown_connections 2024-03-26T13:59:08.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 --2- 172.21.15.137:0/1947104883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa518104ad0 0x7fa518104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 --2- 172.21.15.137:0/1947104883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa518105490 0x7fa51810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 --2- 172.21.15.137:0/1947104883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5181038e0 0x7fa518103d00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 -- 172.21.15.137:0/1947104883 >> 172.21.15.137:0/1947104883 conn(0x7fa5180fee90 msgr2=0x7fa5181012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:08.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 -- 172.21.15.137:0/1947104883 shutdown_connections 2024-03-26T13:59:08.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.360+0000 7fa51d400640 1 -- 172.21.15.137:0/1947104883 wait complete. 2024-03-26T13:59:08.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 Processor -- start 2024-03-26T13:59:08.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 -- start start 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5181038e0 0x7fa518199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 0x7fa51819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa518105490 0x7fa5181a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51810f430 con 0x7fa518104ad0 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51810f2b0 con 0x7fa518105490 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.361+0000 7fa51d400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51810f5b0 con 0x7fa5181038e0 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 0x7fa51819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 0x7fa51819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51624/0 (socket says 172.21.15.137:51624) 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 -- 172.21.15.137:0/4199461894 learned_addr learned my addr 172.21.15.137:0/4199461894 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:08.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa517200640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa518105490 0x7fa5181a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa516800640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5181038e0 0x7fa518199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 -- 172.21.15.137:0/4199461894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5181038e0 msgr2=0x7fa518199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5181038e0 0x7fa518199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 -- 172.21.15.137:0/4199461894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa518105490 msgr2=0x7fa5181a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa518105490 0x7fa5181a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 -- 172.21.15.137:0/4199461894 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5181a1700 con 0x7fa518104ad0 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa515e00640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 0x7fa51819a020 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa50400c890 tx=0x7fa50400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.362+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004220 con 0x7fa518104ad0 2024-03-26T13:59:08.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.363+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5040043c0 con 0x7fa518104ad0 2024-03-26T13:59:08.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.363+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5181a19f0 con 0x7fa518104ad0 2024-03-26T13:59:08.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.363+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504004a00 con 0x7fa518104ad0 2024-03-26T13:59:08.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.363+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181a1f40 con 0x7fa518104ad0 2024-03-26T13:59:08.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.364+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51810ac00 con 0x7fa518104ad0 2024-03-26T13:59:08.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.367+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa504004ba0 con 0x7fa518104ad0 2024-03-26T13:59:08.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.368+0000 7fa50f400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4fc0a2800 0x7fa4fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:08.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.368+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa5040ce3e0 con 0x7fa518104ad0 2024-03-26T13:59:08.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.368+0000 7fa516800640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4fc0a2800 0x7fa4fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:08.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.369+0000 7fa516800640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4fc0a2800 0x7fa4fc0a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5080096c0 tx=0x7fa508005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:08.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.372+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa504014030 con 0x7fa518104ad0 2024-03-26T13:59:08.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:08 smithi179 ceph-mon[35472]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:08.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:08 smithi159 ceph-mon[35501]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:08.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.616+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5180622a0 con 0x7fa518104ad0 2024-03-26T13:59:08.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.617+0000 7fa50f400640 1 -- 172.21.15.137:0/4199461894 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7fa50409b880 con 0x7fa518104ad0 2024-03-26T13:59:08.613 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:08.613 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T13:59:08.614 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 10 2024-03-26T13:59:08.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4fc0a2800 msgr2=0x7fa4fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:08.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4fc0a2800 0x7fa4fc0a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5080096c0 tx=0x7fa508005fd0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 msgr2=0x7fa51819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:08.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 0x7fa51819a020 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa50400c890 tx=0x7fa50400cd50 comp rx=0 tx=0).stop 2024-03-26T13:59:08.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 shutdown_connections 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4fc0a2800 0x7fa4fc0a4cc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa518105490 0x7fa5181a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa518104ad0 0x7fa51819a020 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 --2- 172.21.15.137:0/4199461894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5181038e0 0x7fa518199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 >> 172.21.15.137:0/4199461894 conn(0x7fa5180fee90 msgr2=0x7fa518107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 shutdown_connections 2024-03-26T13:59:08.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:08.620+0000 7fa51d400640 1 -- 172.21.15.137:0/4199461894 wait complete. 2024-03-26T13:59:09.008 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 13:59:08 smithi137 systemd[1]: Started Ceph mds.g for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4199461894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:09.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:09.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4199461894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:09.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:09.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:09.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4199461894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:09.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:10 smithi137 ceph-mon[31539]: Deploying daemon mds.a on smithi137 2024-03-26T13:59:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:10 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.137:6838/2542329898,v1:172.21.15.137:6839/2542329898] up:boot 2024-03-26T13:59:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:10 smithi137 ceph-mon[31539]: fsmap 10 up:standby 2024-03-26T13:59:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T13:59:10.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:10 smithi137 ceph-mon[31539]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:10 smithi179 ceph-mon[35472]: Deploying daemon mds.a on smithi137 2024-03-26T13:59:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:10 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.137:6838/2542329898,v1:172.21.15.137:6839/2542329898] up:boot 2024-03-26T13:59:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:10 smithi179 ceph-mon[35472]: fsmap 10 up:standby 2024-03-26T13:59:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:10 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T13:59:10.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:10 smithi179 ceph-mon[35472]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:10.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:10 smithi159 ceph-mon[35501]: Deploying daemon mds.a on smithi137 2024-03-26T13:59:10.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:10 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.137:6838/2542329898,v1:172.21.15.137:6839/2542329898] up:boot 2024-03-26T13:59:10.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:10 smithi159 ceph-mon[35501]: fsmap 10 up:standby 2024-03-26T13:59:10.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:10 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-26T13:59:10.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:10 smithi159 ceph-mon[35501]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:10.633 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.863+0000 7fc993e00640 1 -- 172.21.15.137:0/102541990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc994101f70 msgr2=0x7fc994104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.863+0000 7fc993e00640 1 --2- 172.21.15.137:0/102541990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc994101f70 0x7fc994104360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc984009900 tx=0x7fc98402bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 -- 172.21.15.137:0/102541990 shutdown_connections 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 --2- 172.21.15.137:0/102541990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9941048a0 0x7fc994106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 --2- 172.21.15.137:0/102541990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941071d0 0x7fc994109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 --2- 172.21.15.137:0/102541990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc994101f70 0x7fc994104360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 -- 172.21.15.137:0/102541990 >> 172.21.15.137:0/102541990 conn(0x7fc9940fb980 msgr2=0x7fc9940fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 -- 172.21.15.137:0/102541990 shutdown_connections 2024-03-26T13:59:10.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.864+0000 7fc993e00640 1 -- 172.21.15.137:0/102541990 wait complete. 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 Processor -- start 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 -- start start 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101f70 0x7fc994199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941048a0 0x7fc994199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 0x7fc9941a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99410d080 con 0x7fc9941071d0 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99410cf00 con 0x7fc994101f70 2024-03-26T13:59:10.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99410d200 con 0x7fc9941048a0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 0x7fc9941a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 0x7fc9941a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51660/0 (socket says 172.21.15.137:51660) 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.865+0000 7fc993400640 1 -- 172.21.15.137:0/3370691819 learned_addr learned my addr 172.21.15.137:0/3370691819 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc992a00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101f70 0x7fc994199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc992000640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941048a0 0x7fc994199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc993400640 1 -- 172.21.15.137:0/3370691819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941048a0 msgr2=0x7fc994199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc993400640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941048a0 0x7fc994199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc993400640 1 -- 172.21.15.137:0/3370691819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101f70 msgr2=0x7fc994199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc993400640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101f70 0x7fc994199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc993400640 1 -- 172.21.15.137:0/3370691819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9941a1580 con 0x7fc9941071d0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc992000640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941048a0 0x7fc994199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.866+0000 7fc993400640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 0x7fc9941a0f10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00cad0 tx=0x7fc98c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.867+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00ae20 con 0x7fc9941071d0 2024-03-26T13:59:10.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.867+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc98c005020 con 0x7fc9941071d0 2024-03-26T13:59:10.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.867+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00f6b0 con 0x7fc9941071d0 2024-03-26T13:59:10.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.867+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9941a1870 con 0x7fc9941071d0 2024-03-26T13:59:10.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.867+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9941a1dc0 con 0x7fc9941071d0 2024-03-26T13:59:10.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.868+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc994108d20 con 0x7fc9941071d0 2024-03-26T13:59:10.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.870+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc98c0051d0 con 0x7fc9941071d0 2024-03-26T13:59:10.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.871+0000 7fc998a00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9780a2880 0x7fc9780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:10.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.871+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc98c0ce680 con 0x7fc9941071d0 2024-03-26T13:59:10.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.872+0000 7fc992a00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9780a2880 0x7fc9780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:10.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.872+0000 7fc992a00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9780a2880 0x7fc9780a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc98402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:10.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:10.876+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc98c014030 con 0x7fc9941071d0 2024-03-26T13:59:11.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.121+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9940622a0 con 0x7fc9941071d0 2024-03-26T13:59:11.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.126+0000 7fc998a00640 1 -- 172.21.15.137:0/3370691819 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7fc98c09bb20 con 0x7fc9941071d0 2024-03-26T13:59:11.122 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:11.122 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T13:59:11.123 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 11 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9780a2880 msgr2=0x7fc9780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9780a2880 0x7fc9780a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc984000c00 tx=0x7fc98402c7e0 comp rx=0 tx=0).stop 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 msgr2=0x7fc9941a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 0x7fc9941a0f10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00cad0 tx=0x7fc98c00cf90 comp rx=0 tx=0).stop 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 shutdown_connections 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9780a2880 0x7fc9780a4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9941071d0 0x7fc9941a0f10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:11.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9941048a0 0x7fc994199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:11.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 --2- 172.21.15.137:0/3370691819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc994101f70 0x7fc994199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:11.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 >> 172.21.15.137:0/3370691819 conn(0x7fc9940fb980 msgr2=0x7fc9941016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:11.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 shutdown_connections 2024-03-26T13:59:11.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:11.129+0000 7fc993e00640 1 -- 172.21.15.137:0/3370691819 wait complete. 2024-03-26T13:59:12.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:12 smithi137 ceph-mon[31539]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:12.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3370691819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:12.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:12 smithi179 ceph-mon[35472]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:12.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3370691819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:12.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:12 smithi159 ceph-mon[35501]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:12.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3370691819' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:12.660 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 podman[73342]: 2024-03-26 13:59:12.384068113 +0000 UTC m=+0.158741040 container create cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T13:59:12.660 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 podman[73342]: 2024-03-26 13:59:12.663277968 +0000 UTC m=+0.437950902 container init cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T13:59:13.032 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 podman[73342]: 2024-03-26 13:59:12.674094413 +0000 UTC m=+0.448767377 container start cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:59:13.032 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 ceph-mds[73361]: -- 172.21.15.137:0/478984531 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5631c6929a40 con 0x5631c68fcc00 2024-03-26T13:59:13.032 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a[73357]: starting mds.a at 2024-03-26T13:59:13.032 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 bash[73342]: cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 2024-03-26T13:59:13.032 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 13:59:12 smithi137 systemd[1]: Started Ceph mds.a for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:59:13.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.370+0000 7f7ae8a00640 1 -- 172.21.15.137:0/749732355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4077c40 msgr2=0x7f7ae40784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.370+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/749732355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4077c40 0x7f7ae40784e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009ca0 tx=0x7f7adc02c100 comp rx=0 tx=0).stop 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 -- 172.21.15.137:0/749732355 shutdown_connections 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/749732355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4077c40 0x7f7ae40784e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/749732355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae40772a0 0x7f7ae4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/749732355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4076060 0x7f7ae4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 -- 172.21.15.137:0/749732355 >> 172.21.15.137:0/749732355 conn(0x7f7ae40fee50 msgr2=0x7f7ae4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 -- 172.21.15.137:0/749732355 shutdown_connections 2024-03-26T13:59:13.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.371+0000 7f7ae8a00640 1 -- 172.21.15.137:0/749732355 wait complete. 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 Processor -- start 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 -- start start 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 0x7f7ae419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae40772a0 0x7f7ae419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4077c40 0x7f7ae41a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4113820 con 0x7f7ae4076060 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae41136a0 con 0x7f7ae4077c40 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae8a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae41139a0 con 0x7f7ae40772a0 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 0x7f7ae419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 0x7f7ae419ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51704/0 (socket says 172.21.15.137:51704) 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.372+0000 7f7ae1e00640 1 -- 172.21.15.137:0/319127646 learned_addr learned my addr 172.21.15.137:0/319127646 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:13.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae2800640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4077c40 0x7f7ae41a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1400640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae40772a0 0x7f7ae419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1e00640 1 -- 172.21.15.137:0/319127646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae40772a0 msgr2=0x7f7ae419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1e00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae40772a0 0x7f7ae419e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1e00640 1 -- 172.21.15.137:0/319127646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4077c40 msgr2=0x7f7ae41a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1e00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4077c40 0x7f7ae41a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1e00640 1 -- 172.21.15.137:0/319127646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae41a5a10 con 0x7f7ae4076060 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae2800640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4077c40 0x7f7ae41a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae1e00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 0x7f7ae419ddf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400d950 tx=0x7f7ad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4017070 con 0x7f7ae4076060 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7ad4009c30 con 0x7f7ae4076060 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400f620 con 0x7f7ae4076060 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.373+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae41a5d00 con 0x7f7ae4076060 2024-03-26T13:59:13.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.374+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae41a61a0 con 0x7f7ae4076060 2024-03-26T13:59:13.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.375+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae4110390 con 0x7f7ae4076060 2024-03-26T13:59:13.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.377+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7ad40109a0 con 0x7f7ae4076060 2024-03-26T13:59:13.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.378+0000 7f7adaa00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7ac80a2880 0x7f7ac80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:13.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.378+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7ad40ce2e0 con 0x7f7ae4076060 2024-03-26T13:59:13.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.378+0000 7f7ae1400640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7ac80a2880 0x7f7ac80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:13.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.379+0000 7f7ae1400640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7ac80a2880 0x7f7ac80a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ae419f300 tx=0x7f7ad0005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:13.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.382+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ad4009de0 con 0x7f7ae4076060 2024-03-26T13:59:13.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.623+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ae402d070 con 0x7f7ae4076060 2024-03-26T13:59:13.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.625+0000 7f7adaa00640 1 -- 172.21.15.137:0/319127646 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7f7ad409b780 con 0x7f7ae4076060 2024-03-26T13:59:13.622 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:13.622 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T13:59:13.622 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 12 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.627+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7ac80a2880 msgr2=0x7f7ac80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.627+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7ac80a2880 0x7f7ac80a4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7ae419f300 tx=0x7f7ad0005cc0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.627+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 msgr2=0x7f7ae419ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.627+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 0x7f7ae419ddf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400d950 tx=0x7f7ad400de10 comp rx=0 tx=0).stop 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 shutdown_connections 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7ac80a2880 0x7f7ac80a4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ae4077c40 0x7f7ae41a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ae40772a0 0x7f7ae419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 --2- 172.21.15.137:0/319127646 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ae4076060 0x7f7ae419ddf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 >> 172.21.15.137:0/319127646 conn(0x7f7ae40fee50 msgr2=0x7f7ae4100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:13.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 shutdown_connections 2024-03-26T13:59:13.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:13.628+0000 7f7ae8a00640 1 -- 172.21.15.137:0/319127646 wait complete. 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: Deploying daemon mds.b on smithi159 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.137:6840/2350775776,v1:172.21.15.137:6841/2350775776] up:boot 2024-03-26T13:59:14.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: fsmap 11 up:standby 2024-03-26T13:59:14.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T13:59:14.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:13 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/319127646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:14.187 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.187 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.187 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: Deploying daemon mds.b on smithi159 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.137:6840/2350775776,v1:172.21.15.137:6841/2350775776] up:boot 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: fsmap 11 up:standby 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T13:59:14.188 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:13 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/319127646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: Deploying daemon mds.b on smithi159 2024-03-26T13:59:14.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:14.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.137:6840/2350775776,v1:172.21.15.137:6841/2350775776] up:boot 2024-03-26T13:59:14.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: fsmap 11 up:standby 2024-03-26T13:59:14.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-26T13:59:14.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:13 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/319127646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:15.645 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:15.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 -- 172.21.15.137:0/3876610866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 msgr2=0x7ff684103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:15.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 --2- 172.21.15.137:0/3876610866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684103d00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff674009900 tx=0x7ff67402bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:15.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 -- 172.21.15.137:0/3876610866 shutdown_connections 2024-03-26T13:59:15.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 --2- 172.21.15.137:0/3876610866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684105490 0x7ff68410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:15.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 --2- 172.21.15.137:0/3876610866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684104ad0 0x7ff684104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:15.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 --2- 172.21.15.137:0/3876610866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684103d00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:15.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 -- 172.21.15.137:0/3876610866 >> 172.21.15.137:0/3876610866 conn(0x7ff6840fee90 msgr2=0x7ff6841012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:15.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 -- 172.21.15.137:0/3876610866 shutdown_connections 2024-03-26T13:59:15.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.873+0000 7ff689a00640 1 -- 172.21.15.137:0/3876610866 wait complete. 2024-03-26T13:59:15.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 Processor -- start 2024-03-26T13:59:15.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 -- start start 2024-03-26T13:59:15.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:15.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684104ad0 0x7ff68419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:15.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684105490 0x7ff6841a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:15.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410f450 con 0x7ff6841038e0 2024-03-26T13:59:15.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410f2d0 con 0x7ff684104ad0 2024-03-26T13:59:15.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.874+0000 7ff689a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68410f5d0 con 0x7ff684105490 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51728/0 (socket says 172.21.15.137:51728) 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 -- 172.21.15.137:0/347636409 learned_addr learned my addr 172.21.15.137:0/347636409 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff67a000640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684104ad0 0x7ff68419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff683400640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684105490 0x7ff6841a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 -- 172.21.15.137:0/347636409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684105490 msgr2=0x7ff6841a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684105490 0x7ff6841a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 -- 172.21.15.137:0/347636409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684104ad0 msgr2=0x7ff68419a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684104ad0 0x7ff68419a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:15.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 -- 172.21.15.137:0/347636409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6841a1770 con 0x7ff6841038e0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff683400640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684105490 0x7ff6841a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff67a000640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684104ad0 0x7ff68419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.875+0000 7ff682a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684199b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff6740040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.876+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674043020 con 0x7ff6841038e0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.876+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6740044c0 con 0x7ff6841038e0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.876+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674003c60 con 0x7ff6841038e0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.876+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6841a1a00 con 0x7ff6841038e0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.876+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6841a1f50 con 0x7ff6841038e0 2024-03-26T13:59:15.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.877+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68410ac70 con 0x7ff6841038e0 2024-03-26T13:59:15.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.879+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff674046020 con 0x7ff6841038e0 2024-03-26T13:59:15.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.881+0000 7ff67be00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6680a2800 0x7ff6680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:15.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.881+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6740f08e0 con 0x7ff6841038e0 2024-03-26T13:59:15.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.881+0000 7ff67a000640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6680a2800 0x7ff6680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:15.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.882+0000 7ff67a000640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6680a2800 0x7ff6680a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff68419b060 tx=0x7ff66c009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:15.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:15.886+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff67404d020 con 0x7ff6841038e0 2024-03-26T13:59:16.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.127+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff68402d070 con 0x7ff6841038e0 2024-03-26T13:59:16.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.132+0000 7ff67be00640 1 -- 172.21.15.137:0/347636409 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7ff6740bde20 con 0x7ff6841038e0 2024-03-26T13:59:16.129 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:16.130 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-26T13:59:16.130 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 12 2024-03-26T13:59:16.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.134+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6680a2800 msgr2=0x7ff6680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:16.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.134+0000 7ff689a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6680a2800 0x7ff6680a4cc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff68419b060 tx=0x7ff66c009520 comp rx=0 tx=0).stop 2024-03-26T13:59:16.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.134+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 msgr2=0x7ff684199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:16.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.134+0000 7ff689a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684199b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff6740040c0 comp rx=0 tx=0).stop 2024-03-26T13:59:16.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 shutdown_connections 2024-03-26T13:59:16.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6680a2800 0x7ff6680a4cc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:16.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff684104ad0 0x7ff68419a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:16.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff684105490 0x7ff6841a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:16.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 --2- 172.21.15.137:0/347636409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6841038e0 0x7ff684199b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:16.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 >> 172.21.15.137:0/347636409 conn(0x7ff6840fee90 msgr2=0x7ff684107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:16.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 shutdown_connections 2024-03-26T13:59:16.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:16.135+0000 7ff689a00640 1 -- 172.21.15.137:0/347636409 wait complete. 2024-03-26T13:59:16.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:16 smithi159 ceph-mon[35501]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:16.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:16 smithi179 ceph-mon[35472]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:16.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:16 smithi137 ceph-mon[31539]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:16.570 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 systemd[1]: Starting Ceph mds.b for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T13:59:16.869 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 podman[65796]: 2024-03-26 13:59:16.492527458 +0000 UTC m=+0.032912751 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T13:59:16.869 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 podman[65796]: 2024-03-26 13:59:16.616233983 +0000 UTC m=+0.156619309 container create 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-26T13:59:17.135 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 podman[65796]: 2024-03-26 13:59:16.920046333 +0000 UTC m=+0.460431646 container init 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T13:59:17.136 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 podman[65796]: 2024-03-26 13:59:16.931538892 +0000 UTC m=+0.471924186 container start 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T13:59:17.136 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 ceph-mds[65823]: -- 172.21.15.159:0/481327855 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563e385e7a40 con 0x563e385bb000 2024-03-26T13:59:17.136 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: starting mds.b at 2024-03-26T13:59:17.136 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:17 smithi159 bash[65796]: 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c 2024-03-26T13:59:17.136 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:17 smithi159 systemd[1]: Started Ceph mds.b for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T13:59:17.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:17 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/347636409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:17.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:17 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/347636409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:17.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:17 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/347636409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:18.153 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 -- 172.21.15.137:0/3122260853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c441016e0 msgr2=0x7f1c44101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3122260853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c441016e0 0x7f1c44101b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1c38009900 tx=0x7f1c3802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 -- 172.21.15.137:0/3122260853 shutdown_connections 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3122260853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c44103290 0x7f1c44109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3122260853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c441028d0 0x7f1c44102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3122260853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c441016e0 0x7f1c44101b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 -- 172.21.15.137:0/3122260853 >> 172.21.15.137:0/3122260853 conn(0x7f1c440fcc50 msgr2=0x7f1c440ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:18.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 -- 172.21.15.137:0/3122260853 shutdown_connections 2024-03-26T13:59:18.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.372+0000 7f1c4b400640 1 -- 172.21.15.137:0/3122260853 wait complete. 2024-03-26T13:59:18.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.374+0000 7f1c4b400640 1 Processor -- start 2024-03-26T13:59:18.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.374+0000 7f1c4b400640 1 -- start start 2024-03-26T13:59:18.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.374+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c441016e0 0x7f1c44075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c441028d0 0x7f1c44073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c4b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 0x7f1c44073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c4b400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4410d250 con 0x7f1c44103290 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c4b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4410d0d0 con 0x7f1c441028d0 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c4b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4410d3d0 con 0x7f1c441016e0 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 0x7f1c44073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c43e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c441028d0 0x7f1c44073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 0x7f1c44073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35826/0 (socket says 172.21.15.137:35826) 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 -- 172.21.15.137:0/3014213039 learned_addr learned my addr 172.21.15.137:0/3014213039 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c48a00640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c441016e0 0x7f1c44075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 -- 172.21.15.137:0/3014213039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c441016e0 msgr2=0x7f1c44075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c441016e0 0x7f1c44075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 -- 172.21.15.137:0/3014213039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c441028d0 msgr2=0x7f1c44073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c441028d0 0x7f1c44073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.375+0000 7f1c49400640 1 -- 172.21.15.137:0/3014213039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c44074860 con 0x7f1c44103290 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.376+0000 7f1c49400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 0x7f1c44073fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00eb10 tx=0x7f1c3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.376+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c004b20 con 0x7f1c44103290 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.376+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c3c004cc0 con 0x7f1c44103290 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.376+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c441a1a70 con 0x7f1c44103290 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.376+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c010690 con 0x7f1c44103290 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.376+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c441a1f70 con 0x7f1c44103290 2024-03-26T13:59:18.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.377+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c44108ad0 con 0x7f1c44103290 2024-03-26T13:59:18.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.379+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1c3c005020 con 0x7f1c44103290 2024-03-26T13:59:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.381+0000 7f1c41600640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c2c0a2800 0x7f1c2c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.381+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1c3c0cec60 con 0x7f1c44103290 2024-03-26T13:59:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.381+0000 7f1c48a00640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c2c0a2800 0x7f1c2c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.382+0000 7f1c48a00640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c2c0a2800 0x7f1c2c0a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c3802c240 tx=0x7f1c3802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:18.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.385+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c3c09c100 con 0x7f1c44103290 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] up:boot 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: fsmap 12 up:standby 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:59:18.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] up:boot 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: fsmap 12 up:standby 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:59:18.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] up:boot 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: fsmap 12 up:standby 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:59:18.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:18.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.653+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c440622a0 con 0x7f1c44103290 2024-03-26T13:59:18.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.655+0000 7f1c41600640 1 -- 172.21.15.137:0/3014213039 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10052 (secure 0 0 0) 0x7f1c3c09ba00 con 0x7f1c44103290 2024-03-26T13:59:18.652 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:18.652 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":34496,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-26T13:59:18.653 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 13 2024-03-26T13:59:18.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c2c0a2800 msgr2=0x7f1c2c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c2c0a2800 0x7f1c2c0a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1c3802c240 tx=0x7f1c3802c7e0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 msgr2=0x7f1c44073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 0x7f1c44073fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00eb10 tx=0x7f1c3c00c2d0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 shutdown_connections 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c2c0a2800 0x7f1c2c0a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c441028d0 0x7f1c44073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c44103290 0x7f1c44073fd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 --2- 172.21.15.137:0/3014213039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c441016e0 0x7f1c44075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 >> 172.21.15.137:0/3014213039 conn(0x7f1c440fcc50 msgr2=0x7f1c44105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 shutdown_connections 2024-03-26T13:59:18.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.658+0000 7f1c4b400640 1 -- 172.21.15.137:0/3014213039 wait complete. 2024-03-26T13:59:18.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 -- 172.21.15.137:0/1185002028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 msgr2=0x7f081c10bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 --2- 172.21.15.137:0/1185002028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c10bb30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0814009be0 tx=0x7f081402bef0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 -- 172.21.15.137:0/1185002028 shutdown_connections 2024-03-26T13:59:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 --2- 172.21.15.137:0/1185002028 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c10bb30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 --2- 172.21.15.137:0/1185002028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f081c1048e0 0x7f081c104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 --2- 172.21.15.137:0/1185002028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c1036f0 0x7f081c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 -- 172.21.15.137:0/1185002028 >> 172.21.15.137:0/1185002028 conn(0x7f081c0fec80 msgr2=0x7f081c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:18.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 -- 172.21.15.137:0/1185002028 shutdown_connections 2024-03-26T13:59:18.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.870+0000 7f081b200640 1 -- 172.21.15.137:0/1185002028 wait complete. 2024-03-26T13:59:18.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 Processor -- start 2024-03-26T13:59:18.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 -- start start 2024-03-26T13:59:18.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c1036f0 0x7f081c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f081c1048e0 0x7f081c199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c1a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c10f270 con 0x7f081c1052a0 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c10f0f0 con 0x7f081c1036f0 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.871+0000 7f081b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c10f3f0 con 0x7f081c1048e0 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c1a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c1a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35846/0 (socket says 172.21.15.137:35846) 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 -- 172.21.15.137:0/2938315733 learned_addr learned my addr 172.21.15.137:0/2938315733 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:18.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f0819e00640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c1036f0 0x7f081c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f0819400640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f081c1048e0 0x7f081c199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 -- 172.21.15.137:0/2938315733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f081c1048e0 msgr2=0x7f081c199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f081c1048e0 0x7f081c199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 -- 172.21.15.137:0/2938315733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c1036f0 msgr2=0x7f081c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c1036f0 0x7f081c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.872+0000 7f081a800640 1 -- 172.21.15.137:0/2938315733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c1a14d0 con 0x7f081c1052a0 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.873+0000 7f081a800640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c1a0e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0814034fd0 tx=0x7f081402cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.873+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814043020 con 0x7f081c1052a0 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.873+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0814003e20 con 0x7f081c1052a0 2024-03-26T13:59:18.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.873+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c1a1760 con 0x7f081c1052a0 2024-03-26T13:59:18.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.873+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814034690 con 0x7f081c1052a0 2024-03-26T13:59:18.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.873+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c1a1d00 con 0x7f081c1052a0 2024-03-26T13:59:18.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.874+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c10aa10 con 0x7f081c1052a0 2024-03-26T13:59:18.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.876+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0814046020 con 0x7f081c1052a0 2024-03-26T13:59:18.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.878+0000 7f0812a00640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08000a2880 0x7f08000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:18.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.878+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f08140f0d60 con 0x7f081c1052a0 2024-03-26T13:59:18.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.878+0000 7f0819e00640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08000a2880 0x7f08000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:18.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.879+0000 7f0819e00640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08000a2880 0x7f08000a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f080c009530 tx=0x7f080c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:18.884+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f081404d020 con 0x7f081c1052a0 2024-03-26T13:59:19.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.092+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f081c0622a0 con 0x7f081c1052a0 2024-03-26T13:59:19.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.094+0000 7f0812a00640 1 -- 172.21.15.137:0/2938315733 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f08140be2a0 con 0x7f081c1052a0 2024-03-26T13:59:19.092 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":82,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:58:17.300101+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08000a2880 msgr2=0x7f08000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08000a2880 0x7f08000a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f080c009530 tx=0x7f080c009260 comp rx=0 tx=0).stop 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 msgr2=0x7f081c1a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c1a0e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0814034fd0 tx=0x7f081402cbc0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 shutdown_connections 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08000a2880 0x7f08000a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f081c1052a0 0x7f081c1a0e60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f081c1048e0 0x7f081c199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 --2- 172.21.15.137:0/2938315733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f081c1036f0 0x7f081c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 >> 172.21.15.137:0/2938315733 conn(0x7f081c0fec80 msgr2=0x7f081c107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:19.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 shutdown_connections 2024-03-26T13:59:19.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.097+0000 7f081b200640 1 -- 172.21.15.137:0/2938315733 wait complete. 2024-03-26T13:59:19.114 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-26T13:59:19.115 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T13:59:19.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 -- 172.21.15.137:0/1908617540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310077c40 msgr2=0x7f13100784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 --2- 172.21.15.137:0/1908617540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310077c40 0x7f13100784e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1308009ca0 tx=0x7f130802c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 -- 172.21.15.137:0/1908617540 shutdown_connections 2024-03-26T13:59:19.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 --2- 172.21.15.137:0/1908617540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1310077c40 0x7f13100784e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 --2- 172.21.15.137:0/1908617540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13100772a0 0x7f1310077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 --2- 172.21.15.137:0/1908617540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 0x7f1310076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 -- 172.21.15.137:0/1908617540 >> 172.21.15.137:0/1908617540 conn(0x7f13100fee50 msgr2=0x7f1310101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:19.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 -- 172.21.15.137:0/1908617540 shutdown_connections 2024-03-26T13:59:19.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.316+0000 7f1314e00640 1 -- 172.21.15.137:0/1908617540 wait complete. 2024-03-26T13:59:19.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.317+0000 7f1314e00640 1 Processor -- start 2024-03-26T13:59:19.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.317+0000 7f1314e00640 1 -- start start 2024-03-26T13:59:19.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.317+0000 7f1314e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 0x7f131019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.317+0000 7f1314e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 0x7f131019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.317+0000 7f1314e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1310077c40 0x7f13101a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1314e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1310113820 con 0x7f13100772a0 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1314e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13101136a0 con 0x7f1310076060 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1314e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13101139a0 con 0x7f1310077c40 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 0x7f131019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 0x7f131019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35870/0 (socket says 172.21.15.137:35870) 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 -- 172.21.15.137:0/1605418141 learned_addr learned my addr 172.21.15.137:0/1605418141 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f130de00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 0x7f131019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f130e800640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1310077c40 0x7f13101a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 -- 172.21.15.137:0/1605418141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1310077c40 msgr2=0x7f13101a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1310077c40 0x7f13101a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 -- 172.21.15.137:0/1605418141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 msgr2=0x7f131019def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 0x7f131019def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f1307e00640 1 -- 172.21.15.137:0/1605418141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13101a5b10 con 0x7f13100772a0 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.318+0000 7f130de00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 0x7f131019def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.319+0000 7f1307e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 0x7f131019e430 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d770 tx=0x7f12f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.319+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8017070 con 0x7f13100772a0 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.319+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f12f80054d0 con 0x7f13100772a0 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.319+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13101a5e00 con 0x7f13100772a0 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.319+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8004cf0 con 0x7f13100772a0 2024-03-26T13:59:19.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.319+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13101a6270 con 0x7f13100772a0 2024-03-26T13:59:19.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.320+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1310110250 con 0x7f13100772a0 2024-03-26T13:59:19.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.323+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f12f8004e90 con 0x7f13100772a0 2024-03-26T13:59:19.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.325+0000 7f1306a00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f12f40a2800 0x7f12f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.325+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f12f80ce0d0 con 0x7f13100772a0 2024-03-26T13:59:19.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.325+0000 7f130de00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f12f40a2800 0x7f12f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.325+0000 7f130de00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f12f40a2800 0x7f12f40a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13000058c0 tx=0x7f1300005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:19.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.333+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f12f809b5f0 con 0x7f13100772a0 2024-03-26T13:59:19.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:19 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3014213039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:19.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:19 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2938315733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:19.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:19 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3014213039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:19.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:19 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2938315733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:19.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:19 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3014213039' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:19.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:19 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2938315733' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:19.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.542+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f131002d2b0 con 0x7f13100772a0 2024-03-26T13:59:19.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.543+0000 7f1306a00640 1 -- 172.21.15.137:0/1605418141 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12f809aef0 con 0x7f13100772a0 2024-03-26T13:59:19.540 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f12f40a2800 msgr2=0x7f12f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f12f40a2800 0x7f12f40a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f13000058c0 tx=0x7f1300005850 comp rx=0 tx=0).stop 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 msgr2=0x7f131019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 0x7f131019e430 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f12f800d770 tx=0x7f12f800dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 shutdown_connections 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f12f40a2800 0x7f12f40a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1310077c40 0x7f13101a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f13100772a0 0x7f131019e430 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 --2- 172.21.15.137:0/1605418141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1310076060 0x7f131019def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 >> 172.21.15.137:0/1605418141 conn(0x7f13100fee50 msgr2=0x7f1310100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:19.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 shutdown_connections 2024-03-26T13:59:19.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.545+0000 7f1314e00640 1 -- 172.21.15.137:0/1605418141 wait complete. 2024-03-26T13:59:19.561 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 -- 172.21.15.137:0/2284097112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e8103270 msgr2=0x7f63e8109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 --2- 172.21.15.137:0/2284097112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e8103270 0x7f63e8109b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63e0009ca0 tx=0x7f63e002c100 comp rx=0 tx=0).stop 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 -- 172.21.15.137:0/2284097112 shutdown_connections 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 --2- 172.21.15.137:0/2284097112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63e81028b0 0x7f63e8102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 --2- 172.21.15.137:0/2284097112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e8103270 0x7f63e8109b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 --2- 172.21.15.137:0/2284097112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63e81016c0 0x7f63e8101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 -- 172.21.15.137:0/2284097112 >> 172.21.15.137:0/2284097112 conn(0x7f63e80fcc50 msgr2=0x7f63e80ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 -- 172.21.15.137:0/2284097112 shutdown_connections 2024-03-26T13:59:19.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.751+0000 7f63ed200640 1 -- 172.21.15.137:0/2284097112 wait complete. 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.752+0000 7f63ed200640 1 Processor -- start 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.752+0000 7f63ed200640 1 -- start start 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63ed200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 0x7f63e8075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63ed200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63e81028b0 0x7f63e8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63ed200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63e8103270 0x7f63e8073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63ed200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e810d1f0 con 0x7f63e81016c0 2024-03-26T13:59:19.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63ed200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e810d070 con 0x7f63e81028b0 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63ed200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e810d370 con 0x7f63e8103270 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 0x7f63e8075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 0x7f63e8075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35886/0 (socket says 172.21.15.137:35886) 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 -- 172.21.15.137:0/1511555682 learned_addr learned my addr 172.21.15.137:0/1511555682 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e6800640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63e8103270 0x7f63e8073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5400640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63e81028b0 0x7f63e8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 -- 172.21.15.137:0/1511555682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63e8103270 msgr2=0x7f63e8073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63e8103270 0x7f63e8073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 -- 172.21.15.137:0/1511555682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63e81028b0 msgr2=0x7f63e8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63e81028b0 0x7f63e8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.753+0000 7f63e5e00640 1 -- 172.21.15.137:0/1511555682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e8074860 con 0x7f63e81016c0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.754+0000 7f63e5e00640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 0x7f63e8075450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63d800e9c0 tx=0x7f63d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.754+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800cd80 con 0x7f63e81016c0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.754+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f63d80035f0 con 0x7f63e81016c0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.754+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e81a19e0 con 0x7f63e81016c0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.754+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8010690 con 0x7f63e81016c0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.754+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e81a1f30 con 0x7f63e81016c0 2024-03-26T13:59:19.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.755+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e8108a30 con 0x7f63e81016c0 2024-03-26T13:59:19.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.756+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f63d80037a0 con 0x7f63e81016c0 2024-03-26T13:59:19.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.758+0000 7f63dea00640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63cc0a2880 0x7f63cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:19.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.758+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63d80ceb80 con 0x7f63e81016c0 2024-03-26T13:59:19.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.758+0000 7f63e5400640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63cc0a2880 0x7f63cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:19.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.760+0000 7f63e5400640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63cc0a2880 0x7f63cc0a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63e80751c0 tx=0x7f63d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:19.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.763+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f63d809c020 con 0x7f63e81016c0 2024-03-26T13:59:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.981+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f63e80622a0 con 0x7f63e81016c0 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.981+0000 7f63dea00640 1 -- 172.21.15.137:0/1511555682 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f63d809b920 con 0x7f63e81016c0 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stdout:[ 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "poolnum": 1, 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: "poolname": ".mgr" 2024-03-26T13:59:19.978 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T13:59:19.979 INFO:teuthology.orchestra.run.smithi137.stdout:] 2024-03-26T13:59:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63cc0a2880 msgr2=0x7f63cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63cc0a2880 0x7f63cc0a4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63e80751c0 tx=0x7f63d400b410 comp rx=0 tx=0).stop 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 msgr2=0x7f63e8075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 0x7f63e8075450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f63d800e9c0 tx=0x7f63d800ee80 comp rx=0 tx=0).stop 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 shutdown_connections 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63cc0a2880 0x7f63cc0a4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63e81028b0 0x7f63e8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63e8103270 0x7f63e8073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 --2- 172.21.15.137:0/1511555682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63e81016c0 0x7f63e8075450 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 >> 172.21.15.137:0/1511555682 conn(0x7f63e80fcc50 msgr2=0x7f63e8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 shutdown_connections 2024-03-26T13:59:19.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:19.984+0000 7f63ed200640 1 -- 172.21.15.137:0/1511555682 wait complete. 2024-03-26T13:59:19.998 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-26T13:59:19.998 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-26T13:59:20.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 -- 172.21.15.137:0/2917070264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc102150 msgr2=0x7f62dc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 --2- 172.21.15.137:0/2917070264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc102150 0x7f62dc1089e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009ca0 tx=0x7f62d402c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 -- 172.21.15.137:0/2917070264 shutdown_connections 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 --2- 172.21.15.137:0/2917070264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc101790 0x7f62dc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 --2- 172.21.15.137:0/2917070264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc102150 0x7f62dc1089e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 --2- 172.21.15.137:0/2917070264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc1005a0 0x7f62dc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 -- 172.21.15.137:0/2917070264 >> 172.21.15.137:0/2917070264 conn(0x7f62dc0fbb30 msgr2=0x7f62dc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 -- 172.21.15.137:0/2917070264 shutdown_connections 2024-03-26T13:59:20.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.189+0000 7f62e1600640 1 -- 172.21.15.137:0/2917070264 wait complete. 2024-03-26T13:59:20.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.190+0000 7f62e1600640 1 Processor -- start 2024-03-26T13:59:20.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.190+0000 7f62e1600640 1 -- start start 2024-03-26T13:59:20.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62e1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc1005a0 0x7f62dc199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:20.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62e1600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 0x7f62dc199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:20.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62e1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc102150 0x7f62dc1a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:20.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62e1600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc10c0e0 con 0x7f62dc101790 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62e1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc10bf60 con 0x7f62dc1005a0 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62e1600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc10c260 con 0x7f62dc102150 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 0x7f62dc199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 0x7f62dc199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35904/0 (socket says 172.21.15.137:35904) 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 -- 172.21.15.137:0/775651352 learned_addr learned my addr 172.21.15.137:0/775651352 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62daa00640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc1005a0 0x7f62dc199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62db400640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc102150 0x7f62dc1a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 -- 172.21.15.137:0/775651352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc102150 msgr2=0x7f62dc1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc102150 0x7f62dc1a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:20.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 -- 172.21.15.137:0/775651352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc1005a0 msgr2=0x7f62dc199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc1005a0 0x7f62dc199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62d2000640 1 -- 172.21.15.137:0/775651352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc1a1630 con 0x7f62dc101790 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.191+0000 7f62db400640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc102150 0x7f62dc1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.192+0000 7f62d2000640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 0x7f62dc199f50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c920 tx=0x7f62c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.192+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4007c70 con 0x7f62dc101790 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.192+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc1a1920 con 0x7f62dc101790 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.192+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62c4007e10 con 0x7f62dc101790 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.192+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4018820 con 0x7f62dc101790 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.192+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc1a1ec0 con 0x7f62dc101790 2024-03-26T13:59:20.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.193+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc1a2150 con 0x7f62dc101790 2024-03-26T13:59:20.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.195+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f62c40189c0 con 0x7f62dc101790 2024-03-26T13:59:20.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.196+0000 7f62d3e00640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:20.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.196+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62c40ce840 con 0x7f62dc101790 2024-03-26T13:59:20.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.196+0000 7f62daa00640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:20.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.197+0000 7f62daa00640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62c00a2800 0x7f62c00a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62cc005fd0 tx=0x7f62cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:20.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.200+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f62c4014030 con 0x7f62dc101790 2024-03-26T13:59:20.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:20.407+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f62dc1a2320 con 0x7f62dc101790 2024-03-26T13:59:20.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:20 smithi179 ceph-mon[35472]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:20.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:20 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1605418141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:20.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:20 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1511555682' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T13:59:20.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:20 smithi159 ceph-mon[35501]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:20.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:20 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1605418141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:20.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:20 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1511555682' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T13:59:20.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:20 smithi137 ceph-mon[31539]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:20.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:20 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1605418141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:20.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:20 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1511555682' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-26T13:59:21.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.235+0000 7f62d3e00640 1 -- 172.21.15.137:0/775651352 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f62c400de20 con 0x7f62dc101790 2024-03-26T13:59:21.232 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_metadata' created 2024-03-26T13:59:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.240+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62c00a2800 msgr2=0x7f62c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.240+0000 7f62e1600640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62c00a2800 0x7f62c00a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62cc005fd0 tx=0x7f62cc005850 comp rx=0 tx=0).stop 2024-03-26T13:59:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.240+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 msgr2=0x7f62dc199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 0x7f62dc199f50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c920 tx=0x7f62c400cde0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 shutdown_connections 2024-03-26T13:59:21.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62c00a2800 0x7f62c00a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62dc102150 0x7f62dc1a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62dc101790 0x7f62dc199f50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 --2- 172.21.15.137:0/775651352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62dc1005a0 0x7f62dc199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.241+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 >> 172.21.15.137:0/775651352 conn(0x7f62dc0fbb30 msgr2=0x7f62dc0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:21.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.242+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 shutdown_connections 2024-03-26T13:59:21.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.243+0000 7f62e1600640 1 -- 172.21.15.137:0/775651352 wait complete. 2024-03-26T13:59:21.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-26T13:59:21.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 -- 172.21.15.137:0/2190484563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 msgr2=0x7fb86c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:21.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 --2- 172.21.15.137:0/2190484563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c10bd20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb864009ca0 tx=0x7fb86402c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 -- 172.21.15.137:0/2190484563 shutdown_connections 2024-03-26T13:59:21.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 --2- 172.21.15.137:0/2190484563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 0x7fb86c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 --2- 172.21.15.137:0/2190484563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c10bd20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 --2- 172.21.15.137:0/2190484563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86c1038e0 0x7fb86c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 -- 172.21.15.137:0/2190484563 >> 172.21.15.137:0/2190484563 conn(0x7fb86c0fee50 msgr2=0x7fb86c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:21.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 -- 172.21.15.137:0/2190484563 shutdown_connections 2024-03-26T13:59:21.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.459+0000 7fb870a00640 1 -- 172.21.15.137:0/2190484563 wait complete. 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 Processor -- start 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 -- start start 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86c1038e0 0x7fb86c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 0x7fb86c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c10f410 con 0x7fb86c105490 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c10f290 con 0x7fb86c104ad0 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb870a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86c10f590 con 0x7fb86c1038e0 2024-03-26T13:59:21.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb86a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb86a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c1a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35932/0 (socket says 172.21.15.137:35932) 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.460+0000 7fb86a800640 1 -- 172.21.15.137:0/977598183 learned_addr learned my addr 172.21.15.137:0/977598183 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb869e00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86c1038e0 0x7fb86c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb861400640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 0x7fb86c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb86a800640 1 -- 172.21.15.137:0/977598183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86c1038e0 msgr2=0x7fb86c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb86a800640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86c1038e0 0x7fb86c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb86a800640 1 -- 172.21.15.137:0/977598183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 msgr2=0x7fb86c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb86a800640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 0x7fb86c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb86a800640 1 -- 172.21.15.137:0/977598183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c1a1710 con 0x7fb86c105490 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb861400640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 0x7fb86c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb86a800640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c1a10a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb864032380 tx=0x7fb864032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:21.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864041020 con 0x7fb86c105490 2024-03-26T13:59:21.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb864003e20 con 0x7fb86c105490 2024-03-26T13:59:21.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86c1a19a0 con 0x7fb86c105490 2024-03-26T13:59:21.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.461+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864039040 con 0x7fb86c105490 2024-03-26T13:59:21.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.462+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c1a1e90 con 0x7fb86c105490 2024-03-26T13:59:21.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.463+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86c10ac20 con 0x7fb86c105490 2024-03-26T13:59:21.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.465+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb864044020 con 0x7fb86c105490 2024-03-26T13:59:21.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.466+0000 7fb863400640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8500a2800 0x7fb8500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:21.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.466+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fb8640f0f20 con 0x7fb86c105490 2024-03-26T13:59:21.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.466+0000 7fb869e00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8500a2800 0x7fb8500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:21.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.467+0000 7fb869e00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8500a2800 0x7fb8500a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005fd0 tx=0x7fb85c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:21.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.469+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb86404b020 con 0x7fb86c105490 2024-03-26T13:59:21.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/775651352' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T13:59:21.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:21.669+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fb86c02d070 con 0x7fb86c105490 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/775651352' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/775651352' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:21.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:22.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.241+0000 7fb863400640 1 -- 172.21.15.137:0/977598183 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) v1 ==== 171+0+0 (secure 0 0 0) 0x7fb8640333a0 con 0x7fb86c105490 2024-03-26T13:59:22.237 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_data' created 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8500a2800 msgr2=0x7fb8500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8500a2800 0x7fb8500a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005fd0 tx=0x7fb85c005850 comp rx=0 tx=0).stop 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 msgr2=0x7fb86c1a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c1a10a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb864032380 tx=0x7fb864032820 comp rx=0 tx=0).stop 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 shutdown_connections 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8500a2800 0x7fb8500a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb86c104ad0 0x7fb86c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb86c105490 0x7fb86c1a10a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 --2- 172.21.15.137:0/977598183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86c1038e0 0x7fb86c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.244+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 >> 172.21.15.137:0/977598183 conn(0x7fb86c0fee50 msgr2=0x7fb86c107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:22.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.245+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 shutdown_connections 2024-03-26T13:59:22.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.245+0000 7fb870a00640 1 -- 172.21.15.137:0/977598183 wait complete. 2024-03-26T13:59:22.278 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-26T13:59:22.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.465+0000 7ff442e00640 1 -- 172.21.15.137:0/2599784124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c104ad0 msgr2=0x7ff43c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.465+0000 7ff442e00640 1 --2- 172.21.15.137:0/2599784124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c104ad0 0x7ff43c104f50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff428009900 tx=0x7ff42802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 -- 172.21.15.137:0/2599784124 shutdown_connections 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 --2- 172.21.15.137:0/2599784124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c105490 0x7ff43c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 --2- 172.21.15.137:0/2599784124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c104ad0 0x7ff43c104f50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 --2- 172.21.15.137:0/2599784124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff43c1038e0 0x7ff43c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 -- 172.21.15.137:0/2599784124 >> 172.21.15.137:0/2599784124 conn(0x7ff43c0fee50 msgr2=0x7ff43c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 -- 172.21.15.137:0/2599784124 shutdown_connections 2024-03-26T13:59:22.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 -- 172.21.15.137:0/2599784124 wait complete. 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.466+0000 7ff442e00640 1 Processor -- start 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 -- start start 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c1038e0 0x7ff43c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff43c104ad0 0x7ff43c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 0x7ff43c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c10f440 con 0x7ff43c105490 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c10f2c0 con 0x7ff43c104ad0 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff442e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c10f5c0 con 0x7ff43c1038e0 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff440e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 0x7ff43c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff440e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 0x7ff43c1a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35950/0 (socket says 172.21.15.137:35950) 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff440e00640 1 -- 172.21.15.137:0/2475522652 learned_addr learned my addr 172.21.15.137:0/2475522652 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.467+0000 7ff43be00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c1038e0 0x7ff43c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff43b400640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff43c104ad0 0x7ff43c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:22.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff440e00640 1 -- 172.21.15.137:0/2475522652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c1038e0 msgr2=0x7ff43c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:22.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff440e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c1038e0 0x7ff43c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff440e00640 1 -- 172.21.15.137:0/2475522652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff43c104ad0 msgr2=0x7ff43c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:22.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff440e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff43c104ad0 0x7ff43c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:22.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff440e00640 1 -- 172.21.15.137:0/2475522652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c1a16b0 con 0x7ff43c105490 2024-03-26T13:59:22.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff440e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 0x7ff43c1a1040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff43400cce0 tx=0x7ff43400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:22.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43400f450 con 0x7ff43c105490 2024-03-26T13:59:22.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff43400faa0 con 0x7ff43c105490 2024-03-26T13:59:22.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434018b70 con 0x7ff43c105490 2024-03-26T13:59:22.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c1a19a0 con 0x7ff43c105490 2024-03-26T13:59:22.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.468+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c1a1e70 con 0x7ff43c105490 2024-03-26T13:59:22.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.470+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c10ac00 con 0x7ff43c105490 2024-03-26T13:59:22.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.473+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff434018d10 con 0x7ff43c105490 2024-03-26T13:59:22.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.474+0000 7ff438c00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff4200a2880 0x7ff4200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:22.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.474+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7ff4340cee70 con 0x7ff43c105490 2024-03-26T13:59:22.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.474+0000 7ff43be00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff4200a2880 0x7ff4200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:22.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.475+0000 7ff43be00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff4200a2880 0x7ff4200a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff42c005fd0 tx=0x7ff42c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:22.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.487+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff434014030 con 0x7ff43c105490 2024-03-26T13:59:22.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:22 smithi137 ceph-mon[31539]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:22.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:22 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/775651352' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T13:59:22.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:22 smithi137 ceph-mon[31539]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T13:59:22.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:22 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/977598183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T13:59:22.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:22 smithi179 ceph-mon[35472]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:22.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:22 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/775651352' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T13:59:22.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:22 smithi179 ceph-mon[35472]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T13:59:22.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:22 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/977598183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T13:59:22.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:22 smithi159 ceph-mon[35501]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:22.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:22 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/775651352' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-26T13:59:22.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:22 smithi159 ceph-mon[35501]: osdmap e83: 12 total, 12 up, 12 in 2024-03-26T13:59:22.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:22 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/977598183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-03-26T13:59:22.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:22.725+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff43c02d070 con 0x7ff43c105490 2024-03-26T13:59:23.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.290+0000 7ff438c00640 1 -- 172.21.15.137:0/2475522652 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff43409c130 con 0x7ff43c105490 2024-03-26T13:59:23.288 INFO:teuthology.orchestra.run.smithi137.stderr:new fs with metadata pool 2 and data pool 3 2024-03-26T13:59:23.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.296+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff4200a2880 msgr2=0x7ff4200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:23.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.296+0000 7ff442e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff4200a2880 0x7ff4200a4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff42c005fd0 tx=0x7ff42c005850 comp rx=0 tx=0).stop 2024-03-26T13:59:23.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.296+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 msgr2=0x7ff43c1a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:23.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.296+0000 7ff442e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 0x7ff43c1a1040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff43400cce0 tx=0x7ff43400a620 comp rx=0 tx=0).stop 2024-03-26T13:59:23.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.296+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 shutdown_connections 2024-03-26T13:59:23.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff4200a2880 0x7ff4200a4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff43c104ad0 0x7ff43c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff43c105490 0x7ff43c1a1040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 --2- 172.21.15.137:0/2475522652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c1038e0 0x7ff43c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 >> 172.21.15.137:0/2475522652 conn(0x7ff43c0fee50 msgr2=0x7ff43c107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:23.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 shutdown_connections 2024-03-26T13:59:23.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.297+0000 7ff442e00640 1 -- 172.21.15.137:0/2475522652 wait complete. 2024-03-26T13:59:23.326 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-26T13:59:23.326 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-26T13:59:23.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/977598183' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T13:59:23.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T13:59:23.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:23.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:59:23.387 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:23.388 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2475522652' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T13:59:23.388 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T13:59:23.388 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:59:23.388 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:23 smithi137 ceph-mon[31539]: pgmap v359: 97 pgs: 26 creating+peering, 68 unknown, 3 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:23.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 -- 172.21.15.137:0/4007034203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 msgr2=0x7ff1b0103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:23.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 --2- 172.21.15.137:0/4007034203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0103d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009900 tx=0x7ff1a002bc50 comp rx=0 tx=0).stop 2024-03-26T13:59:23.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 -- 172.21.15.137:0/4007034203 shutdown_connections 2024-03-26T13:59:23.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 --2- 172.21.15.137:0/4007034203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1b0105490 0x7ff1b010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 --2- 172.21.15.137:0/4007034203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0104ad0 0x7ff1b0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 --2- 172.21.15.137:0/4007034203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0103d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 -- 172.21.15.137:0/4007034203 >> 172.21.15.137:0/4007034203 conn(0x7ff1b00fee50 msgr2=0x7ff1b0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:23.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 -- 172.21.15.137:0/4007034203 shutdown_connections 2024-03-26T13:59:23.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.528+0000 7ff1b5800640 1 -- 172.21.15.137:0/4007034203 wait complete. 2024-03-26T13:59:23.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 Processor -- start 2024-03-26T13:59:23.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 -- start start 2024-03-26T13:59:23.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:23.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1b0104ad0 0x7ff1b019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:23.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0105490 0x7ff1b01a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:23.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b010f430 con 0x7ff1b0105490 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b010f2b0 con 0x7ff1b01038e0 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.529+0000 7ff1b5800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b010f5b0 con 0x7ff1b0104ad0 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0105490 0x7ff1b01a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55966/0 (socket says 172.21.15.137:55966) 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 -- 172.21.15.137:0/1439358100 learned_addr learned my addr 172.21.15.137:0/1439358100 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1ae000640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1b0104ad0 0x7ff1b019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 -- 172.21.15.137:0/1439358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1b0104ad0 msgr2=0x7ff1b019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1b0104ad0 0x7ff1b019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 -- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0105490 msgr2=0x7ff1b01a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0105490 0x7ff1b01a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1aea00640 1 -- 172.21.15.137:0/1439358100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b01a1750 con 0x7ff1b01038e0 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.530+0000 7ff1af400640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0105490 0x7ff1b01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T13:59:23.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.531+0000 7ff1aea00640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0199b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0000c00 tx=0x7ff1a00040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:23.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.531+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0043020 con 0x7ff1b01038e0 2024-03-26T13:59:23.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.531+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b01a19e0 con 0x7ff1b01038e0 2024-03-26T13:59:23.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.531+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff1a00044c0 con 0x7ff1b01038e0 2024-03-26T13:59:23.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.531+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0003c00 con 0x7ff1b01038e0 2024-03-26T13:59:23.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.531+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b01a1f80 con 0x7ff1b01038e0 2024-03-26T13:59:23.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.532+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b00688e0 con 0x7ff1b01038e0 2024-03-26T13:59:23.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.534+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff1a0046020 con 0x7ff1b01038e0 2024-03-26T13:59:23.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.535+0000 7ff1a7400640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff1940a2800 0x7ff1940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:23.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.536+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1a00f0f70 con 0x7ff1b01038e0 2024-03-26T13:59:23.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.536+0000 7ff1ae000640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff1940a2800 0x7ff1940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:23.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.536+0000 7ff1ae000640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff1940a2800 0x7ff1940a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff1b019b040 tx=0x7ff19c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:23.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.538+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff1a004d020 con 0x7ff1b01038e0 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/977598183' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2475522652' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:59:23.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:23 smithi179 ceph-mon[35472]: pgmap v359: 97 pgs: 26 creating+peering, 68 unknown, 3 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/977598183' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: osdmap e84: 12 total, 12 up, 12 in 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2475522652' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T13:59:23.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:23 smithi159 ceph-mon[35501]: pgmap v359: 97 pgs: 26 creating+peering, 68 unknown, 3 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:23.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:23.791+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ff1b002d100 con 0x7ff1b01038e0 2024-03-26T13:59:24.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.272+0000 7ff1a7400640 1 -- 172.21.15.137:0/1439358100 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) v1 ==== 157+0+0 (secure 0 0 0) 0x7ff1a00be0c0 con 0x7ff1b01038e0 2024-03-26T13:59:24.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff1940a2800 msgr2=0x7ff1940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff1940a2800 0x7ff1940a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff1b019b040 tx=0x7ff19c00b410 comp rx=0 tx=0).stop 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 msgr2=0x7ff1b0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0199b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0000c00 tx=0x7ff1a00040c0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 shutdown_connections 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff1940a2800 0x7ff1940a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1b0105490 0x7ff1b01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff1b0104ad0 0x7ff1b019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 --2- 172.21.15.137:0/1439358100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff1b01038e0 0x7ff1b0199b30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 >> 172.21.15.137:0/1439358100 conn(0x7ff1b00fee50 msgr2=0x7ff1b0107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:24.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.276+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 shutdown_connections 2024-03-26T13:59:24.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.277+0000 7ff1b5800640 1 -- 172.21.15.137:0/1439358100 wait complete. 2024-03-26T13:59:24.291 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-26T13:59:24.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T13:59:24.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2475522652' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] up:active 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1439358100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T13:59:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:24 smithi159 ceph-mon[35501]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 -- 172.21.15.137:0/1660569708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241036f0 msgr2=0x7fca24103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1660569708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241036f0 0x7fca24103b10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fca18009900 tx=0x7fca1802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 -- 172.21.15.137:0/1660569708 shutdown_connections 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1660569708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241048e0 0x7fca24104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1660569708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca241052a0 0x7fca2410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1660569708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241036f0 0x7fca24103b10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 -- 172.21.15.137:0/1660569708 >> 172.21.15.137:0/1660569708 conn(0x7fca240fec80 msgr2=0x7fca241010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:24.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 -- 172.21.15.137:0/1660569708 shutdown_connections 2024-03-26T13:59:24.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.496+0000 7fca2ac00640 1 -- 172.21.15.137:0/1660569708 wait complete. 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 Processor -- start 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 -- start start 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241036f0 0x7fca241998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca241048e0 0x7fca24199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 0x7fca241a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2410f270 con 0x7fca241052a0 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2410f0f0 con 0x7fca241036f0 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.497+0000 7fca2ac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2410f3f0 con 0x7fca241048e0 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 0x7fca241a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 0x7fca241a0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35984/0 (socket says 172.21.15.137:35984) 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 -- 172.21.15.137:0/1566463911 learned_addr learned my addr 172.21.15.137:0/1566463911 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:24.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca29800640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241036f0 0x7fca241998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca28e00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca241048e0 0x7fca24199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 -- 172.21.15.137:0/1566463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca241048e0 msgr2=0x7fca24199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca241048e0 0x7fca24199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 -- 172.21.15.137:0/1566463911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241036f0 msgr2=0x7fca241998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241036f0 0x7fca241998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 -- 172.21.15.137:0/1566463911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca241a14e0 con 0x7fca241052a0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca29800640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241036f0 0x7fca241998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.498+0000 7fca2a200640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 0x7fca241a0e70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00eb10 tx=0x7fca1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.499+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c004b20 con 0x7fca241052a0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.499+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca1c004cc0 con 0x7fca241052a0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.499+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca241a17d0 con 0x7fca241052a0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.499+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c010690 con 0x7fca241052a0 2024-03-26T13:59:24.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.499+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca241a1d20 con 0x7fca241052a0 2024-03-26T13:59:24.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.500+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2410aa10 con 0x7fca241052a0 2024-03-26T13:59:24.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.502+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fca1c005020 con 0x7fca241052a0 2024-03-26T13:59:24.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.503+0000 7fca22000640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:24.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.504+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8277+0+0 (secure 0 0 0) 0x7fca1c0cf260 con 0x7fca241052a0 2024-03-26T13:59:24.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.504+0000 7fca29800640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:24.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.504+0000 7fca29800640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca1802c240 tx=0x7fca18005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:24.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.508+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fca1c09c270 con 0x7fca241052a0 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2475522652' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] up:active 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1439358100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T13:59:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:24 smithi137 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: osdmap e85: 12 total, 12 up, 12 in 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2475522652' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] up:active 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1439358100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T13:59:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:24 smithi179 ceph-mon[35472]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-03-26T13:59:24.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:24.726+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fca240622a0 con 0x7fca241052a0 2024-03-26T13:59:26.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:25 smithi179 ceph-mon[35472]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T13:59:26.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:25 smithi179 ceph-mon[35472]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T13:59:26.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:25 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1566463911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T13:59:26.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:25 smithi179 ceph-mon[35472]: pgmap v362: 97 pgs: 34 creating+peering, 48 unknown, 15 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-03-26T13:59:26.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:25 smithi159 ceph-mon[35501]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T13:59:26.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:25 smithi159 ceph-mon[35501]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T13:59:26.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:25 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1566463911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T13:59:26.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:25 smithi159 ceph-mon[35501]: pgmap v362: 97 pgs: 34 creating+peering, 48 unknown, 15 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-03-26T13:59:26.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.284+0000 7fca22000640 1 -- 172.21.15.137:0/1566463911 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) v1 ==== 228+0+0 (secure 0 0 0) 0x7fca1c09bb70 con 0x7fca241052a0 2024-03-26T13:59:26.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:25 smithi137 ceph-mon[31539]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-03-26T13:59:26.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:25 smithi137 ceph-mon[31539]: osdmap e86: 12 total, 12 up, 12 in 2024-03-26T13:59:26.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:25 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1566463911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-26T13:59:26.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:25 smithi137 ceph-mon[31539]: pgmap v362: 97 pgs: 34 creating+peering, 48 unknown, 15 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2024-03-26T13:59:26.282 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_data_ec' created 2024-03-26T13:59:26.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca0c0a2800 msgr2=0x7fca0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:26.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca1802c240 tx=0x7fca18005fd0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 msgr2=0x7fca241a0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:26.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 0x7fca241a0e70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00eb10 tx=0x7fca1c00c2d0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 shutdown_connections 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca0c0a2800 0x7fca0c0a4cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca241052a0 0x7fca241a0e70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca241048e0 0x7fca24199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 --2- 172.21.15.137:0/1566463911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca241036f0 0x7fca241998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.289+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 >> 172.21.15.137:0/1566463911 conn(0x7fca240fec80 msgr2=0x7fca24107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.290+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 shutdown_connections 2024-03-26T13:59:26.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.290+0000 7fca2ac00640 1 -- 172.21.15.137:0/1566463911 wait complete. 2024-03-26T13:59:26.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.503+0000 7f59e4200640 1 -- 172.21.15.137:0/817222030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc1038c0 msgr2=0x7f59dc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 --2- 172.21.15.137:0/817222030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc1038c0 0x7f59dc103ce0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009900 tx=0x7f59d002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 -- 172.21.15.137:0/817222030 shutdown_connections 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 --2- 172.21.15.137:0/817222030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 0x7f59dc10bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 --2- 172.21.15.137:0/817222030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59dc104ab0 0x7f59dc104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 --2- 172.21.15.137:0/817222030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc1038c0 0x7f59dc103ce0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 -- 172.21.15.137:0/817222030 >> 172.21.15.137:0/817222030 conn(0x7f59dc0fee50 msgr2=0x7f59dc101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 -- 172.21.15.137:0/817222030 shutdown_connections 2024-03-26T13:59:26.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.504+0000 7f59e4200640 1 -- 172.21.15.137:0/817222030 wait complete. 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 Processor -- start 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 -- start start 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59dc1038c0 0x7f59dc199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 0x7f59dc19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 0x7f59dc1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc10f430 con 0x7f59dc104ab0 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc10f2b0 con 0x7f59dc105470 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e4200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc10f5b0 con 0x7f59dc1038c0 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 0x7f59dc19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 0x7f59dc19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36012/0 (socket says 172.21.15.137:36012) 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.505+0000 7f59e0e00640 1 -- 172.21.15.137:0/1313462617 learned_addr learned my addr 172.21.15.137:0/1313462617 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:26.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e2200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 0x7f59dc1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e1800640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59dc1038c0 0x7f59dc199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e0e00640 1 -- 172.21.15.137:0/1313462617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59dc1038c0 msgr2=0x7f59dc199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e0e00640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59dc1038c0 0x7f59dc199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e0e00640 1 -- 172.21.15.137:0/1313462617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 msgr2=0x7f59dc1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e0e00640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 0x7f59dc1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e0e00640 1 -- 172.21.15.137:0/1313462617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc1a1750 con 0x7f59dc104ab0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e2200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 0x7f59dc1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e0e00640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 0x7f59dc19a070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59c800d770 tx=0x7f59c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8017070 con 0x7f59dc104ab0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59c80054d0 con 0x7f59dc104ab0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc1a1a40 con 0x7f59dc104ab0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.506+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c8004cf0 con 0x7f59dc104ab0 2024-03-26T13:59:26.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.507+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc1a1f40 con 0x7f59dc104ab0 2024-03-26T13:59:26.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.507+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc10ac80 con 0x7f59dc104ab0 2024-03-26T13:59:26.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.509+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f59c8004e90 con 0x7f59dc104ab0 2024-03-26T13:59:26.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.512+0000 7f59da000640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f59c40a2800 0x7f59c40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:26.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.512+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f59c80ceba0 con 0x7f59dc104ab0 2024-03-26T13:59:26.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.512+0000 7f59e1800640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f59c40a2800 0x7f59c40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:26.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.513+0000 7f59e1800640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f59c40a2800 0x7f59c40a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59d002c240 tx=0x7f59d0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:26.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.515+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f59c80cee20 con 0x7f59dc104ab0 2024-03-26T13:59:26.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:26.699+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f59dc062310 con 0x7f59dc104ab0 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: Cluster is now healthy 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:27.155 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:59:27.156 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.156 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1566463911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T13:59:27.156 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T13:59:27.156 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:26 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1313462617' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: Cluster is now healthy 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:27.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:59:27.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1566463911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T13:59:27.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T13:59:27.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:26 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1313462617' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T13:59:27.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-26T13:59:27.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: Cluster is now healthy 2024-03-26T13:59:27.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: osdmap e87: 12 total, 12 up, 12 in 2024-03-26T13:59:27.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1566463911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: osdmap e88: 12 total, 12 up, 12 in 2024-03-26T13:59:27.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:26 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1313462617' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-03-26T13:59:27.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.288+0000 7f59da000640 1 -- 172.21.15.137:0/1313462617 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7f59c809b9c0 con 0x7f59dc104ab0 2024-03-26T13:59:27.285 INFO:teuthology.orchestra.run.smithi137.stderr:set pool 4 allow_ec_overwrites to true 2024-03-26T13:59:27.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.294+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f59c40a2800 msgr2=0x7f59c40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:27.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.294+0000 7f59e4200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f59c40a2800 0x7f59c40a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59d002c240 tx=0x7f59d0005fd0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.294+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 msgr2=0x7f59dc19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:27.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.294+0000 7f59e4200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 0x7f59dc19a070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59c800d770 tx=0x7f59c800dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 shutdown_connections 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f59c40a2800 0x7f59c40a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59dc105470 0x7f59dc1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f59dc104ab0 0x7f59dc19a070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 --2- 172.21.15.137:0/1313462617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59dc1038c0 0x7f59dc199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 >> 172.21.15.137:0/1313462617 conn(0x7f59dc0fee50 msgr2=0x7f59dc107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:27.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 shutdown_connections 2024-03-26T13:59:27.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.295+0000 7f59e4200640 1 -- 172.21.15.137:0/1313462617 wait complete. 2024-03-26T13:59:27.325 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-26T13:59:27.449 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: 2024-03-26T13:59:27.155+0000 7fe59f200700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 -- 172.21.15.137:0/1344780128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680109430 msgr2=0x7f568010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 --2- 172.21.15.137:0/1344780128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680109430 0x7f568010b8c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5678009ca0 tx=0x7f567802c190 comp rx=0 tx=0).stop 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 -- 172.21.15.137:0/1344780128 shutdown_connections 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 --2- 172.21.15.137:0/1344780128 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680109430 0x7f568010b8c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 --2- 172.21.15.137:0/1344780128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5680074080 0x7f56800744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 --2- 172.21.15.137:0/1344780128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5680075660 0x7f5680073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 -- 172.21.15.137:0/1344780128 >> 172.21.15.137:0/1344780128 conn(0x7f56800fcb10 msgr2=0x7f56800fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 -- 172.21.15.137:0/1344780128 shutdown_connections 2024-03-26T13:59:27.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.504+0000 7f5685e00640 1 -- 172.21.15.137:0/1344780128 wait complete. 2024-03-26T13:59:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.505+0000 7f5685e00640 1 Processor -- start 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.505+0000 7f5685e00640 1 -- start start 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5680074080 0x7f56801998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 0x7f5680199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5680109430 0x7f56801a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568010f2c0 con 0x7f5680075660 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568010f140 con 0x7f5680074080 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568010f440 con 0x7f5680109430 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 0x7f5680199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 0x7f5680199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36044/0 (socket says 172.21.15.137:36044) 2024-03-26T13:59:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 -- 172.21.15.137:0/1848776623 learned_addr learned my addr 172.21.15.137:0/1848776623 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5684a00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5680074080 0x7f56801998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5685400640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5680109430 0x7f56801a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 -- 172.21.15.137:0/1848776623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5680109430 msgr2=0x7f56801a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5680109430 0x7f56801a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 -- 172.21.15.137:0/1848776623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5680074080 msgr2=0x7f56801998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5680074080 0x7f56801998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.506+0000 7f5677e00640 1 -- 172.21.15.137:0/1848776623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56801a1510 con 0x7f5680075660 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.507+0000 7f5677e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 0x7f5680199e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f566000c970 tx=0x7f566000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.507+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660007bb0 con 0x7f5680075660 2024-03-26T13:59:27.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.507+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5660007d50 con 0x7f5680075660 2024-03-26T13:59:27.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.507+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566000f450 con 0x7f5680075660 2024-03-26T13:59:27.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.507+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56801a1800 con 0x7f5680075660 2024-03-26T13:59:27.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.507+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56801a1d50 con 0x7f5680075660 2024-03-26T13:59:27.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.508+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56801081f0 con 0x7f5680075660 2024-03-26T13:59:27.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.510+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5660004f00 con 0x7f5680075660 2024-03-26T13:59:27.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.511+0000 7f567e000640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56700a2880 0x7f56700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:27.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.511+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8770+0+0 (secure 0 0 0) 0x7f56600cee50 con 0x7f5680075660 2024-03-26T13:59:27.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.511+0000 7f5684a00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56700a2880 0x7f56700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:27.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.512+0000 7f5684a00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56700a2880 0x7f56700a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5668005e10 tx=0x7f5668005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:27.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.514+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5660014030 con 0x7f5680075660 2024-03-26T13:59:27.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:27.735+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f56800622a0 con 0x7f5680075660 2024-03-26T13:59:28.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:27 smithi179 ceph-mon[35472]: pgmap v365: 129 pgs: 30 creating+peering, 35 unknown, 64 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-26T13:59:28.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:27 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1313462617' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T13:59:28.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:27 smithi179 ceph-mon[35472]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T13:59:28.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:27 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1848776623' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T13:59:28.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:27 smithi159 ceph-mon[35501]: pgmap v365: 129 pgs: 30 creating+peering, 35 unknown, 64 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-26T13:59:28.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:27 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1313462617' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T13:59:28.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:27 smithi159 ceph-mon[35501]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T13:59:28.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:27 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1848776623' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T13:59:28.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:27 smithi137 ceph-mon[31539]: pgmap v365: 129 pgs: 30 creating+peering, 35 unknown, 64 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-03-26T13:59:28.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:27 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1313462617' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-03-26T13:59:28.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:27 smithi137 ceph-mon[31539]: osdmap e89: 12 total, 12 up, 12 in 2024-03-26T13:59:28.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:27 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1848776623' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-03-26T13:59:28.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.292+0000 7f567e000640 1 -- 172.21.15.137:0/1848776623 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7f566009bcf0 con 0x7f5680075660 2024-03-26T13:59:28.289 INFO:teuthology.orchestra.run.smithi137.stderr:added data pool 4 to fsmap 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56700a2880 msgr2=0x7f56700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56700a2880 0x7f56700a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5668005e10 tx=0x7f5668005da0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 msgr2=0x7f5680199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 0x7f5680199e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f566000c970 tx=0x7f566000ce30 comp rx=0 tx=0).stop 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 shutdown_connections 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56700a2880 0x7f56700a4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5680109430 0x7f56801a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5680075660 0x7f5680199e30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 --2- 172.21.15.137:0/1848776623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5680074080 0x7f56801998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.296+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 >> 172.21.15.137:0/1848776623 conn(0x7f56800fcb10 msgr2=0x7f56801053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:28.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.297+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 shutdown_connections 2024-03-26T13:59:28.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.297+0000 7f5685e00640 1 -- 172.21.15.137:0/1848776623 wait complete. 2024-03-26T13:59:28.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 -- 172.21.15.137:0/1138664227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144076060 msgr2=0x7fd144076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 --2- 172.21.15.137:0/1138664227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144076060 0x7fd144076480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd138009900 tx=0x7fd13802bc50 comp rx=0 tx=0).stop 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 -- 172.21.15.137:0/1138664227 shutdown_connections 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 --2- 172.21.15.137:0/1138664227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1440772a0 0x7fd144077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 --2- 172.21.15.137:0/1138664227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd144077c40 0x7fd1440784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 --2- 172.21.15.137:0/1138664227 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144076060 0x7fd144076480 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 -- 172.21.15.137:0/1138664227 >> 172.21.15.137:0/1138664227 conn(0x7fd1440fee50 msgr2=0x7fd144101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:28.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 -- 172.21.15.137:0/1138664227 shutdown_connections 2024-03-26T13:59:28.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.530+0000 7fd14c000640 1 -- 172.21.15.137:0/1138664227 wait complete. 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.531+0000 7fd14c000640 1 Processor -- start 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.531+0000 7fd14c000640 1 -- start start 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14c000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd144076060 0x7fd14419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14c000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440772a0 0x7fd14419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14c000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 0x7fd1441a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14c000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd144113820 con 0x7fd144077c40 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14c000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441136a0 con 0x7fd144076060 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14c000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1441139a0 con 0x7fd1440772a0 2024-03-26T13:59:28.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 0x7fd1441a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 0x7fd1441a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34456/0 (socket says 172.21.15.137:34456) 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 -- 172.21.15.137:0/2821461854 learned_addr learned my addr 172.21.15.137:0/2821461854 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd148c00640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440772a0 0x7fd14419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd149600640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd144076060 0x7fd14419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 -- 172.21.15.137:0/2821461854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440772a0 msgr2=0x7fd14419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440772a0 0x7fd14419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 -- 172.21.15.137:0/2821461854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd144076060 msgr2=0x7fd14419ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd144076060 0x7fd14419ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.532+0000 7fd14a000640 1 -- 172.21.15.137:0/2821461854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1441a5af0 con 0x7fd144077c40 2024-03-26T13:59:28.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd149600640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd144076060 0x7fd14419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd14a000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 0x7fd1441a5480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00eaa0 tx=0x7fd13c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00cdb0 con 0x7fd144077c40 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd13c005020 con 0x7fd144077c40 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1441a5de0 con 0x7fd144077c40 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13c0116d0 con 0x7fd144077c40 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.533+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1441a62a0 con 0x7fd144077c40 2024-03-26T13:59:28.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.534+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd144110450 con 0x7fd144077c40 2024-03-26T13:59:28.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.536+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd13c0051d0 con 0x7fd144077c40 2024-03-26T13:59:28.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.537+0000 7fd142000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd12c0a2800 0x7fd12c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:28.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.538+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd13c0ceec0 con 0x7fd144077c40 2024-03-26T13:59:28.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.538+0000 7fd149600640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd12c0a2800 0x7fd12c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:28.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.538+0000 7fd149600640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd12c0a2800 0x7fd12c0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1380096c0 tx=0x7fd138000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:28.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.542+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd13c016030 con 0x7fd144077c40 2024-03-26T13:59:28.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.781+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd14402d070 con 0x7fd144077c40 2024-03-26T13:59:28.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.782+0000 7fd142000640 1 -- 172.21.15.137:0/2821461854 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11336 (secure 0 0 0) 0x7fd13c09bd20 con 0x7fd144077c40 2024-03-26T13:59:28.779 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:28.779 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:27.738005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34496,"qdb_cluster":[34496]},"id":1}]} 2024-03-26T13:59:28.779 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 16 2024-03-26T13:59:28.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.784+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd12c0a2800 msgr2=0x7fd12c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.784+0000 7fd14c000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd12c0a2800 0x7fd12c0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1380096c0 tx=0x7fd138000bc0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.784+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 msgr2=0x7fd1441a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:28.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.784+0000 7fd14c000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 0x7fd1441a5480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd13c00eaa0 tx=0x7fd13c00ef60 comp rx=0 tx=0).stop 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 shutdown_connections 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd12c0a2800 0x7fd12c0a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd144077c40 0x7fd1441a5480 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1440772a0 0x7fd14419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 --2- 172.21.15.137:0/2821461854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd144076060 0x7fd14419ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 >> 172.21.15.137:0/2821461854 conn(0x7fd1440fee50 msgr2=0x7fd1441005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 shutdown_connections 2024-03-26T13:59:28.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:28.785+0000 7fd14c000640 1 -- 172.21.15.137:0/2821461854 wait complete. 2024-03-26T13:59:28.800 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:29.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3280092682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0108a40 msgr2=0x7ff7f0108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3280092682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0108a40 0x7ff7f0108e60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009900 tx=0x7ff7e002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:29.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3280092682 shutdown_connections 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3280092682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f010a0f0 0x7ff7f01109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3280092682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f0109750 0x7ff7f0109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3280092682 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0108a40 0x7ff7f0108e60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3280092682 >> 172.21.15.137:0/3280092682 conn(0x7ff7f006e8b0 msgr2=0x7ff7f0070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3280092682 shutdown_connections 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.016+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3280092682 wait complete. 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.017+0000 7ff7f6e00640 1 Processor -- start 2024-03-26T13:59:29.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.017+0000 7ff7f6e00640 1 -- start start 2024-03-26T13:59:29.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f6e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f0108a40 0x7ff7f01a2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f6e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 0x7ff7f01a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f6e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f010a0f0 0x7ff7f01a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f6e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f01137a0 con 0x7ff7f0109750 2024-03-26T13:59:29.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f6e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0113620 con 0x7ff7f0108a40 2024-03-26T13:59:29.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f6e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0113920 con 0x7ff7f010a0f0 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 0x7ff7f01a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 0x7ff7f01a2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34476/0 (socket says 172.21.15.137:34476) 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7ef400640 1 -- 172.21.15.137:0/3795329658 learned_addr learned my addr 172.21.15.137:0/3795329658 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7efe00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f0108a40 0x7ff7f01a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.018+0000 7ff7f4e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f010a0f0 0x7ff7f01a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ef400640 1 -- 172.21.15.137:0/3795329658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f010a0f0 msgr2=0x7ff7f01a98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ef400640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f010a0f0 0x7ff7f01a98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ef400640 1 -- 172.21.15.137:0/3795329658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f0108a40 msgr2=0x7ff7f01a2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ef400640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f0108a40 0x7ff7f01a2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ef400640 1 -- 172.21.15.137:0/3795329658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f01a9f60 con 0x7ff7f0109750 2024-03-26T13:59:29.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7f4e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f010a0f0 0x7ff7f01a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ef400640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 0x7ff7f01a2880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c890 tx=0x7ff7dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc004220 con 0x7ff7f0109750 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff7dc0043c0 con 0x7ff7f0109750 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f01aa250 con 0x7ff7f0109750 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc004a00 con 0x7ff7f0109750 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.019+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f01aa7a0 con 0x7ff7f0109750 2024-03-26T13:59:29.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.020+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f010fb00 con 0x7ff7f0109750 2024-03-26T13:59:29.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.023+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff7dc004ba0 con 0x7ff7f0109750 2024-03-26T13:59:29.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.024+0000 7ff7ecc00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff7d40a2800 0x7ff7d40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.024+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff7dc0ce9d0 con 0x7ff7f0109750 2024-03-26T13:59:29.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.025+0000 7ff7efe00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff7d40a2800 0x7ff7d40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.025+0000 7ff7efe00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff7d40a2800 0x7ff7d40a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009230 tx=0x7ff7e0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:29.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.029+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff7dc014030 con 0x7ff7f0109750 2024-03-26T13:59:29.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:28 smithi179 ceph-mon[35472]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T13:59:29.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:28 smithi179 ceph-mon[35472]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T13:59:29.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:28 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1848776623' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T13:59:29.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:28 smithi179 ceph-mon[35472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:29.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:28 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2821461854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:29.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:28 smithi159 ceph-mon[35501]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T13:59:29.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:28 smithi159 ceph-mon[35501]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T13:59:29.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:28 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1848776623' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T13:59:29.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:28 smithi159 ceph-mon[35501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:29.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:28 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2821461854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:29.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.228+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7f00622a0 con 0x7ff7f0109750 2024-03-26T13:59:29.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.231+0000 7ff7ecc00640 1 -- 172.21.15.137:0/3795329658 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7ff7dc09b7d0 con 0x7ff7f0109750 2024-03-26T13:59:29.228 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:29.228 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:29.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff7d40a2800 msgr2=0x7ff7d40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff7d40a2800 0x7ff7d40a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009230 tx=0x7ff7e0005fd0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 msgr2=0x7ff7f01a2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 0x7ff7f01a2880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c890 tx=0x7ff7dc00cd50 comp rx=0 tx=0).stop 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 shutdown_connections 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff7d40a2800 0x7ff7d40a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7f010a0f0 0x7ff7f01a98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7f0109750 0x7ff7f01a2880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 --2- 172.21.15.137:0/3795329658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f0108a40 0x7ff7f01a2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 >> 172.21.15.137:0/3795329658 conn(0x7ff7f006e8b0 msgr2=0x7ff7f010ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 shutdown_connections 2024-03-26T13:59:29.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.234+0000 7ff7f6e00640 1 -- 172.21.15.137:0/3795329658 wait complete. 2024-03-26T13:59:29.245 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:28 smithi137 ceph-mon[31539]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-26T13:59:29.246 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:28 smithi137 ceph-mon[31539]: osdmap e90: 12 total, 12 up, 12 in 2024-03-26T13:59:29.246 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:28 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1848776623' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-03-26T13:59:29.246 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:28 smithi137 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:29.246 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:28 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2821461854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:29.248 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 -- 172.21.15.137:0/2380146132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80681038e0 msgr2=0x7f8068103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 --2- 172.21.15.137:0/2380146132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80681038e0 0x7f8068103d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f805c009900 tx=0x7f805c02bc50 comp rx=0 tx=0).stop 2024-03-26T13:59:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 -- 172.21.15.137:0/2380146132 shutdown_connections 2024-03-26T13:59:29.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 --2- 172.21.15.137:0/2380146132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068104ad0 0x7f8068104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 --2- 172.21.15.137:0/2380146132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8068105490 0x7f806810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 --2- 172.21.15.137:0/2380146132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80681038e0 0x7f8068103d00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 -- 172.21.15.137:0/2380146132 >> 172.21.15.137:0/2380146132 conn(0x7f80680fee90 msgr2=0x7f80681012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:29.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 -- 172.21.15.137:0/2380146132 shutdown_connections 2024-03-26T13:59:29.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.455+0000 7f8070600640 1 -- 172.21.15.137:0/2380146132 wait complete. 2024-03-26T13:59:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.456+0000 7f8070600640 1 Processor -- start 2024-03-26T13:59:29.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.456+0000 7f8070600640 1 -- start start 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f8070600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80681038e0 0x7f8068199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f8070600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 0x7f806819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f8070600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068105490 0x7f80681a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f8070600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806810f410 con 0x7f8068104ad0 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f8070600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806810f290 con 0x7f8068105490 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f8070600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806810f590 con 0x7f80681038e0 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f806d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 0x7f806819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f806d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 0x7f806819a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34500/0 (socket says 172.21.15.137:34500) 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f806d200640 1 -- 172.21.15.137:0/1351762668 learned_addr learned my addr 172.21.15.137:0/1351762668 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f806dc00640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80681038e0 0x7f8068199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.457+0000 7f806e600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068105490 0x7f80681a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806d200640 1 -- 172.21.15.137:0/1351762668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80681038e0 msgr2=0x7f8068199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806d200640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80681038e0 0x7f8068199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806d200640 1 -- 172.21.15.137:0/1351762668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068105490 msgr2=0x7f80681a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806d200640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068105490 0x7f80681a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806d200640 1 -- 172.21.15.137:0/1351762668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80681a16e0 con 0x7f8068104ad0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806e600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068105490 0x7f80681a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f806d200640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 0x7f806819a000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f805400c920 tx=0x7f805400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054007c70 con 0x7f8068104ad0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8054007e10 con 0x7f8068104ad0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054018820 con 0x7f8068104ad0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80681a19d0 con 0x7f8068104ad0 2024-03-26T13:59:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.458+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80681a1f70 con 0x7f8068104ad0 2024-03-26T13:59:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.460+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806810ac00 con 0x7f8068104ad0 2024-03-26T13:59:29.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.462+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f80540189c0 con 0x7f8068104ad0 2024-03-26T13:59:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.463+0000 7f8066000640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80500a2880 0x7f80500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.463+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f80540cee60 con 0x7f8068104ad0 2024-03-26T13:59:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.464+0000 7f806dc00640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80500a2880 0x7f80500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.464+0000 7f806dc00640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80500a2880 0x7f80500a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f805c036040 tx=0x7f805c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:29.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.468+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8054014030 con 0x7f8068104ad0 2024-03-26T13:59:29.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.655+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80680622d0 con 0x7f8068104ad0 2024-03-26T13:59:29.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.659+0000 7f8066000640 1 -- 172.21.15.137:0/1351762668 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7f805400de20 con 0x7f8068104ad0 2024-03-26T13:59:29.656 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:29.656 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:29.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80500a2880 msgr2=0x7f80500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80500a2880 0x7f80500a4d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f805c036040 tx=0x7f805c000bc0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 msgr2=0x7f806819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 0x7f806819a000 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f805400c920 tx=0x7f805400cde0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 shutdown_connections 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80500a2880 0x7f80500a4d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8068105490 0x7f80681a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8068104ad0 0x7f806819a000 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 --2- 172.21.15.137:0/1351762668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80681038e0 0x7f8068199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 >> 172.21.15.137:0/1351762668 conn(0x7f80680fee90 msgr2=0x7f8068107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 shutdown_connections 2024-03-26T13:59:29.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.662+0000 7f8070600640 1 -- 172.21.15.137:0/1351762668 wait complete. 2024-03-26T13:59:29.694 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:29.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 -- 172.21.15.137:0/3151344970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8076060 msgr2=0x7f90d8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 --2- 172.21.15.137:0/3151344970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8076060 0x7f90d8076480 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009900 tx=0x7f90c802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 -- 172.21.15.137:0/3151344970 shutdown_connections 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 --2- 172.21.15.137:0/3151344970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8077c40 0x7f90d80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 --2- 172.21.15.137:0/3151344970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d80772a0 0x7f90d8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 --2- 172.21.15.137:0/3151344970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8076060 0x7f90d8076480 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 -- 172.21.15.137:0/3151344970 >> 172.21.15.137:0/3151344970 conn(0x7f90d80fee50 msgr2=0x7f90d8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 -- 172.21.15.137:0/3151344970 shutdown_connections 2024-03-26T13:59:29.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.904+0000 7f90dee00640 1 -- 172.21.15.137:0/3151344970 wait complete. 2024-03-26T13:59:29.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.905+0000 7f90dee00640 1 Processor -- start 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.905+0000 7f90dee00640 1 -- start start 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8076060 0x7f90d819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d80772a0 0x7f90d819e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dee00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 0x7f90d81a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dee00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8113920 con 0x7f90d8077c40 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d81137a0 con 0x7f90d8076060 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d8113aa0 con 0x7f90d80772a0 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 0x7f90d81a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 0x7f90d81a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34520/0 (socket says 172.21.15.137:34520) 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90dce00640 1 -- 172.21.15.137:0/3882922598 learned_addr learned my addr 172.21.15.137:0/3882922598 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90d7400640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d80772a0 0x7f90d819e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.906+0000 7f90d7e00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8076060 0x7f90d819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dce00640 1 -- 172.21.15.137:0/3882922598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d80772a0 msgr2=0x7f90d819e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dce00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d80772a0 0x7f90d819e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dce00640 1 -- 172.21.15.137:0/3882922598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8076060 msgr2=0x7f90d819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dce00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8076060 0x7f90d819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dce00640 1 -- 172.21.15.137:0/3882922598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d81a5cb0 con 0x7f90d8077c40 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dce00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 0x7f90d81a5640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90d000b970 tx=0x7f90d000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0019070 con 0x7f90d8077c40 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f90d0014440 con 0x7f90d8077c40 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d81a5fa0 con 0x7f90d8077c40 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00075d0 con 0x7f90d8077c40 2024-03-26T13:59:29.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.907+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d81a6490 con 0x7f90d8077c40 2024-03-26T13:59:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.909+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d81104f0 con 0x7f90d8077c40 2024-03-26T13:59:29.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.910+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f90d001c070 con 0x7f90d8077c40 2024-03-26T13:59:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.913+0000 7f90d4c00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90bc0a2800 0x7f90bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.913+0000 7f90d7e00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90bc0a2800 0x7f90bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.913+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f90d00ce2e0 con 0x7f90d8077c40 2024-03-26T13:59:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.913+0000 7f90d7e00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90bc0a2800 0x7f90bc0a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f90c802c840 tx=0x7f90c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:29.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:29.917+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f90d009b140 con 0x7f90d8077c40 2024-03-26T13:59:30.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:29 smithi137 ceph-mon[31539]: pgmap v368: 129 pgs: 13 creating+peering, 8 unknown, 108 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-03-26T13:59:30.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:29 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3795329658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:29 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1351762668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.156+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90d802d070 con 0x7f90d8077c40 2024-03-26T13:59:30.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.157+0000 7f90d4c00640 1 -- 172.21.15.137:0/3882922598 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+11336 (secure 0 0 0) 0x7f90d00aee20 con 0x7f90d8077c40 2024-03-26T13:59:30.153 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:30.153 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24481,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34394,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34424,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34451,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:27.738005+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34496,"qdb_cluster":[34496]},"id":1}]} 2024-03-26T13:59:30.154 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 16 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90bc0a2800 msgr2=0x7f90bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90bc0a2800 0x7f90bc0a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f90c802c840 tx=0x7f90c8000bc0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 msgr2=0x7f90d81a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 0x7f90d81a5640 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90d000b970 tx=0x7f90d000be30 comp rx=0 tx=0).stop 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 shutdown_connections 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f90bc0a2800 0x7f90bc0a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f90d8077c40 0x7f90d81a5640 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f90d80772a0 0x7f90d819e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 --2- 172.21.15.137:0/3882922598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d8076060 0x7f90d819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.159+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 >> 172.21.15.137:0/3882922598 conn(0x7f90d80fee50 msgr2=0x7f90d8100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.160+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 shutdown_connections 2024-03-26T13:59:30.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.160+0000 7f90dee00640 1 -- 172.21.15.137:0/3882922598 wait complete. 2024-03-26T13:59:30.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:29 smithi179 ceph-mon[35472]: pgmap v368: 129 pgs: 13 creating+peering, 8 unknown, 108 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-03-26T13:59:30.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:29 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3795329658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:29 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1351762668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.174 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:30.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:29 smithi159 ceph-mon[35501]: pgmap v368: 129 pgs: 13 creating+peering, 8 unknown, 108 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 9 op/s 2024-03-26T13:59:30.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:29 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3795329658' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:29 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1351762668' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 -- 172.21.15.137:0/576586748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c701038e0 msgr2=0x7f6c70103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 --2- 172.21.15.137:0/576586748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c701038e0 0x7f6c70103d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009900 tx=0x7f6c6002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 -- 172.21.15.137:0/576586748 shutdown_connections 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 --2- 172.21.15.137:0/576586748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 0x7f6c70104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 --2- 172.21.15.137:0/576586748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c70105490 0x7f6c7010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 --2- 172.21.15.137:0/576586748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c701038e0 0x7f6c70103d00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 -- 172.21.15.137:0/576586748 >> 172.21.15.137:0/576586748 conn(0x7f6c700fee50 msgr2=0x7f6c70101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 -- 172.21.15.137:0/576586748 shutdown_connections 2024-03-26T13:59:30.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.392+0000 7f6c77800640 1 -- 172.21.15.137:0/576586748 wait complete. 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.393+0000 7f6c77800640 1 Processor -- start 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 -- start start 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c701038e0 0x7f6c70199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 0x7f6c7019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 0x7f6c701a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7010f460 con 0x7f6c70105490 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7010f2e0 con 0x7f6c70104ad0 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c77800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7010f5e0 con 0x7f6c701038e0 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c75800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 0x7f6c701a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c75800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 0x7f6c701a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34546/0 (socket says 172.21.15.137:34546) 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.394+0000 7f6c75800640 1 -- 172.21.15.137:0/3737311124 learned_addr learned my addr 172.21.15.137:0/3737311124 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c67e00640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 0x7f6c7019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c74e00640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c701038e0 0x7f6c70199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c75800640 1 -- 172.21.15.137:0/3737311124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c701038e0 msgr2=0x7f6c70199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c75800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c701038e0 0x7f6c70199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c75800640 1 -- 172.21.15.137:0/3737311124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 msgr2=0x7f6c7019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c75800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 0x7f6c7019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c75800640 1 -- 172.21.15.137:0/3737311124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c701a1720 con 0x7f6c70105490 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c67e00640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 0x7f6c7019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c75800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 0x7f6c701a10b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800eaa0 tx=0x7f6c6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800cdb0 con 0x7f6c70105490 2024-03-26T13:59:30.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c68005020 con 0x7f6c70105490 2024-03-26T13:59:30.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c701a1a10 con 0x7f6c70105490 2024-03-26T13:59:30.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.395+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680116d0 con 0x7f6c70105490 2024-03-26T13:59:30.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.396+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c701a1fb0 con 0x7f6c70105490 2024-03-26T13:59:30.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.397+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7010ac80 con 0x7f6c70105490 2024-03-26T13:59:30.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.400+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6c680051d0 con 0x7f6c70105490 2024-03-26T13:59:30.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.401+0000 7f6c6e000640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c540a2880 0x7f6c540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.402+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f6c680cf290 con 0x7f6c70105490 2024-03-26T13:59:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.402+0000 7f6c74e00640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c540a2880 0x7f6c540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.403+0000 7f6c74e00640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c540a2880 0x7f6c540a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c600096c0 tx=0x7f6c60005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.404+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6c68016030 con 0x7f6c70105490 2024-03-26T13:59:30.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.605+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c700622a0 con 0x7f6c70105490 2024-03-26T13:59:30.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.608+0000 7f6c6e000640 1 -- 172.21.15.137:0/3737311124 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7f6c6809c110 con 0x7f6c70105490 2024-03-26T13:59:30.605 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:30.605 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:30.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.610+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c540a2880 msgr2=0x7f6c540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.610+0000 7f6c77800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c540a2880 0x7f6c540a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c600096c0 tx=0x7f6c60005fd0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.610+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 msgr2=0x7f6c701a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.610+0000 7f6c77800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 0x7f6c701a10b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800eaa0 tx=0x7f6c6800ef60 comp rx=0 tx=0).stop 2024-03-26T13:59:30.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 shutdown_connections 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c540a2880 0x7f6c540a4d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70104ad0 0x7f6c7019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c70105490 0x7f6c701a10b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 --2- 172.21.15.137:0/3737311124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c701038e0 0x7f6c70199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 >> 172.21.15.137:0/3737311124 conn(0x7f6c700fee50 msgr2=0x7f6c70107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 shutdown_connections 2024-03-26T13:59:30.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.611+0000 7f6c77800640 1 -- 172.21.15.137:0/3737311124 wait complete. 2024-03-26T13:59:30.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-26T13:59:22.728231+0000', 'modified': '2024-03-26T13:59:27.738005+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34496': {'gid': 34496, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/2964052982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2964052982}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2964052982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34496, 'qdb_cluster': [34496]} 2024-03-26T13:59:30.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-26T13:59:30.628 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T13:59:30.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 -- 172.21.15.137:0/3223562203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 msgr2=0x7f5bb810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3223562203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb810bd00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009ca0 tx=0x7f5bb002c100 comp rx=0 tx=0).stop 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 -- 172.21.15.137:0/3223562203 shutdown_connections 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3223562203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb8104ab0 0x7f5bb8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3223562203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb810bd00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3223562203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 0x7f5bb8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 -- 172.21.15.137:0/3223562203 >> 172.21.15.137:0/3223562203 conn(0x7f5bb80fee50 msgr2=0x7f5bb8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 -- 172.21.15.137:0/3223562203 shutdown_connections 2024-03-26T13:59:30.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.844+0000 7f5bbe400640 1 -- 172.21.15.137:0/3223562203 wait complete. 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 Processor -- start 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 -- start start 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 0x7f5bb8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb8104ab0 0x7f5bb8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb81a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb810f3f0 con 0x7f5bb8105470 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb810f270 con 0x7f5bb8104ab0 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.845+0000 7f5bbe400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb810f570 con 0x7f5bb81038c0 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb81a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb81a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34554/0 (socket says 172.21.15.137:34554) 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 -- 172.21.15.137:0/3129942145 learned_addr learned my addr 172.21.15.137:0/3129942145 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:30.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bafe00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb8104ab0 0x7f5bb8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 0x7f5bb8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 -- 172.21.15.137:0/3129942145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 msgr2=0x7f5bb8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 0x7f5bb8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 -- 172.21.15.137:0/3129942145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb8104ab0 msgr2=0x7f5bb8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb8104ab0 0x7f5bb8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 -- 172.21.15.137:0/3129942145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb81a16b0 con 0x7f5bb8105470 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 0x7f5bb8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.846+0000 7f5bb7e00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb81a1040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0032520 tx=0x7f5bb0034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.847+0000 7f5bb4c00640 1 -- 172.21.15.137:0/3129942145 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0039040 con 0x7f5bb8105470 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.847+0000 7f5bb4c00640 1 -- 172.21.15.137:0/3129942145 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5bb0044070 con 0x7f5bb8105470 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.847+0000 7f5bb4c00640 1 -- 172.21.15.137:0/3129942145 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004430 con 0x7f5bb8105470 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.847+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb81a1940 con 0x7f5bb8105470 2024-03-26T13:59:30.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.847+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb81a1de0 con 0x7f5bb8105470 2024-03-26T13:59:30.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.850+0000 7f5bb4c00640 1 -- 172.21.15.137:0/3129942145 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5bb0003bc0 con 0x7f5bb8105470 2024-03-26T13:59:30.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.850+0000 7f5bc121a740 1 Processor -- start 2024-03-26T13:59:30.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.850+0000 7f5bc121a740 1 -- start start 2024-03-26T13:59:30.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd37fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd3d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd3fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bc121a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cc9cc90 con 0x55fb7cd37ba0 2024-03-26T13:59:30.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bc121a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cc45980 con 0x55fb7cd38590 2024-03-26T13:59:30.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bc121a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cc98d30 con 0x55fb7cd3d700 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd37fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd37fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34556/0 (socket says 172.21.15.137:34556) 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 -- 172.21.15.137:0/3865621862 learned_addr learned my addr 172.21.15.137:0/3865621862 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bb7400640 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd3d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd3fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bb4c00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b9c0a2880 0x7f5b9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 -- 172.21.15.137:0/3865621862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 msgr2=0x55fb7cd3fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd3fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 -- 172.21.15.137:0/3865621862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 msgr2=0x55fb7cd3d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd3d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.851+0000 7f5bafe00640 1 -- 172.21.15.137:0/3865621862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb7cc5d510 con 0x55fb7cd37ba0 2024-03-26T13:59:30.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.852+0000 7f5bb4c00640 1 -- 172.21.15.137:0/3129942145 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5bb00f13d0 con 0x7f5bb8105470 2024-03-26T13:59:30.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.852+0000 7f5bb7e00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b9c0a2880 0x7f5b9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.852+0000 7f5bafe00640 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd37fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0013520 tx=0x7f5ba000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0b47a9feef791b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.852+0000 7f5bacc00640 1 -- 172.21.15.137:0/3865621862 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0019030 con 0x55fb7cd37ba0 2024-03-26T13:59:30.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.852+0000 7f5bacc00640 1 -- 172.21.15.137:0/3865621862 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ba0011970 con 0x55fb7cd37ba0 2024-03-26T13:59:30.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.852+0000 7f5bacc00640 1 -- 172.21.15.137:0/3865621862 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba001d820 con 0x55fb7cd37ba0 2024-03-26T13:59:30.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 -- 172.21.15.137:0/3865621862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 msgr2=0x55fb7cd37fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd37fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0013520 tx=0x7f5ba000b6b0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bb7e00640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b9c0a2880 0x7f5b9c0a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00bdac0 tx=0x7f5bb00095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 -- 172.21.15.137:0/3865621862 shutdown_connections 2024-03-26T13:59:30.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd3fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 --2- 172.21.15.137:0/3865621862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd37fc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 -- 172.21.15.137:0/3865621862 >> 172.21.15.137:0/3865621862 conn(0x55fb7cd33110 msgr2=0x55fb7cd33510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 -- 172.21.15.137:0/3865621862 shutdown_connections 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.853+0000 7f5bc121a740 1 -- 172.21.15.137:0/3865621862 wait complete. 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 -- start start 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 Processor -- start 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd42600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd49640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd406b0 con 0x55fb7cd37ba0 2024-03-26T13:59:30.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cc58dd0 con 0x55fb7cd38590 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd40830 con 0x55fb7cd3d700 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.854+0000 7f5bc121a740 10 client.0 _ll_register_callbacks cb 0x55fb7c9b0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34568/0 (socket says 172.21.15.137:34568) 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 -- 172.21.15.137:0/400708506 learned_addr learned my addr 172.21.15.137:0/400708506 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bb7400640 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd42600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 -- 172.21.15.137:0/400708506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 msgr2=0x55fb7cd49640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd49640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 -- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 msgr2=0x55fb7cd42600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd42600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 -- 172.21.15.137:0/400708506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb7c9b12d0 con 0x55fb7cd37ba0 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5bafe00640 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd420c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000b390 tx=0x7f5ba000b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba7738c0e895eab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.855+0000 7f5ba5600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T13:59:30.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5ba5600640 1 -- 172.21.15.137:0/400708506 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0019040 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5ba5600640 1 -- 172.21.15.137:0/400708506 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5ba0011d20 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ba0011970 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5ba5600640 1 -- 172.21.15.137:0/400708506 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ba0030050 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5bc121a740 10 client.14781 fetch_fsmap learned FSMap version 16 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x55fb7ca7b6b0 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5ba5600640 1 -- 172.21.15.137:0/400708506 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ba0011970 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.857+0000 7f5ba5600640 1 -- 172.21.15.137:0/400708506 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 16) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5ba00046c0 con 0x55fb7cd37ba0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bc121a740 10 client.14781 fetch_fsmap finished waiting for FSMap version 16 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bc121a740 10 client.14781 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bc121a740 20 client.14781 populate_metadata read hostname 'smithi137' 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bc121a740 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cd4d8e0 0x55fb7cd529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bc121a740 4 client.14781 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb7cc3e520 con 0x55fb7cd4d8e0 2024-03-26T13:59:30.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.858+0000 7f5bb7400640 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cd4d8e0 0x55fb7cd529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.860+0000 7f5bb7400640 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cd4d8e0 0x55fb7cd529a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42b6c478cfb18e server_cookie=94f49ad4b9f224e8 in_seq=0 out_seq=0 2024-03-26T13:59:30.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.860+0000 7f5ba5600640 10 client.14781 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T13:59:30.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5ba5600640 1 -- 172.21.15.137:0/400708506 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fb7cc3e520 con 0x55fb7cd4d8e0 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5ba5600640 10 client.14781 handle_command_reply: tid=0 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 client.14781 shutdown 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 msgr2=0x55fb7cd420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd420c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000b390 tx=0x7f5ba000b3c0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 shutdown_connections 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cd4d8e0 0x55fb7cd529a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd42600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd3d700 0x55fb7cd49640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 --2- 172.21.15.137:0/400708506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd420c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 >> 172.21.15.137:0/400708506 conn(0x55fb7cd33110 msgr2=0x55fb7cd32310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5ba5600640 0 client.14781 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 shutdown_connections 2024-03-26T13:59:30.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.863+0000 7f5bc121a740 1 -- 172.21.15.137:0/400708506 wait complete. 2024-03-26T13:59:30.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.864+0000 7f5bc121a740 20 client.14781 trim_cache size 0 max 16384 2024-03-26T13:59:30.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 Processor -- start 2024-03-26T13:59:30.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 -- start start 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd2de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd38590 0x55fb7cd495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd4dd40 con 0x55fb7cd3d700 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd4e3a0 con 0x55fb7cd37ba0 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.882+0000 7f5bc121a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd4ea00 con 0x55fb7cd38590 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34580/0 (socket says 172.21.15.137:34580) 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bb7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd38590 0x55fb7cd495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 -- 172.21.15.137:0/582418535 learned_addr learned my addr 172.21.15.137:0/582418535 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:30.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd2de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 -- 172.21.15.137:0/582418535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd38590 msgr2=0x55fb7cd495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd38590 0x55fb7cd495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 -- 172.21.15.137:0/582418535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd37ba0 msgr2=0x55fb7cd2de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd2de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 -- 172.21.15.137:0/582418535 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb7cd4a730 con 0x55fb7cd3d700 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.883+0000 7f5bafe00640 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd334f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c010 tx=0x7f5ba000b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d140617e6e835132 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5ba7e00640 1 -- 172.21.15.137:0/582418535 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0019040 con 0x55fb7cd3d700 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5ba7e00640 1 -- 172.21.15.137:0/582418535 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ba0024070 con 0x55fb7cd3d700 2024-03-26T13:59:30.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5ba7e00640 1 -- 172.21.15.137:0/582418535 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00053c0 con 0x55fb7cd3d700 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5bc121a740 1 -- 172.21.15.137:0/582418535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 msgr2=0x55fb7cd334f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5bc121a740 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd334f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000c010 tx=0x7f5ba000b9e0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5bc121a740 1 -- 172.21.15.137:0/582418535 shutdown_connections 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5bc121a740 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd334f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.884+0000 7f5bc121a740 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd38590 0x55fb7cd495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.885+0000 7f5bc121a740 1 --2- 172.21.15.137:0/582418535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd2de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.885+0000 7f5bc121a740 1 -- 172.21.15.137:0/582418535 >> 172.21.15.137:0/582418535 conn(0x55fb7cd33110 msgr2=0x55fb7cd40c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.885+0000 7f5bc121a740 1 -- 172.21.15.137:0/582418535 shutdown_connections 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.885+0000 7f5bc121a740 1 -- 172.21.15.137:0/582418535 wait complete. 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.885+0000 7f5bc121a740 1 -- start start 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.885+0000 7f5bc121a740 1 Processor -- start 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd48dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd49310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd45870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd4f6c0 con 0x55fb7cd3d700 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd4f060 con 0x55fb7cd38590 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb7cd4fd20 con 0x55fb7cd37ba0 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bc121a740 10 client.0 _ll_register_callbacks cb 0x7f5ba0013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T13:59:30.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd45870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd45870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34594/0 (socket says 172.21.15.137:34594) 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 -- 172.21.15.137:0/4274751610 learned_addr learned my addr 172.21.15.137:0/4274751610 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd48dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bb7400640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd49310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 -- 172.21.15.137:0/4274751610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd37ba0 msgr2=0x55fb7cd48dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd48dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 -- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 msgr2=0x55fb7cd49310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd49310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bafe00640 1 -- 172.21.15.137:0/4274751610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb7cd45ee0 con 0x55fb7cd3d700 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.886+0000 7f5bb7400640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd49310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5bafe00640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd45870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000bd30 tx=0x7f5ba0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663c73bafcb3d62b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5ba5600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T13:59:30.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5ba5600640 1 -- 172.21.15.137:0/4274751610 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0019070 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5ba5600640 1 -- 172.21.15.137:0/4274751610 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5ba0011b00 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ba0024070 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5ba5600640 1 -- 172.21.15.137:0/4274751610 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ba0004890 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5bc121a740 10 client.14787 fetch_fsmap learned FSMap version 16 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x55fb7cdaa680 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5ba5600640 1 -- 172.21.15.137:0/4274751610 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ba0024070 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5ba5600640 1 -- 172.21.15.137:0/4274751610 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 16) v1 ==== 7922+0+0 (secure 0 0 0) 0x7f5ba00297a0 con 0x55fb7cd3d700 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.887+0000 7f5bc121a740 10 client.14787 fetch_fsmap finished waiting for FSMap version 16 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.888+0000 7f5bc121a740 10 client.14787 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.888+0000 7f5bc121a740 20 client.14787 populate_metadata read hostname 'smithi137' 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.888+0000 7f5bc121a740 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cdae910 0x55fb7cdaa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.888+0000 7f5bc121a740 4 client.14787 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.888+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb7ccaa1d0 con 0x55fb7cdae910 2024-03-26T13:59:30.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.888+0000 7f5bb7400640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cdae910 0x55fb7cdaa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:30.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.889+0000 7f5bb7400640 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cdae910 0x55fb7cdaa8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d7d1a100aac341 server_cookie=ecb16a7b34c97d7a in_seq=0 out_seq=0 2024-03-26T13:59:30.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.889+0000 7f5ba5600640 10 client.14787 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5ba5600640 1 -- 172.21.15.137:0/4274751610 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fb7ccaa1d0 con 0x55fb7cdae910 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5ba5600640 10 client.14787 handle_command_reply: tid=0 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 client.14787 shutdown 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 msgr2=0x55fb7cd45870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd45870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000bd30 tx=0x7f5ba0000bc0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 shutdown_connections 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fb7cdae910 0x55fb7cdaa8a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb7cd38590 0x55fb7cd49310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5ba5600640 0 client.14787 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T13:59:30.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb7cd3d700 0x55fb7cd45870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 --2- 172.21.15.137:0/4274751610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb7cd37ba0 0x55fb7cd48dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 >> 172.21.15.137:0/4274751610 conn(0x55fb7cd33110 msgr2=0x55fb7cd2de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 shutdown_connections 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 1 -- 172.21.15.137:0/4274751610 wait complete. 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.890+0000 7f5bc121a740 20 client.14787 trim_cache size 0 max 16384 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 0, 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34496, 2024-03-26T13:59:30.891 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 7.5950573669999999, 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 16, 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 90, 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 13.90811708 2024-03-26T13:59:30.892 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b9c0a2880 msgr2=0x7f5b9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b9c0a2880 0x7f5b9c0a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00bdac0 tx=0x7f5bb00095c0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 msgr2=0x7f5bb81a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb81a1040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0032520 tx=0x7f5bb0034880 comp rx=0 tx=0).stop 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 shutdown_connections 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b9c0a2880 0x7f5b9c0a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bb8104ab0 0x7f5bb8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5bb8105470 0x7f5bb81a1040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 --2- 172.21.15.137:0/3129942145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5bb81038c0 0x7f5bb8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 >> 172.21.15.137:0/3129942145 conn(0x7f5bb80fee50 msgr2=0x7f5bb8107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 shutdown_connections 2024-03-26T13:59:30.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:30.891+0000 7f5bbe400640 1 -- 172.21.15.137:0/3129942145 wait complete. 2024-03-26T13:59:30.902 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:30 smithi137 ceph-mon[31539]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T13:59:30.902 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:30 smithi137 ceph-mon[31539]: Cluster is now healthy 2024-03-26T13:59:30.902 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3882922598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.902 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3737311124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:30.905 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T13:59:30.905 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:30.905 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T13:59:31.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:30 smithi179 ceph-mon[35472]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T13:59:31.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:30 smithi179 ceph-mon[35472]: Cluster is now healthy 2024-03-26T13:59:31.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3882922598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:31.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3737311124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:31.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:30 smithi159 ceph-mon[35501]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-26T13:59:31.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:30 smithi159 ceph-mon[35501]: Cluster is now healthy 2024-03-26T13:59:31.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3882922598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:31.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3737311124' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:31.332 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T13:59:32.031 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:31 smithi137 ceph-mon[31539]: pgmap v369: 129 pgs: 13 creating+peering, 4 unknown, 112 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-03-26T13:59:32.170 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:31 smithi179 ceph-mon[35472]: pgmap v369: 129 pgs: 13 creating+peering, 4 unknown, 112 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-03-26T13:59:32.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:31 smithi159 ceph-mon[35501]: pgmap v369: 129 pgs: 13 creating+peering, 4 unknown, 112 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 6 op/s 2024-03-26T13:59:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.508+0000 7fa2c9000700 1 -- 172.21.15.137:0/2285014131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4102440 msgr2=0x7fa2c4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.508+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2285014131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4102440 0x7fa2c4102820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa2b40099c0 tx=0x7fa2b4009cd0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.508+0000 7fa2c9000700 1 -- 172.21.15.137:0/2285014131 shutdown_connections 2024-03-26T13:59:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.508+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2285014131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4107510 0x7fa2c410f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.508+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2285014131 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4102440 0x7fa2c4102820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.508+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2285014131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2c4102df0 0x7fa2c4106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.509+0000 7fa2c9000700 1 -- 172.21.15.137:0/2285014131 >> 172.21.15.137:0/2285014131 conn(0x7fa2c40792f0 msgr2=0x7fa2c4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.509+0000 7fa2c9000700 1 -- 172.21.15.137:0/2285014131 shutdown_connections 2024-03-26T13:59:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.509+0000 7fa2c9000700 1 -- 172.21.15.137:0/2285014131 wait complete. 2024-03-26T13:59:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 Processor -- start 2024-03-26T13:59:33.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 -- start start 2024-03-26T13:59:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2c4102440 0x7fa2c41a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4102df0 0x7fa2c41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 0x7fa2c419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41123d0 con 0x7fa2c4107510 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4112250 con 0x7fa2c4102440 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.510+0000 7fa2c9000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c4112550 con 0x7fa2c4102df0 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 0x7fa2c419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 0x7fa2c419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34608/0 (socket says 172.21.15.137:34608) 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 -- 172.21.15.137:0/2353608474 learned_addr learned my addr 172.21.15.137:0/2353608474 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c1e00700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2c4102440 0x7fa2c41a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c1400700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4102df0 0x7fa2c41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 -- 172.21.15.137:0/2353608474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4102df0 msgr2=0x7fa2c41a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4102df0 0x7fa2c41a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 -- 172.21.15.137:0/2353608474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2c4102440 msgr2=0x7fa2c41a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2c4102440 0x7fa2c41a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 -- 172.21.15.137:0/2353608474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c419b0b0 con 0x7fa2c4107510 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c1400700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4102df0 0x7fa2c41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.511+0000 7fa2c2800700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 0x7fa2c419a950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007f10 tx=0x7fa2bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.512+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc00dbd0 con 0x7fa2c4107510 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.512+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2bc00dd70 con 0x7fa2c4107510 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.512+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c419b3a0 con 0x7fa2c4107510 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.512+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc005990 con 0x7fa2c4107510 2024-03-26T13:59:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.512+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c41a7660 con 0x7fa2c4107510 2024-03-26T13:59:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.513+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c4068e50 con 0x7fa2c4107510 2024-03-26T13:59:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.515+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa2bc01a020 con 0x7fa2c4107510 2024-03-26T13:59:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.516+0000 7fa2baa00700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2a80a2910 0x7fa2a80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.516+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fa2bc0ce370 con 0x7fa2c4107510 2024-03-26T13:59:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.517+0000 7fa2c1e00700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2a80a2910 0x7fa2a80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.517+0000 7fa2c1e00700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2a80a2910 0x7fa2a80a4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4000c00 tx=0x7fa2b4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.521+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2bc010070 con 0x7fa2c4107510 2024-03-26T13:59:33.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.793+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa2c41a7890 con 0x7fa2c4107510 2024-03-26T13:59:33.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.793+0000 7fa2baa00700 1 -- 172.21.15.137:0/2353608474 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa2bc09b170 con 0x7fa2c4107510 2024-03-26T13:59:33.790 INFO:teuthology.orchestra.run.smithi137.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-26T13:59:33.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.796+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2a80a2910 msgr2=0x7fa2a80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:33.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.796+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2a80a2910 0x7fa2a80a4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4000c00 tx=0x7fa2b4005fd0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.796+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 msgr2=0x7fa2c419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.796+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 0x7fa2c419a950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc007f10 tx=0x7fa2bc00d3b0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 shutdown_connections 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2a80a2910 0x7fa2a80a4dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2c4107510 0x7fa2c419a950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2c4102440 0x7fa2c41a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 --2- 172.21.15.137:0/2353608474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2c4102df0 0x7fa2c41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 >> 172.21.15.137:0/2353608474 conn(0x7fa2c40792f0 msgr2=0x7fa2c41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 shutdown_connections 2024-03-26T13:59:33.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:33.797+0000 7fa2c9000700 1 -- 172.21.15.137:0/2353608474 wait complete. 2024-03-26T13:59:34.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:34 smithi179 ceph-mon[35472]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 452 B/s wr, 4 op/s 2024-03-26T13:59:34.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2353608474' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T13:59:34.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:34 smithi159 ceph-mon[35501]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 452 B/s wr, 4 op/s 2024-03-26T13:59:34.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2353608474' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T13:59:34.465 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:34 smithi137 ceph-mon[31539]: pgmap v370: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 452 B/s wr, 4 op/s 2024-03-26T13:59:34.465 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2353608474' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T13:59:34.674 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-26T13:59:34.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T13:59:34.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi137.front.sepia.ceph.com 2024-03-26T13:59:34.674 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-26T13:59:34.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:34.674 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-26T13:59:34.675 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T13:59:34.675 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T13:59:34.675 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-26T13:59:34.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:34.675 DEBUG:teuthology.orchestra.run.smithi137:> ip addr 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-03-26T13:59:34.734 INFO:teuthology.orchestra.run.smithi137.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7e brd ff:ff:ff:ff:ff:ff 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f0 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7f brd ff:ff:ff:ff:ff:ff 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f1 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:36 brd ff:ff:ff:ff:ff:ff 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:37 brd ff:ff:ff:ff:ff:ff 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: inet 172.21.15.137/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-26T13:59:34.735 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft 41891sec preferred_lft 41891sec 2024-03-26T13:59:34.736 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-03-26T13:59:34.736 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-03-26T13:59:34.737 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T13:59:34.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add name ceph-brx type bridge 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr flush dev ceph-brx 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set ceph-brx up 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T13:59:34.738 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T13:59:35.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:35.168 DEBUG:teuthology.orchestra.run.smithi137:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T13:59:35.245 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-03-26T13:59:35.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:35.249 DEBUG:teuthology.orchestra.run.smithi137:> route 2024-03-26T13:59:35.306 INFO:teuthology.orchestra.run.smithi137.stdout:Kernel IP routing table 2024-03-26T13:59:35.306 INFO:teuthology.orchestra.run.smithi137.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T13:59:35.306 INFO:teuthology.orchestra.run.smithi137.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-26T13:59:35.306 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-26T13:59:35.307 INFO:teuthology.orchestra.run.smithi137.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T13:59:35.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:35.307 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T13:59:35.307 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T13:59:35.308 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-26T13:59:35.308 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-26T13:59:35.308 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-26T13:59:35.308 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T13:59:36.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:36.069 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-03-26T13:59:36.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:36.124 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list-id 2024-03-26T13:59:36.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:36.176 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T13:59:36.176 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T13:59:36.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:36.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-26T13:59:36.176 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T13:59:36.314 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:36 smithi137 ceph-mon[31539]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-26T13:59:36.318 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-26T13:59:36.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:36.318 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T13:59:36.318 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T13:59:36.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-26T13:59:36.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T13:59:36.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-26T13:59:36.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-26T13:59:36.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-26T13:59:36.319 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T13:59:36.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:36 smithi179 ceph-mon[35472]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-26T13:59:36.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:36 smithi159 ceph-mon[35501]: pgmap v371: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-03-26T13:59:36.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:36.619 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T13:59:36.619 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T13:59:36.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set brx.0 up 2024-03-26T13:59:36.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T13:59:36.619 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T13:59:36.784 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-26T13:59:36.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T13:59:36.784 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:36.830 INFO:teuthology.orchestra.run.smithi137.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-26T13:59:36.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T13:59:36.830 DEBUG:teuthology.orchestra.run.smithi137:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:36.925 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-03-26T13:59:37.021 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T13:59:37.069 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/6d5176c50bb0924b5697fe484de22ac7910bd0182cb6638d95cb41ddc003720a/merged 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/ef7699130a1edbef8eca4ceb5dee06744f5f7dce4795f904cf09d267f7a491af/merged 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/0 2024-03-26T13:59:37.070 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/aa69ad5c31bdc2ed88bdc366aaadc85a8ea7d3a0b1728d83208028cec057bb5f/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/f5ef712a5805662c301f59e342c4dc1c0b4d4d0dbe51bc5c1a22a3290c7704d8/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/5f4b573d55b2740cad2b22819d0378791b0e9bab5aa9b4a7cf8915ce4a2892ae/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/b68dcbdc09abd5f92e66e58475d97fd1488aa21a66a54942044551525518fa32/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/e553abe0ad1f21fb92e8b35c38f139a5239ff1495e80ff77e44f7036b09abc4d/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/525c7d55962104ce64112cfaef76e05acc035d714ebccbdd99e5d57e87cbf5b8/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/dac0fd409288119dad4627664a2cf3f341d8f8920adf422a09110be7380bd00f/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/194c9c7a044b837c9d3bb06ae7d20494d1994a0ae34421cefa18bccdd79adf3b/merged 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:37.071 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:37.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:37.073 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-03-26T13:59:37.118 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-26T13:59:37.118 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-26T13:59:37.163 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-03-26T13:59:37.236 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-03-26T13:59:37.278 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay 2024-03-26T13:59:37.279 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/6d5176c50bb0924b5697fe484de22ac7910bd0182cb6638d95cb41ddc003720a/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/ef7699130a1edbef8eca4ceb5dee06744f5f7dce4795f904cf09d267f7a491af/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/0 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/aa69ad5c31bdc2ed88bdc366aaadc85a8ea7d3a0b1728d83208028cec057bb5f/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/f5ef712a5805662c301f59e342c4dc1c0b4d4d0dbe51bc5c1a22a3290c7704d8/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/5f4b573d55b2740cad2b22819d0378791b0e9bab5aa9b4a7cf8915ce4a2892ae/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/b68dcbdc09abd5f92e66e58475d97fd1488aa21a66a54942044551525518fa32/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/e553abe0ad1f21fb92e8b35c38f139a5239ff1495e80ff77e44f7036b09abc4d/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/525c7d55962104ce64112cfaef76e05acc035d714ebccbdd99e5d57e87cbf5b8/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/dac0fd409288119dad4627664a2cf3f341d8f8920adf422a09110be7380bd00f/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/194c9c7a044b837c9d3bb06ae7d20494d1994a0ae34421cefa18bccdd79adf3b/merged 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:37.280 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:37.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:37.281 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-03-26T13:59:37.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi137.stderr:2024-03-26T13:59:37.373+0000 7f1a98840480 -1 init, newargv = 0x5613b8682ee0 newargc=15 2024-03-26T13:59:37.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi137.stderr:ceph-fuse[75619]: starting ceph client 2024-03-26T13:59:37.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi137.stderr:ceph-fuse[75619]: starting fuse 2024-03-26T13:59:38.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:38 smithi179 ceph-mon[35472]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:38.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:38 smithi159 ceph-mon[35501]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:38.468 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-03-26T13:59:38.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:38 smithi137 ceph-mon[31539]: pgmap v372: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:38.560 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T13:59:38.605 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-03-26T13:59:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T13:59:38.607 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/6d5176c50bb0924b5697fe484de22ac7910bd0182cb6638d95cb41ddc003720a/merged 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/ef7699130a1edbef8eca4ceb5dee06744f5f7dce4795f904cf09d267f7a491af/merged 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/0 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/aa69ad5c31bdc2ed88bdc366aaadc85a8ea7d3a0b1728d83208028cec057bb5f/merged 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/f5ef712a5805662c301f59e342c4dc1c0b4d4d0dbe51bc5c1a22a3290c7704d8/merged 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/5f4b573d55b2740cad2b22819d0378791b0e9bab5aa9b4a7cf8915ce4a2892ae/merged 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/b68dcbdc09abd5f92e66e58475d97fd1488aa21a66a54942044551525518fa32/merged 2024-03-26T13:59:38.608 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/e553abe0ad1f21fb92e8b35c38f139a5239ff1495e80ff77e44f7036b09abc4d/merged 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/525c7d55962104ce64112cfaef76e05acc035d714ebccbdd99e5d57e87cbf5b8/merged 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/dac0fd409288119dad4627664a2cf3f341d8f8920adf422a09110be7380bd00f/merged 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/194c9c7a044b837c9d3bb06ae7d20494d1994a0ae34421cefa18bccdd79adf3b/merged 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T13:59:38.609 INFO:teuthology.orchestra.run.smithi137.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:38.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:38.610 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-03-26T13:59:38.702 INFO:teuthology.orchestra.run.smithi137.stdout:118 2024-03-26T13:59:38.703 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-26T13:59:38.703 DEBUG:teuthology.orchestra.run.smithi137:> sudo stdin-killer -- python3 -c ' 2024-03-26T13:59:38.703 DEBUG:teuthology.orchestra.run.smithi137:> import glob 2024-03-26T13:59:38.703 DEBUG:teuthology.orchestra.run.smithi137:> import re 2024-03-26T13:59:38.703 DEBUG:teuthology.orchestra.run.smithi137:> import os 2024-03-26T13:59:38.703 DEBUG:teuthology.orchestra.run.smithi137:> import subprocess 2024-03-26T13:59:38.703 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> def _find_admin_socket(client_name): 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> files = glob.glob(asok_path) 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> # Given a non-glob path, it better be there 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> if "*" not in asok_path: 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> assert(len(files) == 1) 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> return files[0] 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> for f in files: 2024-03-26T13:59:38.704 DEBUG:teuthology.orchestra.run.smithi137:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> contents = proc_f.read() 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> if mountpoint in contents: 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> return f 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> print(_find_admin_socket("client.admin")) 2024-03-26T13:59:38.705 DEBUG:teuthology.orchestra.run.smithi137:> ' 2024-03-26T13:59:38.873 INFO:teuthology.orchestra.run.smithi137.stdout:/var/run/ceph/ceph-client.admin.75619.asok 2024-03-26T13:59:38.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T13:59:38.891 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75619.asok 2024-03-26T13:59:38.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:38.892 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75619.asok status 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "metadata": { 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "entity_id": "admin", 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": "smithi137", 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "pid": "75619", 2024-03-26T13:59:39.124 INFO:teuthology.orchestra.run.smithi137.stdout: "root": "/" 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_count": 0, 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_pinned_count": 0, 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 14799, 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "inst": { 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "name": { 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "client", 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "num": 14799 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T13:59:39.125 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3814462907 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3814462907 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T13:59:39.126 INFO:teuthology.orchestra.run.smithi137.stdout: "inst_str": "client.14799 192.168.0.1:0/3814462907", 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "addr_str": "192.168.0.1:0/3814462907", 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "inode_count": 1, 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_epoch": 16, 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch": 90, 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch_barrier": 0, 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "blocklisted": false, 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs" 2024-03-26T13:59:39.127 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T13:59:39.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:39.139 DEBUG:teuthology.orchestra.run.smithi137:> stat --file-system '--printf=%T 2024-03-26T13:59:39.139 DEBUG:teuthology.orchestra.run.smithi137:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:39.185 INFO:teuthology.orchestra.run.smithi137.stdout:fuseblk 2024-03-26T13:59:39.185 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:39.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:39.185 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:39.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:39.321 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-26T13:59:39.411 INFO:teuthology.orchestra.run.smithi137.stdout:# file: . 2024-03-26T13:59:39.411 INFO:teuthology.orchestra.run.smithi137.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-26T13:59:39.411 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:39.412 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:59:39.412 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T13:59:39.463 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:39.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 -- 172.21.15.137:0/295307454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 msgr2=0x7fd2a8102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:39.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/295307454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8102b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd290009900 tx=0x7fd29002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:39.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 -- 172.21.15.137:0/295307454 shutdown_connections 2024-03-26T13:59:39.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/295307454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2a81030c0 0x7fd2a8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/295307454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8102b80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/295307454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8101510 0x7fd2a8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 -- 172.21.15.137:0/295307454 >> 172.21.15.137:0/295307454 conn(0x7fd2a80fca80 msgr2=0x7fd2a80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:39.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 -- 172.21.15.137:0/295307454 shutdown_connections 2024-03-26T13:59:39.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.722+0000 7fd2a7e00640 1 -- 172.21.15.137:0/295307454 wait complete. 2024-03-26T13:59:39.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 Processor -- start 2024-03-26T13:59:39.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 -- start start 2024-03-26T13:59:39.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2a8101510 0x7fd2a8197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:39.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a81030c0 0x7fd2a819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a810d050 con 0x7fd2a8102700 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a810ced0 con 0x7fd2a8101510 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.723+0000 7fd2a7e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a810d1d0 con 0x7fd2a81030c0 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48810/0 (socket says 172.21.15.137:48810) 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 -- 172.21.15.137:0/2417673720 learned_addr learned my addr 172.21.15.137:0/2417673720 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6a00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2a8101510 0x7fd2a8197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a7400640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a81030c0 0x7fd2a819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 -- 172.21.15.137:0/2417673720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a81030c0 msgr2=0x7fd2a819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:39.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a81030c0 0x7fd2a819ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 -- 172.21.15.137:0/2417673720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2a8101510 msgr2=0x7fd2a8197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2a8101510 0x7fd2a8197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 -- 172.21.15.137:0/2417673720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a819f360 con 0x7fd2a8102700 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.724+0000 7fd2a6000640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8197cd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd290006010 tx=0x7fd2900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.725+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290043020 con 0x7fd2a8102700 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.725+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a819f5f0 con 0x7fd2a8102700 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.725+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd290004aa0 con 0x7fd2a8102700 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.725+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd290003c60 con 0x7fd2a8102700 2024-03-26T13:59:39.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.725+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a819fb90 con 0x7fd2a8102700 2024-03-26T13:59:39.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.726+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a8108950 con 0x7fd2a8102700 2024-03-26T13:59:39.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.728+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd290046020 con 0x7fd2a8102700 2024-03-26T13:59:39.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.730+0000 7fd29f400640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd28c0a2880 0x7fd28c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:39.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.730+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd2900f1770 con 0x7fd2a8102700 2024-03-26T13:59:39.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.730+0000 7fd2a6a00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd28c0a2880 0x7fd28c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:39.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.731+0000 7fd2a6a00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd28c0a2880 0x7fd28c0a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8102560 tx=0x7fd298009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:39.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.733+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd29004d020 con 0x7fd2a8102700 2024-03-26T13:59:39.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.932+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2a802d2c0 con 0x7fd2a8102700 2024-03-26T13:59:39.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.935+0000 7fd29f400640 1 -- 172.21.15.137:0/2417673720 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7fd2900be570 con 0x7fd2a8102700 2024-03-26T13:59:39.933 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:39.933 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.937+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd28c0a2880 msgr2=0x7fd28c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.937+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd28c0a2880 0x7fd28c0a4d40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8102560 tx=0x7fd298009870 comp rx=0 tx=0).stop 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.937+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 msgr2=0x7fd2a8197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.937+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8197cd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd290006010 tx=0x7fd2900042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 shutdown_connections 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd28c0a2880 0x7fd28c0a4d40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a81030c0 0x7fd2a819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2a8102700 0x7fd2a8197cd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 --2- 172.21.15.137:0/2417673720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2a8101510 0x7fd2a8197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:39.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 >> 172.21.15.137:0/2417673720 conn(0x7fd2a80fca80 msgr2=0x7fd2a8105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:39.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 shutdown_connections 2024-03-26T13:59:39.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:39.938+0000 7fd2a7e00640 1 -- 172.21.15.137:0/2417673720 wait complete. 2024-03-26T13:59:39.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T13:59:39.955 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T13:59:40.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 -- 172.21.15.137:0/1255817502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701028a0 msgr2=0x7fc170102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 --2- 172.21.15.137:0/1255817502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701028a0 0x7fc170102d20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc158009900 tx=0x7fc15802bc50 comp rx=0 tx=0).stop 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 -- 172.21.15.137:0/1255817502 shutdown_connections 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 --2- 172.21.15.137:0/1255817502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc170103260 0x7fc170109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 --2- 172.21.15.137:0/1255817502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1701028a0 0x7fc170102d20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 --2- 172.21.15.137:0/1255817502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1701016b0 0x7fc170101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 -- 172.21.15.137:0/1255817502 >> 172.21.15.137:0/1255817502 conn(0x7fc1700fcc40 msgr2=0x7fc1700ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 -- 172.21.15.137:0/1255817502 shutdown_connections 2024-03-26T13:59:40.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.179+0000 7fc175800640 1 -- 172.21.15.137:0/1255817502 wait complete. 2024-03-26T13:59:40.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 Processor -- start 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 -- start start 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1701016b0 0x7fc170199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1701028a0 0x7fc17019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 0x7fc1701a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17010d240 con 0x7fc170103260 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17010d0c0 con 0x7fc1701016b0 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.180+0000 7fc175800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17010d3c0 con 0x7fc1701028a0 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 0x7fc1701a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 0x7fc1701a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48838/0 (socket says 172.21.15.137:48838) 2024-03-26T13:59:40.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 -- 172.21.15.137:0/2228632483 learned_addr learned my addr 172.21.15.137:0/2228632483 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16ea00640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1701016b0 0x7fc170199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16e000640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1701028a0 0x7fc17019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 -- 172.21.15.137:0/2228632483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1701028a0 msgr2=0x7fc17019a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1701028a0 0x7fc17019a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 -- 172.21.15.137:0/2228632483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1701016b0 msgr2=0x7fc170199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1701016b0 0x7fc170199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.181+0000 7fc16f400640 1 -- 172.21.15.137:0/2228632483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1701a1770 con 0x7fc170103260 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.182+0000 7fc16f400640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 0x7fc1701a1100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc160009fd0 tx=0x7fc16000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.182+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160011830 con 0x7fc170103260 2024-03-26T13:59:40.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.182+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1701a1a60 con 0x7fc170103260 2024-03-26T13:59:40.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.182+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc160016070 con 0x7fc170103260 2024-03-26T13:59:40.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.182+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160010de0 con 0x7fc170103260 2024-03-26T13:59:40.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.182+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701a1fb0 con 0x7fc170103260 2024-03-26T13:59:40.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.183+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1701a2240 con 0x7fc170103260 2024-03-26T13:59:40.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.185+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc1600119d0 con 0x7fc170103260 2024-03-26T13:59:40.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.186+0000 7fc167400640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc1540a2880 0x7fc1540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.187+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc1600ce3d0 con 0x7fc170103260 2024-03-26T13:59:40.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.187+0000 7fc16ea00640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc1540a2880 0x7fc1540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.188+0000 7fc16ea00640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc1540a2880 0x7fc1540a4d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc16800a300 tx=0x7fc16800a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:40.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.191+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc16009b230 con 0x7fc170103260 2024-03-26T13:59:40.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:40 smithi137 ceph-mon[31539]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:40.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:40 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2417673720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:40.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.392+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1701a24b0 con 0x7fc170103260 2024-03-26T13:59:40.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.393+0000 7fc167400640 1 -- 172.21.15.137:0/2228632483 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc16009ab30 con 0x7fc170103260 2024-03-26T13:59:40.390 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T13:59:40.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc1540a2880 msgr2=0x7fc1540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc1540a2880 0x7fc1540a4d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc16800a300 tx=0x7fc16800a290 comp rx=0 tx=0).stop 2024-03-26T13:59:40.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 msgr2=0x7fc1701a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 0x7fc1701a1100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc160009fd0 tx=0x7fc16000b8e0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 shutdown_connections 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc1540a2880 0x7fc1540a4d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc170103260 0x7fc1701a1100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc1701028a0 0x7fc17019a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 --2- 172.21.15.137:0/2228632483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1701016b0 0x7fc170199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 >> 172.21.15.137:0/2228632483 conn(0x7fc1700fcc40 msgr2=0x7fc1700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 shutdown_connections 2024-03-26T13:59:40.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.396+0000 7fc175800640 1 -- 172.21.15.137:0/2228632483 wait complete. 2024-03-26T13:59:40.412 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T13:59:40.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:40 smithi179 ceph-mon[35472]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:40.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:40 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2417673720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:40.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:40 smithi159 ceph-mon[35501]: pgmap v373: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:40.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:40 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2417673720' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:40.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 -- 172.21.15.137:0/1587560591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780756a0 msgr2=0x7f5778073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 --2- 172.21.15.137:0/1587560591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780756a0 0x7f5778073af0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5768009900 tx=0x7f576802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:40.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 -- 172.21.15.137:0/1587560591 shutdown_connections 2024-03-26T13:59:40.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 --2- 172.21.15.137:0/1587560591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778109560 0x7f577810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 --2- 172.21.15.137:0/1587560591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57780740c0 0x7f5778074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 --2- 172.21.15.137:0/1587560591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780756a0 0x7f5778073af0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 -- 172.21.15.137:0/1587560591 >> 172.21.15.137:0/1587560591 conn(0x7f57780fcc40 msgr2=0x7f57780ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:40.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 -- 172.21.15.137:0/1587560591 shutdown_connections 2024-03-26T13:59:40.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.631+0000 7f577e600640 1 -- 172.21.15.137:0/1587560591 wait complete. 2024-03-26T13:59:40.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 Processor -- start 2024-03-26T13:59:40.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 -- start start 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 0x7f5778199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57780756a0 0x7f577819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778109560 0x7f57781a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810f410 con 0x7f57780740c0 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810f290 con 0x7f57780756a0 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.632+0000 7f577e600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810f590 con 0x7f5778109560 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 0x7f5778199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 0x7f5778199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48858/0 (socket says 172.21.15.137:48858) 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 -- 172.21.15.137:0/3435708419 learned_addr learned my addr 172.21.15.137:0/3435708419 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5776a00640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57780756a0 0x7f577819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777e00640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778109560 0x7f57781a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 -- 172.21.15.137:0/3435708419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778109560 msgr2=0x7f57781a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778109560 0x7f57781a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 -- 172.21.15.137:0/3435708419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57780756a0 msgr2=0x7f577819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57780756a0 0x7f577819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.633+0000 7f5777400640 1 -- 172.21.15.137:0/3435708419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57781a1750 con 0x7f57780740c0 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.634+0000 7f5777400640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 0x7f5778199b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5768000c00 tx=0x7f57680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:40.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.634+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768043020 con 0x7f57780740c0 2024-03-26T13:59:40.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.634+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57680054c0 con 0x7f57780740c0 2024-03-26T13:59:40.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.634+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781a19e0 con 0x7f57780740c0 2024-03-26T13:59:40.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.634+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768003d70 con 0x7f57780740c0 2024-03-26T13:59:40.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.634+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781a1f30 con 0x7f57780740c0 2024-03-26T13:59:40.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.635+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57780688e0 con 0x7f57780740c0 2024-03-26T13:59:40.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.637+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5768046020 con 0x7f57780740c0 2024-03-26T13:59:40.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.638+0000 7f576fe00640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57580a2800 0x7f57580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:40.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.639+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f57680f12b0 con 0x7f57780740c0 2024-03-26T13:59:40.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.639+0000 7f5776a00640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57580a2800 0x7f57580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:40.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.639+0000 7f5776a00640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57580a2800 0x7f57580a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f577819b040 tx=0x7f5760009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:40.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.643+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f576804d020 con 0x7f57780740c0 2024-03-26T13:59:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.835+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57780622a0 con 0x7f57780740c0 2024-03-26T13:59:40.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.836+0000 7f576fe00640 1 -- 172.21.15.137:0/3435708419 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57680be230 con 0x7f57780740c0 2024-03-26T13:59:40.833 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57580a2800 msgr2=0x7f57580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57580a2800 0x7f57580a4cc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f577819b040 tx=0x7f5760009870 comp rx=0 tx=0).stop 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 msgr2=0x7f5778199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 0x7f5778199b30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5768000c00 tx=0x7f57680042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 shutdown_connections 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57580a2800 0x7f57580a4cc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57780756a0 0x7f577819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778109560 0x7f57781a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 --2- 172.21.15.137:0/3435708419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780740c0 0x7f5778199b30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 >> 172.21.15.137:0/3435708419 conn(0x7f57780fcc40 msgr2=0x7f57781054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 shutdown_connections 2024-03-26T13:59:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:40.839+0000 7f577e600640 1 -- 172.21.15.137:0/3435708419 wait complete. 2024-03-26T13:59:40.855 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T13:59:41.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.076+0000 7ff22b200640 1 -- 172.21.15.137:0/2163216820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241049b0 msgr2=0x7ff224106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.076+0000 7ff22b200640 1 --2- 172.21.15.137:0/2163216820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241049b0 0x7ff224106da0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff214009900 tx=0x7ff21402bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:41.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.076+0000 7ff22b200640 1 -- 172.21.15.137:0/2163216820 shutdown_connections 2024-03-26T13:59:41.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.076+0000 7ff22b200640 1 --2- 172.21.15.137:0/2163216820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff2241072e0 0x7ff2241096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.076+0000 7ff22b200640 1 --2- 172.21.15.137:0/2163216820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241049b0 0x7ff224106da0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.077+0000 7ff22b200640 1 --2- 172.21.15.137:0/2163216820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff224102080 0x7ff224104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.077+0000 7ff22b200640 1 -- 172.21.15.137:0/2163216820 >> 172.21.15.137:0/2163216820 conn(0x7ff2240fba10 msgr2=0x7ff2240fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:41.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.077+0000 7ff22b200640 1 -- 172.21.15.137:0/2163216820 shutdown_connections 2024-03-26T13:59:41.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.077+0000 7ff22b200640 1 -- 172.21.15.137:0/2163216820 wait complete. 2024-03-26T13:59:41.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.077+0000 7ff22b200640 1 Processor -- start 2024-03-26T13:59:41.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 -- start start 2024-03-26T13:59:41.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff224102080 0x7ff2241019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2241049b0 0x7ff224100030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 0x7ff224100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22410d110 con 0x7ff2241072e0 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22410cf90 con 0x7ff224102080 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22410d290 con 0x7ff2241049b0 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 0x7ff224100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 0x7ff224100570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48880/0 (socket says 172.21.15.137:48880) 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.078+0000 7ff22a800640 1 -- 172.21.15.137:0/1498432437 learned_addr learned my addr 172.21.15.137:0/1498432437 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff229e00640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff224102080 0x7ff2241019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff229400640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2241049b0 0x7ff224100030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22a800640 1 -- 172.21.15.137:0/1498432437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2241049b0 msgr2=0x7ff224100030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22a800640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2241049b0 0x7ff224100030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22a800640 1 -- 172.21.15.137:0/1498432437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff224102080 msgr2=0x7ff2241019f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22a800640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff224102080 0x7ff2241019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22a800640 1 -- 172.21.15.137:0/1498432437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224100e00 con 0x7ff2241072e0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22a800640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 0x7ff224100570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff22000cce0 tx=0x7ff22000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000f450 con 0x7ff2241072e0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff22000faa0 con 0x7ff2241072e0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2241a18c0 con 0x7ff2241072e0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220018b70 con 0x7ff2241072e0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.079+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2241a1dc0 con 0x7ff2241072e0 2024-03-26T13:59:41.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.081+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2240fd500 con 0x7ff2241072e0 2024-03-26T13:59:41.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.082+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff220018d10 con 0x7ff2241072e0 2024-03-26T13:59:41.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.084+0000 7ff21ea00640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff20c0a2800 0x7ff20c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.084+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7ff2200cf240 con 0x7ff2241072e0 2024-03-26T13:59:41.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.084+0000 7ff229e00640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff20c0a2800 0x7ff20c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.084+0000 7ff229e00640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff20c0a2800 0x7ff20c0a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff218005fd0 tx=0x7ff218005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:41.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.088+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff220014030 con 0x7ff2241072e0 2024-03-26T13:59:41.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.294+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2240622a0 con 0x7ff2241072e0 2024-03-26T13:59:41.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.294+0000 7ff21ea00640 1 -- 172.21.15.137:0/1498432437 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff22009c040 con 0x7ff2241072e0 2024-03-26T13:59:41.291 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T13:59:41.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff20c0a2800 msgr2=0x7ff20c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff20c0a2800 0x7ff20c0a4cc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff218005fd0 tx=0x7ff218005850 comp rx=0 tx=0).stop 2024-03-26T13:59:41.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 msgr2=0x7ff224100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 0x7ff224100570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff22000cce0 tx=0x7ff22000a620 comp rx=0 tx=0).stop 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 shutdown_connections 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff20c0a2800 0x7ff20c0a4cc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff2241072e0 0x7ff224100570 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff2241049b0 0x7ff224100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 --2- 172.21.15.137:0/1498432437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff224102080 0x7ff2241019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 >> 172.21.15.137:0/1498432437 conn(0x7ff2240fba10 msgr2=0x7ff2241029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 shutdown_connections 2024-03-26T13:59:41.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.297+0000 7ff22b200640 1 -- 172.21.15.137:0/1498432437 wait complete. 2024-03-26T13:59:41.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T13:59:41.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:41 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2228632483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:41.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:41 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3435708419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T13:59:41.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:41 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2228632483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:41.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:41 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3435708419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T13:59:41.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 -- 172.21.15.137:0/2186262815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 msgr2=0x7f4558109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 --2- 172.21.15.137:0/2186262815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f4558109230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4548009900 tx=0x7f454802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:41.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 -- 172.21.15.137:0/2186262815 shutdown_connections 2024-03-26T13:59:41.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 --2- 172.21.15.137:0/2186262815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558109770 0x7f4558110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 --2- 172.21.15.137:0/2186262815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f4558109230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 --2- 172.21.15.137:0/2186262815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558107bc0 0x7f4558107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.517+0000 7f455e000640 1 -- 172.21.15.137:0/2186262815 >> 172.21.15.137:0/2186262815 conn(0x7f4558076b70 msgr2=0x7f4558078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.518+0000 7f455e000640 1 -- 172.21.15.137:0/2186262815 shutdown_connections 2024-03-26T13:59:41.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.518+0000 7f455e000640 1 -- 172.21.15.137:0/2186262815 wait complete. 2024-03-26T13:59:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.518+0000 7f455e000640 1 Processor -- start 2024-03-26T13:59:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 -- start start 2024-03-26T13:59:41.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558107bc0 0x7f455819ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f455819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558109770 0x7f45581a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45581136d0 con 0x7f4558108db0 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558113550 con 0x7f4558107bc0 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f455e000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558113850 con 0x7f4558109770 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f454fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f455819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f454fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f455819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48890/0 (socket says 172.21.15.137:48890) 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.519+0000 7f454fe00640 1 -- 172.21.15.137:0/472198763 learned_addr learned my addr 172.21.15.137:0/472198763 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f455cc00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558107bc0 0x7f455819ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f455d600640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558109770 0x7f45581a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f454fe00640 1 -- 172.21.15.137:0/472198763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558109770 msgr2=0x7f45581a5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f454fe00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558109770 0x7f45581a5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f454fe00640 1 -- 172.21.15.137:0/472198763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558107bc0 msgr2=0x7f455819ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f454fe00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558107bc0 0x7f455819ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f454fe00640 1 -- 172.21.15.137:0/472198763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45581a5930 con 0x7f4558108db0 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f455cc00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558107bc0 0x7f455819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f455d600640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558109770 0x7f45581a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f454fe00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f455819e300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4548006010 tx=0x7f45480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548043020 con 0x7f4558108db0 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4548004aa0 con 0x7f4558108db0 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45581a5bc0 con 0x7f4558108db0 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.520+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4548003c60 con 0x7f4558108db0 2024-03-26T13:59:41.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.521+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45581a6160 con 0x7f4558108db0 2024-03-26T13:59:41.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.522+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455810eee0 con 0x7f4558108db0 2024-03-26T13:59:41.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.524+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4548046020 con 0x7f4558108db0 2024-03-26T13:59:41.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.525+0000 7f4556000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f453c0a2880 0x7f453c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.525+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f45480f1170 con 0x7f4558108db0 2024-03-26T13:59:41.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.525+0000 7f455cc00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f453c0a2880 0x7f453c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.526+0000 7f455cc00640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f453c0a2880 0x7f453c0a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f455819f1f0 tx=0x7f4540005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:41.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.529+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f454804d020 con 0x7f4558108db0 2024-03-26T13:59:41.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2228632483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:41.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:41 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3435708419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T13:59:41.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.727+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f45580622a0 con 0x7f4558108db0 2024-03-26T13:59:41.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.728+0000 7f4556000640 1 -- 172.21.15.137:0/472198763 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f45480bdfd0 con 0x7f4558108db0 2024-03-26T13:59:41.725 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T13:59:41.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f453c0a2880 msgr2=0x7f453c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f453c0a2880 0x7f453c0a4d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f455819f1f0 tx=0x7f4540005dd0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 msgr2=0x7f455819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f455819e300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4548006010 tx=0x7f45480042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 shutdown_connections 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f453c0a2880 0x7f453c0a4d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4558109770 0x7f45581a5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4558108db0 0x7f455819e300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 --2- 172.21.15.137:0/472198763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4558107bc0 0x7f455819ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 >> 172.21.15.137:0/472198763 conn(0x7f4558076b70 msgr2=0x7f455810c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 shutdown_connections 2024-03-26T13:59:41.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.731+0000 7f455e000640 1 -- 172.21.15.137:0/472198763 wait complete. 2024-03-26T13:59:41.746 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T13:59:41.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 -- 172.21.15.137:0/1673163886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210109870 msgr2=0x7f7210110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 --2- 172.21.15.137:0/1673163886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210109870 0x7f7210110100 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7208009ca0 tx=0x7f720802c100 comp rx=0 tx=0).stop 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 -- 172.21.15.137:0/1673163886 shutdown_connections 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 --2- 172.21.15.137:0/1673163886 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210109870 0x7f7210110100 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 --2- 172.21.15.137:0/1673163886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210108eb0 0x7f7210109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 --2- 172.21.15.137:0/1673163886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7210107cc0 0x7f72101080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 -- 172.21.15.137:0/1673163886 >> 172.21.15.137:0/1673163886 conn(0x7f7210076c60 msgr2=0x7f72100790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 -- 172.21.15.137:0/1673163886 shutdown_connections 2024-03-26T13:59:41.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.960+0000 7f7217e00640 1 -- 172.21.15.137:0/1673163886 wait complete. 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 Processor -- start 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 -- start start 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 0x7f721019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7210108eb0 0x7f721019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210109870 0x7f72101a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101137f0 con 0x7f7210107cc0 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210113670 con 0x7f7210108eb0 2024-03-26T13:59:41.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.961+0000 7f7217e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210113970 con 0x7f7210109870 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 0x7f721019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 0x7f721019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48904/0 (socket says 172.21.15.137:48904) 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 -- 172.21.15.137:0/3409450991 learned_addr learned my addr 172.21.15.137:0/3409450991 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7214a00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7210108eb0 0x7f721019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210109870 0x7f72101a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 -- 172.21.15.137:0/3409450991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210109870 msgr2=0x7f72101a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210109870 0x7f72101a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 -- 172.21.15.137:0/3409450991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7210108eb0 msgr2=0x7f721019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7210108eb0 0x7f721019e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215400640 1 -- 172.21.15.137:0/3409450991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72101a5a80 con 0x7f7210107cc0 2024-03-26T13:59:41.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.962+0000 7f7215e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210109870 0x7f72101a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.963+0000 7f7215400640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 0x7f721019de60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f720400d950 tx=0x7f720400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.963+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204017020 con 0x7f7210107cc0 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.963+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7204009c30 con 0x7f7210107cc0 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.963+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400f6b0 con 0x7f7210107cc0 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.963+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72101a5d70 con 0x7f7210107cc0 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.963+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101a6200 con 0x7f7210107cc0 2024-03-26T13:59:41.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.964+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721010efe0 con 0x7f7210107cc0 2024-03-26T13:59:41.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.966+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f720401a020 con 0x7f7210107cc0 2024-03-26T13:59:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.968+0000 7f720e000640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71f80a2800 0x7f71f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.968+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f72040ced10 con 0x7f7210107cc0 2024-03-26T13:59:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.968+0000 7f7214a00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71f80a2800 0x7f71f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.968+0000 7f7214a00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71f80a2800 0x7f71f80a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005fd0 tx=0x7f71fc00b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:41.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:41.972+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f72040d4020 con 0x7f7210107cc0 2024-03-26T13:59:42.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.183+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f721002d070 con 0x7f7210107cc0 2024-03-26T13:59:42.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.184+0000 7f720e000640 1 -- 172.21.15.137:0/3409450991 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+600 (secure 0 0 0) 0x7f721002d070 con 0x7f7210107cc0 2024-03-26T13:59:42.180 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:42.180 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:42.180 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/460394311 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1426358112 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2012111099 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1602182932 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3592180121 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/487629871 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:42.181 INFO:teuthology.orchestra.run.smithi137.stderr:listed 10 entries 2024-03-26T13:59:42.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.186+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71f80a2800 msgr2=0x7f71f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.186+0000 7f7217e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71f80a2800 0x7f71f80a4cc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005fd0 tx=0x7f71fc00b510 comp rx=0 tx=0).stop 2024-03-26T13:59:42.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.186+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 msgr2=0x7f721019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 0x7f721019de60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f720400d950 tx=0x7f720400de10 comp rx=0 tx=0).stop 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 shutdown_connections 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f71f80a2800 0x7f71f80a4cc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7210108eb0 0x7f721019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7210109870 0x7f72101a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 --2- 172.21.15.137:0/3409450991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7210107cc0 0x7f721019de60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 >> 172.21.15.137:0/3409450991 conn(0x7f7210076c60 msgr2=0x7f721010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 shutdown_connections 2024-03-26T13:59:42.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.187+0000 7f7217e00640 1 -- 172.21.15.137:0/3409450991 wait complete. 2024-03-26T13:59:42.202 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T13:59:42.202 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T13:59:42.251 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:42.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:42 smithi179 ceph-mon[35472]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:42.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:42 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1498432437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T13:59:42.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:42 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/472198763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T13:59:42.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:42 smithi159 ceph-mon[35501]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:42.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:42 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1498432437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T13:59:42.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:42 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/472198763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T13:59:42.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 -- 172.21.15.137:0/50156254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 msgr2=0x7f1f3c109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 --2- 172.21.15.137:0/50156254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c109350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009900 tx=0x7f1f2402bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 -- 172.21.15.137:0/50156254 shutdown_connections 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 --2- 172.21.15.137:0/50156254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f3c109890 0x7f1f3c110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 --2- 172.21.15.137:0/50156254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c109350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 --2- 172.21.15.137:0/50156254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3c107ce0 0x7f1f3c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 -- 172.21.15.137:0/50156254 >> 172.21.15.137:0/50156254 conn(0x7f1f3c076cb0 msgr2=0x7f1f3c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 -- 172.21.15.137:0/50156254 shutdown_connections 2024-03-26T13:59:42.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.511+0000 7f1f43400640 1 -- 172.21.15.137:0/50156254 wait complete. 2024-03-26T13:59:42.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 Processor -- start 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 -- start start 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3c107ce0 0x7f1f3c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f3c109890 0x7f1f3c1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c113860 con 0x7f1f3c108ed0 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1136e0 con 0x7f1f3c109890 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.512+0000 7f1f43400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3c1139e0 con 0x7f1f3c107ce0 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c19e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48924/0 (socket says 172.21.15.137:48924) 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 -- 172.21.15.137:0/1901148844 learned_addr learned my addr 172.21.15.137:0/1901148844 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:42.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f41400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f3c109890 0x7f1f3c1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f40a00640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3c107ce0 0x7f1f3c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 -- 172.21.15.137:0/1901148844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3c107ce0 msgr2=0x7f1f3c19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3c107ce0 0x7f1f3c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 -- 172.21.15.137:0/1901148844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f3c109890 msgr2=0x7f1f3c1a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f3c109890 0x7f1f3c1a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 -- 172.21.15.137:0/1901148844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c1a5b30 con 0x7f1f3c108ed0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.513+0000 7f1f33e00640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c19e450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.514+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24043020 con 0x7f1f3c108ed0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.514+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f240054c0 con 0x7f1f3c108ed0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.514+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24003d70 con 0x7f1f3c108ed0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.514+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3c1a5dc0 con 0x7f1f3c108ed0 2024-03-26T13:59:42.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.514+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c1a6280 con 0x7f1f3c108ed0 2024-03-26T13:59:42.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.515+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3c10f020 con 0x7f1f3c108ed0 2024-03-26T13:59:42.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.517+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1f24046020 con 0x7f1f3c108ed0 2024-03-26T13:59:42.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.518+0000 7f1f3a000640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1f200a2800 0x7f1f200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.519+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f1f240f1210 con 0x7f1f3c108ed0 2024-03-26T13:59:42.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.519+0000 7f1f40a00640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1f200a2800 0x7f1f200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.519+0000 7f1f40a00640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1f200a2800 0x7f1f200a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c005fd0 tx=0x7f1f2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:42.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.522+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f2404d020 con 0x7f1f3c108ed0 2024-03-26T13:59:42.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:42 smithi137 ceph-mon[31539]: pgmap v374: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T13:59:42.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:42 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1498432437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T13:59:42.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:42 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/472198763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T13:59:42.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.709+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f3c062340 con 0x7f1f3c108ed0 2024-03-26T13:59:42.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.712+0000 7f1f3a000640 1 -- 172.21.15.137:0/1901148844 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7f1f240be130 con 0x7f1f3c108ed0 2024-03-26T13:59:42.709 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:42.709 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.715+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1f200a2800 msgr2=0x7f1f200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.715+0000 7f1f43400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1f200a2800 0x7f1f200a4cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c005fd0 tx=0x7f1f2c005850 comp rx=0 tx=0).stop 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.715+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 msgr2=0x7f1f3c19e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.715+0000 7f1f43400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c19e450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000c00 tx=0x7f1f240042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 shutdown_connections 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1f200a2800 0x7f1f200a4cc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f3c109890 0x7f1f3c1a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f3c108ed0 0x7f1f3c19e450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 --2- 172.21.15.137:0/1901148844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f3c107ce0 0x7f1f3c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 >> 172.21.15.137:0/1901148844 conn(0x7f1f3c076cb0 msgr2=0x7f1f3c10c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:42.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 shutdown_connections 2024-03-26T13:59:42.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.716+0000 7f1f43400640 1 -- 172.21.15.137:0/1901148844 wait complete. 2024-03-26T13:59:42.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T13:59:42.732 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 -- 172.21.15.137:0/2052391569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8077c40 msgr2=0x7fc7e80784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 --2- 172.21.15.137:0/2052391569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8077c40 0x7fc7e80784e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009900 tx=0x7fc7e002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 -- 172.21.15.137:0/2052391569 shutdown_connections 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 --2- 172.21.15.137:0/2052391569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8077c40 0x7fc7e80784e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 --2- 172.21.15.137:0/2052391569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e80772a0 0x7fc7e8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 --2- 172.21.15.137:0/2052391569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7e8076060 0x7fc7e8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 -- 172.21.15.137:0/2052391569 >> 172.21.15.137:0/2052391569 conn(0x7fc7e80fee50 msgr2=0x7fc7e8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 -- 172.21.15.137:0/2052391569 shutdown_connections 2024-03-26T13:59:42.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.944+0000 7fc7ee000640 1 -- 172.21.15.137:0/2052391569 wait complete. 2024-03-26T13:59:42.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.945+0000 7fc7ee000640 1 Processor -- start 2024-03-26T13:59:42.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.945+0000 7fc7ee000640 1 -- start start 2024-03-26T13:59:42.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7ee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 0x7fc7e819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7ee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7e80772a0 0x7fc7e819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7ee000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e8077c40 0x7fc7e81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7ee000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8113860 con 0x7fc7e8076060 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7ee000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81136e0 con 0x7fc7e80772a0 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7ee000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e81139e0 con 0x7fc7e8077c40 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 0x7fc7e819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 0x7fc7e819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48936/0 (socket says 172.21.15.137:48936) 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 -- 172.21.15.137:0/4244469460 learned_addr learned my addr 172.21.15.137:0/4244469460 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7e00640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e8077c40 0x7fc7e81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e6a00640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7e80772a0 0x7fc7e819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 -- 172.21.15.137:0/4244469460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e8077c40 msgr2=0x7fc7e81a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e8077c40 0x7fc7e81a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 -- 172.21.15.137:0/4244469460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7e80772a0 msgr2=0x7fc7e819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7e80772a0 0x7fc7e819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.946+0000 7fc7e7400640 1 -- 172.21.15.137:0/4244469460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e81a5aa0 con 0x7fc7e8076060 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.947+0000 7fc7e7400640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 0x7fc7e819de80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800db80 tx=0x7fc7d800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.947+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8017070 con 0x7fc7e8076060 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.947+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc7d8003b70 con 0x7fc7e8076060 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.947+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e81a5d90 con 0x7fc7e8076060 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.947+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80104c0 con 0x7fc7e8076060 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.947+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e81a6250 con 0x7fc7e8076060 2024-03-26T13:59:42.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.948+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e81103e0 con 0x7fc7e8076060 2024-03-26T13:59:42.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.950+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc7d8004020 con 0x7fc7e8076060 2024-03-26T13:59:42.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.952+0000 7fc7dfe00640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7cc0a2880 0x7fc7cc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:42.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.952+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc7d80cee60 con 0x7fc7e8076060 2024-03-26T13:59:42.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.953+0000 7fc7e6a00640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7cc0a2880 0x7fc7cc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:42.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.953+0000 7fc7e6a00640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7cc0a2880 0x7fc7cc0a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005fd0 tx=0x7fc7d000b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:42.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:42.957+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc7d809bc60 con 0x7fc7e8076060 2024-03-26T13:59:43.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.156+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7e80623a0 con 0x7fc7e8076060 2024-03-26T13:59:43.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.157+0000 7fc7dfe00640 1 -- 172.21.15.137:0/4244469460 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7d809b560 con 0x7fc7e8076060 2024-03-26T13:59:43.153 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T13:59:43.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7cc0a2880 msgr2=0x7fc7cc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7cc0a2880 0x7fc7cc0a4d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005fd0 tx=0x7fc7d000b510 comp rx=0 tx=0).stop 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 msgr2=0x7fc7e819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 0x7fc7e819de80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800db80 tx=0x7fc7d800b630 comp rx=0 tx=0).stop 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 shutdown_connections 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7cc0a2880 0x7fc7cc0a4d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7e80772a0 0x7fc7e819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7e8077c40 0x7fc7e81a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 --2- 172.21.15.137:0/4244469460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7e8076060 0x7fc7e819de80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 >> 172.21.15.137:0/4244469460 conn(0x7fc7e80fee50 msgr2=0x7fc7e8100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 shutdown_connections 2024-03-26T13:59:43.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.160+0000 7fc7ee000640 1 -- 172.21.15.137:0/4244469460 wait complete. 2024-03-26T13:59:43.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.396+0000 7f9eeb000640 1 -- 172.21.15.137:0/1131462752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4107bc0 msgr2=0x7f9ee4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.396+0000 7f9eeb000640 1 --2- 172.21.15.137:0/1131462752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4107bc0 0x7f9ee4107fe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 -- 172.21.15.137:0/1131462752 shutdown_connections 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 --2- 172.21.15.137:0/1131462752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4108db0 0x7f9ee4109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 --2- 172.21.15.137:0/1131462752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4109770 0x7f9ee4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 --2- 172.21.15.137:0/1131462752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4107bc0 0x7f9ee4107fe0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 -- 172.21.15.137:0/1131462752 >> 172.21.15.137:0/1131462752 conn(0x7f9ee4076b70 msgr2=0x7f9ee4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 -- 172.21.15.137:0/1131462752 shutdown_connections 2024-03-26T13:59:43.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 -- 172.21.15.137:0/1131462752 wait complete. 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.397+0000 7f9eeb000640 1 Processor -- start 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 -- start start 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4107bc0 0x7f9ee419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4108db0 0x7f9ee419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 0x7f9ee41a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4113710 con 0x7f9ee4109770 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4113590 con 0x7f9ee4107bc0 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eeb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4113890 con 0x7f9ee4108db0 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eea600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 0x7f9ee41a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eea600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 0x7f9ee41a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48956/0 (socket says 172.21.15.137:48956) 2024-03-26T13:59:43.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9eea600640 1 -- 172.21.15.137:0/2120429051 learned_addr learned my addr 172.21.15.137:0/2120429051 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.398+0000 7f9ee9c00640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4107bc0 0x7f9ee419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9ee9200640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4108db0 0x7f9ee419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eea600640 1 -- 172.21.15.137:0/2120429051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4108db0 msgr2=0x7f9ee419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eea600640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4108db0 0x7f9ee419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eea600640 1 -- 172.21.15.137:0/2120429051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4107bc0 msgr2=0x7f9ee419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eea600640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4107bc0 0x7f9ee419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eea600640 1 -- 172.21.15.137:0/2120429051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee41a59b0 con 0x7f9ee4109770 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9ee9c00640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4107bc0 0x7f9ee419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eea600640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 0x7f9ee41a5310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00cce0 tx=0x7f9edc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00f450 con 0x7f9ee4109770 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9edc00faa0 con 0x7f9ee4109770 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee41a5ca0 con 0x7f9ee4109770 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.399+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc018b70 con 0x7f9ee4109770 2024-03-26T13:59:43.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.400+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee41a6170 con 0x7f9ee4109770 2024-03-26T13:59:43.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.401+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee410eee0 con 0x7f9ee4109770 2024-03-26T13:59:43.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.403+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9edc018d10 con 0x7f9ee4109770 2024-03-26T13:59:43.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.404+0000 7f9ee2000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ecc0a2800 0x7f9ecc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.404+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9edc0cf2b0 con 0x7f9ee4109770 2024-03-26T13:59:43.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.404+0000 7f9ee9c00640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ecc0a2800 0x7f9ecc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.405+0000 7f9ee9c00640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ecc0a2800 0x7f9ecc0a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802c240 tx=0x7f9ed802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:43.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.408+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9edc014030 con 0x7f9ee4109770 2024-03-26T13:59:43.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:43 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3409450991' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T13:59:43.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:43 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1901148844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:43.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:43 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4244469460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:43.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:43 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3409450991' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T13:59:43.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:43 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1901148844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:43.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:43 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4244469460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:43.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:43 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3409450991' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T13:59:43.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:43 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1901148844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:43.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:43 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4244469460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T13:59:43.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.608+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ee40622a0 con 0x7f9ee4109770 2024-03-26T13:59:43.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.609+0000 7f9ee2000640 1 -- 172.21.15.137:0/2120429051 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9edc09c0b0 con 0x7f9ee4109770 2024-03-26T13:59:43.605 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T13:59:43.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.611+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ecc0a2800 msgr2=0x7f9ecc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ecc0a2800 0x7f9ecc0a4cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802c240 tx=0x7f9ed802c7e0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 msgr2=0x7f9ee41a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 0x7f9ee41a5310 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00cce0 tx=0x7f9edc00a620 comp rx=0 tx=0).stop 2024-03-26T13:59:43.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 shutdown_connections 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ecc0a2800 0x7f9ecc0a4cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ee4109770 0x7f9ee41a5310 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ee4108db0 0x7f9ee419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 --2- 172.21.15.137:0/2120429051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ee4107bc0 0x7f9ee419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 >> 172.21.15.137:0/2120429051 conn(0x7f9ee4076b70 msgr2=0x7f9ee410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 shutdown_connections 2024-03-26T13:59:43.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.612+0000 7f9eeb000640 1 -- 172.21.15.137:0/2120429051 wait complete. 2024-03-26T13:59:43.626 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 -- 172.21.15.137:0/1759118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec107bc0 msgr2=0x7fc4ec107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 --2- 172.21.15.137:0/1759118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec107bc0 0x7fc4ec107fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009900 tx=0x7fc4dc02bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 -- 172.21.15.137:0/1759118011 shutdown_connections 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 --2- 172.21.15.137:0/1759118011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4ec108db0 0x7fc4ec109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 --2- 172.21.15.137:0/1759118011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4ec109770 0x7fc4ec110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 --2- 172.21.15.137:0/1759118011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec107bc0 0x7fc4ec107fe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 -- 172.21.15.137:0/1759118011 >> 172.21.15.137:0/1759118011 conn(0x7fc4ec076b70 msgr2=0x7fc4ec078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:43.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 -- 172.21.15.137:0/1759118011 shutdown_connections 2024-03-26T13:59:43.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.843+0000 7fc4eb200640 1 -- 172.21.15.137:0/1759118011 wait complete. 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.844+0000 7fc4eb200640 1 Processor -- start 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 -- start start 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4ec107bc0 0x7fc4ec19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 0x7fc4ec19e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4ec109770 0x7fc4ec1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1136f0 con 0x7fc4ec108db0 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec113570 con 0x7fc4ec109770 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4eb200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec113870 con 0x7fc4ec107bc0 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4e3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 0x7fc4ec19e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4e3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 0x7fc4ec19e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48984/0 (socket says 172.21.15.137:48984) 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4e3e00640 1 -- 172.21.15.137:0/2510585480 learned_addr learned my addr 172.21.15.137:0/2510585480 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4e9e00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4ec107bc0 0x7fc4ec19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.845+0000 7fc4ea800640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4ec109770 0x7fc4ec1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e3e00640 1 -- 172.21.15.137:0/2510585480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4ec107bc0 msgr2=0x7fc4ec19de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e3e00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4ec107bc0 0x7fc4ec19de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e3e00640 1 -- 172.21.15.137:0/2510585480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4ec109770 msgr2=0x7fc4ec1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e3e00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4ec109770 0x7fc4ec1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e3e00640 1 -- 172.21.15.137:0/2510585480 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec1a5a50 con 0x7fc4ec108db0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e3e00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 0x7fc4ec19e3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400d950 tx=0x7fc4d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4017070 con 0x7fc4ec108db0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc4d4009c30 con 0x7fc4ec108db0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec1a5d40 con 0x7fc4ec108db0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400f620 con 0x7fc4ec108db0 2024-03-26T13:59:43.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.846+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec1a6280 con 0x7fc4ec108db0 2024-03-26T13:59:43.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.847+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec10f000 con 0x7fc4ec108db0 2024-03-26T13:59:43.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.849+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc4d40109a0 con 0x7fc4ec108db0 2024-03-26T13:59:43.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.851+0000 7fc4e2a00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4d00a2880 0x7fc4d00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:43.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.851+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fc4d40ce910 con 0x7fc4ec108db0 2024-03-26T13:59:43.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.851+0000 7fc4e9e00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4d00a2880 0x7fc4d00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:43.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.852+0000 7fc4e9e00640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4d00a2880 0x7fc4d00a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc006010 tx=0x7fc4dc005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:43.855+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4d4009de0 con 0x7fc4ec108db0 2024-03-26T13:59:44.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.054+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4ec0622a0 con 0x7fc4ec108db0 2024-03-26T13:59:44.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.055+0000 7fc4e2a00640 1 -- 172.21.15.137:0/2510585480 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4d409b770 con 0x7fc4ec108db0 2024-03-26T13:59:44.052 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4d00a2880 msgr2=0x7fc4d00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4d00a2880 0x7fc4d00a4d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc006010 tx=0x7fc4dc005be0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 msgr2=0x7fc4ec19e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 0x7fc4ec19e3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400d950 tx=0x7fc4d400de10 comp rx=0 tx=0).stop 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 shutdown_connections 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4d00a2880 0x7fc4d00a4d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4ec109770 0x7fc4ec1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ec108db0 0x7fc4ec19e3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 --2- 172.21.15.137:0/2510585480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4ec107bc0 0x7fc4ec19de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.058+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 >> 172.21.15.137:0/2510585480 conn(0x7fc4ec076b70 msgr2=0x7fc4ec10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:44.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.059+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 shutdown_connections 2024-03-26T13:59:44.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.059+0000 7fc4eb200640 1 -- 172.21.15.137:0/2510585480 wait complete. 2024-03-26T13:59:44.072 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T13:59:44.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.286+0000 7f5b56800640 1 -- 172.21.15.137:0/30464405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50105120 msgr2=0x7f5b501055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.286+0000 7f5b56800640 1 --2- 172.21.15.137:0/30464405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50105120 0x7f5b501055a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009c90 tx=0x7f5b4802c0a0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 -- 172.21.15.137:0/30464405 shutdown_connections 2024-03-26T13:59:44.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 --2- 172.21.15.137:0/30464405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50105120 0x7f5b501055a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 --2- 172.21.15.137:0/30464405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50102770 0x7f5b50104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 --2- 172.21.15.137:0/30464405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b500ffe00 0x7f5b50102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 -- 172.21.15.137:0/30464405 >> 172.21.15.137:0/30464405 conn(0x7f5b500fba20 msgr2=0x7f5b500fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:44.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 -- 172.21.15.137:0/30464405 shutdown_connections 2024-03-26T13:59:44.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.287+0000 7f5b56800640 1 -- 172.21.15.137:0/30464405 wait complete. 2024-03-26T13:59:44.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 Processor -- start 2024-03-26T13:59:44.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 -- start start 2024-03-26T13:59:44.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b500ffe00 0x7f5b5019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 0x7f5b5019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50105120 0x7f5b501a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b501136b0 con 0x7f5b50102770 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b50113530 con 0x7f5b500ffe00 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b56800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b50113830 con 0x7f5b50105120 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b54a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 0x7f5b5019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b54a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 0x7f5b5019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48996/0 (socket says 172.21.15.137:48996) 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.288+0000 7f5b54a00640 1 -- 172.21.15.137:0/3768956385 learned_addr learned my addr 172.21.15.137:0/3768956385 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b55400640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b500ffe00 0x7f5b5019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b55e00640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50105120 0x7f5b501a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b54a00640 1 -- 172.21.15.137:0/3768956385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50105120 msgr2=0x7f5b501a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b54a00640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50105120 0x7f5b501a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b54a00640 1 -- 172.21.15.137:0/3768956385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b500ffe00 msgr2=0x7f5b5019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b54a00640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b500ffe00 0x7f5b5019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b54a00640 1 -- 172.21.15.137:0/3768956385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b501a59e0 con 0x7f5b50102770 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b55e00640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50105120 0x7f5b501a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:44.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b54a00640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 0x7f5b5019e300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d770 tx=0x7f5b3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c017070 con 0x7f5b50102770 2024-03-26T13:59:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5b3c0054d0 con 0x7f5b50102770 2024-03-26T13:59:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.289+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b501a5cd0 con 0x7f5b50102770 2024-03-26T13:59:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.290+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c004cf0 con 0x7f5b50102770 2024-03-26T13:59:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.290+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b501a61d0 con 0x7f5b50102770 2024-03-26T13:59:44.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.291+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b500fd510 con 0x7f5b50102770 2024-03-26T13:59:44.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.293+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5b3c004e90 con 0x7f5b50102770 2024-03-26T13:59:44.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.295+0000 7f5b4e000640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b380a2800 0x7f5b380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.295+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f5b3c0ce7b0 con 0x7f5b50102770 2024-03-26T13:59:44.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.295+0000 7f5b55400640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b380a2800 0x7f5b380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.296+0000 7f5b55400640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b380a2800 0x7f5b380a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005fd0 tx=0x7f5b44005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:44.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.299+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5b3c09b610 con 0x7f5b50102770 2024-03-26T13:59:44.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:44 smithi159 ceph-mon[35501]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-26T13:59:44.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:44 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2120429051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T13:59:44.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:44 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2510585480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T13:59:44.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.498+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b500622a0 con 0x7f5b50102770 2024-03-26T13:59:44.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.499+0000 7f5b4e000640 1 -- 172.21.15.137:0/3768956385 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b3c09af10 con 0x7f5b50102770 2024-03-26T13:59:44.495 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b380a2800 msgr2=0x7f5b380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b380a2800 0x7f5b380a4cc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b44005fd0 tx=0x7f5b44005850 comp rx=0 tx=0).stop 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 msgr2=0x7f5b5019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 0x7f5b5019e300 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d770 tx=0x7f5b3c00dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 shutdown_connections 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5b380a2800 0x7f5b380a4cc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b50105120 0x7f5b501a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b50102770 0x7f5b5019e300 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 --2- 172.21.15.137:0/3768956385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b500ffe00 0x7f5b5019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.501+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 >> 172.21.15.137:0/3768956385 conn(0x7f5b500fba20 msgr2=0x7f5b501007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.502+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 shutdown_connections 2024-03-26T13:59:44.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.502+0000 7f5b56800640 1 -- 172.21.15.137:0/3768956385 wait complete. 2024-03-26T13:59:44.513 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:44 smithi137 ceph-mon[31539]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-26T13:59:44.513 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:44 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2120429051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T13:59:44.513 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:44 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2510585480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T13:59:44.517 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T13:59:44.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:44 smithi179 ceph-mon[35472]: pgmap v375: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-26T13:59:44.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:44 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2120429051' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T13:59:44.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:44 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2510585480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T13:59:44.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 -- 172.21.15.137:0/2020489093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441049b0 msgr2=0x7f7e44104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 --2- 172.21.15.137:0/2020489093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441049b0 0x7f7e44104e30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009900 tx=0x7f7e2c02bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:44.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 -- 172.21.15.137:0/2020489093 shutdown_connections 2024-03-26T13:59:44.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 --2- 172.21.15.137:0/2020489093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e44105370 0x7f7e4410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 --2- 172.21.15.137:0/2020489093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441049b0 0x7f7e44104e30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 --2- 172.21.15.137:0/2020489093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e441037c0 0x7f7e44103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 -- 172.21.15.137:0/2020489093 >> 172.21.15.137:0/2020489093 conn(0x7f7e440fed50 msgr2=0x7f7e44101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 -- 172.21.15.137:0/2020489093 shutdown_connections 2024-03-26T13:59:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.732+0000 7f7e43200640 1 -- 172.21.15.137:0/2020489093 wait complete. 2024-03-26T13:59:44.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 Processor -- start 2024-03-26T13:59:44.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 -- start start 2024-03-26T13:59:44.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 0x7f7e441999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e441049b0 0x7f7e44199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e44105370 0x7f7e441a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4410f340 con 0x7f7e441037c0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4410f1c0 con 0x7f7e441049b0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.733+0000 7f7e43200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4410f4c0 con 0x7f7e44105370 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 0x7f7e441999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 0x7f7e441999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49008/0 (socket says 172.21.15.137:49008) 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 -- 172.21.15.137:0/3428682723 learned_addr learned my addr 172.21.15.137:0/3428682723 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e3be00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e441049b0 0x7f7e44199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e42800640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e44105370 0x7f7e441a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 -- 172.21.15.137:0/3428682723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e44105370 msgr2=0x7f7e441a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e44105370 0x7f7e441a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 -- 172.21.15.137:0/3428682723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e441049b0 msgr2=0x7f7e44199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e441049b0 0x7f7e44199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 -- 172.21.15.137:0/3428682723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e441a15e0 con 0x7f7e441037c0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.734+0000 7f7e41e00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 0x7f7e441999c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400d770 tx=0x7f7e3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:44.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.735+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34017070 con 0x7f7e441037c0 2024-03-26T13:59:44.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.735+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e34009c30 con 0x7f7e441037c0 2024-03-26T13:59:44.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.735+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e441a18d0 con 0x7f7e441037c0 2024-03-26T13:59:44.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.735+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34003d00 con 0x7f7e441037c0 2024-03-26T13:59:44.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.735+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e441a1e40 con 0x7f7e441037c0 2024-03-26T13:59:44.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.736+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4410ab60 con 0x7f7e441037c0 2024-03-26T13:59:44.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.738+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7e34003ea0 con 0x7f7e441037c0 2024-03-26T13:59:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.739+0000 7f7e3aa00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e280a2880 0x7f7e280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.739+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7e340cea20 con 0x7f7e441037c0 2024-03-26T13:59:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.739+0000 7f7e3be00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e280a2880 0x7f7e280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.740+0000 7f7e3be00640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e280a2880 0x7f7e280a4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c006010 tx=0x7f7e2c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.743+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e3409b880 con 0x7f7e441037c0 2024-03-26T13:59:44.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.954+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e4402d070 con 0x7f7e441037c0 2024-03-26T13:59:44.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.955+0000 7f7e3aa00640 1 -- 172.21.15.137:0/3428682723 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) v1 ==== 81+0+600 (secure 0 0 0) 0x7f7e34009de0 con 0x7f7e441037c0 2024-03-26T13:59:44.951 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:44.951 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:44.951 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/460394311 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1426358112 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2012111099 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1602182932 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3592180121 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/487629871 2024-03-27T13:49:52.910355+0000 2024-03-26T13:59:44.952 INFO:teuthology.orchestra.run.smithi137.stderr:listed 10 entries 2024-03-26T13:59:44.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e280a2880 msgr2=0x7f7e280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e280a2880 0x7f7e280a4d40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c006010 tx=0x7f7e2c000bc0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 msgr2=0x7f7e441999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:44.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 0x7f7e441999c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400d770 tx=0x7f7e3400dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:44.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 shutdown_connections 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e280a2880 0x7f7e280a4d40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e441049b0 0x7f7e44199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e44105370 0x7f7e441a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 --2- 172.21.15.137:0/3428682723 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e441037c0 0x7f7e441999c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 >> 172.21.15.137:0/3428682723 conn(0x7f7e440fed50 msgr2=0x7f7e44107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 shutdown_connections 2024-03-26T13:59:44.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:44.958+0000 7f7e43200640 1 -- 172.21.15.137:0/3428682723 wait complete. 2024-03-26T13:59:44.971 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi137.front.sepia.ceph.com... 2024-03-26T13:59:44.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:44.971 DEBUG:teuthology.orchestra.run.smithi137:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-26T13:59:45.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi137.stderr:ceph-fuse[75619]: fuse finished with error 0 and tester_r 0 2024-03-26T13:59:45.060 INFO:teuthology.orchestra.run:waiting for 300 2024-03-26T13:59:45.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T13:59:45.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T13:59:45.152 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-26T13:59:45.218 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:45.237 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:45 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3768956385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T13:59:45.237 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:45 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3428682723' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T13:59:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.436+0000 7fdbc2800640 1 -- 172.21.15.137:0/4281354315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 msgr2=0x7fdbbc10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.436+0000 7fdbc2800640 1 --2- 172.21.15.137:0/4281354315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc10bd20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009ca0 tx=0x7fdbb402c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 -- 172.21.15.137:0/4281354315 shutdown_connections 2024-03-26T13:59:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 --2- 172.21.15.137:0/4281354315 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc10bd20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 --2- 172.21.15.137:0/4281354315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc104ad0 0x7fdbbc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 --2- 172.21.15.137:0/4281354315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbbc1038e0 0x7fdbbc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 -- 172.21.15.137:0/4281354315 >> 172.21.15.137:0/4281354315 conn(0x7fdbbc0fee90 msgr2=0x7fdbbc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 -- 172.21.15.137:0/4281354315 shutdown_connections 2024-03-26T13:59:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.437+0000 7fdbc2800640 1 -- 172.21.15.137:0/4281354315 wait complete. 2024-03-26T13:59:45.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 Processor -- start 2024-03-26T13:59:45.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 -- start start 2024-03-26T13:59:45.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbbc1038e0 0x7fdbbc199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc104ad0 0x7fdbbc199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc10f3f0 con 0x7fdbbc105490 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc10f270 con 0x7fdbbc1038e0 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbc2800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc10f570 con 0x7fdbbc104ad0 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.438+0000 7fdbbbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc1a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49022/0 (socket says 172.21.15.137:49022) 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 -- 172.21.15.137:0/751307490 learned_addr learned my addr 172.21.15.137:0/751307490 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbb400640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbbc1038e0 0x7fdbbc199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbb3e00640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc104ad0 0x7fdbbc199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 -- 172.21.15.137:0/751307490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc104ad0 msgr2=0x7fdbbc199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc104ad0 0x7fdbbc199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 -- 172.21.15.137:0/751307490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbbc1038e0 msgr2=0x7fdbbc199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbbc1038e0 0x7fdbbc199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 -- 172.21.15.137:0/751307490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc1a1620 con 0x7fdbbc105490 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbbbe00640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc1a0fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4032380 tx=0x7fdbb4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.439+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4041020 con 0x7fdbbc105490 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.440+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdbb4003e20 con 0x7fdbbc105490 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.440+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc1a18b0 con 0x7fdbbc105490 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.440+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4039040 con 0x7fdbbc105490 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.440+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc1a1da0 con 0x7fdbbc105490 2024-03-26T13:59:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.441+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc10ac00 con 0x7fdbbc105490 2024-03-26T13:59:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.443+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdbb4044020 con 0x7fdbbc105490 2024-03-26T13:59:45.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.444+0000 7fdbb8c00640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdba00a2800 0x7fdba00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.444+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fdbb40f1060 con 0x7fdbbc105490 2024-03-26T13:59:45.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.444+0000 7fdbbb400640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdba00a2800 0x7fdba00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.445+0000 7fdbbb400640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdba00a2800 0x7fdba00a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005fd0 tx=0x7fdbac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:45.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.448+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdbb404b020 con 0x7fdbbc105490 2024-03-26T13:59:45.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:45 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3768956385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T13:59:45.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:45 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3428682723' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T13:59:45.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.648+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbbc062340 con 0x7fdbbc105490 2024-03-26T13:59:45.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.651+0000 7fdbb8c00640 1 -- 172.21.15.137:0/751307490 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7fdbb40bde60 con 0x7fdbbc105490 2024-03-26T13:59:45.648 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:45.648 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:45.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdba00a2800 msgr2=0x7fdba00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdba00a2800 0x7fdba00a4cc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005fd0 tx=0x7fdbac005850 comp rx=0 tx=0).stop 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 msgr2=0x7fdbbc1a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc1a0fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4032380 tx=0x7fdbb4032820 comp rx=0 tx=0).stop 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 shutdown_connections 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdba00a2800 0x7fdba00a4cc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdbbc105490 0x7fdbbc1a0fb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdbbc104ad0 0x7fdbbc199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 --2- 172.21.15.137:0/751307490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbbc1038e0 0x7fdbbc199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 >> 172.21.15.137:0/751307490 conn(0x7fdbbc0fee90 msgr2=0x7fdbbc107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.654+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 shutdown_connections 2024-03-26T13:59:45.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.655+0000 7fdbc2800640 1 -- 172.21.15.137:0/751307490 wait complete. 2024-03-26T13:59:45.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:45.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:45 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3768956385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T13:59:45.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:45 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3428682723' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T13:59:45.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.929+0000 7f0eba000640 1 -- 172.21.15.137:0/594784636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb40756e0 msgr2=0x7f0eb4073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.929+0000 7f0eba000640 1 --2- 172.21.15.137:0/594784636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb40756e0 0x7f0eb4073b30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009cd0 tx=0x7f0eac02c0e0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.929+0000 7f0eba000640 1 -- 172.21.15.137:0/594784636 shutdown_connections 2024-03-26T13:59:45.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.929+0000 7f0eba000640 1 --2- 172.21.15.137:0/594784636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb410dae0 0x7f0eb410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.929+0000 7f0eba000640 1 --2- 172.21.15.137:0/594784636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb4074100 0x7f0eb4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.929+0000 7f0eba000640 1 --2- 172.21.15.137:0/594784636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb40756e0 0x7f0eb4073b30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.930+0000 7f0eba000640 1 -- 172.21.15.137:0/594784636 >> 172.21.15.137:0/594784636 conn(0x7f0eb40fca80 msgr2=0x7f0eb40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:45.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.930+0000 7f0eba000640 1 -- 172.21.15.137:0/594784636 shutdown_connections 2024-03-26T13:59:45.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.930+0000 7f0eba000640 1 -- 172.21.15.137:0/594784636 wait complete. 2024-03-26T13:59:45.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.930+0000 7f0eba000640 1 Processor -- start 2024-03-26T13:59:45.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 -- start start 2024-03-26T13:59:45.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 0x7f0eb419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb40756e0 0x7f0eb419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb410dae0 0x7f0eb41a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4113630 con 0x7f0eb4074100 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb41134b0 con 0x7f0eb410dae0 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eba000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb41137b0 con 0x7f0eb40756e0 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eb8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 0x7f0eb419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eb8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 0x7f0eb419dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49036/0 (socket says 172.21.15.137:49036) 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.931+0000 7f0eb8c00640 1 -- 172.21.15.137:0/1644607925 learned_addr learned my addr 172.21.15.137:0/1644607925 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb3e00640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb40756e0 0x7f0eb419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb8c00640 1 -- 172.21.15.137:0/1644607925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb40756e0 msgr2=0x7f0eb419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb9600640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb410dae0 0x7f0eb41a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb8c00640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb40756e0 0x7f0eb419e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb8c00640 1 -- 172.21.15.137:0/1644607925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb410dae0 msgr2=0x7f0eb41a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb8c00640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb410dae0 0x7f0eb41a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb8c00640 1 -- 172.21.15.137:0/1644607925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb41a5960 con 0x7f0eb4074100 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb8c00640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 0x7f0eb419dd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000f80 tx=0x7f0eac035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac039040 con 0x7f0eb4074100 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0eac044070 con 0x7f0eb4074100 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac004560 con 0x7f0eb4074100 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb41a5bf0 con 0x7f0eb4074100 2024-03-26T13:59:45.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.932+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb41a60e0 con 0x7f0eb4074100 2024-03-26T13:59:45.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.934+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb4103cb0 con 0x7f0eb4074100 2024-03-26T13:59:45.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.936+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0eac004700 con 0x7f0eb4074100 2024-03-26T13:59:45.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.938+0000 7f0eb1600640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2800 0x7f0e980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:45.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.938+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f0eac0f14d0 con 0x7f0eb4074100 2024-03-26T13:59:45.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.938+0000 7f0eb3e00640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2800 0x7f0e980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:45.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.939+0000 7f0eb3e00640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2800 0x7f0e980a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005fd0 tx=0x7f0e9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:45.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:45.942+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0eac0be2d0 con 0x7f0eb4074100 2024-03-26T13:59:46.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.142+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0eb40622d0 con 0x7f0eb4074100 2024-03-26T13:59:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.144+0000 7f0eb1600640 1 -- 172.21.15.137:0/1644607925 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24198 (secure 0 0 0) 0x7f0eac0bdbd0 con 0x7f0eb4074100 2024-03-26T13:59:46.141 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:46.141 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":90,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:27.737432+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2800 msgr2=0x7f0e980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2800 0x7f0e980a4cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005fd0 tx=0x7f0e9c005850 comp rx=0 tx=0).stop 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 msgr2=0x7f0eb419dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 0x7f0eb419dd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000f80 tx=0x7f0eac035d60 comp rx=0 tx=0).stop 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 shutdown_connections 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2800 0x7f0e980a4cc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb410dae0 0x7f0eb41a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb40756e0 0x7f0eb419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 --2- 172.21.15.137:0/1644607925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4074100 0x7f0eb419dd50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 >> 172.21.15.137:0/1644607925 conn(0x7f0eb40fca80 msgr2=0x7f0eb4100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 shutdown_connections 2024-03-26T13:59:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.147+0000 7f0eba000640 1 -- 172.21.15.137:0/1644607925 wait complete. 2024-03-26T13:59:46.162 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-26T13:59:46.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 -- 172.21.15.137:0/831668951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327010db70 msgr2=0x7f327010ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:46.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 --2- 172.21.15.137:0/831668951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327010db70 0x7f327010ff60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3268009930 tx=0x7f326802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 -- 172.21.15.137:0/831668951 shutdown_connections 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 --2- 172.21.15.137:0/831668951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f327010db70 0x7f327010ff60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 --2- 172.21.15.137:0/831668951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3270074100 0x7f3270074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 --2- 172.21.15.137:0/831668951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32700756e0 0x7f3270073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 -- 172.21.15.137:0/831668951 >> 172.21.15.137:0/831668951 conn(0x7f32700fcb10 msgr2=0x7f32700fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 -- 172.21.15.137:0/831668951 shutdown_connections 2024-03-26T13:59:46.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.380+0000 7f3275a00640 1 -- 172.21.15.137:0/831668951 wait complete. 2024-03-26T13:59:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 Processor -- start 2024-03-26T13:59:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 -- start start 2024-03-26T13:59:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3270074100 0x7f327019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 0x7f327019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327010db70 0x7f32701a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3270113710 con 0x7f32700756e0 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3270113590 con 0x7f327010db70 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.381+0000 7f3275a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3270113890 con 0x7f3270074100 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 0x7f327019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 0x7f327019e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49060/0 (socket says 172.21.15.137:49060) 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 -- 172.21.15.137:0/2365422726 learned_addr learned my addr 172.21.15.137:0/2365422726 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f3275000640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327010db70 0x7f32701a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326fe00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3270074100 0x7f327019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 -- 172.21.15.137:0/2365422726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3270074100 msgr2=0x7f327019dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3270074100 0x7f327019dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 -- 172.21.15.137:0/2365422726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327010db70 msgr2=0x7f32701a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327010db70 0x7f32701a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.382+0000 7f326f400640 1 -- 172.21.15.137:0/2365422726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32701a5a00 con 0x7f32700756e0 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.383+0000 7f326f400640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 0x7f327019e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f325800e9f0 tx=0x7f325800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.383+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325800cd80 con 0x7f32700756e0 2024-03-26T13:59:46.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.383+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32701a5cf0 con 0x7f32700756e0 2024-03-26T13:59:46.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.383+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32580054d0 con 0x7f32700756e0 2024-03-26T13:59:46.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.383+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258010490 con 0x7f32700756e0 2024-03-26T13:59:46.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.383+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32701a61e0 con 0x7f32700756e0 2024-03-26T13:59:46.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.384+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3270103d40 con 0x7f32700756e0 2024-03-26T13:59:46.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.386+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3258005020 con 0x7f32700756e0 2024-03-26T13:59:46.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.388+0000 7f326cc00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32540a2800 0x7f32540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:46.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.388+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f32580cf1a0 con 0x7f32700756e0 2024-03-26T13:59:46.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.388+0000 7f326fe00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32540a2800 0x7f32540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:46.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.388+0000 7f326fe00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32540a2800 0x7f32540a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f3260009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.392+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f325809bfa0 con 0x7f32700756e0 2024-03-26T13:59:46.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:46 smithi159 ceph-mon[35501]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-26T13:59:46.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:46 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/751307490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:46.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:46 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1644607925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:46.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:46 smithi179 ceph-mon[35472]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-26T13:59:46.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:46 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/751307490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:46.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:46 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1644607925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:46.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:46 smithi137 ceph-mon[31539]: pgmap v376: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-03-26T13:59:46.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:46 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/751307490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:46.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:46 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1644607925' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:46.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:46.629+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f32700622a0 con 0x7f32700756e0 2024-03-26T13:59:47.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.208+0000 7f326cc00640 1 -- 172.21.15.137:0/2365422726 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f325809b8a0 con 0x7f32700756e0 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.211+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32540a2800 msgr2=0x7f32540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.211+0000 7f3275a00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32540a2800 0x7f32540a4cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f3260009500 comp rx=0 tx=0).stop 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.211+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 msgr2=0x7f327019e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 0x7f327019e320 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f325800e9f0 tx=0x7f325800edf0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 shutdown_connections 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32540a2800 0x7f32540a4cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f327010db70 0x7f32701a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32700756e0 0x7f327019e320 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 --2- 172.21.15.137:0/2365422726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3270074100 0x7f327019dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 >> 172.21.15.137:0/2365422726 conn(0x7f32700fcb10 msgr2=0x7f3270100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:47.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 shutdown_connections 2024-03-26T13:59:47.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.212+0000 7f3275a00640 1 -- 172.21.15.137:0/2365422726 wait complete. 2024-03-26T13:59:47.232 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-03-26T13:59:47.233 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-26T13:59:47.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:47 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2365422726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.450+0000 7f1525a00640 1 -- 172.21.15.137:0/1339249895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520076060 msgr2=0x7f1520076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.450+0000 7f1525a00640 1 --2- 172.21.15.137:0/1339249895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520076060 0x7f1520076480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1510009900 tx=0x7f151002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.450+0000 7f1525a00640 1 -- 172.21.15.137:0/1339249895 shutdown_connections 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.451+0000 7f1525a00640 1 --2- 172.21.15.137:0/1339249895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15200772a0 0x7f1520077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.451+0000 7f1525a00640 1 --2- 172.21.15.137:0/1339249895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1520077c40 0x7f15200784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.451+0000 7f1525a00640 1 --2- 172.21.15.137:0/1339249895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1520076060 0x7f1520076480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.451+0000 7f1525a00640 1 -- 172.21.15.137:0/1339249895 >> 172.21.15.137:0/1339249895 conn(0x7f15200fee50 msgr2=0x7f1520101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.451+0000 7f1525a00640 1 -- 172.21.15.137:0/1339249895 shutdown_connections 2024-03-26T13:59:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.451+0000 7f1525a00640 1 -- 172.21.15.137:0/1339249895 wait complete. 2024-03-26T13:59:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 Processor -- start 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 -- start start 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1520076060 0x7f152019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 0x7f152019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1520077c40 0x7f15201a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1520113820 con 0x7f15200772a0 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15201136a0 con 0x7f1520077c40 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1525a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15201139a0 con 0x7f1520076060 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1516000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 0x7f152019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1516000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 0x7f152019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49068/0 (socket says 172.21.15.137:49068) 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.452+0000 7f1516000640 1 -- 172.21.15.137:0/3935617929 learned_addr learned my addr 172.21.15.137:0/3935617929 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f151ea00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1520076060 0x7f152019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1516000640 1 -- 172.21.15.137:0/3935617929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1520076060 msgr2=0x7f152019df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1516000640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1520076060 0x7f152019df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1516000640 1 -- 172.21.15.137:0/3935617929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1520077c40 msgr2=0x7f15201a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1516000640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1520077c40 0x7f15201a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1516000640 1 -- 172.21.15.137:0/3935617929 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15201a5b50 con 0x7f15200772a0 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1516000640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 0x7f152019e470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f150800d950 tx=0x7f150800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508017070 con 0x7f15200772a0 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1508009c30 con 0x7f15200772a0 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800f620 con 0x7f15200772a0 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15201a5e40 con 0x7f15200772a0 2024-03-26T13:59:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.453+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15201a6300 con 0x7f15200772a0 2024-03-26T13:59:47.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:47 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2365422726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T13:59:47.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.455+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1520110250 con 0x7f15200772a0 2024-03-26T13:59:47.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.456+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f15080109a0 con 0x7f15200772a0 2024-03-26T13:59:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.458+0000 7f1517e00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15040a2800 0x7f15040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.458+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f15080cea00 con 0x7f15200772a0 2024-03-26T13:59:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.458+0000 7f151ea00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15040a2800 0x7f15040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:47.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.459+0000 7f151ea00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15040a2800 0x7f15040a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15100096c0 tx=0x7f1510000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:47.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.462+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1508009de0 con 0x7f15200772a0 2024-03-26T13:59:47.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:47 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2365422726' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-26T13:59:47.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:47.703+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f1520062340 con 0x7f15200772a0 2024-03-26T13:59:48.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.211+0000 7f1517e00640 1 -- 172.21.15.137:0/3935617929 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f150809b800 con 0x7f15200772a0 2024-03-26T13:59:48.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15040a2800 msgr2=0x7f15040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:48.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15040a2800 0x7f15040a4cc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15100096c0 tx=0x7f1510000bc0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 msgr2=0x7f152019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:48.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 0x7f152019e470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f150800d950 tx=0x7f150800de10 comp rx=0 tx=0).stop 2024-03-26T13:59:48.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 shutdown_connections 2024-03-26T13:59:48.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15040a2800 0x7f15040a4cc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1520077c40 0x7f15201a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15200772a0 0x7f152019e470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 --2- 172.21.15.137:0/3935617929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1520076060 0x7f152019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 >> 172.21.15.137:0/3935617929 conn(0x7f15200fee50 msgr2=0x7f1520100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:48.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 shutdown_connections 2024-03-26T13:59:48.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.217+0000 7f1525a00640 1 -- 172.21.15.137:0/3935617929 wait complete. 2024-03-26T13:59:48.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-26T13:59:48.314 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:48 smithi137 ceph-mon[31539]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-26T13:59:48.314 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:48 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2365422726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T13:59:48.314 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:48 smithi137 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:48.314 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:48 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3935617929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-26T13:59:48.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 -- 172.21.15.137:0/1456753802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 msgr2=0x7fb438102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 --2- 172.21.15.137:0/1456753802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438102d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb424009900 tx=0x7fb42402bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 -- 172.21.15.137:0/1456753802 shutdown_connections 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 --2- 172.21.15.137:0/1456753802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103260 0x7fb438109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 --2- 172.21.15.137:0/1456753802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438102d20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 --2- 172.21.15.137:0/1456753802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 0x7fb438101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 -- 172.21.15.137:0/1456753802 >> 172.21.15.137:0/1456753802 conn(0x7fb4380fcc40 msgr2=0x7fb4380ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 -- 172.21.15.137:0/1456753802 shutdown_connections 2024-03-26T13:59:48.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.449+0000 7fb43d800640 1 -- 172.21.15.137:0/1456753802 wait complete. 2024-03-26T13:59:48.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.450+0000 7fb43d800640 1 Processor -- start 2024-03-26T13:59:48.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 -- start start 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 0x7fb438197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103260 0x7fb43819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43810d230 con 0x7fb4381028a0 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43810d0b0 con 0x7fb4381016b0 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb43d800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43810d3b0 con 0x7fb438103260 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb436000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb436000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36246/0 (socket says 172.21.15.137:36246) 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.451+0000 7fb436000640 1 -- 172.21.15.137:0/2878863173 learned_addr learned my addr 172.21.15.137:0/2878863173 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436a00640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 0x7fb438197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb437400640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103260 0x7fb43819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436000640 1 -- 172.21.15.137:0/2878863173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103260 msgr2=0x7fb43819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436000640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103260 0x7fb43819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436000640 1 -- 172.21.15.137:0/2878863173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 msgr2=0x7fb438197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436000640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 0x7fb438197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436000640 1 -- 172.21.15.137:0/2878863173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43819f540 con 0x7fb4381028a0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436a00640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 0x7fb438197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb436000640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438197e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb424000c00 tx=0x7fb4240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424043020 con 0x7fb4381028a0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43819f7d0 con 0x7fb4381028a0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4240054c0 con 0x7fb4381028a0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb424003d40 con 0x7fb4381028a0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.452+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43819fd20 con 0x7fb4381028a0 2024-03-26T13:59:48.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.454+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438108ac0 con 0x7fb4381028a0 2024-03-26T13:59:48.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.455+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb424046020 con 0x7fb4381028a0 2024-03-26T13:59:48.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.457+0000 7fb42f400640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb41c0a2800 0x7fb41c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:48.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.457+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fb4240f11a0 con 0x7fb4381028a0 2024-03-26T13:59:48.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.457+0000 7fb436a00640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb41c0a2800 0x7fb41c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:48.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.458+0000 7fb436a00640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb41c0a2800 0x7fb41c0a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb438102700 tx=0x7fb428005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:48.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.461+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb42404d020 con 0x7fb4381028a0 2024-03-26T13:59:48.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:48 smithi179 ceph-mon[35472]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-26T13:59:48.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:48 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2365422726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T13:59:48.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:48 smithi179 ceph-mon[35472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:48.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:48 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3935617929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-26T13:59:48.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:48.701+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fb43802d070 con 0x7fb4381028a0 2024-03-26T13:59:48.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:48 smithi159 ceph-mon[35501]: pgmap v377: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-03-26T13:59:48.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:48 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2365422726' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-26T13:59:48.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:48 smithi159 ceph-mon[35501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:48.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:48 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3935617929' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-03-26T13:59:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.224+0000 7fb42f400640 1 -- 172.21.15.137:0/2878863173 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fb4240be0c0 con 0x7fb4381028a0 2024-03-26T13:59:49.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.227+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb41c0a2800 msgr2=0x7fb41c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:49.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.227+0000 7fb43d800640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb41c0a2800 0x7fb41c0a4cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb438102700 tx=0x7fb428005e70 comp rx=0 tx=0).stop 2024-03-26T13:59:49.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.227+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 msgr2=0x7fb438197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438197e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb424000c00 tx=0x7fb4240042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 shutdown_connections 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb41c0a2800 0x7fb41c0a4cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb438103260 0x7fb43819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4381028a0 0x7fb438197e60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 --2- 172.21.15.137:0/2878863173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4381016b0 0x7fb438197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 >> 172.21.15.137:0/2878863173 conn(0x7fb4380fcc40 msgr2=0x7fb438105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 shutdown_connections 2024-03-26T13:59:49.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.228+0000 7fb43d800640 1 -- 172.21.15.137:0/2878863173 wait complete. 2024-03-26T13:59:49.254 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 -- 172.21.15.137:0/1466605724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 msgr2=0x7f9a28104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 --2- 172.21.15.137:0/1466605724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28104e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009900 tx=0x7f9a1002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 -- 172.21.15.137:0/1466605724 shutdown_connections 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 --2- 172.21.15.137:0/1466605724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a28105370 0x7f9a2810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 --2- 172.21.15.137:0/1466605724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28104e30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 --2- 172.21.15.137:0/1466605724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 0x7f9a28103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 -- 172.21.15.137:0/1466605724 >> 172.21.15.137:0/1466605724 conn(0x7f9a280fed50 msgr2=0x7f9a28101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 -- 172.21.15.137:0/1466605724 shutdown_connections 2024-03-26T13:59:49.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.455+0000 7f9a2d200640 1 -- 172.21.15.137:0/1466605724 wait complete. 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.456+0000 7f9a2d200640 1 Processor -- start 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 -- start start 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 0x7f9a281999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a28105370 0x7f9a281a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810f340 con 0x7f9a281049b0 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810f1c0 con 0x7f9a281037c0 2024-03-26T13:59:49.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a2d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2810f4c0 con 0x7f9a28105370 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a1fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a1fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36264/0 (socket says 172.21.15.137:36264) 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.457+0000 7f9a1fe00640 1 -- 172.21.15.137:0/177098781 learned_addr learned my addr 172.21.15.137:0/177098781 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a27400640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 0x7f9a281999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a27e00640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a28105370 0x7f9a281a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a1fe00640 1 -- 172.21.15.137:0/177098781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a28105370 msgr2=0x7f9a281a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a1fe00640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a28105370 0x7f9a281a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a1fe00640 1 -- 172.21.15.137:0/177098781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 msgr2=0x7f9a281999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a1fe00640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 0x7f9a281999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a1fe00640 1 -- 172.21.15.137:0/177098781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a281a15e0 con 0x7f9a281049b0 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a27400640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 0x7f9a281999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a1fe00640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28199f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:49.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10043020 con 0x7f9a281049b0 2024-03-26T13:59:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a10004aa0 con 0x7f9a281049b0 2024-03-26T13:59:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.458+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10003d00 con 0x7f9a281049b0 2024-03-26T13:59:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.459+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a281a1870 con 0x7f9a281049b0 2024-03-26T13:59:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.459+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a281a1d70 con 0x7f9a281049b0 2024-03-26T13:59:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.460+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2810ab60 con 0x7f9a281049b0 2024-03-26T13:59:49.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.462+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9a10046020 con 0x7f9a281049b0 2024-03-26T13:59:49.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.463+0000 7f9a24c00640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2880 0x7f9a0c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:49.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.463+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f9a100f1080 con 0x7f9a281049b0 2024-03-26T13:59:49.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.463+0000 7f9a27400640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2880 0x7f9a0c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:49.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.464+0000 7f9a27400640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2880 0x7f9a0c0a4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a28104810 tx=0x7f9a18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:49.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.468+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9a1004d020 con 0x7f9a281049b0 2024-03-26T13:59:49.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T13:59:49.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3935617929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-26T13:59:49.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T13:59:49.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:49.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T13:59:49.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T13:59:49.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:49 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2878863173' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T13:59:49.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T13:59:49.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3935617929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-26T13:59:49.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T13:59:49.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:49.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T13:59:49.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T13:59:49.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:49 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2878863173' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T13:59:49.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-26T13:59:49.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3935617929' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-03-26T13:59:49.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-26T13:59:49.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-26T13:59:49.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-26T13:59:49.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-26T13:59:49.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:49 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2878863173' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-26T13:59:49.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:49.708+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f9a280622a0 con 0x7f9a281049b0 2024-03-26T13:59:50.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.239+0000 7f9a24c00640 1 -- 172.21.15.137:0/177098781 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f9a100bdfa0 con 0x7f9a281049b0 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2880 msgr2=0x7f9a0c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2880 0x7f9a0c0a4d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a28104810 tx=0x7f9a18005f90 comp rx=0 tx=0).stop 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 msgr2=0x7f9a28199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28199f00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a100042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 shutdown_connections 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2880 0x7f9a0c0a4d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a28105370 0x7f9a281a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a281049b0 0x7f9a28199f00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 --2- 172.21.15.137:0/177098781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a281037c0 0x7f9a281999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.242+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 >> 172.21.15.137:0/177098781 conn(0x7f9a280fed50 msgr2=0x7f9a28107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:50.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.243+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 shutdown_connections 2024-03-26T13:59:50.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.243+0000 7f9a2d200640 1 -- 172.21.15.137:0/177098781 wait complete. 2024-03-26T13:59:50.261 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-03-26T13:59:50.261 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-26T13:59:50.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.476+0000 7fd2e9000640 1 -- 172.21.15.137:0/1038417452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4107ce0 msgr2=0x7fd2e4108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.476+0000 7fd2e9000640 1 --2- 172.21.15.137:0/1038417452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4107ce0 0x7fd2e4108100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009860 tx=0x7fd2cc02bb70 comp rx=0 tx=0).stop 2024-03-26T13:59:50.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 -- 172.21.15.137:0/1038417452 shutdown_connections 2024-03-26T13:59:50.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 --2- 172.21.15.137:0/1038417452 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4109890 0x7fd2e4110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 --2- 172.21.15.137:0/1038417452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2e4108ed0 0x7fd2e4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 --2- 172.21.15.137:0/1038417452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4107ce0 0x7fd2e4108100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 -- 172.21.15.137:0/1038417452 >> 172.21.15.137:0/1038417452 conn(0x7fd2e4076c90 msgr2=0x7fd2e40790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:50.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 -- 172.21.15.137:0/1038417452 shutdown_connections 2024-03-26T13:59:50.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.477+0000 7fd2e9000640 1 -- 172.21.15.137:0/1038417452 wait complete. 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.478+0000 7fd2e9000640 1 Processor -- start 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.478+0000 7fd2e9000640 1 -- start start 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 0x7fd2e419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4108ed0 0x7fd2e419e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e9000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2e4109890 0x7fd2e41a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e9000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4113800 con 0x7fd2e4107ce0 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4113680 con 0x7fd2e4108ed0 2024-03-26T13:59:50.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e9000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4113980 con 0x7fd2e4109890 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 0x7fd2e419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 0x7fd2e419df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36272/0 (socket says 172.21.15.137:36272) 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 -- 172.21.15.137:0/3941388913 learned_addr learned my addr 172.21.15.137:0/3941388913 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2d9400640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4108ed0 0x7fd2e419e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e2800640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2e4109890 0x7fd2e41a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 -- 172.21.15.137:0/3941388913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2e4109890 msgr2=0x7fd2e41a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2e4109890 0x7fd2e41a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 -- 172.21.15.137:0/3941388913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4108ed0 msgr2=0x7fd2e419e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4108ed0 0x7fd2e419e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.479+0000 7fd2e1e00640 1 -- 172.21.15.137:0/3941388913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e41a5b20 con 0x7fd2e4107ce0 2024-03-26T13:59:50.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.480+0000 7fd2e1e00640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 0x7fd2e419df50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0057c0 tx=0x7fd2cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:50.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.480+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc041030 con 0x7fd2e4107ce0 2024-03-26T13:59:50.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.480+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2cc02c9c0 con 0x7fd2e4107ce0 2024-03-26T13:59:50.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.480+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e41a5db0 con 0x7fd2e4107ce0 2024-03-26T13:59:50.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.480+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc039040 con 0x7fd2e4107ce0 2024-03-26T13:59:50.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.480+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e41a61e0 con 0x7fd2e4107ce0 2024-03-26T13:59:50.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.482+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e410f0c0 con 0x7fd2e4107ce0 2024-03-26T13:59:50.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.483+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd2cc044070 con 0x7fd2e4107ce0 2024-03-26T13:59:50.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.485+0000 7fd2db400640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2c80a2800 0x7fd2c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.485+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7fd2cc0f03f0 con 0x7fd2e4107ce0 2024-03-26T13:59:50.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.485+0000 7fd2d9400640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2c80a2800 0x7fd2c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.486+0000 7fd2d9400640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2c80a2800 0x7fd2c80a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005fd0 tx=0x7fd2d4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:50.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.489+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd2cc0bd250 con 0x7fd2e4107ce0 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2878863173' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] up:active 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: Cluster is now healthy 2024-03-26T13:59:50.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T13:59:50.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-26T13:59:50.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-26T13:59:50.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/177098781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T13:59:50.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2878863173' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] up:active 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: Cluster is now healthy 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-26T13:59:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:50 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/177098781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: pgmap v378: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 0 op/s 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2878863173' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] up:active 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: Cluster is now healthy 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-26T13:59:50.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:50 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/177098781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-26T13:59:50.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.718+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2e4061ae0 con 0x7fd2c80a2800 2024-03-26T13:59:50.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.759+0000 7fd2db400640 1 -- 172.21.15.137:0/3941388913 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd2e4061ae0 con 0x7fd2c80a2800 2024-03-26T13:59:50.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2c80a2800 msgr2=0x7fd2c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2c80a2800 0x7fd2c80a4cc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005fd0 tx=0x7fd2d4005850 comp rx=0 tx=0).stop 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 msgr2=0x7fd2e419df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 0x7fd2e419df50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0057c0 tx=0x7fd2cc0042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 shutdown_connections 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2c80a2800 0x7fd2c80a4cc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2e4108ed0 0x7fd2e419e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2e4109890 0x7fd2e41a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 --2- 172.21.15.137:0/3941388913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd2e4107ce0 0x7fd2e419df50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 >> 172.21.15.137:0/3941388913 conn(0x7fd2e4076c90 msgr2=0x7fd2e410c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 shutdown_connections 2024-03-26T13:59:50.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.762+0000 7fd2e9000640 1 -- 172.21.15.137:0/3941388913 wait complete. 2024-03-26T13:59:50.779 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-26T13:59:50.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.995+0000 7efcf0000640 1 -- 172.21.15.137:0/727653519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce8105470 msgr2=0x7efce810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.995+0000 7efcf0000640 1 --2- 172.21.15.137:0/727653519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce8105470 0x7efce810bd00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efce0009ca0 tx=0x7efce002c100 comp rx=0 tx=0).stop 2024-03-26T13:59:50.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 -- 172.21.15.137:0/727653519 shutdown_connections 2024-03-26T13:59:50.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 --2- 172.21.15.137:0/727653519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8104ab0 0x7efce8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 --2- 172.21.15.137:0/727653519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce8105470 0x7efce810bd00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 --2- 172.21.15.137:0/727653519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efce81038c0 0x7efce8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 -- 172.21.15.137:0/727653519 >> 172.21.15.137:0/727653519 conn(0x7efce80fee50 msgr2=0x7efce8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:50.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 -- 172.21.15.137:0/727653519 shutdown_connections 2024-03-26T13:59:50.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.996+0000 7efcf0000640 1 -- 172.21.15.137:0/727653519 wait complete. 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 Processor -- start 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 -- start start 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 0x7efce8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efce8104ab0 0x7efce8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8105470 0x7efce81a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce810f3f0 con 0x7efce81038c0 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce810f270 con 0x7efce8105470 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efcf0000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce810f570 con 0x7efce8104ab0 2024-03-26T13:59:50.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efced600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 0x7efce8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efced600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 0x7efce8199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36300/0 (socket says 172.21.15.137:36300) 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.997+0000 7efced600640 1 -- 172.21.15.137:0/4111871578 learned_addr learned my addr 172.21.15.137:0/4111871578 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efcee000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8105470 0x7efce81a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efcecc00640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efce8104ab0 0x7efce8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efced600640 1 -- 172.21.15.137:0/4111871578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efce8104ab0 msgr2=0x7efce8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efced600640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efce8104ab0 0x7efce8199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efced600640 1 -- 172.21.15.137:0/4111871578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8105470 msgr2=0x7efce81a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efced600640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8105470 0x7efce81a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efced600640 1 -- 172.21.15.137:0/4111871578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce81a16b0 con 0x7efce81038c0 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efcee000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8105470 0x7efce81a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efced600640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 0x7efce8199a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d950 tx=0x7efcdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:50.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc017070 con 0x7efce81038c0 2024-03-26T13:59:50.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efcdc009c30 con 0x7efce81038c0 2024-03-26T13:59:50.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00f620 con 0x7efce81038c0 2024-03-26T13:59:50.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.998+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce81a19a0 con 0x7efce81038c0 2024-03-26T13:59:50.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:50.999+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce81a1ea0 con 0x7efce81038c0 2024-03-26T13:59:50.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.000+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce810abe0 con 0x7efce81038c0 2024-03-26T13:59:50.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.001+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7efcdc01a070 con 0x7efce81038c0 2024-03-26T13:59:50.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.003+0000 7efce6000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efcd00a2800 0x7efcd00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:50.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.003+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7efcdc0ce7e0 con 0x7efce81038c0 2024-03-26T13:59:50.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.003+0000 7efcecc00640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efcd00a2800 0x7efcd00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:51.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.004+0000 7efcecc00640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efcd00a2800 0x7efcd00a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efcd8005fd0 tx=0x7efcd8009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:51.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.007+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efcdc009de0 con 0x7efce81038c0 2024-03-26T13:59:51.032 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:59:50.729+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T13:59:51.032 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:59:50.729+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T13:59:51.032 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:59:50.729+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T13:59:51.032 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:59:50.729+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T13:59:51.032 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 13:59:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T13:59:50.729+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T13:59:51.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.236+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7efce8061b70 con 0x7efcd00a2800 2024-03-26T13:59:51.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.291+0000 7efce6000640 1 -- 172.21.15.137:0/4111871578 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efce8061b70 con 0x7efcd00a2800 2024-03-26T13:59:51.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efcd00a2800 msgr2=0x7efcd00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efcd00a2800 0x7efcd00a4cc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efcd8005fd0 tx=0x7efcd8009470 comp rx=0 tx=0).stop 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 msgr2=0x7efce8199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 0x7efce8199a90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00d950 tx=0x7efcdc00de10 comp rx=0 tx=0).stop 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 shutdown_connections 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efcd00a2800 0x7efcd00a4cc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efce8105470 0x7efce81a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efce8104ab0 0x7efce8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 --2- 172.21.15.137:0/4111871578 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efce81038c0 0x7efce8199a90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 >> 172.21.15.137:0/4111871578 conn(0x7efce80fee50 msgr2=0x7efce8107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 shutdown_connections 2024-03-26T13:59:51.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.295+0000 7efcf0000640 1 -- 172.21.15.137:0/4111871578 wait complete. 2024-03-26T13:59:51.312 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-26T13:59:51.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 -- 172.21.15.137:0/3319702130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 msgr2=0x7f761c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 --2- 172.21.15.137:0/3319702130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c104f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7608009900 tx=0x7f760802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 -- 172.21.15.137:0/3319702130 shutdown_connections 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 --2- 172.21.15.137:0/3319702130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c105490 0x7f761c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 --2- 172.21.15.137:0/3319702130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c104f50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 --2- 172.21.15.137:0/3319702130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c1038e0 0x7f761c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 -- 172.21.15.137:0/3319702130 >> 172.21.15.137:0/3319702130 conn(0x7f761c0fee50 msgr2=0x7f761c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 -- 172.21.15.137:0/3319702130 shutdown_connections 2024-03-26T13:59:51.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.528+0000 7f7624000640 1 -- 172.21.15.137:0/3319702130 wait complete. 2024-03-26T13:59:51.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 Processor -- start 2024-03-26T13:59:51.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 -- start start 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c1038e0 0x7f761c199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c105490 0x7f761c1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c10f460 con 0x7f761c104ad0 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c10f2e0 con 0x7f761c105490 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.529+0000 7f7624000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761c10f5e0 con 0x7f761c1038e0 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36308/0 (socket says 172.21.15.137:36308) 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 -- 172.21.15.137:0/945972834 learned_addr learned my addr 172.21.15.137:0/945972834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7622000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c105490 0x7f761c1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7621600640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c1038e0 0x7f761c199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 -- 172.21.15.137:0/945972834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c1038e0 msgr2=0x7f761c199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c1038e0 0x7f761c199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 -- 172.21.15.137:0/945972834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c105490 msgr2=0x7f761c1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c105490 0x7f761c1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 -- 172.21.15.137:0/945972834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c1a1720 con 0x7f761c104ad0 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7622000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c105490 0x7f761c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.530+0000 7f7620c00640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c19a040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7608000c00 tx=0x7f76080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.531+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608043020 con 0x7f761c104ad0 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.531+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f761c1a19b0 con 0x7f761c104ad0 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.531+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7608004aa0 con 0x7f761c104ad0 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.531+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608003d00 con 0x7f761c104ad0 2024-03-26T13:59:51.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.531+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f761c1a1f50 con 0x7f761c104ad0 2024-03-26T13:59:51.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.532+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f761c0688e0 con 0x7f761c104ad0 2024-03-26T13:59:51.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.534+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7608046020 con 0x7f761c104ad0 2024-03-26T13:59:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/177098781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T13:59:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] up:active 2024-03-26T13:59:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-26T13:59:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T13:59:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:51.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-03-26T13:59:51.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:51 smithi137 ceph-mon[31539]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:51.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.535+0000 7f761a000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76040a2800 0x7f76040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:51.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.535+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f76080f10c0 con 0x7f761c104ad0 2024-03-26T13:59:51.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.536+0000 7f7621600640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76040a2800 0x7f76040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:51.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.536+0000 7f7621600640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76040a2800 0x7f76040a4cc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f761c104930 tx=0x7f7610005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:51.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.539+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f760804d020 con 0x7f761c104ad0 2024-03-26T13:59:51.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/177098781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T13:59:51.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] up:active 2024-03-26T13:59:51.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-26T13:59:51.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T13:59:51.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:51.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-03-26T13:59:51.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:51 smithi179 ceph-mon[35472]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:51.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/177098781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-26T13:59:51.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] up:active 2024-03-26T13:59:51.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-26T13:59:51.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T13:59:51.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:51.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: pgmap v379: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 1 op/s 2024-03-26T13:59:51.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:51 smithi159 ceph-mon[35501]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:51.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.770+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f761c061ba0 con 0x7f76040a2800 2024-03-26T13:59:51.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.782+0000 7f761a000640 1 -- 172.21.15.137:0/945972834 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f761c061ba0 con 0x7f76040a2800 2024-03-26T13:59:51.778 INFO:teuthology.orchestra.run.smithi137.stdout:/volumes/qa/sv_0/9f3c23c9-2b6e-40ba-9f8e-0d9d1fbd6b47 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.784+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76040a2800 msgr2=0x7f76040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.784+0000 7f7624000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76040a2800 0x7f76040a4cc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f761c104930 tx=0x7f7610005f90 comp rx=0 tx=0).stop 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.784+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 msgr2=0x7f761c19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.784+0000 7f7624000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c19a040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7608000c00 tx=0x7f76080042f0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 shutdown_connections 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76040a2800 0x7f76040a4cc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f761c105490 0x7f761c1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761c104ad0 0x7f761c19a040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 --2- 172.21.15.137:0/945972834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f761c1038e0 0x7f761c199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:51.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 >> 172.21.15.137:0/945972834 conn(0x7f761c0fee50 msgr2=0x7f761c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:51.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 shutdown_connections 2024-03-26T13:59:51.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:51.785+0000 7f7624000640 1 -- 172.21.15.137:0/945972834 wait complete. 2024-03-26T13:59:51.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 -- 172.21.15.137:0/687085647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca8102160 msgr2=0x7f7ca81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 --2- 172.21.15.137:0/687085647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca8102160 0x7f7ca81089f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009be0 tx=0x7f7ca002bef0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 -- 172.21.15.137:0/687085647 shutdown_connections 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 --2- 172.21.15.137:0/687085647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ca81017a0 0x7f7ca8101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 --2- 172.21.15.137:0/687085647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca8102160 0x7f7ca81089f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 --2- 172.21.15.137:0/687085647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81005b0 0x7f7ca81009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 -- 172.21.15.137:0/687085647 >> 172.21.15.137:0/687085647 conn(0x7f7ca80fbb40 msgr2=0x7f7ca80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 -- 172.21.15.137:0/687085647 shutdown_connections 2024-03-26T13:59:52.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.026+0000 7f7caf600640 1 -- 172.21.15.137:0/687085647 wait complete. 2024-03-26T13:59:52.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.027+0000 7f7caf600640 1 Processor -- start 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.027+0000 7f7caf600640 1 -- start start 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7caf600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 0x7f7ca8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7caf600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81017a0 0x7f7ca819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7caf600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ca8102160 0x7f7ca81a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7caf600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca810c130 con 0x7f7ca81005b0 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7caf600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca810bfb0 con 0x7f7ca8102160 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7caf600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca810c2b0 con 0x7f7ca81017a0 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7cacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 0x7f7ca8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7cacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 0x7f7ca8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36318/0 (socket says 172.21.15.137:36318) 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7cacc00640 1 -- 172.21.15.137:0/4029903316 learned_addr learned my addr 172.21.15.137:0/4029903316 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7cad600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ca8102160 0x7f7ca81a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7ca7e00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81017a0 0x7f7ca819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7cacc00640 1 -- 172.21.15.137:0/4029903316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81017a0 msgr2=0x7f7ca819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.028+0000 7f7cacc00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81017a0 0x7f7ca819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7cacc00640 1 -- 172.21.15.137:0/4029903316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ca8102160 msgr2=0x7f7ca81a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7cacc00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ca8102160 0x7f7ca81a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7cacc00640 1 -- 172.21.15.137:0/4029903316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca81a16d0 con 0x7f7ca81005b0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7ca7e00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81017a0 0x7f7ca819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7cacc00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 0x7f7ca8199b20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00cd80 con 0x7f7ca81005b0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca81a19c0 con 0x7f7ca81005b0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a1f10 con 0x7f7ca81005b0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7c9c0035f0 con 0x7f7ca81005b0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.029+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c010690 con 0x7f7ca81005b0 2024-03-26T13:59:52.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.030+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c880052c0 con 0x7f7ca81005b0 2024-03-26T13:59:52.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.032+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7c9c0037a0 con 0x7f7ca81005b0 2024-03-26T13:59:52.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.034+0000 7f7ca5600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7c900a2880 0x7f7c900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.034+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f7c9c0cf2d0 con 0x7f7ca81005b0 2024-03-26T13:59:52.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.034+0000 7f7ca7e00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7c900a2880 0x7f7c900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.035+0000 7f7ca7e00640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7c900a2880 0x7f7c900a4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7ca819afc0 tx=0x7f7c94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:52.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.038+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7c9c09c0d0 con 0x7f7ca81005b0 2024-03-26T13:59:52.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.268+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c88000bf0 con 0x7f7c900a2880 2024-03-26T13:59:52.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.284+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f7c9c098040 con 0x7f7ca81005b0 2024-03-26T13:59:52.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.328+0000 7f7ca5600640 1 -- 172.21.15.137:0/4029903316 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7c88000bf0 con 0x7f7c900a2880 2024-03-26T13:59:52.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7c900a2880 msgr2=0x7f7c900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7c900a2880 0x7f7c900a4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7ca819afc0 tx=0x7f7c94009260 comp rx=0 tx=0).stop 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 msgr2=0x7f7ca8199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 0x7f7ca8199b20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9c0 tx=0x7f7c9c00ee80 comp rx=0 tx=0).stop 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 shutdown_connections 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7c900a2880 0x7f7c900a4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ca8102160 0x7f7ca81a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ca81017a0 0x7f7ca819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 --2- 172.21.15.137:0/4029903316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ca81005b0 0x7f7ca8199b20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 >> 172.21.15.137:0/4029903316 conn(0x7f7ca80fbb40 msgr2=0x7f7ca80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 shutdown_connections 2024-03-26T13:59:52.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.331+0000 7f7caf600640 1 -- 172.21.15.137:0/4029903316 wait complete. 2024-03-26T13:59:52.348 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-26T13:59:52.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:52 smithi159 ceph-mon[35501]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:52.449 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: 2024-03-26T13:59:52.159+0000 7fe59f200700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-26T13:59:52.449 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 13:59:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: 2024-03-26T13:59:52.159+0000 7fe59f200700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 -- 172.21.15.137:0/4041298255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 msgr2=0x7f3ce4109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 --2- 172.21.15.137:0/4041298255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce4109af0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009ca0 tx=0x7f3cdc02c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 -- 172.21.15.137:0/4041298255 shutdown_connections 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 --2- 172.21.15.137:0/4041298255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce4109af0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 --2- 172.21.15.137:0/4041298255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce41028a0 0x7f3ce4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 --2- 172.21.15.137:0/4041298255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ce41016b0 0x7f3ce4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 -- 172.21.15.137:0/4041298255 >> 172.21.15.137:0/4041298255 conn(0x7f3ce40fcc40 msgr2=0x7f3ce40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:52.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 -- 172.21.15.137:0/4041298255 shutdown_connections 2024-03-26T13:59:52.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.552+0000 7f3ce9800640 1 -- 172.21.15.137:0/4041298255 wait complete. 2024-03-26T13:59:52.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 Processor -- start 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 -- start start 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce41016b0 0x7f3ce4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ce41028a0 0x7f3ce419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce41a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce410d1e0 con 0x7f3ce4103260 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce410d060 con 0x7f3ce41028a0 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.553+0000 7f3ce9800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce410d360 con 0x7f3ce41016b0 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce41a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce41a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36338/0 (socket says 172.21.15.137:36338) 2024-03-26T13:59:52.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 -- 172.21.15.137:0/1897219405 learned_addr learned my addr 172.21.15.137:0/1897219405 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:52.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce2000640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ce41028a0 0x7f3ce419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce2a00640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce41016b0 0x7f3ce4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 -- 172.21.15.137:0/1897219405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce41016b0 msgr2=0x7f3ce4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce41016b0 0x7f3ce4199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 -- 172.21.15.137:0/1897219405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ce41028a0 msgr2=0x7f3ce419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ce41028a0 0x7f3ce419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 -- 172.21.15.137:0/1897219405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce41a1710 con 0x7f3ce4103260 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.554+0000 7f3ce3400640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce41a10a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc032380 tx=0x7f3cdc032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.555+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc041020 con 0x7f3ce4103260 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.555+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3cdc003e20 con 0x7f3ce4103260 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.555+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce41a19a0 con 0x7f3ce4103260 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.555+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc039040 con 0x7f3ce4103260 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.555+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce41a1ee0 con 0x7f3ce4103260 2024-03-26T13:59:52.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.556+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce41a2110 con 0x7f3ce4103260 2024-03-26T13:59:52.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.558+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3cdc044020 con 0x7f3ce4103260 2024-03-26T13:59:52.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.559+0000 7f3cdb400640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cc80a2910 0x7f3cc80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:52.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.559+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f3cdc0f1000 con 0x7f3ce4103260 2024-03-26T13:59:52.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.560+0000 7f3ce2a00640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cc80a2910 0x7f3cc80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:52.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.560+0000 7f3ce2a00640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cc80a2910 0x7f3cc80a4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd4005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:52.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.564+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3cdc04b020 con 0x7f3ce4103260 2024-03-26T13:59:52.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:52 smithi179 ceph-mon[35472]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:52.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:52 smithi137 ceph-mon[31539]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:52.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.797+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ce4074ff0 con 0x7f3cc80a2910 2024-03-26T13:59:52.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.813+0000 7f3cdb400640 1 -- 172.21.15.137:0/1897219405 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f3ce4074ff0 con 0x7f3cc80a2910 2024-03-26T13:59:52.809 INFO:teuthology.orchestra.run.smithi137.stdout:/volumes/qa/sv_1/37cb2355-7ebb-4842-9d75-642cb02f7cca 2024-03-26T13:59:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.815+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cc80a2910 msgr2=0x7f3cc80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.815+0000 7f3ce9800640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cc80a2910 0x7f3cc80a4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd4005c70 comp rx=0 tx=0).stop 2024-03-26T13:59:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.815+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 msgr2=0x7f3ce41a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.815+0000 7f3ce9800640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce41a10a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc032380 tx=0x7f3cdc032820 comp rx=0 tx=0).stop 2024-03-26T13:59:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 shutdown_connections 2024-03-26T13:59:52.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cc80a2910 0x7f3cc80a4dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ce41028a0 0x7f3ce419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ce4103260 0x7f3ce41a10a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 --2- 172.21.15.137:0/1897219405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce41016b0 0x7f3ce4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 >> 172.21.15.137:0/1897219405 conn(0x7f3ce40fcc40 msgr2=0x7f3ce40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 shutdown_connections 2024-03-26T13:59:52.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:52.816+0000 7f3ce9800640 1 -- 172.21.15.137:0/1897219405 wait complete. 2024-03-26T13:59:52.831 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T13:59:53.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 -- 172.21.15.137:0/389076320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08107420 msgr2=0x7f2a08109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 --2- 172.21.15.137:0/389076320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08107420 0x7f2a08109810 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009ca0 tx=0x7f2a0002c100 comp rx=0 tx=0).stop 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 -- 172.21.15.137:0/389076320 shutdown_connections 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 --2- 172.21.15.137:0/389076320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a08104af0 0x7f2a08106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 --2- 172.21.15.137:0/389076320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a08107420 0x7f2a08109810 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 --2- 172.21.15.137:0/389076320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a081021c0 0x7f2a081045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 -- 172.21.15.137:0/389076320 >> 172.21.15.137:0/389076320 conn(0x7f2a080fbb50 msgr2=0x7f2a080fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 -- 172.21.15.137:0/389076320 shutdown_connections 2024-03-26T13:59:53.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.036+0000 7f2a0e200640 1 -- 172.21.15.137:0/389076320 wait complete. 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.037+0000 7f2a0e200640 1 Processor -- start 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.037+0000 7f2a0e200640 1 -- start start 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a0e200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 0x7f2a08199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a0e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a08104af0 0x7f2a0819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a0e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a08107420 0x7f2a081a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a0e200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0810d230 con 0x7f2a081021c0 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a0e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0810d0b0 con 0x7f2a08107420 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a0e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0810d3b0 con 0x7f2a08104af0 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a07400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 0x7f2a08199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a07400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 0x7f2a08199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36356/0 (socket says 172.21.15.137:36356) 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a07400640 1 -- 172.21.15.137:0/663977605 learned_addr learned my addr 172.21.15.137:0/663977605 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f2a07e00640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a08107420 0x7f2a081a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.038+0000 7f29fea00640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a08104af0 0x7f2a0819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a07400640 1 -- 172.21.15.137:0/663977605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a08104af0 msgr2=0x7f2a0819a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a07400640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a08104af0 0x7f2a0819a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a07400640 1 -- 172.21.15.137:0/663977605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a08107420 msgr2=0x7f2a081a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a07400640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a08107420 0x7f2a081a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a07400640 1 -- 172.21.15.137:0/663977605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a081a16e0 con 0x7f2a081021c0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a07400640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 0x7f2a08199ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f29f800d770 tx=0x7f29f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8017070 con 0x7f2a081021c0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29f80054d0 con 0x7f2a081021c0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f8004cf0 con 0x7f2a081021c0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.039+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a081a19d0 con 0x7f2a081021c0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.040+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a081a1ed0 con 0x7f2a081021c0 2024-03-26T13:59:53.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.041+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08108f10 con 0x7f2a081021c0 2024-03-26T13:59:53.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.043+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f29f8004e90 con 0x7f2a081021c0 2024-03-26T13:59:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.044+0000 7f2a04c00640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29ec0a2890 0x7f29ec0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.044+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8802+0+0 (secure 0 0 0) 0x7f29f80ce8f0 con 0x7f2a081021c0 2024-03-26T13:59:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.044+0000 7f29fea00640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29ec0a2890 0x7f29ec0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.045+0000 7f29fea00640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29ec0a2890 0x7f29ec0a4d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f29f0005fd0 tx=0x7f29f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:53.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.049+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f29f809b590 con 0x7f2a081021c0 2024-03-26T13:59:53.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.284+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a0802d070 con 0x7f2a081021c0 2024-03-26T13:59:53.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.286+0000 7f2a04c00640 1 -- 172.21.15.137:0/663977605 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f29f809ae90 con 0x7f2a081021c0 2024-03-26T13:59:53.282 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:53.282 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:50.245808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34424,34451,34496]},"id":1}]} 2024-03-26T13:59:53.283 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29ec0a2890 msgr2=0x7f29ec0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29ec0a2890 0x7f29ec0a4d50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f29f0005fd0 tx=0x7f29f0005850 comp rx=0 tx=0).stop 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 msgr2=0x7f2a08199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 0x7f2a08199ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f29f800d770 tx=0x7f29f800dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 shutdown_connections 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29ec0a2890 0x7f29ec0a4d50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a08107420 0x7f2a081a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a08104af0 0x7f2a0819a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 --2- 172.21.15.137:0/663977605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a081021c0 0x7f2a08199ac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 >> 172.21.15.137:0/663977605 conn(0x7f2a080fbb50 msgr2=0x7f2a081018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 shutdown_connections 2024-03-26T13:59:53.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.289+0000 7f2a0e200640 1 -- 172.21.15.137:0/663977605 wait complete. 2024-03-26T13:59:53.302 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 -- 172.21.15.137:0/4227194281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe270104ad0 msgr2=0x7fe270104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 --2- 172.21.15.137:0/4227194281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe270104ad0 0x7fe270104f50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009900 tx=0x7fe25c02bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 -- 172.21.15.137:0/4227194281 shutdown_connections 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 --2- 172.21.15.137:0/4227194281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 0x7fe27010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 --2- 172.21.15.137:0/4227194281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe270104ad0 0x7fe270104f50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 --2- 172.21.15.137:0/4227194281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2701038e0 0x7fe270103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 -- 172.21.15.137:0/4227194281 >> 172.21.15.137:0/4227194281 conn(0x7fe2700fee50 msgr2=0x7fe270101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:53.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.492+0000 7fe277400640 1 -- 172.21.15.137:0/4227194281 shutdown_connections 2024-03-26T13:59:53.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.493+0000 7fe277400640 1 -- 172.21.15.137:0/4227194281 wait complete. 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.493+0000 7fe277400640 1 Processor -- start 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.493+0000 7fe277400640 1 -- start start 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe277400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 0x7fe270199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe277400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe270104ad0 0x7fe27019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe277400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 0x7fe2701a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe277400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27010f3f0 con 0x7fe2701038e0 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe277400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27010f270 con 0x7fe270104ad0 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe277400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27010f570 con 0x7fe270105490 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 0x7fe270199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 0x7fe270199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36366/0 (socket says 172.21.15.137:36366) 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 -- 172.21.15.137:0/2929915629 learned_addr learned my addr 172.21.15.137:0/2929915629 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:53.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe26fe00640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe270104ad0 0x7fe27019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe275400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 0x7fe2701a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 -- 172.21.15.137:0/2929915629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 msgr2=0x7fe2701a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 0x7fe2701a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 -- 172.21.15.137:0/2929915629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe270104ad0 msgr2=0x7fe27019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe270104ad0 0x7fe27019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.494+0000 7fe274a00640 1 -- 172.21.15.137:0/2929915629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2701a1740 con 0x7fe2701038e0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe275400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 0x7fe2701a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe274a00640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 0x7fe270199b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe26400d770 tx=0x7fe26400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264017070 con 0x7fe2701038e0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2701a1a30 con 0x7fe2701038e0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe264009c30 con 0x7fe2701038e0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264020060 con 0x7fe2701038e0 2024-03-26T13:59:53.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.495+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2701a1fd0 con 0x7fe2701038e0 2024-03-26T13:59:53.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.496+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27010ac70 con 0x7fe2701038e0 2024-03-26T13:59:53.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.498+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fe264003ca0 con 0x7fe2701038e0 2024-03-26T13:59:53.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.500+0000 7fe26d600640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2580a2890 0x7fe2580a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.500+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe2640ce9d0 con 0x7fe2701038e0 2024-03-26T13:59:53.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.500+0000 7fe26fe00640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2580a2890 0x7fe2580a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.501+0000 7fe26fe00640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2580a2890 0x7fe2580a4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe27019b030 tx=0x7fe25c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:53.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.504+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe26409b5b0 con 0x7fe2701038e0 2024-03-26T13:59:53.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: mgrmap e17: x(active, since 9m), standbys: y, z 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-03-26T13:59:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: pgmap v380: 129 pgs: 129 active+clean; 589 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-03-26T13:59:53.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:53 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/663977605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:53.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.697+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe27002d320 con 0x7fe2701038e0 2024-03-26T13:59:53.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.699+0000 7fe26d600640 1 -- 172.21.15.137:0/2929915629 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24668 (secure 0 0 0) 0x7fe264009de0 con 0x7fe2701038e0 2024-03-26T13:59:53.696 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:53.696 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":91,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:53.294925+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2580a2890 msgr2=0x7fe2580a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2580a2890 0x7fe2580a4d50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe27019b030 tx=0x7fe25c005be0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 msgr2=0x7fe270199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 0x7fe270199b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe26400d770 tx=0x7fe26400dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 shutdown_connections 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2580a2890 0x7fe2580a4d50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe270104ad0 0x7fe27019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe270105490 0x7fe2701a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 --2- 172.21.15.137:0/2929915629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2701038e0 0x7fe270199b20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 >> 172.21.15.137:0/2929915629 conn(0x7fe2700fee50 msgr2=0x7fe270107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:53.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 shutdown_connections 2024-03-26T13:59:53.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.702+0000 7fe277400640 1 -- 172.21.15.137:0/2929915629 wait complete. 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: mgrmap e17: x(active, since 9m), standbys: y, z 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T13:59:53.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-26T13:59:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-26T13:59:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-26T13:59:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-26T13:59:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-03-26T13:59:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: pgmap v380: 129 pgs: 129 active+clean; 589 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-03-26T13:59:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:53 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/663977605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: mgrmap e17: x(active, since 9m), standbys: y, z 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-03-26T13:59:53.713 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: pgmap v380: 129 pgs: 129 active+clean; 589 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-03-26T13:59:53.714 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:53 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/663977605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T13:59:53.716 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-26T13:59:53.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 -- 172.21.15.137:0/3359243440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5778101500 msgr2=0x7f5778101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 --2- 172.21.15.137:0/3359243440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5778101500 0x7f5778101920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5768009ca0 tx=0x7f576802c100 comp rx=0 tx=0).stop 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 -- 172.21.15.137:0/3359243440 shutdown_connections 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 --2- 172.21.15.137:0/3359243440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57781030b0 0x7f5778109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 --2- 172.21.15.137:0/3359243440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781026f0 0x7f5778102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 --2- 172.21.15.137:0/3359243440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5778101500 0x7f5778101920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.921+0000 7f5777e00640 1 -- 172.21.15.137:0/3359243440 >> 172.21.15.137:0/3359243440 conn(0x7f57780fca70 msgr2=0x7f57780feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.922+0000 7f5777e00640 1 -- 172.21.15.137:0/3359243440 shutdown_connections 2024-03-26T13:59:53.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.922+0000 7f5777e00640 1 -- 172.21.15.137:0/3359243440 wait complete. 2024-03-26T13:59:53.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.922+0000 7f5777e00640 1 Processor -- start 2024-03-26T13:59:53.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.922+0000 7f5777e00640 1 -- start start 2024-03-26T13:59:53.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778101500 0x7f5778075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 0x7f5778073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781030b0 0x7f5778073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810d040 con 0x7f57781026f0 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810cec0 con 0x7f5778101500 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577810d1c0 con 0x7f57781030b0 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 0x7f5778073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 0x7f5778073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36394/0 (socket says 172.21.15.137:36394) 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 -- 172.21.15.137:0/1697221785 learned_addr learned my addr 172.21.15.137:0/1697221785 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5776a00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778101500 0x7f5778075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777400640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781030b0 0x7f5778073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 -- 172.21.15.137:0/1697221785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781030b0 msgr2=0x7f5778073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781030b0 0x7f5778073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 -- 172.21.15.137:0/1697221785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778101500 msgr2=0x7f5778075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778101500 0x7f5778075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f576e000640 1 -- 172.21.15.137:0/1697221785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778074860 con 0x7f57781026f0 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5777400640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781030b0 0x7f5778073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:53.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.923+0000 7f5776a00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778101500 0x7f5778075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.924+0000 7f576e000640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 0x7f5778073a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f576000c920 tx=0x7f576000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.924+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760007c70 con 0x7f57781026f0 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.924+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781a17c0 con 0x7f57781026f0 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.924+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5760007e10 con 0x7f57781026f0 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.924+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760018820 con 0x7f57781026f0 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.924+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781a1d10 con 0x7f57781026f0 2024-03-26T13:59:53.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.926+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778108820 con 0x7f57781026f0 2024-03-26T13:59:53.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.927+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f57600189c0 con 0x7f57781026f0 2024-03-26T13:59:53.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.929+0000 7f577ca00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f575c0a2890 0x7f575c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:53.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.929+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f57600cf1b0 con 0x7f57781026f0 2024-03-26T13:59:53.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.929+0000 7f5776a00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f575c0a2890 0x7f575c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:53.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.930+0000 7f5776a00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f575c0a2890 0x7f575c0a4d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5768006010 tx=0x7f5768032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:53.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:53.938+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5760014030 con 0x7f57781026f0 2024-03-26T13:59:54.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.126+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5778061ae0 con 0x7f575c0a2890 2024-03-26T13:59:54.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.161+0000 7f577ca00640 1 -- 172.21.15.137:0/1697221785 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340699 (secure 0 0 0) 0x7f5778061ae0 con 0x7f575c0a2890 2024-03-26T13:59:54.162 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:54.164 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f575c0a2890 msgr2=0x7f575c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f575c0a2890 0x7f575c0a4d50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5768006010 tx=0x7f5768032340 comp rx=0 tx=0).stop 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 msgr2=0x7f5778073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 0x7f5778073a90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f576000c920 tx=0x7f576000cde0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 shutdown_connections 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f575c0a2890 0x7f575c0a4d50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57781030b0 0x7f5778073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57781026f0 0x7f5778073a90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 --2- 172.21.15.137:0/1697221785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778101500 0x7f5778075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.168+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 >> 172.21.15.137:0/1697221785 conn(0x7f57780fca70 msgr2=0x7f57781059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:54.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.169+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 shutdown_connections 2024-03-26T13:59:54.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.169+0000 7f5777e00640 1 -- 172.21.15.137:0/1697221785 wait complete. 2024-03-26T13:59:54.181 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-03-26T13:59:53.309642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603333,"num_objects":56,"num_object_clones":0,"num_object_copies":169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":145,"num_read_kb":128,"num_write":303,"num_write_kb":2987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":244,"ondisk_log_size":244,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360944,"kb_used_data":11780,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580880,"statfs":{"total":1151940427776,"available":1151570821120,"internally_reserved":0,"allocated":12062720,"data_stored":7594718,"data_compressed":283341,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8466,"num_objects":32,"num_object_clones":0,"num_object_copies":97,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":74,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.230817"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235192+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:30.235192+0000","last_peered":"2024-03-26T13:59:30.235192+0000","last_clean":"2024-03-26T13:59:30.235192+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:30.235192+0000","last_undegraded":"2024-03-26T13:59:30.235192+0000","last_fullsized":"2024-03-26T13:59:30.235192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364396+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:29.364396+0000","last_peered":"2024-03-26T13:59:29.364396+0000","last_clean":"2024-03-26T13:59:29.364396+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:29.364396+0000","last_undegraded":"2024-03-26T13:59:29.364396+0000","last_fullsized":"2024-03-26T13:59:29.364396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364118+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:29.364118+0000","last_peered":"2024-03-26T13:59:29.364118+0000","last_clean":"2024-03-26T13:59:29.364118+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:29.364118+0000","last_undegraded":"2024-03-26T13:59:29.364118+0000","last_fullsized":"2024-03-26T13:59:29.364118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.547427+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:28.547427+0000","last_peered":"2024-03-26T13:59:28.547427+0000","last_clean":"2024-03-26T13:59:28.547427+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:28.547427+0000","last_undegraded":"2024-03-26T13:59:28.547427+0000","last_fullsized":"2024-03-26T13:59:28.547427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469440+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:29.469440+0000","last_peered":"2024-03-26T13:59:29.469440+0000","last_clean":"2024-03-26T13:59:29.469440+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:29.469440+0000","last_undegraded":"2024-03-26T13:59:29.469440+0000","last_fullsized":"2024-03-26T13:59:29.469440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781472+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:28.781472+0000","last_peered":"2024-03-26T13:59:28.781472+0000","last_clean":"2024-03-26T13:59:28.781472+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:28.781472+0000","last_undegraded":"2024-03-26T13:59:28.781472+0000","last_fullsized":"2024-03-26T13:59:28.781472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839948+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:27.839948+0000","last_peered":"2024-03-26T13:59:27.839948+0000","last_clean":"2024-03-26T13:59:27.839948+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:27.839948+0000","last_undegraded":"2024-03-26T13:59:27.839948+0000","last_fullsized":"2024-03-26T13:59:27.839948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349872+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:29.349872+0000","last_peered":"2024-03-26T13:59:29.349872+0000","last_clean":"2024-03-26T13:59:29.349872+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:29.349872+0000","last_undegraded":"2024-03-26T13:59:29.349872+0000","last_fullsized":"2024-03-26T13:59:29.349872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782777+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:28.782777+0000","last_peered":"2024-03-26T13:59:28.782777+0000","last_clean":"2024-03-26T13:59:28.782777+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:28.782777+0000","last_undegraded":"2024-03-26T13:59:28.782777+0000","last_fullsized":"2024-03-26T13:59:28.782777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841218+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:27.841218+0000","last_peered":"2024-03-26T13:59:27.841218+0000","last_clean":"2024-03-26T13:59:27.841218+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:27.841218+0000","last_undegraded":"2024-03-26T13:59:27.841218+0000","last_fullsized":"2024-03-26T13:59:27.841218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.547290+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:28.547290+0000","last_peered":"2024-03-26T13:59:28.547290+0000","last_clean":"2024-03-26T13:59:28.547290+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:28.547290+0000","last_undegraded":"2024-03-26T13:59:28.547290+0000","last_fullsized":"2024-03-26T13:59:28.547290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839733+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:27.839733+0000","last_peered":"2024-03-26T13:59:27.839733+0000","last_clean":"2024-03-26T13:59:27.839733+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:27.839733+0000","last_undegraded":"2024-03-26T13:59:27.839733+0000","last_fullsized":"2024-03-26T13:59:27.839733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363825+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:29.363825+0000","last_peered":"2024-03-26T13:59:29.363825+0000","last_clean":"2024-03-26T13:59:29.363825+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:29.363825+0000","last_undegraded":"2024-03-26T13:59:29.363825+0000","last_fullsized":"2024-03-26T13:59:29.363825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363266+0000","last_change":"2024-03-26T13:59:23.312717+0000","last_active":"2024-03-26T13:59:29.363266+0000","last_peered":"2024-03-26T13:59:29.363266+0000","last_clean":"2024-03-26T13:59:29.363266+0000","last_became_active":"2024-03-26T13:59:23.312122+0000","last_became_peered":"2024-03-26T13:59:23.312122+0000","last_unstale":"2024-03-26T13:59:29.363266+0000","last_undegraded":"2024-03-26T13:59:29.363266+0000","last_fullsized":"2024-03-26T13:59:29.363266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:54.487004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348001+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:29.348001+0000","last_peered":"2024-03-26T13:59:29.348001+0000","last_clean":"2024-03-26T13:59:29.348001+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:29.348001+0000","last_undegraded":"2024-03-26T13:59:29.348001+0000","last_fullsized":"2024-03-26T13:59:29.348001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234897+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:30.234897+0000","last_peered":"2024-03-26T13:59:30.234897+0000","last_clean":"2024-03-26T13:59:30.234897+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:30.234897+0000","last_undegraded":"2024-03-26T13:59:30.234897+0000","last_fullsized":"2024-03-26T13:59:30.234897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231510+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:30.231510+0000","last_peered":"2024-03-26T13:59:30.231510+0000","last_clean":"2024-03-26T13:59:30.231510+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:30.231510+0000","last_undegraded":"2024-03-26T13:59:30.231510+0000","last_fullsized":"2024-03-26T13:59:30.231510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.236555+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:30.236555+0000","last_peered":"2024-03-26T13:59:30.236555+0000","last_clean":"2024-03-26T13:59:30.236555+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:30.236555+0000","last_undegraded":"2024-03-26T13:59:30.236555+0000","last_fullsized":"2024-03-26T13:59:30.236555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233326+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:30.233326+0000","last_peered":"2024-03-26T13:59:30.233326+0000","last_clean":"2024-03-26T13:59:30.233326+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:30.233326+0000","last_undegraded":"2024-03-26T13:59:30.233326+0000","last_fullsized":"2024-03-26T13:59:30.233326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264793+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:49.264793+0000","last_peered":"2024-03-26T13:59:49.264793+0000","last_clean":"2024-03-26T13:59:49.264793+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:49.264793+0000","last_undegraded":"2024-03-26T13:59:49.264793+0000","last_fullsized":"2024-03-26T13:59:49.264793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325746+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:28.325746+0000","last_peered":"2024-03-26T13:59:28.325746+0000","last_clean":"2024-03-26T13:59:28.325746+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:28.325746+0000","last_undegraded":"2024-03-26T13:59:28.325746+0000","last_fullsized":"2024-03-26T13:59:28.325746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.546383+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:28.546383+0000","last_peered":"2024-03-26T13:59:28.546383+0000","last_clean":"2024-03-26T13:59:28.546383+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:28.546383+0000","last_undegraded":"2024-03-26T13:59:28.546383+0000","last_fullsized":"2024-03-26T13:59:28.546383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.326299+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:28.326299+0000","last_peered":"2024-03-26T13:59:28.326299+0000","last_clean":"2024-03-26T13:59:28.326299+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:28.326299+0000","last_undegraded":"2024-03-26T13:59:28.326299+0000","last_fullsized":"2024-03-26T13:59:28.326299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364222+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:29.364222+0000","last_peered":"2024-03-26T13:59:29.364222+0000","last_clean":"2024-03-26T13:59:29.364222+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:29.364222+0000","last_undegraded":"2024-03-26T13:59:29.364222+0000","last_fullsized":"2024-03-26T13:59:29.364222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.546743+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:28.546743+0000","last_peered":"2024-03-26T13:59:28.546743+0000","last_clean":"2024-03-26T13:59:28.546743+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:28.546743+0000","last_undegraded":"2024-03-26T13:59:28.546743+0000","last_fullsized":"2024-03-26T13:59:28.546743+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264016+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:49.264016+0000","last_peered":"2024-03-26T13:59:49.264016+0000","last_clean":"2024-03-26T13:59:49.264016+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:49.264016+0000","last_undegraded":"2024-03-26T13:59:49.264016+0000","last_fullsized":"2024-03-26T13:59:49.264016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349048+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:29.349048+0000","last_peered":"2024-03-26T13:59:29.349048+0000","last_clean":"2024-03-26T13:59:29.349048+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:29.349048+0000","last_undegraded":"2024-03-26T13:59:29.349048+0000","last_fullsized":"2024-03-26T13:59:29.349048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.546610+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:28.546610+0000","last_peered":"2024-03-26T13:59:28.546610+0000","last_clean":"2024-03-26T13:59:28.546610+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:28.546610+0000","last_undegraded":"2024-03-26T13:59:28.546610+0000","last_fullsized":"2024-03-26T13:59:28.546610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469895+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:29.469895+0000","last_peered":"2024-03-26T13:59:29.469895+0000","last_clean":"2024-03-26T13:59:29.469895+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:29.469895+0000","last_undegraded":"2024-03-26T13:59:29.469895+0000","last_fullsized":"2024-03-26T13:59:29.469895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841812+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:27.841812+0000","last_peered":"2024-03-26T13:59:27.841812+0000","last_clean":"2024-03-26T13:59:27.841812+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:27.841812+0000","last_undegraded":"2024-03-26T13:59:27.841812+0000","last_fullsized":"2024-03-26T13:59:27.841812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.236151+0000","last_change":"2024-03-26T13:59:27.324095+0000","last_active":"2024-03-26T13:59:30.236151+0000","last_peered":"2024-03-26T13:59:30.236151+0000","last_clean":"2024-03-26T13:59:30.236151+0000","last_became_active":"2024-03-26T13:59:27.323585+0000","last_became_peered":"2024-03-26T13:59:27.323585+0000","last_unstale":"2024-03-26T13:59:30.236151+0000","last_undegraded":"2024-03-26T13:59:30.236151+0000","last_fullsized":"2024-03-26T13:59:30.236151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:35:22.699658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.265450+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:50.265450+0000","last_peered":"2024-03-26T13:59:50.265450+0000","last_clean":"2024-03-26T13:59:50.265450+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:50.265450+0000","last_undegraded":"2024-03-26T13:59:50.265450+0000","last_fullsized":"2024-03-26T13:59:50.265450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469799+0000","last_change":"2024-03-26T13:59:23.299805+0000","last_active":"2024-03-26T13:59:29.469799+0000","last_peered":"2024-03-26T13:59:29.469799+0000","last_clean":"2024-03-26T13:59:29.469799+0000","last_became_active":"2024-03-26T13:59:23.295723+0000","last_became_peered":"2024-03-26T13:59:23.295723+0000","last_unstale":"2024-03-26T13:59:29.469799+0000","last_undegraded":"2024-03-26T13:59:29.469799+0000","last_fullsized":"2024-03-26T13:59:29.469799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:25:05.112905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234146+0000","last_change":"2024-03-26T13:59:27.325199+0000","last_active":"2024-03-26T13:59:30.234146+0000","last_peered":"2024-03-26T13:59:30.234146+0000","last_clean":"2024-03-26T13:59:30.234146+0000","last_became_active":"2024-03-26T13:59:27.324762+0000","last_became_peered":"2024-03-26T13:59:27.324762+0000","last_unstale":"2024-03-26T13:59:30.234146+0000","last_undegraded":"2024-03-26T13:59:30.234146+0000","last_fullsized":"2024-03-26T13:59:30.234146+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:44:06.542149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.261450+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:49.261450+0000","last_peered":"2024-03-26T13:59:49.261450+0000","last_clean":"2024-03-26T13:59:49.261450+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:49.261450+0000","last_undegraded":"2024-03-26T13:59:49.261450+0000","last_fullsized":"2024-03-26T13:59:49.261450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841919+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:27.841919+0000","last_peered":"2024-03-26T13:59:27.841919+0000","last_clean":"2024-03-26T13:59:27.841919+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:27.841919+0000","last_undegraded":"2024-03-26T13:59:27.841919+0000","last_fullsized":"2024-03-26T13:59:27.841919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235034+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:30.235034+0000","last_peered":"2024-03-26T13:59:30.235034+0000","last_clean":"2024-03-26T13:59:30.235034+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:30.235034+0000","last_undegraded":"2024-03-26T13:59:30.235034+0000","last_fullsized":"2024-03-26T13:59:30.235034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264064+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:49.264064+0000","last_peered":"2024-03-26T13:59:49.264064+0000","last_clean":"2024-03-26T13:59:49.264064+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:49.264064+0000","last_undegraded":"2024-03-26T13:59:49.264064+0000","last_fullsized":"2024-03-26T13:59:49.264064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324313+0000","last_change":"2024-03-26T13:59:23.298324+0000","last_active":"2024-03-26T13:59:28.324313+0000","last_peered":"2024-03-26T13:59:28.324313+0000","last_clean":"2024-03-26T13:59:28.324313+0000","last_became_active":"2024-03-26T13:59:23.297577+0000","last_became_peered":"2024-03-26T13:59:23.297577+0000","last_unstale":"2024-03-26T13:59:28.324313+0000","last_undegraded":"2024-03-26T13:59:28.324313+0000","last_fullsized":"2024-03-26T13:59:28.324313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:24.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364431+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:29.364431+0000","last_peered":"2024-03-26T13:59:29.364431+0000","last_clean":"2024-03-26T13:59:29.364431+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:29.364431+0000","last_undegraded":"2024-03-26T13:59:29.364431+0000","last_fullsized":"2024-03-26T13:59:29.364431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840981+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:27.840981+0000","last_peered":"2024-03-26T13:59:27.840981+0000","last_clean":"2024-03-26T13:59:27.840981+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:27.840981+0000","last_undegraded":"2024-03-26T13:59:27.840981+0000","last_fullsized":"2024-03-26T13:59:27.840981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230691+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:30.230691+0000","last_peered":"2024-03-26T13:59:30.230691+0000","last_clean":"2024-03-26T13:59:30.230691+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:30.230691+0000","last_undegraded":"2024-03-26T13:59:30.230691+0000","last_fullsized":"2024-03-26T13:59:30.230691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324666+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:28.324666+0000","last_peered":"2024-03-26T13:59:28.324666+0000","last_clean":"2024-03-26T13:59:28.324666+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:28.324666+0000","last_undegraded":"2024-03-26T13:59:28.324666+0000","last_fullsized":"2024-03-26T13:59:28.324666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.250279+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:48.250279+0000","last_peered":"2024-03-26T13:59:48.250279+0000","last_clean":"2024-03-26T13:59:48.250279+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:48.250279+0000","last_undegraded":"2024-03-26T13:59:48.250279+0000","last_fullsized":"2024-03-26T13:59:48.250279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349771+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:29.349771+0000","last_peered":"2024-03-26T13:59:29.349771+0000","last_clean":"2024-03-26T13:59:29.349771+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:29.349771+0000","last_undegraded":"2024-03-26T13:59:29.349771+0000","last_fullsized":"2024-03-26T13:59:29.349771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231033+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:30.231033+0000","last_peered":"2024-03-26T13:59:30.231033+0000","last_clean":"2024-03-26T13:59:30.231033+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:30.231033+0000","last_undegraded":"2024-03-26T13:59:30.231033+0000","last_fullsized":"2024-03-26T13:59:30.231033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.243150+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:48.243150+0000","last_peered":"2024-03-26T13:59:48.243150+0000","last_clean":"2024-03-26T13:59:48.243150+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:48.243150+0000","last_undegraded":"2024-03-26T13:59:48.243150+0000","last_fullsized":"2024-03-26T13:59:48.243150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364498+0000","last_change":"2024-03-26T13:59:23.314458+0000","last_active":"2024-03-26T13:59:29.364498+0000","last_peered":"2024-03-26T13:59:29.364498+0000","last_clean":"2024-03-26T13:59:29.364498+0000","last_became_active":"2024-03-26T13:59:23.314060+0000","last_became_peered":"2024-03-26T13:59:23.314060+0000","last_unstale":"2024-03-26T13:59:29.364498+0000","last_undegraded":"2024-03-26T13:59:29.364498+0000","last_fullsized":"2024-03-26T13:59:29.364498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:11:40.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.547074+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:28.547074+0000","last_peered":"2024-03-26T13:59:28.547074+0000","last_clean":"2024-03-26T13:59:28.547074+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:28.547074+0000","last_undegraded":"2024-03-26T13:59:28.547074+0000","last_fullsized":"2024-03-26T13:59:28.547074+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.242523+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:48.242523+0000","last_peered":"2024-03-26T13:59:48.242523+0000","last_clean":"2024-03-26T13:59:48.242523+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:48.242523+0000","last_undegraded":"2024-03-26T13:59:48.242523+0000","last_fullsized":"2024-03-26T13:59:48.242523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349320+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:29.349320+0000","last_peered":"2024-03-26T13:59:29.349320+0000","last_clean":"2024-03-26T13:59:29.349320+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:29.349320+0000","last_undegraded":"2024-03-26T13:59:29.349320+0000","last_fullsized":"2024-03-26T13:59:29.349320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230980+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:30.230980+0000","last_peered":"2024-03-26T13:59:30.230980+0000","last_clean":"2024-03-26T13:59:30.230980+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:30.230980+0000","last_undegraded":"2024-03-26T13:59:30.230980+0000","last_fullsized":"2024-03-26T13:59:30.230980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.748837+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:50.748837+0000","last_peered":"2024-03-26T13:59:50.748837+0000","last_clean":"2024-03-26T13:59:50.748837+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:50.748837+0000","last_undegraded":"2024-03-26T13:59:50.748837+0000","last_fullsized":"2024-03-26T13:59:50.748837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.470191+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:29.470191+0000","last_peered":"2024-03-26T13:59:29.470191+0000","last_clean":"2024-03-26T13:59:29.470191+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:29.470191+0000","last_undegraded":"2024-03-26T13:59:29.470191+0000","last_fullsized":"2024-03-26T13:59:29.470191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842554+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:27.842554+0000","last_peered":"2024-03-26T13:59:27.842554+0000","last_clean":"2024-03-26T13:59:27.842554+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:27.842554+0000","last_undegraded":"2024-03-26T13:59:27.842554+0000","last_fullsized":"2024-03-26T13:59:27.842554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.251222+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:48.251222+0000","last_peered":"2024-03-26T13:59:48.251222+0000","last_clean":"2024-03-26T13:59:48.251222+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:48.251222+0000","last_undegraded":"2024-03-26T13:59:48.251222+0000","last_fullsized":"2024-03-26T13:59:48.251222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839344+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:27.839344+0000","last_peered":"2024-03-26T13:59:27.839344+0000","last_clean":"2024-03-26T13:59:27.839344+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:27.839344+0000","last_undegraded":"2024-03-26T13:59:27.839344+0000","last_fullsized":"2024-03-26T13:59:27.839344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233995+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:30.233995+0000","last_peered":"2024-03-26T13:59:30.233995+0000","last_clean":"2024-03-26T13:59:30.233995+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:30.233995+0000","last_undegraded":"2024-03-26T13:59:30.233995+0000","last_fullsized":"2024-03-26T13:59:30.233995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.248560+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:48.248560+0000","last_peered":"2024-03-26T13:59:48.248560+0000","last_clean":"2024-03-26T13:59:48.248560+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:48.248560+0000","last_undegraded":"2024-03-26T13:59:48.248560+0000","last_fullsized":"2024-03-26T13:59:48.248560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840635+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:27.840635+0000","last_peered":"2024-03-26T13:59:27.840635+0000","last_clean":"2024-03-26T13:59:27.840635+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:27.840635+0000","last_undegraded":"2024-03-26T13:59:27.840635+0000","last_fullsized":"2024-03-26T13:59:27.840635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231524+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:30.231524+0000","last_peered":"2024-03-26T13:59:30.231524+0000","last_clean":"2024-03-26T13:59:30.231524+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:30.231524+0000","last_undegraded":"2024-03-26T13:59:30.231524+0000","last_fullsized":"2024-03-26T13:59:30.231524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.545994+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:28.545994+0000","last_peered":"2024-03-26T13:59:28.545994+0000","last_clean":"2024-03-26T13:59:28.545994+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:28.545994+0000","last_undegraded":"2024-03-26T13:59:28.545994+0000","last_fullsized":"2024-03-26T13:59:28.545994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840102+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:27.840102+0000","last_peered":"2024-03-26T13:59:27.840102+0000","last_clean":"2024-03-26T13:59:27.840102+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:27.840102+0000","last_undegraded":"2024-03-26T13:59:27.840102+0000","last_fullsized":"2024-03-26T13:59:27.840102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840430+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:27.840430+0000","last_peered":"2024-03-26T13:59:27.840430+0000","last_clean":"2024-03-26T13:59:27.840430+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:27.840430+0000","last_undegraded":"2024-03-26T13:59:27.840430+0000","last_fullsized":"2024-03-26T13:59:27.840430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.266019+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:49.266019+0000","last_peered":"2024-03-26T13:59:49.266019+0000","last_clean":"2024-03-26T13:59:49.266019+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:49.266019+0000","last_undegraded":"2024-03-26T13:59:49.266019+0000","last_fullsized":"2024-03-26T13:59:49.266019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840790+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:27.840790+0000","last_peered":"2024-03-26T13:59:27.840790+0000","last_clean":"2024-03-26T13:59:27.840790+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:27.840790+0000","last_undegraded":"2024-03-26T13:59:27.840790+0000","last_fullsized":"2024-03-26T13:59:27.840790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841011+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:27.841011+0000","last_peered":"2024-03-26T13:59:27.841011+0000","last_clean":"2024-03-26T13:59:27.841011+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:27.841011+0000","last_undegraded":"2024-03-26T13:59:27.841011+0000","last_fullsized":"2024-03-26T13:59:27.841011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.246505+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:50.246505+0000","last_peered":"2024-03-26T13:59:50.246505+0000","last_clean":"2024-03-26T13:59:50.246505+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:50.246505+0000","last_undegraded":"2024-03-26T13:59:50.246505+0000","last_fullsized":"2024-03-26T13:59:50.246505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234557+0000","last_change":"2024-03-26T13:59:23.314700+0000","last_active":"2024-03-26T13:59:30.234557+0000","last_peered":"2024-03-26T13:59:30.234557+0000","last_clean":"2024-03-26T13:59:30.234557+0000","last_became_active":"2024-03-26T13:59:23.314372+0000","last_became_peered":"2024-03-26T13:59:23.314372+0000","last_unstale":"2024-03-26T13:59:30.234557+0000","last_undegraded":"2024-03-26T13:59:30.234557+0000","last_fullsized":"2024-03-26T13:59:30.234557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:32.631858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469875+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:29.469875+0000","last_peered":"2024-03-26T13:59:29.469875+0000","last_clean":"2024-03-26T13:59:29.469875+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:29.469875+0000","last_undegraded":"2024-03-26T13:59:29.469875+0000","last_fullsized":"2024-03-26T13:59:29.469875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231159+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:30.231159+0000","last_peered":"2024-03-26T13:59:30.231159+0000","last_clean":"2024-03-26T13:59:30.231159+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:30.231159+0000","last_undegraded":"2024-03-26T13:59:30.231159+0000","last_fullsized":"2024-03-26T13:59:30.231159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782231+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:28.782231+0000","last_peered":"2024-03-26T13:59:28.782231+0000","last_clean":"2024-03-26T13:59:28.782231+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:28.782231+0000","last_undegraded":"2024-03-26T13:59:28.782231+0000","last_fullsized":"2024-03-26T13:59:28.782231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839740+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:27.839740+0000","last_peered":"2024-03-26T13:59:27.839740+0000","last_clean":"2024-03-26T13:59:27.839740+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:27.839740+0000","last_undegraded":"2024-03-26T13:59:27.839740+0000","last_fullsized":"2024-03-26T13:59:27.839740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840640+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:27.840640+0000","last_peered":"2024-03-26T13:59:27.840640+0000","last_clean":"2024-03-26T13:59:27.840640+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:27.840640+0000","last_undegraded":"2024-03-26T13:59:27.840640+0000","last_fullsized":"2024-03-26T13:59:27.840640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781589+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:28.781589+0000","last_peered":"2024-03-26T13:59:28.781589+0000","last_clean":"2024-03-26T13:59:28.781589+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:28.781589+0000","last_undegraded":"2024-03-26T13:59:28.781589+0000","last_fullsized":"2024-03-26T13:59:28.781589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840495+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:27.840495+0000","last_peered":"2024-03-26T13:59:27.840495+0000","last_clean":"2024-03-26T13:59:27.840495+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:27.840495+0000","last_undegraded":"2024-03-26T13:59:27.840495+0000","last_fullsized":"2024-03-26T13:59:27.840495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841110+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:27.841110+0000","last_peered":"2024-03-26T13:59:27.841110+0000","last_clean":"2024-03-26T13:59:27.841110+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:27.841110+0000","last_undegraded":"2024-03-26T13:59:27.841110+0000","last_fullsized":"2024-03-26T13:59:27.841110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.263864+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:50.263864+0000","last_peered":"2024-03-26T13:59:50.263864+0000","last_clean":"2024-03-26T13:59:50.263864+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:50.263864+0000","last_undegraded":"2024-03-26T13:59:50.263864+0000","last_fullsized":"2024-03-26T13:59:50.263864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841168+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:27.841168+0000","last_peered":"2024-03-26T13:59:27.841168+0000","last_clean":"2024-03-26T13:59:27.841168+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:27.841168+0000","last_undegraded":"2024-03-26T13:59:27.841168+0000","last_fullsized":"2024-03-26T13:59:27.841168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.247190+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:48.247190+0000","last_peered":"2024-03-26T13:59:48.247190+0000","last_clean":"2024-03-26T13:59:48.247190+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:48.247190+0000","last_undegraded":"2024-03-26T13:59:48.247190+0000","last_fullsized":"2024-03-26T13:59:48.247190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839877+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:27.839877+0000","last_peered":"2024-03-26T13:59:27.839877+0000","last_clean":"2024-03-26T13:59:27.839877+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:27.839877+0000","last_undegraded":"2024-03-26T13:59:27.839877+0000","last_fullsized":"2024-03-26T13:59:27.839877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842500+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:27.842500+0000","last_peered":"2024-03-26T13:59:27.842500+0000","last_clean":"2024-03-26T13:59:27.842500+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:27.842500+0000","last_undegraded":"2024-03-26T13:59:27.842500+0000","last_fullsized":"2024-03-26T13:59:27.842500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.838817+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:27.838817+0000","last_peered":"2024-03-26T13:59:27.838817+0000","last_clean":"2024-03-26T13:59:27.838817+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:27.838817+0000","last_undegraded":"2024-03-26T13:59:27.838817+0000","last_fullsized":"2024-03-26T13:59:27.838817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264433+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:49.264433+0000","last_peered":"2024-03-26T13:59:49.264433+0000","last_clean":"2024-03-26T13:59:49.264433+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:49.264433+0000","last_undegraded":"2024-03-26T13:59:49.264433+0000","last_fullsized":"2024-03-26T13:59:49.264433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233597+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:30.233597+0000","last_peered":"2024-03-26T13:59:30.233597+0000","last_clean":"2024-03-26T13:59:30.233597+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:30.233597+0000","last_undegraded":"2024-03-26T13:59:30.233597+0000","last_fullsized":"2024-03-26T13:59:30.233597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246587+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:48.246587+0000","last_peered":"2024-03-26T13:59:48.246587+0000","last_clean":"2024-03-26T13:59:48.246587+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:48.246587+0000","last_undegraded":"2024-03-26T13:59:48.246587+0000","last_fullsized":"2024-03-26T13:59:48.246587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839930+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:27.839930+0000","last_peered":"2024-03-26T13:59:27.839930+0000","last_clean":"2024-03-26T13:59:27.839930+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:27.839930+0000","last_undegraded":"2024-03-26T13:59:27.839930+0000","last_fullsized":"2024-03-26T13:59:27.839930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324348+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:28.324348+0000","last_peered":"2024-03-26T13:59:28.324348+0000","last_clean":"2024-03-26T13:59:28.324348+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:28.324348+0000","last_undegraded":"2024-03-26T13:59:28.324348+0000","last_fullsized":"2024-03-26T13:59:28.324348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.260579+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:49.260579+0000","last_peered":"2024-03-26T13:59:49.260579+0000","last_clean":"2024-03-26T13:59:49.260579+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:49.260579+0000","last_undegraded":"2024-03-26T13:59:49.260579+0000","last_fullsized":"2024-03-26T13:59:49.260579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325372+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:28.325372+0000","last_peered":"2024-03-26T13:59:28.325372+0000","last_clean":"2024-03-26T13:59:28.325372+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:28.325372+0000","last_undegraded":"2024-03-26T13:59:28.325372+0000","last_fullsized":"2024-03-26T13:59:28.325372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232712+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:30.232712+0000","last_peered":"2024-03-26T13:59:30.232712+0000","last_clean":"2024-03-26T13:59:30.232712+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:30.232712+0000","last_undegraded":"2024-03-26T13:59:30.232712+0000","last_fullsized":"2024-03-26T13:59:30.232712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.243940+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:48.243940+0000","last_peered":"2024-03-26T13:59:48.243940+0000","last_clean":"2024-03-26T13:59:48.243940+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:48.243940+0000","last_undegraded":"2024-03-26T13:59:48.243940+0000","last_fullsized":"2024-03-26T13:59:48.243940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364591+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:29.364591+0000","last_peered":"2024-03-26T13:59:29.364591+0000","last_clean":"2024-03-26T13:59:29.364591+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:29.364591+0000","last_undegraded":"2024-03-26T13:59:29.364591+0000","last_fullsized":"2024-03-26T13:59:29.364591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.546435+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:28.546435+0000","last_peered":"2024-03-26T13:59:28.546435+0000","last_clean":"2024-03-26T13:59:28.546435+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:28.546435+0000","last_undegraded":"2024-03-26T13:59:28.546435+0000","last_fullsized":"2024-03-26T13:59:28.546435+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.249700+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:48.249700+0000","last_peered":"2024-03-26T13:59:48.249700+0000","last_clean":"2024-03-26T13:59:48.249700+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:48.249700+0000","last_undegraded":"2024-03-26T13:59:48.249700+0000","last_fullsized":"2024-03-26T13:59:48.249700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469234+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:29.469234+0000","last_peered":"2024-03-26T13:59:29.469234+0000","last_clean":"2024-03-26T13:59:29.469234+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:29.469234+0000","last_undegraded":"2024-03-26T13:59:29.469234+0000","last_fullsized":"2024-03-26T13:59:29.469234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231634+0000","last_change":"2024-03-26T13:59:27.332943+0000","last_active":"2024-03-26T13:59:30.231634+0000","last_peered":"2024-03-26T13:59:30.231634+0000","last_clean":"2024-03-26T13:59:30.231634+0000","last_became_active":"2024-03-26T13:59:27.331677+0000","last_became_peered":"2024-03-26T13:59:27.331677+0000","last_unstale":"2024-03-26T13:59:30.231634+0000","last_undegraded":"2024-03-26T13:59:30.231634+0000","last_fullsized":"2024-03-26T13:59:30.231634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:19.990935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246514+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:48.246514+0000","last_peered":"2024-03-26T13:59:48.246514+0000","last_clean":"2024-03-26T13:59:48.246514+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:48.246514+0000","last_undegraded":"2024-03-26T13:59:48.246514+0000","last_fullsized":"2024-03-26T13:59:48.246514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232606+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:30.232606+0000","last_peered":"2024-03-26T13:59:30.232606+0000","last_clean":"2024-03-26T13:59:30.232606+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:30.232606+0000","last_undegraded":"2024-03-26T13:59:30.232606+0000","last_fullsized":"2024-03-26T13:59:30.232606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.547262+0000","last_change":"2024-03-26T13:59:27.326558+0000","last_active":"2024-03-26T13:59:28.547262+0000","last_peered":"2024-03-26T13:59:28.547262+0000","last_clean":"2024-03-26T13:59:28.547262+0000","last_became_active":"2024-03-26T13:59:27.326077+0000","last_became_peered":"2024-03-26T13:59:27.326077+0000","last_unstale":"2024-03-26T13:59:28.547262+0000","last_undegraded":"2024-03-26T13:59:28.547262+0000","last_fullsized":"2024-03-26T13:59:28.547262+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:50:58.679616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233322+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:30.233322+0000","last_peered":"2024-03-26T13:59:30.233322+0000","last_clean":"2024-03-26T13:59:30.233322+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:30.233322+0000","last_undegraded":"2024-03-26T13:59:30.233322+0000","last_fullsized":"2024-03-26T13:59:30.233322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325092+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:28.325092+0000","last_peered":"2024-03-26T13:59:28.325092+0000","last_clean":"2024-03-26T13:59:28.325092+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:28.325092+0000","last_undegraded":"2024-03-26T13:59:28.325092+0000","last_fullsized":"2024-03-26T13:59:28.325092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.546513+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:28.546513+0000","last_peered":"2024-03-26T13:59:28.546513+0000","last_clean":"2024-03-26T13:59:28.546513+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:28.546513+0000","last_undegraded":"2024-03-26T13:59:28.546513+0000","last_fullsized":"2024-03-26T13:59:28.546513+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.252376+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:48.252376+0000","last_peered":"2024-03-26T13:59:48.252376+0000","last_clean":"2024-03-26T13:59:48.252376+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:48.252376+0000","last_undegraded":"2024-03-26T13:59:48.252376+0000","last_fullsized":"2024-03-26T13:59:48.252376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840788+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:27.840788+0000","last_peered":"2024-03-26T13:59:27.840788+0000","last_clean":"2024-03-26T13:59:27.840788+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:27.840788+0000","last_undegraded":"2024-03-26T13:59:27.840788+0000","last_fullsized":"2024-03-26T13:59:27.840788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348985+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:29.348985+0000","last_peered":"2024-03-26T13:59:29.348985+0000","last_clean":"2024-03-26T13:59:29.348985+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:29.348985+0000","last_undegraded":"2024-03-26T13:59:29.348985+0000","last_fullsized":"2024-03-26T13:59:29.348985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235703+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:30.235703+0000","last_peered":"2024-03-26T13:59:30.235703+0000","last_clean":"2024-03-26T13:59:30.235703+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:30.235703+0000","last_undegraded":"2024-03-26T13:59:30.235703+0000","last_fullsized":"2024-03-26T13:59:30.235703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841263+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:27.841263+0000","last_peered":"2024-03-26T13:59:27.841263+0000","last_clean":"2024-03-26T13:59:27.841263+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:27.841263+0000","last_undegraded":"2024-03-26T13:59:27.841263+0000","last_fullsized":"2024-03-26T13:59:27.841263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363505+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:29.363505+0000","last_peered":"2024-03-26T13:59:29.363505+0000","last_clean":"2024-03-26T13:59:29.363505+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:29.363505+0000","last_undegraded":"2024-03-26T13:59:29.363505+0000","last_fullsized":"2024-03-26T13:59:29.363505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842212+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:27.842212+0000","last_peered":"2024-03-26T13:59:27.842212+0000","last_clean":"2024-03-26T13:59:27.842212+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:27.842212+0000","last_undegraded":"2024-03-26T13:59:27.842212+0000","last_fullsized":"2024-03-26T13:59:27.842212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841024+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:27.841024+0000","last_peered":"2024-03-26T13:59:27.841024+0000","last_clean":"2024-03-26T13:59:27.841024+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:27.841024+0000","last_undegraded":"2024-03-26T13:59:27.841024+0000","last_fullsized":"2024-03-26T13:59:27.841024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840326+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:27.840326+0000","last_peered":"2024-03-26T13:59:27.840326+0000","last_clean":"2024-03-26T13:59:27.840326+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:27.840326+0000","last_undegraded":"2024-03-26T13:59:27.840326+0000","last_fullsized":"2024-03-26T13:59:27.840326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840337+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:27.840337+0000","last_peered":"2024-03-26T13:59:27.840337+0000","last_clean":"2024-03-26T13:59:27.840337+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:27.840337+0000","last_undegraded":"2024-03-26T13:59:27.840337+0000","last_fullsized":"2024-03-26T13:59:27.840337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841677+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:27.841677+0000","last_peered":"2024-03-26T13:59:27.841677+0000","last_clean":"2024-03-26T13:59:27.841677+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:27.841677+0000","last_undegraded":"2024-03-26T13:59:27.841677+0000","last_fullsized":"2024-03-26T13:59:27.841677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233480+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:30.233480+0000","last_peered":"2024-03-26T13:59:30.233480+0000","last_clean":"2024-03-26T13:59:30.233480+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:30.233480+0000","last_undegraded":"2024-03-26T13:59:30.233480+0000","last_fullsized":"2024-03-26T13:59:30.233480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232873+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:30.232873+0000","last_peered":"2024-03-26T13:59:30.232873+0000","last_clean":"2024-03-26T13:59:30.232873+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:30.232873+0000","last_undegraded":"2024-03-26T13:59:30.232873+0000","last_fullsized":"2024-03-26T13:59:30.232873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232999+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:30.232999+0000","last_peered":"2024-03-26T13:59:30.232999+0000","last_clean":"2024-03-26T13:59:30.232999+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:30.232999+0000","last_undegraded":"2024-03-26T13:59:30.232999+0000","last_fullsized":"2024-03-26T13:59:30.232999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363485+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:29.363485+0000","last_peered":"2024-03-26T13:59:29.363485+0000","last_clean":"2024-03-26T13:59:29.363485+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:29.363485+0000","last_undegraded":"2024-03-26T13:59:29.363485+0000","last_fullsized":"2024-03-26T13:59:29.363485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840283+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:27.840283+0000","last_peered":"2024-03-26T13:59:27.840283+0000","last_clean":"2024-03-26T13:59:27.840283+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:27.840283+0000","last_undegraded":"2024-03-26T13:59:27.840283+0000","last_fullsized":"2024-03-26T13:59:27.840283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234703+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:30.234703+0000","last_peered":"2024-03-26T13:59:30.234703+0000","last_clean":"2024-03-26T13:59:30.234703+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:30.234703+0000","last_undegraded":"2024-03-26T13:59:30.234703+0000","last_fullsized":"2024-03-26T13:59:30.234703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.236407+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:30.236407+0000","last_peered":"2024-03-26T13:59:30.236407+0000","last_clean":"2024-03-26T13:59:30.236407+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:30.236407+0000","last_undegraded":"2024-03-26T13:59:30.236407+0000","last_fullsized":"2024-03-26T13:59:30.236407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325560+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:28.325560+0000","last_peered":"2024-03-26T13:59:28.325560+0000","last_clean":"2024-03-26T13:59:28.325560+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:28.325560+0000","last_undegraded":"2024-03-26T13:59:28.325560+0000","last_fullsized":"2024-03-26T13:59:28.325560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840768+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:27.840768+0000","last_peered":"2024-03-26T13:59:27.840768+0000","last_clean":"2024-03-26T13:59:27.840768+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:27.840768+0000","last_undegraded":"2024-03-26T13:59:27.840768+0000","last_fullsized":"2024-03-26T13:59:27.840768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324700+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:28.324700+0000","last_peered":"2024-03-26T13:59:28.324700+0000","last_clean":"2024-03-26T13:59:28.324700+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:28.324700+0000","last_undegraded":"2024-03-26T13:59:28.324700+0000","last_fullsized":"2024-03-26T13:59:28.324700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348523+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:29.348523+0000","last_peered":"2024-03-26T13:59:29.348523+0000","last_clean":"2024-03-26T13:59:29.348523+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:29.348523+0000","last_undegraded":"2024-03-26T13:59:29.348523+0000","last_fullsized":"2024-03-26T13:59:29.348523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839530+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:27.839530+0000","last_peered":"2024-03-26T13:59:27.839530+0000","last_clean":"2024-03-26T13:59:27.839530+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:27.839530+0000","last_undegraded":"2024-03-26T13:59:27.839530+0000","last_fullsized":"2024-03-26T13:59:27.839530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234803+0000","last_change":"2024-03-26T13:59:23.308422+0000","last_active":"2024-03-26T13:59:30.234803+0000","last_peered":"2024-03-26T13:59:30.234803+0000","last_clean":"2024-03-26T13:59:30.234803+0000","last_became_active":"2024-03-26T13:59:23.308011+0000","last_became_peered":"2024-03-26T13:59:23.308011+0000","last_unstale":"2024-03-26T13:59:30.234803+0000","last_undegraded":"2024-03-26T13:59:30.234803+0000","last_fullsized":"2024-03-26T13:59:30.234803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:08.894340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":12848,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":121,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":58311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1083045,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481084,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":481004,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1072817,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":512012,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":484764,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":480288,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1076206,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":33731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T13:59:54.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T13:59:54.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 -- 172.21.15.137:0/3715695218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 msgr2=0x7fbfc410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/3715695218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc410bd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009900 tx=0x7fbfbc02bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:54.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 -- 172.21.15.137:0/3715695218 shutdown_connections 2024-03-26T13:59:54.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/3715695218 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc410bd20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/3715695218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfc4104ad0 0x7fbfc4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/3715695218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfc41038e0 0x7fbfc4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 -- 172.21.15.137:0/3715695218 >> 172.21.15.137:0/3715695218 conn(0x7fbfc40fee50 msgr2=0x7fbfc4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:54.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 -- 172.21.15.137:0/3715695218 shutdown_connections 2024-03-26T13:59:54.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.445+0000 7fbfc9e00640 1 -- 172.21.15.137:0/3715695218 wait complete. 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.446+0000 7fbfc9e00640 1 Processor -- start 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.446+0000 7fbfc9e00640 1 -- start start 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc9e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfc41038e0 0x7fbfc4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfc4104ad0 0x7fbfc4199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc41a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc9e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc410f440 con 0x7fbfc4105490 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc9e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc410f2c0 con 0x7fbfc4104ad0 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc9e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc410f5c0 con 0x7fbfc41038e0 2024-03-26T13:59:54.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc41a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc41a1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36412/0 (socket says 172.21.15.137:36412) 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc3400640 1 -- 172.21.15.137:0/2761125111 learned_addr learned my addr 172.21.15.137:0/2761125111 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc2000640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfc4104ad0 0x7fbfc4199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc2a00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfc41038e0 0x7fbfc4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.447+0000 7fbfc3400640 1 -- 172.21.15.137:0/2761125111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfc41038e0 msgr2=0x7fbfc4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc3400640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfc41038e0 0x7fbfc4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc3400640 1 -- 172.21.15.137:0/2761125111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfc4104ad0 msgr2=0x7fbfc4199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc3400640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfc4104ad0 0x7fbfc4199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc3400640 1 -- 172.21.15.137:0/2761125111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc41a16a0 con 0x7fbfc4105490 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc3400640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc41a1030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc036040 tx=0x7fbfbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc043020 con 0x7fbfc4105490 2024-03-26T13:59:54.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbfbc004490 con 0x7fbfc4105490 2024-03-26T13:59:54.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc41a1930 con 0x7fbfc4105490 2024-03-26T13:59:54.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc003d10 con 0x7fbfc4105490 2024-03-26T13:59:54.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.448+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc41a1e80 con 0x7fbfc4105490 2024-03-26T13:59:54.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.450+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc410aba0 con 0x7fbfc4105490 2024-03-26T13:59:54.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.451+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fbfbc046020 con 0x7fbfc4105490 2024-03-26T13:59:54.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.452+0000 7fbfc8a00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbfa80a2890 0x7fbfa80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.453+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbfbc0f1260 con 0x7fbfc4105490 2024-03-26T13:59:54.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.453+0000 7fbfc2a00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbfa80a2890 0x7fbfa80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.453+0000 7fbfc2a00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbfa80a2890 0x7fbfa80a4d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4104930 tx=0x7fbfb4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:54.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.457+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbfbc04d020 con 0x7fbfc4105490 2024-03-26T13:59:54.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.638+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfc4061ae0 con 0x7fbfa80a2890 2024-03-26T13:59:54.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.666+0000 7fbfc8a00640 1 -- 172.21.15.137:0/2761125111 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340636 (secure 0 0 0) 0x7fbfc4061ae0 con 0x7fbfa80a2890 2024-03-26T13:59:54.668 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:54.670 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T13:59:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-26T13:59:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2929915629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:54 smithi179 ceph-mon[35472]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:54.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbfa80a2890 msgr2=0x7fbfa80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbfa80a2890 0x7fbfa80a4d50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4104930 tx=0x7fbfb4009870 comp rx=0 tx=0).stop 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 msgr2=0x7fbfc41a1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc41a1030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc036040 tx=0x7fbfbc004270 comp rx=0 tx=0).stop 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 shutdown_connections 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbfa80a2890 0x7fbfa80a4d50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbfc4104ad0 0x7fbfc4199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbfc4105490 0x7fbfc41a1030 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 --2- 172.21.15.137:0/2761125111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfc41038e0 0x7fbfc4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.673+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 >> 172.21.15.137:0/2761125111 conn(0x7fbfc40fee50 msgr2=0x7fbfc4107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.674+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 shutdown_connections 2024-03-26T13:59:54.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.674+0000 7fbfc9e00640 1 -- 172.21.15.137:0/2761125111 wait complete. 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2929915629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:54.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:54 smithi137 ceph-mon[31539]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:54.686 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-03-26T13:59:54.323608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605045,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":316,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361008,"kb_used_data":11856,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580816,"statfs":{"total":1151940427776,"available":1151570755584,"internally_reserved":0,"allocated":12140544,"data_stored":7653133,"data_compressed":283341,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10178,"num_objects":40,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":87,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.243509"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235192+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:30.235192+0000","last_peered":"2024-03-26T13:59:30.235192+0000","last_clean":"2024-03-26T13:59:30.235192+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:30.235192+0000","last_undegraded":"2024-03-26T13:59:30.235192+0000","last_fullsized":"2024-03-26T13:59:30.235192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364396+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:29.364396+0000","last_peered":"2024-03-26T13:59:29.364396+0000","last_clean":"2024-03-26T13:59:29.364396+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:29.364396+0000","last_undegraded":"2024-03-26T13:59:29.364396+0000","last_fullsized":"2024-03-26T13:59:29.364396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364118+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:29.364118+0000","last_peered":"2024-03-26T13:59:29.364118+0000","last_clean":"2024-03-26T13:59:29.364118+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:29.364118+0000","last_undegraded":"2024-03-26T13:59:29.364118+0000","last_fullsized":"2024-03-26T13:59:29.364118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315175+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:53.315175+0000","last_peered":"2024-03-26T13:59:53.315175+0000","last_clean":"2024-03-26T13:59:53.315175+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:53.315175+0000","last_undegraded":"2024-03-26T13:59:53.315175+0000","last_fullsized":"2024-03-26T13:59:53.315175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469440+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:29.469440+0000","last_peered":"2024-03-26T13:59:29.469440+0000","last_clean":"2024-03-26T13:59:29.469440+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:29.469440+0000","last_undegraded":"2024-03-26T13:59:29.469440+0000","last_fullsized":"2024-03-26T13:59:29.469440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781472+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:28.781472+0000","last_peered":"2024-03-26T13:59:28.781472+0000","last_clean":"2024-03-26T13:59:28.781472+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:28.781472+0000","last_undegraded":"2024-03-26T13:59:28.781472+0000","last_fullsized":"2024-03-26T13:59:28.781472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839948+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:27.839948+0000","last_peered":"2024-03-26T13:59:27.839948+0000","last_clean":"2024-03-26T13:59:27.839948+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:27.839948+0000","last_undegraded":"2024-03-26T13:59:27.839948+0000","last_fullsized":"2024-03-26T13:59:27.839948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349872+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:29.349872+0000","last_peered":"2024-03-26T13:59:29.349872+0000","last_clean":"2024-03-26T13:59:29.349872+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:29.349872+0000","last_undegraded":"2024-03-26T13:59:29.349872+0000","last_fullsized":"2024-03-26T13:59:29.349872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782777+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:28.782777+0000","last_peered":"2024-03-26T13:59:28.782777+0000","last_clean":"2024-03-26T13:59:28.782777+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:28.782777+0000","last_undegraded":"2024-03-26T13:59:28.782777+0000","last_fullsized":"2024-03-26T13:59:28.782777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841218+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:27.841218+0000","last_peered":"2024-03-26T13:59:27.841218+0000","last_clean":"2024-03-26T13:59:27.841218+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:27.841218+0000","last_undegraded":"2024-03-26T13:59:27.841218+0000","last_fullsized":"2024-03-26T13:59:27.841218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315062+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:53.315062+0000","last_peered":"2024-03-26T13:59:53.315062+0000","last_clean":"2024-03-26T13:59:53.315062+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:53.315062+0000","last_undegraded":"2024-03-26T13:59:53.315062+0000","last_fullsized":"2024-03-26T13:59:53.315062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839733+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:27.839733+0000","last_peered":"2024-03-26T13:59:27.839733+0000","last_clean":"2024-03-26T13:59:27.839733+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:27.839733+0000","last_undegraded":"2024-03-26T13:59:27.839733+0000","last_fullsized":"2024-03-26T13:59:27.839733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363825+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:29.363825+0000","last_peered":"2024-03-26T13:59:29.363825+0000","last_clean":"2024-03-26T13:59:29.363825+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:29.363825+0000","last_undegraded":"2024-03-26T13:59:29.363825+0000","last_fullsized":"2024-03-26T13:59:29.363825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363266+0000","last_change":"2024-03-26T13:59:23.312717+0000","last_active":"2024-03-26T13:59:29.363266+0000","last_peered":"2024-03-26T13:59:29.363266+0000","last_clean":"2024-03-26T13:59:29.363266+0000","last_became_active":"2024-03-26T13:59:23.312122+0000","last_became_peered":"2024-03-26T13:59:23.312122+0000","last_unstale":"2024-03-26T13:59:29.363266+0000","last_undegraded":"2024-03-26T13:59:29.363266+0000","last_fullsized":"2024-03-26T13:59:29.363266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:54.487004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348001+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:29.348001+0000","last_peered":"2024-03-26T13:59:29.348001+0000","last_clean":"2024-03-26T13:59:29.348001+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:29.348001+0000","last_undegraded":"2024-03-26T13:59:29.348001+0000","last_fullsized":"2024-03-26T13:59:29.348001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234897+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:30.234897+0000","last_peered":"2024-03-26T13:59:30.234897+0000","last_clean":"2024-03-26T13:59:30.234897+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:30.234897+0000","last_undegraded":"2024-03-26T13:59:30.234897+0000","last_fullsized":"2024-03-26T13:59:30.234897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231510+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:30.231510+0000","last_peered":"2024-03-26T13:59:30.231510+0000","last_clean":"2024-03-26T13:59:30.231510+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:30.231510+0000","last_undegraded":"2024-03-26T13:59:30.231510+0000","last_fullsized":"2024-03-26T13:59:30.231510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319222+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:53.319222+0000","last_peered":"2024-03-26T13:59:53.319222+0000","last_clean":"2024-03-26T13:59:53.319222+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:53.319222+0000","last_undegraded":"2024-03-26T13:59:53.319222+0000","last_fullsized":"2024-03-26T13:59:53.319222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317072+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:53.317072+0000","last_peered":"2024-03-26T13:59:53.317072+0000","last_clean":"2024-03-26T13:59:53.317072+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:53.317072+0000","last_undegraded":"2024-03-26T13:59:53.317072+0000","last_fullsized":"2024-03-26T13:59:53.317072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264793+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:49.264793+0000","last_peered":"2024-03-26T13:59:49.264793+0000","last_clean":"2024-03-26T13:59:49.264793+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:49.264793+0000","last_undegraded":"2024-03-26T13:59:49.264793+0000","last_fullsized":"2024-03-26T13:59:49.264793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325746+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:28.325746+0000","last_peered":"2024-03-26T13:59:28.325746+0000","last_clean":"2024-03-26T13:59:28.325746+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:28.325746+0000","last_undegraded":"2024-03-26T13:59:28.325746+0000","last_fullsized":"2024-03-26T13:59:28.325746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.313767+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:53.313767+0000","last_peered":"2024-03-26T13:59:53.313767+0000","last_clean":"2024-03-26T13:59:53.313767+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:53.313767+0000","last_undegraded":"2024-03-26T13:59:53.313767+0000","last_fullsized":"2024-03-26T13:59:53.313767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.326299+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:28.326299+0000","last_peered":"2024-03-26T13:59:28.326299+0000","last_clean":"2024-03-26T13:59:28.326299+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:28.326299+0000","last_undegraded":"2024-03-26T13:59:28.326299+0000","last_fullsized":"2024-03-26T13:59:28.326299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364222+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:29.364222+0000","last_peered":"2024-03-26T13:59:29.364222+0000","last_clean":"2024-03-26T13:59:29.364222+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:29.364222+0000","last_undegraded":"2024-03-26T13:59:29.364222+0000","last_fullsized":"2024-03-26T13:59:29.364222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314244+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:53.314244+0000","last_peered":"2024-03-26T13:59:53.314244+0000","last_clean":"2024-03-26T13:59:53.314244+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:53.314244+0000","last_undegraded":"2024-03-26T13:59:53.314244+0000","last_fullsized":"2024-03-26T13:59:53.314244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264016+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:49.264016+0000","last_peered":"2024-03-26T13:59:49.264016+0000","last_clean":"2024-03-26T13:59:49.264016+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:49.264016+0000","last_undegraded":"2024-03-26T13:59:49.264016+0000","last_fullsized":"2024-03-26T13:59:49.264016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349048+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:29.349048+0000","last_peered":"2024-03-26T13:59:29.349048+0000","last_clean":"2024-03-26T13:59:29.349048+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:29.349048+0000","last_undegraded":"2024-03-26T13:59:29.349048+0000","last_fullsized":"2024-03-26T13:59:29.349048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314137+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:53.314137+0000","last_peered":"2024-03-26T13:59:53.314137+0000","last_clean":"2024-03-26T13:59:53.314137+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:53.314137+0000","last_undegraded":"2024-03-26T13:59:53.314137+0000","last_fullsized":"2024-03-26T13:59:53.314137+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469895+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:29.469895+0000","last_peered":"2024-03-26T13:59:29.469895+0000","last_clean":"2024-03-26T13:59:29.469895+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:29.469895+0000","last_undegraded":"2024-03-26T13:59:29.469895+0000","last_fullsized":"2024-03-26T13:59:29.469895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841812+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:27.841812+0000","last_peered":"2024-03-26T13:59:27.841812+0000","last_clean":"2024-03-26T13:59:27.841812+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:27.841812+0000","last_undegraded":"2024-03-26T13:59:27.841812+0000","last_fullsized":"2024-03-26T13:59:27.841812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.236338+0000","last_change":"2024-03-26T13:59:53.317881+0000","last_active":"2024-03-26T13:59:53.317881+0000","last_peered":"2024-03-26T13:59:30.236338+0000","last_clean":"2024-03-26T13:59:30.236338+0000","last_became_active":"2024-03-26T13:59:27.323585+0000","last_became_peered":"2024-03-26T13:59:27.323585+0000","last_unstale":"2024-03-26T13:59:30.236338+0000","last_undegraded":"2024-03-26T13:59:30.236338+0000","last_fullsized":"2024-03-26T13:59:30.236338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7,8,11],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.265450+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:50.265450+0000","last_peered":"2024-03-26T13:59:50.265450+0000","last_clean":"2024-03-26T13:59:50.265450+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:50.265450+0000","last_undegraded":"2024-03-26T13:59:50.265450+0000","last_fullsized":"2024-03-26T13:59:50.265450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469799+0000","last_change":"2024-03-26T13:59:23.299805+0000","last_active":"2024-03-26T13:59:29.469799+0000","last_peered":"2024-03-26T13:59:29.469799+0000","last_clean":"2024-03-26T13:59:29.469799+0000","last_became_active":"2024-03-26T13:59:23.295723+0000","last_became_peered":"2024-03-26T13:59:23.295723+0000","last_unstale":"2024-03-26T13:59:29.469799+0000","last_undegraded":"2024-03-26T13:59:29.469799+0000","last_fullsized":"2024-03-26T13:59:29.469799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:25:05.112905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.234348+0000","last_change":"2024-03-26T13:59:53.316731+0000","last_active":"2024-03-26T13:59:53.316731+0000","last_peered":"2024-03-26T13:59:30.234348+0000","last_clean":"2024-03-26T13:59:30.234348+0000","last_became_active":"2024-03-26T13:59:27.324762+0000","last_became_peered":"2024-03-26T13:59:27.324762+0000","last_unstale":"2024-03-26T13:59:30.234348+0000","last_undegraded":"2024-03-26T13:59:30.234348+0000","last_fullsized":"2024-03-26T13:59:30.234348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.261450+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:49.261450+0000","last_peered":"2024-03-26T13:59:49.261450+0000","last_clean":"2024-03-26T13:59:49.261450+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:49.261450+0000","last_undegraded":"2024-03-26T13:59:49.261450+0000","last_fullsized":"2024-03-26T13:59:49.261450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841919+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:27.841919+0000","last_peered":"2024-03-26T13:59:27.841919+0000","last_clean":"2024-03-26T13:59:27.841919+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:27.841919+0000","last_undegraded":"2024-03-26T13:59:27.841919+0000","last_fullsized":"2024-03-26T13:59:27.841919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235034+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:30.235034+0000","last_peered":"2024-03-26T13:59:30.235034+0000","last_clean":"2024-03-26T13:59:30.235034+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:30.235034+0000","last_undegraded":"2024-03-26T13:59:30.235034+0000","last_fullsized":"2024-03-26T13:59:30.235034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264064+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:49.264064+0000","last_peered":"2024-03-26T13:59:49.264064+0000","last_clean":"2024-03-26T13:59:49.264064+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:49.264064+0000","last_undegraded":"2024-03-26T13:59:49.264064+0000","last_fullsized":"2024-03-26T13:59:49.264064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324313+0000","last_change":"2024-03-26T13:59:23.298324+0000","last_active":"2024-03-26T13:59:28.324313+0000","last_peered":"2024-03-26T13:59:28.324313+0000","last_clean":"2024-03-26T13:59:28.324313+0000","last_became_active":"2024-03-26T13:59:23.297577+0000","last_became_peered":"2024-03-26T13:59:23.297577+0000","last_unstale":"2024-03-26T13:59:28.324313+0000","last_undegraded":"2024-03-26T13:59:28.324313+0000","last_fullsized":"2024-03-26T13:59:28.324313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:24.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364431+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:29.364431+0000","last_peered":"2024-03-26T13:59:29.364431+0000","last_clean":"2024-03-26T13:59:29.364431+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:29.364431+0000","last_undegraded":"2024-03-26T13:59:29.364431+0000","last_fullsized":"2024-03-26T13:59:29.364431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840981+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:27.840981+0000","last_peered":"2024-03-26T13:59:27.840981+0000","last_clean":"2024-03-26T13:59:27.840981+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:27.840981+0000","last_undegraded":"2024-03-26T13:59:27.840981+0000","last_fullsized":"2024-03-26T13:59:27.840981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230691+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:30.230691+0000","last_peered":"2024-03-26T13:59:30.230691+0000","last_clean":"2024-03-26T13:59:30.230691+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:30.230691+0000","last_undegraded":"2024-03-26T13:59:30.230691+0000","last_fullsized":"2024-03-26T13:59:30.230691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324666+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:28.324666+0000","last_peered":"2024-03-26T13:59:28.324666+0000","last_clean":"2024-03-26T13:59:28.324666+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:28.324666+0000","last_undegraded":"2024-03-26T13:59:28.324666+0000","last_fullsized":"2024-03-26T13:59:28.324666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.250279+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:48.250279+0000","last_peered":"2024-03-26T13:59:48.250279+0000","last_clean":"2024-03-26T13:59:48.250279+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:48.250279+0000","last_undegraded":"2024-03-26T13:59:48.250279+0000","last_fullsized":"2024-03-26T13:59:48.250279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349771+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:29.349771+0000","last_peered":"2024-03-26T13:59:29.349771+0000","last_clean":"2024-03-26T13:59:29.349771+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:29.349771+0000","last_undegraded":"2024-03-26T13:59:29.349771+0000","last_fullsized":"2024-03-26T13:59:29.349771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231033+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:30.231033+0000","last_peered":"2024-03-26T13:59:30.231033+0000","last_clean":"2024-03-26T13:59:30.231033+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:30.231033+0000","last_undegraded":"2024-03-26T13:59:30.231033+0000","last_fullsized":"2024-03-26T13:59:30.231033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.243150+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:48.243150+0000","last_peered":"2024-03-26T13:59:48.243150+0000","last_clean":"2024-03-26T13:59:48.243150+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:48.243150+0000","last_undegraded":"2024-03-26T13:59:48.243150+0000","last_fullsized":"2024-03-26T13:59:48.243150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364498+0000","last_change":"2024-03-26T13:59:23.314458+0000","last_active":"2024-03-26T13:59:29.364498+0000","last_peered":"2024-03-26T13:59:29.364498+0000","last_clean":"2024-03-26T13:59:29.364498+0000","last_became_active":"2024-03-26T13:59:23.314060+0000","last_became_peered":"2024-03-26T13:59:23.314060+0000","last_unstale":"2024-03-26T13:59:29.364498+0000","last_undegraded":"2024-03-26T13:59:29.364498+0000","last_fullsized":"2024-03-26T13:59:29.364498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:11:40.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314347+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:53.314347+0000","last_peered":"2024-03-26T13:59:53.314347+0000","last_clean":"2024-03-26T13:59:53.314347+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:53.314347+0000","last_undegraded":"2024-03-26T13:59:53.314347+0000","last_fullsized":"2024-03-26T13:59:53.314347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.242523+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:48.242523+0000","last_peered":"2024-03-26T13:59:48.242523+0000","last_clean":"2024-03-26T13:59:48.242523+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:48.242523+0000","last_undegraded":"2024-03-26T13:59:48.242523+0000","last_fullsized":"2024-03-26T13:59:48.242523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349320+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:29.349320+0000","last_peered":"2024-03-26T13:59:29.349320+0000","last_clean":"2024-03-26T13:59:29.349320+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:29.349320+0000","last_undegraded":"2024-03-26T13:59:29.349320+0000","last_fullsized":"2024-03-26T13:59:29.349320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230980+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:30.230980+0000","last_peered":"2024-03-26T13:59:30.230980+0000","last_clean":"2024-03-26T13:59:30.230980+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:30.230980+0000","last_undegraded":"2024-03-26T13:59:30.230980+0000","last_fullsized":"2024-03-26T13:59:30.230980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.748837+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:50.748837+0000","last_peered":"2024-03-26T13:59:50.748837+0000","last_clean":"2024-03-26T13:59:50.748837+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:50.748837+0000","last_undegraded":"2024-03-26T13:59:50.748837+0000","last_fullsized":"2024-03-26T13:59:50.748837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.470191+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:29.470191+0000","last_peered":"2024-03-26T13:59:29.470191+0000","last_clean":"2024-03-26T13:59:29.470191+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:29.470191+0000","last_undegraded":"2024-03-26T13:59:29.470191+0000","last_fullsized":"2024-03-26T13:59:29.470191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842554+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:27.842554+0000","last_peered":"2024-03-26T13:59:27.842554+0000","last_clean":"2024-03-26T13:59:27.842554+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:27.842554+0000","last_undegraded":"2024-03-26T13:59:27.842554+0000","last_fullsized":"2024-03-26T13:59:27.842554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.318377+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:53.318377+0000","last_peered":"2024-03-26T13:59:53.318377+0000","last_clean":"2024-03-26T13:59:53.318377+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:53.318377+0000","last_undegraded":"2024-03-26T13:59:53.318377+0000","last_fullsized":"2024-03-26T13:59:53.318377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839344+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:27.839344+0000","last_peered":"2024-03-26T13:59:27.839344+0000","last_clean":"2024-03-26T13:59:27.839344+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:27.839344+0000","last_undegraded":"2024-03-26T13:59:27.839344+0000","last_fullsized":"2024-03-26T13:59:27.839344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233995+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:30.233995+0000","last_peered":"2024-03-26T13:59:30.233995+0000","last_clean":"2024-03-26T13:59:30.233995+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:30.233995+0000","last_undegraded":"2024-03-26T13:59:30.233995+0000","last_fullsized":"2024-03-26T13:59:30.233995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.248560+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:48.248560+0000","last_peered":"2024-03-26T13:59:48.248560+0000","last_clean":"2024-03-26T13:59:48.248560+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:48.248560+0000","last_undegraded":"2024-03-26T13:59:48.248560+0000","last_fullsized":"2024-03-26T13:59:48.248560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840635+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:27.840635+0000","last_peered":"2024-03-26T13:59:27.840635+0000","last_clean":"2024-03-26T13:59:27.840635+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:27.840635+0000","last_undegraded":"2024-03-26T13:59:27.840635+0000","last_fullsized":"2024-03-26T13:59:27.840635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231524+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:30.231524+0000","last_peered":"2024-03-26T13:59:30.231524+0000","last_clean":"2024-03-26T13:59:30.231524+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:30.231524+0000","last_undegraded":"2024-03-26T13:59:30.231524+0000","last_fullsized":"2024-03-26T13:59:30.231524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316181+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:53.316181+0000","last_peered":"2024-03-26T13:59:53.316181+0000","last_clean":"2024-03-26T13:59:53.316181+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:53.316181+0000","last_undegraded":"2024-03-26T13:59:53.316181+0000","last_fullsized":"2024-03-26T13:59:53.316181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840102+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:27.840102+0000","last_peered":"2024-03-26T13:59:27.840102+0000","last_clean":"2024-03-26T13:59:27.840102+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:27.840102+0000","last_undegraded":"2024-03-26T13:59:27.840102+0000","last_fullsized":"2024-03-26T13:59:27.840102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840430+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:27.840430+0000","last_peered":"2024-03-26T13:59:27.840430+0000","last_clean":"2024-03-26T13:59:27.840430+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:27.840430+0000","last_undegraded":"2024-03-26T13:59:27.840430+0000","last_fullsized":"2024-03-26T13:59:27.840430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.266019+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:49.266019+0000","last_peered":"2024-03-26T13:59:49.266019+0000","last_clean":"2024-03-26T13:59:49.266019+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:49.266019+0000","last_undegraded":"2024-03-26T13:59:49.266019+0000","last_fullsized":"2024-03-26T13:59:49.266019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840790+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:27.840790+0000","last_peered":"2024-03-26T13:59:27.840790+0000","last_clean":"2024-03-26T13:59:27.840790+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:27.840790+0000","last_undegraded":"2024-03-26T13:59:27.840790+0000","last_fullsized":"2024-03-26T13:59:27.840790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841011+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:27.841011+0000","last_peered":"2024-03-26T13:59:27.841011+0000","last_clean":"2024-03-26T13:59:27.841011+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:27.841011+0000","last_undegraded":"2024-03-26T13:59:27.841011+0000","last_fullsized":"2024-03-26T13:59:27.841011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.246505+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:50.246505+0000","last_peered":"2024-03-26T13:59:50.246505+0000","last_clean":"2024-03-26T13:59:50.246505+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:50.246505+0000","last_undegraded":"2024-03-26T13:59:50.246505+0000","last_fullsized":"2024-03-26T13:59:50.246505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234557+0000","last_change":"2024-03-26T13:59:23.314700+0000","last_active":"2024-03-26T13:59:30.234557+0000","last_peered":"2024-03-26T13:59:30.234557+0000","last_clean":"2024-03-26T13:59:30.234557+0000","last_became_active":"2024-03-26T13:59:23.314372+0000","last_became_peered":"2024-03-26T13:59:23.314372+0000","last_unstale":"2024-03-26T13:59:30.234557+0000","last_undegraded":"2024-03-26T13:59:30.234557+0000","last_fullsized":"2024-03-26T13:59:30.234557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:32.631858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469875+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:29.469875+0000","last_peered":"2024-03-26T13:59:29.469875+0000","last_clean":"2024-03-26T13:59:29.469875+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:29.469875+0000","last_undegraded":"2024-03-26T13:59:29.469875+0000","last_fullsized":"2024-03-26T13:59:29.469875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231159+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:30.231159+0000","last_peered":"2024-03-26T13:59:30.231159+0000","last_clean":"2024-03-26T13:59:30.231159+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:30.231159+0000","last_undegraded":"2024-03-26T13:59:30.231159+0000","last_fullsized":"2024-03-26T13:59:30.231159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782231+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:28.782231+0000","last_peered":"2024-03-26T13:59:28.782231+0000","last_clean":"2024-03-26T13:59:28.782231+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:28.782231+0000","last_undegraded":"2024-03-26T13:59:28.782231+0000","last_fullsized":"2024-03-26T13:59:28.782231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839740+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:27.839740+0000","last_peered":"2024-03-26T13:59:27.839740+0000","last_clean":"2024-03-26T13:59:27.839740+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:27.839740+0000","last_undegraded":"2024-03-26T13:59:27.839740+0000","last_fullsized":"2024-03-26T13:59:27.839740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840640+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:27.840640+0000","last_peered":"2024-03-26T13:59:27.840640+0000","last_clean":"2024-03-26T13:59:27.840640+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:27.840640+0000","last_undegraded":"2024-03-26T13:59:27.840640+0000","last_fullsized":"2024-03-26T13:59:27.840640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781589+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:28.781589+0000","last_peered":"2024-03-26T13:59:28.781589+0000","last_clean":"2024-03-26T13:59:28.781589+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:28.781589+0000","last_undegraded":"2024-03-26T13:59:28.781589+0000","last_fullsized":"2024-03-26T13:59:28.781589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840495+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:27.840495+0000","last_peered":"2024-03-26T13:59:27.840495+0000","last_clean":"2024-03-26T13:59:27.840495+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:27.840495+0000","last_undegraded":"2024-03-26T13:59:27.840495+0000","last_fullsized":"2024-03-26T13:59:27.840495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841110+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:27.841110+0000","last_peered":"2024-03-26T13:59:27.841110+0000","last_clean":"2024-03-26T13:59:27.841110+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:27.841110+0000","last_undegraded":"2024-03-26T13:59:27.841110+0000","last_fullsized":"2024-03-26T13:59:27.841110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.263864+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:50.263864+0000","last_peered":"2024-03-26T13:59:50.263864+0000","last_clean":"2024-03-26T13:59:50.263864+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:50.263864+0000","last_undegraded":"2024-03-26T13:59:50.263864+0000","last_fullsized":"2024-03-26T13:59:50.263864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841168+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:27.841168+0000","last_peered":"2024-03-26T13:59:27.841168+0000","last_clean":"2024-03-26T13:59:27.841168+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:27.841168+0000","last_undegraded":"2024-03-26T13:59:27.841168+0000","last_fullsized":"2024-03-26T13:59:27.841168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314334+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:53.314334+0000","last_peered":"2024-03-26T13:59:53.314334+0000","last_clean":"2024-03-26T13:59:53.314334+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:53.314334+0000","last_undegraded":"2024-03-26T13:59:53.314334+0000","last_fullsized":"2024-03-26T13:59:53.314334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839877+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:27.839877+0000","last_peered":"2024-03-26T13:59:27.839877+0000","last_clean":"2024-03-26T13:59:27.839877+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:27.839877+0000","last_undegraded":"2024-03-26T13:59:27.839877+0000","last_fullsized":"2024-03-26T13:59:27.839877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842500+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:27.842500+0000","last_peered":"2024-03-26T13:59:27.842500+0000","last_clean":"2024-03-26T13:59:27.842500+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:27.842500+0000","last_undegraded":"2024-03-26T13:59:27.842500+0000","last_fullsized":"2024-03-26T13:59:27.842500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.838817+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:27.838817+0000","last_peered":"2024-03-26T13:59:27.838817+0000","last_clean":"2024-03-26T13:59:27.838817+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:27.838817+0000","last_undegraded":"2024-03-26T13:59:27.838817+0000","last_fullsized":"2024-03-26T13:59:27.838817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264433+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:49.264433+0000","last_peered":"2024-03-26T13:59:49.264433+0000","last_clean":"2024-03-26T13:59:49.264433+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:49.264433+0000","last_undegraded":"2024-03-26T13:59:49.264433+0000","last_fullsized":"2024-03-26T13:59:49.264433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233597+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:30.233597+0000","last_peered":"2024-03-26T13:59:30.233597+0000","last_clean":"2024-03-26T13:59:30.233597+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:30.233597+0000","last_undegraded":"2024-03-26T13:59:30.233597+0000","last_fullsized":"2024-03-26T13:59:30.233597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246587+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:48.246587+0000","last_peered":"2024-03-26T13:59:48.246587+0000","last_clean":"2024-03-26T13:59:48.246587+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:48.246587+0000","last_undegraded":"2024-03-26T13:59:48.246587+0000","last_fullsized":"2024-03-26T13:59:48.246587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839930+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:27.839930+0000","last_peered":"2024-03-26T13:59:27.839930+0000","last_clean":"2024-03-26T13:59:27.839930+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:27.839930+0000","last_undegraded":"2024-03-26T13:59:27.839930+0000","last_fullsized":"2024-03-26T13:59:27.839930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324348+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:28.324348+0000","last_peered":"2024-03-26T13:59:28.324348+0000","last_clean":"2024-03-26T13:59:28.324348+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:28.324348+0000","last_undegraded":"2024-03-26T13:59:28.324348+0000","last_fullsized":"2024-03-26T13:59:28.324348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.260579+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:49.260579+0000","last_peered":"2024-03-26T13:59:49.260579+0000","last_clean":"2024-03-26T13:59:49.260579+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:49.260579+0000","last_undegraded":"2024-03-26T13:59:49.260579+0000","last_fullsized":"2024-03-26T13:59:49.260579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325372+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:28.325372+0000","last_peered":"2024-03-26T13:59:28.325372+0000","last_clean":"2024-03-26T13:59:28.325372+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:28.325372+0000","last_undegraded":"2024-03-26T13:59:28.325372+0000","last_fullsized":"2024-03-26T13:59:28.325372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316202+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:53.316202+0000","last_peered":"2024-03-26T13:59:53.316202+0000","last_clean":"2024-03-26T13:59:53.316202+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:53.316202+0000","last_undegraded":"2024-03-26T13:59:53.316202+0000","last_fullsized":"2024-03-26T13:59:53.316202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315881+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:53.315881+0000","last_peered":"2024-03-26T13:59:53.315881+0000","last_clean":"2024-03-26T13:59:53.315881+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:53.315881+0000","last_undegraded":"2024-03-26T13:59:53.315881+0000","last_fullsized":"2024-03-26T13:59:53.315881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364591+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:29.364591+0000","last_peered":"2024-03-26T13:59:29.364591+0000","last_clean":"2024-03-26T13:59:29.364591+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:29.364591+0000","last_undegraded":"2024-03-26T13:59:29.364591+0000","last_fullsized":"2024-03-26T13:59:29.364591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314755+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:53.314755+0000","last_peered":"2024-03-26T13:59:53.314755+0000","last_clean":"2024-03-26T13:59:53.314755+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:53.314755+0000","last_undegraded":"2024-03-26T13:59:53.314755+0000","last_fullsized":"2024-03-26T13:59:53.314755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.249700+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:48.249700+0000","last_peered":"2024-03-26T13:59:48.249700+0000","last_clean":"2024-03-26T13:59:48.249700+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:48.249700+0000","last_undegraded":"2024-03-26T13:59:48.249700+0000","last_fullsized":"2024-03-26T13:59:48.249700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469234+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:29.469234+0000","last_peered":"2024-03-26T13:59:29.469234+0000","last_clean":"2024-03-26T13:59:29.469234+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:29.469234+0000","last_undegraded":"2024-03-26T13:59:29.469234+0000","last_fullsized":"2024-03-26T13:59:29.469234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231634+0000","last_change":"2024-03-26T13:59:27.332943+0000","last_active":"2024-03-26T13:59:30.231634+0000","last_peered":"2024-03-26T13:59:30.231634+0000","last_clean":"2024-03-26T13:59:30.231634+0000","last_became_active":"2024-03-26T13:59:27.331677+0000","last_became_peered":"2024-03-26T13:59:27.331677+0000","last_unstale":"2024-03-26T13:59:30.231634+0000","last_undegraded":"2024-03-26T13:59:30.231634+0000","last_fullsized":"2024-03-26T13:59:30.231634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:19.990935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246514+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:48.246514+0000","last_peered":"2024-03-26T13:59:48.246514+0000","last_clean":"2024-03-26T13:59:48.246514+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:48.246514+0000","last_undegraded":"2024-03-26T13:59:48.246514+0000","last_fullsized":"2024-03-26T13:59:48.246514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232606+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:30.232606+0000","last_peered":"2024-03-26T13:59:30.232606+0000","last_clean":"2024-03-26T13:59:30.232606+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:30.232606+0000","last_undegraded":"2024-03-26T13:59:30.232606+0000","last_fullsized":"2024-03-26T13:59:30.232606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:52.319197+0000","last_change":"2024-03-26T13:59:53.314318+0000","last_active":"2024-03-26T13:59:53.314318+0000","last_peered":"2024-03-26T13:59:52.319197+0000","last_clean":"2024-03-26T13:59:52.319197+0000","last_became_active":"2024-03-26T13:59:27.326077+0000","last_became_peered":"2024-03-26T13:59:27.326077+0000","last_unstale":"2024-03-26T13:59:52.319197+0000","last_undegraded":"2024-03-26T13:59:52.319197+0000","last_fullsized":"2024-03-26T13:59:52.319197+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7,8,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317082+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:53.317082+0000","last_peered":"2024-03-26T13:59:53.317082+0000","last_clean":"2024-03-26T13:59:53.317082+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:53.317082+0000","last_undegraded":"2024-03-26T13:59:53.317082+0000","last_fullsized":"2024-03-26T13:59:53.317082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325092+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:28.325092+0000","last_peered":"2024-03-26T13:59:28.325092+0000","last_clean":"2024-03-26T13:59:28.325092+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:28.325092+0000","last_undegraded":"2024-03-26T13:59:28.325092+0000","last_fullsized":"2024-03-26T13:59:28.325092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.313859+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:53.313859+0000","last_peered":"2024-03-26T13:59:53.313859+0000","last_clean":"2024-03-26T13:59:53.313859+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:53.313859+0000","last_undegraded":"2024-03-26T13:59:53.313859+0000","last_fullsized":"2024-03-26T13:59:53.313859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.252376+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:48.252376+0000","last_peered":"2024-03-26T13:59:48.252376+0000","last_clean":"2024-03-26T13:59:48.252376+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:48.252376+0000","last_undegraded":"2024-03-26T13:59:48.252376+0000","last_fullsized":"2024-03-26T13:59:48.252376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840788+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:27.840788+0000","last_peered":"2024-03-26T13:59:27.840788+0000","last_clean":"2024-03-26T13:59:27.840788+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:27.840788+0000","last_undegraded":"2024-03-26T13:59:27.840788+0000","last_fullsized":"2024-03-26T13:59:27.840788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348985+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:29.348985+0000","last_peered":"2024-03-26T13:59:29.348985+0000","last_clean":"2024-03-26T13:59:29.348985+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:29.348985+0000","last_undegraded":"2024-03-26T13:59:29.348985+0000","last_fullsized":"2024-03-26T13:59:29.348985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319280+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:53.319280+0000","last_peered":"2024-03-26T13:59:53.319280+0000","last_clean":"2024-03-26T13:59:53.319280+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:53.319280+0000","last_undegraded":"2024-03-26T13:59:53.319280+0000","last_fullsized":"2024-03-26T13:59:53.319280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841263+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:27.841263+0000","last_peered":"2024-03-26T13:59:27.841263+0000","last_clean":"2024-03-26T13:59:27.841263+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:27.841263+0000","last_undegraded":"2024-03-26T13:59:27.841263+0000","last_fullsized":"2024-03-26T13:59:27.841263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363505+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:29.363505+0000","last_peered":"2024-03-26T13:59:29.363505+0000","last_clean":"2024-03-26T13:59:29.363505+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:29.363505+0000","last_undegraded":"2024-03-26T13:59:29.363505+0000","last_fullsized":"2024-03-26T13:59:29.363505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842212+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:27.842212+0000","last_peered":"2024-03-26T13:59:27.842212+0000","last_clean":"2024-03-26T13:59:27.842212+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:27.842212+0000","last_undegraded":"2024-03-26T13:59:27.842212+0000","last_fullsized":"2024-03-26T13:59:27.842212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841024+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:27.841024+0000","last_peered":"2024-03-26T13:59:27.841024+0000","last_clean":"2024-03-26T13:59:27.841024+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:27.841024+0000","last_undegraded":"2024-03-26T13:59:27.841024+0000","last_fullsized":"2024-03-26T13:59:27.841024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840326+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:27.840326+0000","last_peered":"2024-03-26T13:59:27.840326+0000","last_clean":"2024-03-26T13:59:27.840326+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:27.840326+0000","last_undegraded":"2024-03-26T13:59:27.840326+0000","last_fullsized":"2024-03-26T13:59:27.840326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840337+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:27.840337+0000","last_peered":"2024-03-26T13:59:27.840337+0000","last_clean":"2024-03-26T13:59:27.840337+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:27.840337+0000","last_undegraded":"2024-03-26T13:59:27.840337+0000","last_fullsized":"2024-03-26T13:59:27.840337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841677+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:27.841677+0000","last_peered":"2024-03-26T13:59:27.841677+0000","last_clean":"2024-03-26T13:59:27.841677+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:27.841677+0000","last_undegraded":"2024-03-26T13:59:27.841677+0000","last_fullsized":"2024-03-26T13:59:27.841677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233480+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:30.233480+0000","last_peered":"2024-03-26T13:59:30.233480+0000","last_clean":"2024-03-26T13:59:30.233480+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:30.233480+0000","last_undegraded":"2024-03-26T13:59:30.233480+0000","last_fullsized":"2024-03-26T13:59:30.233480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316709+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:53.316709+0000","last_peered":"2024-03-26T13:59:53.316709+0000","last_clean":"2024-03-26T13:59:53.316709+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:53.316709+0000","last_undegraded":"2024-03-26T13:59:53.316709+0000","last_fullsized":"2024-03-26T13:59:53.316709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316722+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:53.316722+0000","last_peered":"2024-03-26T13:59:53.316722+0000","last_clean":"2024-03-26T13:59:53.316722+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:53.316722+0000","last_undegraded":"2024-03-26T13:59:53.316722+0000","last_fullsized":"2024-03-26T13:59:53.316722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363485+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:29.363485+0000","last_peered":"2024-03-26T13:59:29.363485+0000","last_clean":"2024-03-26T13:59:29.363485+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:29.363485+0000","last_undegraded":"2024-03-26T13:59:29.363485+0000","last_fullsized":"2024-03-26T13:59:29.363485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840283+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:27.840283+0000","last_peered":"2024-03-26T13:59:27.840283+0000","last_clean":"2024-03-26T13:59:27.840283+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:27.840283+0000","last_undegraded":"2024-03-26T13:59:27.840283+0000","last_fullsized":"2024-03-26T13:59:27.840283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234703+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:30.234703+0000","last_peered":"2024-03-26T13:59:30.234703+0000","last_clean":"2024-03-26T13:59:30.234703+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:30.234703+0000","last_undegraded":"2024-03-26T13:59:30.234703+0000","last_fullsized":"2024-03-26T13:59:30.234703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319745+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:53.319745+0000","last_peered":"2024-03-26T13:59:53.319745+0000","last_clean":"2024-03-26T13:59:53.319745+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:53.319745+0000","last_undegraded":"2024-03-26T13:59:53.319745+0000","last_fullsized":"2024-03-26T13:59:53.319745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325560+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:28.325560+0000","last_peered":"2024-03-26T13:59:28.325560+0000","last_clean":"2024-03-26T13:59:28.325560+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:28.325560+0000","last_undegraded":"2024-03-26T13:59:28.325560+0000","last_fullsized":"2024-03-26T13:59:28.325560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840768+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:27.840768+0000","last_peered":"2024-03-26T13:59:27.840768+0000","last_clean":"2024-03-26T13:59:27.840768+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:27.840768+0000","last_undegraded":"2024-03-26T13:59:27.840768+0000","last_fullsized":"2024-03-26T13:59:27.840768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324700+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:28.324700+0000","last_peered":"2024-03-26T13:59:28.324700+0000","last_clean":"2024-03-26T13:59:28.324700+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:28.324700+0000","last_undegraded":"2024-03-26T13:59:28.324700+0000","last_fullsized":"2024-03-26T13:59:28.324700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348523+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:29.348523+0000","last_peered":"2024-03-26T13:59:29.348523+0000","last_clean":"2024-03-26T13:59:29.348523+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:29.348523+0000","last_undegraded":"2024-03-26T13:59:29.348523+0000","last_fullsized":"2024-03-26T13:59:29.348523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839530+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:27.839530+0000","last_peered":"2024-03-26T13:59:27.839530+0000","last_clean":"2024-03-26T13:59:27.839530+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:27.839530+0000","last_undegraded":"2024-03-26T13:59:27.839530+0000","last_fullsized":"2024-03-26T13:59:27.839530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234803+0000","last_change":"2024-03-26T13:59:23.308422+0000","last_active":"2024-03-26T13:59:30.234803+0000","last_peered":"2024-03-26T13:59:30.234803+0000","last_clean":"2024-03-26T13:59:30.234803+0000","last_became_active":"2024-03-26T13:59:23.308011+0000","last_became_peered":"2024-03-26T13:59:23.308011+0000","last_unstale":"2024-03-26T13:59:30.234803+0000","last_undegraded":"2024-03-26T13:59:30.234803+0000","last_fullsized":"2024-03-26T13:59:30.234803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:08.894340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14443,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":133,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":112630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1083045,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481084,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":481004,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1072817,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":512012,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":538070,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485397,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1076206,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":33731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T13:59:54.691 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: osdmap e91: 12 total, 12 up, 12 in 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2929915629' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T13:59:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:54 smithi159 ceph-mon[35501]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:54.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 -- 172.21.15.137:0/408390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb40772a0 msgr2=0x7f5cb4077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 --2- 172.21.15.137:0/408390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb40772a0 0x7f5cb4077700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009900 tx=0x7f5ca002bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:54.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 -- 172.21.15.137:0/408390963 shutdown_connections 2024-03-26T13:59:54.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 --2- 172.21.15.137:0/408390963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb4077c40 0x7f5cb40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 --2- 172.21.15.137:0/408390963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb40772a0 0x7f5cb4077700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 --2- 172.21.15.137:0/408390963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4076060 0x7f5cb4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.935+0000 7f5cbb600640 1 -- 172.21.15.137:0/408390963 >> 172.21.15.137:0/408390963 conn(0x7f5cb40fee50 msgr2=0x7f5cb4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:54.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.936+0000 7f5cbb600640 1 -- 172.21.15.137:0/408390963 shutdown_connections 2024-03-26T13:59:54.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.936+0000 7f5cbb600640 1 -- 172.21.15.137:0/408390963 wait complete. 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 Processor -- start 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 -- start start 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4076060 0x7f5cb419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb40772a0 0x7f5cb419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 0x7f5cb41a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4113860 con 0x7f5cb4077c40 2024-03-26T13:59:54.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41136e0 con 0x7f5cb4076060 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cbb600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41139e0 con 0x7f5cb40772a0 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.937+0000 7f5cb9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 0x7f5cb41a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 0x7f5cb41a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36434/0 (socket says 172.21.15.137:36434) 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 -- 172.21.15.137:0/163361111 learned_addr learned my addr 172.21.15.137:0/163361111 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb8c00640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4076060 0x7f5cb419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb3e00640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb40772a0 0x7f5cb419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 -- 172.21.15.137:0/163361111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb40772a0 msgr2=0x7f5cb419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb40772a0 0x7f5cb419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 -- 172.21.15.137:0/163361111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4076060 msgr2=0x7f5cb419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4076060 0x7f5cb419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 -- 172.21.15.137:0/163361111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb41a5b30 con 0x7f5cb4077c40 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb3e00640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb40772a0 0x7f5cb419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.938+0000 7f5cb9600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 0x7f5cb41a54c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00cad0 tx=0x7f5cac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.939+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00ae20 con 0x7f5cb4077c40 2024-03-26T13:59:54.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.939+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5cac005020 con 0x7f5cb4077c40 2024-03-26T13:59:54.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.939+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac00f6b0 con 0x7f5cb4077c40 2024-03-26T13:59:54.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.939+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb41a5e20 con 0x7f5cb4077c40 2024-03-26T13:59:54.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.939+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb41a6290 con 0x7f5cb4077c40 2024-03-26T13:59:54.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.940+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb4110450 con 0x7f5cb4077c40 2024-03-26T13:59:54.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.942+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f5cac0051d0 con 0x7f5cb4077c40 2024-03-26T13:59:54.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.943+0000 7f5cb1600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5c9c0a2890 0x7f5c9c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:54.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.943+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f5cac0cef90 con 0x7f5cb4077c40 2024-03-26T13:59:54.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.943+0000 7f5cb8c00640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5c9c0a2890 0x7f5c9c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:54.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.944+0000 7f5cb8c00640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5c9c0a2890 0x7f5c9c0a4d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009bd0 tx=0x7f5ca8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:54.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:54.947+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5cac014030 con 0x7f5cb4077c40 2024-03-26T13:59:55.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.146+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5cb4061ae0 con 0x7f5c9c0a2890 2024-03-26T13:59:55.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.169+0000 7f5cb1600640 1 -- 172.21.15.137:0/163361111 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340636 (secure 0 0 0) 0x7f5cb4061ae0 con 0x7f5c9c0a2890 2024-03-26T13:59:55.171 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:55.173 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5c9c0a2890 msgr2=0x7f5c9c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5c9c0a2890 0x7f5c9c0a4d50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009bd0 tx=0x7f5ca8005f90 comp rx=0 tx=0).stop 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 msgr2=0x7f5cb41a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 0x7f5cb41a54c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00cad0 tx=0x7f5cac00cf90 comp rx=0 tx=0).stop 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 shutdown_connections 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5c9c0a2890 0x7f5c9c0a4d50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5cb4077c40 0x7f5cb41a54c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5cb40772a0 0x7f5cb419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 --2- 172.21.15.137:0/163361111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5cb4076060 0x7f5cb419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 >> 172.21.15.137:0/163361111 conn(0x7f5cb40fee50 msgr2=0x7f5cb41005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.175+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 shutdown_connections 2024-03-26T13:59:55.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.176+0000 7f5cbb600640 1 -- 172.21.15.137:0/163361111 wait complete. 2024-03-26T13:59:55.191 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-03-26T13:59:55.084637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605045,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":316,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361008,"kb_used_data":11856,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580816,"statfs":{"total":1151940427776,"available":1151570755584,"internally_reserved":0,"allocated":12140544,"data_stored":7653133,"data_compressed":283341,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10037,"num_objects":40,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":86,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003855"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235192+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:30.235192+0000","last_peered":"2024-03-26T13:59:30.235192+0000","last_clean":"2024-03-26T13:59:30.235192+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:30.235192+0000","last_undegraded":"2024-03-26T13:59:30.235192+0000","last_fullsized":"2024-03-26T13:59:30.235192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364396+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:29.364396+0000","last_peered":"2024-03-26T13:59:29.364396+0000","last_clean":"2024-03-26T13:59:29.364396+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:29.364396+0000","last_undegraded":"2024-03-26T13:59:29.364396+0000","last_fullsized":"2024-03-26T13:59:29.364396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364118+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:29.364118+0000","last_peered":"2024-03-26T13:59:29.364118+0000","last_clean":"2024-03-26T13:59:29.364118+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:29.364118+0000","last_undegraded":"2024-03-26T13:59:29.364118+0000","last_fullsized":"2024-03-26T13:59:29.364118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315175+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:53.315175+0000","last_peered":"2024-03-26T13:59:53.315175+0000","last_clean":"2024-03-26T13:59:53.315175+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:53.315175+0000","last_undegraded":"2024-03-26T13:59:53.315175+0000","last_fullsized":"2024-03-26T13:59:53.315175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469440+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:29.469440+0000","last_peered":"2024-03-26T13:59:29.469440+0000","last_clean":"2024-03-26T13:59:29.469440+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:29.469440+0000","last_undegraded":"2024-03-26T13:59:29.469440+0000","last_fullsized":"2024-03-26T13:59:29.469440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781472+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:28.781472+0000","last_peered":"2024-03-26T13:59:28.781472+0000","last_clean":"2024-03-26T13:59:28.781472+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:28.781472+0000","last_undegraded":"2024-03-26T13:59:28.781472+0000","last_fullsized":"2024-03-26T13:59:28.781472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839948+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:27.839948+0000","last_peered":"2024-03-26T13:59:27.839948+0000","last_clean":"2024-03-26T13:59:27.839948+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:27.839948+0000","last_undegraded":"2024-03-26T13:59:27.839948+0000","last_fullsized":"2024-03-26T13:59:27.839948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349872+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:29.349872+0000","last_peered":"2024-03-26T13:59:29.349872+0000","last_clean":"2024-03-26T13:59:29.349872+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:29.349872+0000","last_undegraded":"2024-03-26T13:59:29.349872+0000","last_fullsized":"2024-03-26T13:59:29.349872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782777+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:28.782777+0000","last_peered":"2024-03-26T13:59:28.782777+0000","last_clean":"2024-03-26T13:59:28.782777+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:28.782777+0000","last_undegraded":"2024-03-26T13:59:28.782777+0000","last_fullsized":"2024-03-26T13:59:28.782777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841218+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:27.841218+0000","last_peered":"2024-03-26T13:59:27.841218+0000","last_clean":"2024-03-26T13:59:27.841218+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:27.841218+0000","last_undegraded":"2024-03-26T13:59:27.841218+0000","last_fullsized":"2024-03-26T13:59:27.841218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315062+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:53.315062+0000","last_peered":"2024-03-26T13:59:53.315062+0000","last_clean":"2024-03-26T13:59:53.315062+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:53.315062+0000","last_undegraded":"2024-03-26T13:59:53.315062+0000","last_fullsized":"2024-03-26T13:59:53.315062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839733+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:27.839733+0000","last_peered":"2024-03-26T13:59:27.839733+0000","last_clean":"2024-03-26T13:59:27.839733+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:27.839733+0000","last_undegraded":"2024-03-26T13:59:27.839733+0000","last_fullsized":"2024-03-26T13:59:27.839733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363825+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:29.363825+0000","last_peered":"2024-03-26T13:59:29.363825+0000","last_clean":"2024-03-26T13:59:29.363825+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:29.363825+0000","last_undegraded":"2024-03-26T13:59:29.363825+0000","last_fullsized":"2024-03-26T13:59:29.363825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363266+0000","last_change":"2024-03-26T13:59:23.312717+0000","last_active":"2024-03-26T13:59:29.363266+0000","last_peered":"2024-03-26T13:59:29.363266+0000","last_clean":"2024-03-26T13:59:29.363266+0000","last_became_active":"2024-03-26T13:59:23.312122+0000","last_became_peered":"2024-03-26T13:59:23.312122+0000","last_unstale":"2024-03-26T13:59:29.363266+0000","last_undegraded":"2024-03-26T13:59:29.363266+0000","last_fullsized":"2024-03-26T13:59:29.363266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:54.487004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348001+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:29.348001+0000","last_peered":"2024-03-26T13:59:29.348001+0000","last_clean":"2024-03-26T13:59:29.348001+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:29.348001+0000","last_undegraded":"2024-03-26T13:59:29.348001+0000","last_fullsized":"2024-03-26T13:59:29.348001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234897+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:30.234897+0000","last_peered":"2024-03-26T13:59:30.234897+0000","last_clean":"2024-03-26T13:59:30.234897+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:30.234897+0000","last_undegraded":"2024-03-26T13:59:30.234897+0000","last_fullsized":"2024-03-26T13:59:30.234897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231510+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:30.231510+0000","last_peered":"2024-03-26T13:59:30.231510+0000","last_clean":"2024-03-26T13:59:30.231510+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:30.231510+0000","last_undegraded":"2024-03-26T13:59:30.231510+0000","last_fullsized":"2024-03-26T13:59:30.231510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319222+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:53.319222+0000","last_peered":"2024-03-26T13:59:53.319222+0000","last_clean":"2024-03-26T13:59:53.319222+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:53.319222+0000","last_undegraded":"2024-03-26T13:59:53.319222+0000","last_fullsized":"2024-03-26T13:59:53.319222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317072+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:53.317072+0000","last_peered":"2024-03-26T13:59:53.317072+0000","last_clean":"2024-03-26T13:59:53.317072+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:53.317072+0000","last_undegraded":"2024-03-26T13:59:53.317072+0000","last_fullsized":"2024-03-26T13:59:53.317072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264793+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:49.264793+0000","last_peered":"2024-03-26T13:59:49.264793+0000","last_clean":"2024-03-26T13:59:49.264793+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:49.264793+0000","last_undegraded":"2024-03-26T13:59:49.264793+0000","last_fullsized":"2024-03-26T13:59:49.264793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325746+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:28.325746+0000","last_peered":"2024-03-26T13:59:28.325746+0000","last_clean":"2024-03-26T13:59:28.325746+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:28.325746+0000","last_undegraded":"2024-03-26T13:59:28.325746+0000","last_fullsized":"2024-03-26T13:59:28.325746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.313767+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:53.313767+0000","last_peered":"2024-03-26T13:59:53.313767+0000","last_clean":"2024-03-26T13:59:53.313767+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:53.313767+0000","last_undegraded":"2024-03-26T13:59:53.313767+0000","last_fullsized":"2024-03-26T13:59:53.313767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.326299+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:28.326299+0000","last_peered":"2024-03-26T13:59:28.326299+0000","last_clean":"2024-03-26T13:59:28.326299+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:28.326299+0000","last_undegraded":"2024-03-26T13:59:28.326299+0000","last_fullsized":"2024-03-26T13:59:28.326299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364222+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:29.364222+0000","last_peered":"2024-03-26T13:59:29.364222+0000","last_clean":"2024-03-26T13:59:29.364222+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:29.364222+0000","last_undegraded":"2024-03-26T13:59:29.364222+0000","last_fullsized":"2024-03-26T13:59:29.364222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314244+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:53.314244+0000","last_peered":"2024-03-26T13:59:53.314244+0000","last_clean":"2024-03-26T13:59:53.314244+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:53.314244+0000","last_undegraded":"2024-03-26T13:59:53.314244+0000","last_fullsized":"2024-03-26T13:59:53.314244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264016+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:49.264016+0000","last_peered":"2024-03-26T13:59:49.264016+0000","last_clean":"2024-03-26T13:59:49.264016+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:49.264016+0000","last_undegraded":"2024-03-26T13:59:49.264016+0000","last_fullsized":"2024-03-26T13:59:49.264016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349048+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:29.349048+0000","last_peered":"2024-03-26T13:59:29.349048+0000","last_clean":"2024-03-26T13:59:29.349048+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:29.349048+0000","last_undegraded":"2024-03-26T13:59:29.349048+0000","last_fullsized":"2024-03-26T13:59:29.349048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314137+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:53.314137+0000","last_peered":"2024-03-26T13:59:53.314137+0000","last_clean":"2024-03-26T13:59:53.314137+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:53.314137+0000","last_undegraded":"2024-03-26T13:59:53.314137+0000","last_fullsized":"2024-03-26T13:59:53.314137+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469895+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:29.469895+0000","last_peered":"2024-03-26T13:59:29.469895+0000","last_clean":"2024-03-26T13:59:29.469895+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:29.469895+0000","last_undegraded":"2024-03-26T13:59:29.469895+0000","last_fullsized":"2024-03-26T13:59:29.469895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841812+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:27.841812+0000","last_peered":"2024-03-26T13:59:27.841812+0000","last_clean":"2024-03-26T13:59:27.841812+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:27.841812+0000","last_undegraded":"2024-03-26T13:59:27.841812+0000","last_fullsized":"2024-03-26T13:59:27.841812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.236338+0000","last_change":"2024-03-26T13:59:53.317881+0000","last_active":"2024-03-26T13:59:53.317881+0000","last_peered":"2024-03-26T13:59:30.236338+0000","last_clean":"2024-03-26T13:59:30.236338+0000","last_became_active":"2024-03-26T13:59:27.323585+0000","last_became_peered":"2024-03-26T13:59:27.323585+0000","last_unstale":"2024-03-26T13:59:30.236338+0000","last_undegraded":"2024-03-26T13:59:30.236338+0000","last_fullsized":"2024-03-26T13:59:30.236338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7,8,11],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.265450+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:50.265450+0000","last_peered":"2024-03-26T13:59:50.265450+0000","last_clean":"2024-03-26T13:59:50.265450+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:50.265450+0000","last_undegraded":"2024-03-26T13:59:50.265450+0000","last_fullsized":"2024-03-26T13:59:50.265450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469799+0000","last_change":"2024-03-26T13:59:23.299805+0000","last_active":"2024-03-26T13:59:29.469799+0000","last_peered":"2024-03-26T13:59:29.469799+0000","last_clean":"2024-03-26T13:59:29.469799+0000","last_became_active":"2024-03-26T13:59:23.295723+0000","last_became_peered":"2024-03-26T13:59:23.295723+0000","last_unstale":"2024-03-26T13:59:29.469799+0000","last_undegraded":"2024-03-26T13:59:29.469799+0000","last_fullsized":"2024-03-26T13:59:29.469799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:25:05.112905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.234348+0000","last_change":"2024-03-26T13:59:53.316731+0000","last_active":"2024-03-26T13:59:53.316731+0000","last_peered":"2024-03-26T13:59:30.234348+0000","last_clean":"2024-03-26T13:59:30.234348+0000","last_became_active":"2024-03-26T13:59:27.324762+0000","last_became_peered":"2024-03-26T13:59:27.324762+0000","last_unstale":"2024-03-26T13:59:30.234348+0000","last_undegraded":"2024-03-26T13:59:30.234348+0000","last_fullsized":"2024-03-26T13:59:30.234348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.261450+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:49.261450+0000","last_peered":"2024-03-26T13:59:49.261450+0000","last_clean":"2024-03-26T13:59:49.261450+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:49.261450+0000","last_undegraded":"2024-03-26T13:59:49.261450+0000","last_fullsized":"2024-03-26T13:59:49.261450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841919+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:27.841919+0000","last_peered":"2024-03-26T13:59:27.841919+0000","last_clean":"2024-03-26T13:59:27.841919+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:27.841919+0000","last_undegraded":"2024-03-26T13:59:27.841919+0000","last_fullsized":"2024-03-26T13:59:27.841919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235034+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:30.235034+0000","last_peered":"2024-03-26T13:59:30.235034+0000","last_clean":"2024-03-26T13:59:30.235034+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:30.235034+0000","last_undegraded":"2024-03-26T13:59:30.235034+0000","last_fullsized":"2024-03-26T13:59:30.235034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264064+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:49.264064+0000","last_peered":"2024-03-26T13:59:49.264064+0000","last_clean":"2024-03-26T13:59:49.264064+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:49.264064+0000","last_undegraded":"2024-03-26T13:59:49.264064+0000","last_fullsized":"2024-03-26T13:59:49.264064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324313+0000","last_change":"2024-03-26T13:59:23.298324+0000","last_active":"2024-03-26T13:59:28.324313+0000","last_peered":"2024-03-26T13:59:28.324313+0000","last_clean":"2024-03-26T13:59:28.324313+0000","last_became_active":"2024-03-26T13:59:23.297577+0000","last_became_peered":"2024-03-26T13:59:23.297577+0000","last_unstale":"2024-03-26T13:59:28.324313+0000","last_undegraded":"2024-03-26T13:59:28.324313+0000","last_fullsized":"2024-03-26T13:59:28.324313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:24.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364431+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:29.364431+0000","last_peered":"2024-03-26T13:59:29.364431+0000","last_clean":"2024-03-26T13:59:29.364431+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:29.364431+0000","last_undegraded":"2024-03-26T13:59:29.364431+0000","last_fullsized":"2024-03-26T13:59:29.364431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840981+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:27.840981+0000","last_peered":"2024-03-26T13:59:27.840981+0000","last_clean":"2024-03-26T13:59:27.840981+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:27.840981+0000","last_undegraded":"2024-03-26T13:59:27.840981+0000","last_fullsized":"2024-03-26T13:59:27.840981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230691+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:30.230691+0000","last_peered":"2024-03-26T13:59:30.230691+0000","last_clean":"2024-03-26T13:59:30.230691+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:30.230691+0000","last_undegraded":"2024-03-26T13:59:30.230691+0000","last_fullsized":"2024-03-26T13:59:30.230691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324666+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:28.324666+0000","last_peered":"2024-03-26T13:59:28.324666+0000","last_clean":"2024-03-26T13:59:28.324666+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:28.324666+0000","last_undegraded":"2024-03-26T13:59:28.324666+0000","last_fullsized":"2024-03-26T13:59:28.324666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.250279+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:48.250279+0000","last_peered":"2024-03-26T13:59:48.250279+0000","last_clean":"2024-03-26T13:59:48.250279+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:48.250279+0000","last_undegraded":"2024-03-26T13:59:48.250279+0000","last_fullsized":"2024-03-26T13:59:48.250279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349771+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:29.349771+0000","last_peered":"2024-03-26T13:59:29.349771+0000","last_clean":"2024-03-26T13:59:29.349771+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:29.349771+0000","last_undegraded":"2024-03-26T13:59:29.349771+0000","last_fullsized":"2024-03-26T13:59:29.349771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231033+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:30.231033+0000","last_peered":"2024-03-26T13:59:30.231033+0000","last_clean":"2024-03-26T13:59:30.231033+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:30.231033+0000","last_undegraded":"2024-03-26T13:59:30.231033+0000","last_fullsized":"2024-03-26T13:59:30.231033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.243150+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:48.243150+0000","last_peered":"2024-03-26T13:59:48.243150+0000","last_clean":"2024-03-26T13:59:48.243150+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:48.243150+0000","last_undegraded":"2024-03-26T13:59:48.243150+0000","last_fullsized":"2024-03-26T13:59:48.243150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364498+0000","last_change":"2024-03-26T13:59:23.314458+0000","last_active":"2024-03-26T13:59:29.364498+0000","last_peered":"2024-03-26T13:59:29.364498+0000","last_clean":"2024-03-26T13:59:29.364498+0000","last_became_active":"2024-03-26T13:59:23.314060+0000","last_became_peered":"2024-03-26T13:59:23.314060+0000","last_unstale":"2024-03-26T13:59:29.364498+0000","last_undegraded":"2024-03-26T13:59:29.364498+0000","last_fullsized":"2024-03-26T13:59:29.364498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:11:40.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314347+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:53.314347+0000","last_peered":"2024-03-26T13:59:53.314347+0000","last_clean":"2024-03-26T13:59:53.314347+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:53.314347+0000","last_undegraded":"2024-03-26T13:59:53.314347+0000","last_fullsized":"2024-03-26T13:59:53.314347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.242523+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:48.242523+0000","last_peered":"2024-03-26T13:59:48.242523+0000","last_clean":"2024-03-26T13:59:48.242523+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:48.242523+0000","last_undegraded":"2024-03-26T13:59:48.242523+0000","last_fullsized":"2024-03-26T13:59:48.242523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349320+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:29.349320+0000","last_peered":"2024-03-26T13:59:29.349320+0000","last_clean":"2024-03-26T13:59:29.349320+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:29.349320+0000","last_undegraded":"2024-03-26T13:59:29.349320+0000","last_fullsized":"2024-03-26T13:59:29.349320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230980+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:30.230980+0000","last_peered":"2024-03-26T13:59:30.230980+0000","last_clean":"2024-03-26T13:59:30.230980+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:30.230980+0000","last_undegraded":"2024-03-26T13:59:30.230980+0000","last_fullsized":"2024-03-26T13:59:30.230980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.748837+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:50.748837+0000","last_peered":"2024-03-26T13:59:50.748837+0000","last_clean":"2024-03-26T13:59:50.748837+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:50.748837+0000","last_undegraded":"2024-03-26T13:59:50.748837+0000","last_fullsized":"2024-03-26T13:59:50.748837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.470191+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:29.470191+0000","last_peered":"2024-03-26T13:59:29.470191+0000","last_clean":"2024-03-26T13:59:29.470191+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:29.470191+0000","last_undegraded":"2024-03-26T13:59:29.470191+0000","last_fullsized":"2024-03-26T13:59:29.470191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842554+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:27.842554+0000","last_peered":"2024-03-26T13:59:27.842554+0000","last_clean":"2024-03-26T13:59:27.842554+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:27.842554+0000","last_undegraded":"2024-03-26T13:59:27.842554+0000","last_fullsized":"2024-03-26T13:59:27.842554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.318377+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:53.318377+0000","last_peered":"2024-03-26T13:59:53.318377+0000","last_clean":"2024-03-26T13:59:53.318377+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:53.318377+0000","last_undegraded":"2024-03-26T13:59:53.318377+0000","last_fullsized":"2024-03-26T13:59:53.318377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839344+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:27.839344+0000","last_peered":"2024-03-26T13:59:27.839344+0000","last_clean":"2024-03-26T13:59:27.839344+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:27.839344+0000","last_undegraded":"2024-03-26T13:59:27.839344+0000","last_fullsized":"2024-03-26T13:59:27.839344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233995+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:30.233995+0000","last_peered":"2024-03-26T13:59:30.233995+0000","last_clean":"2024-03-26T13:59:30.233995+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:30.233995+0000","last_undegraded":"2024-03-26T13:59:30.233995+0000","last_fullsized":"2024-03-26T13:59:30.233995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.248560+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:48.248560+0000","last_peered":"2024-03-26T13:59:48.248560+0000","last_clean":"2024-03-26T13:59:48.248560+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:48.248560+0000","last_undegraded":"2024-03-26T13:59:48.248560+0000","last_fullsized":"2024-03-26T13:59:48.248560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840635+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:27.840635+0000","last_peered":"2024-03-26T13:59:27.840635+0000","last_clean":"2024-03-26T13:59:27.840635+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:27.840635+0000","last_undegraded":"2024-03-26T13:59:27.840635+0000","last_fullsized":"2024-03-26T13:59:27.840635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231524+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:30.231524+0000","last_peered":"2024-03-26T13:59:30.231524+0000","last_clean":"2024-03-26T13:59:30.231524+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:30.231524+0000","last_undegraded":"2024-03-26T13:59:30.231524+0000","last_fullsized":"2024-03-26T13:59:30.231524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316181+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:53.316181+0000","last_peered":"2024-03-26T13:59:53.316181+0000","last_clean":"2024-03-26T13:59:53.316181+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:53.316181+0000","last_undegraded":"2024-03-26T13:59:53.316181+0000","last_fullsized":"2024-03-26T13:59:53.316181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840102+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:27.840102+0000","last_peered":"2024-03-26T13:59:27.840102+0000","last_clean":"2024-03-26T13:59:27.840102+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:27.840102+0000","last_undegraded":"2024-03-26T13:59:27.840102+0000","last_fullsized":"2024-03-26T13:59:27.840102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840430+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:27.840430+0000","last_peered":"2024-03-26T13:59:27.840430+0000","last_clean":"2024-03-26T13:59:27.840430+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:27.840430+0000","last_undegraded":"2024-03-26T13:59:27.840430+0000","last_fullsized":"2024-03-26T13:59:27.840430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.266019+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:49.266019+0000","last_peered":"2024-03-26T13:59:49.266019+0000","last_clean":"2024-03-26T13:59:49.266019+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:49.266019+0000","last_undegraded":"2024-03-26T13:59:49.266019+0000","last_fullsized":"2024-03-26T13:59:49.266019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840790+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:27.840790+0000","last_peered":"2024-03-26T13:59:27.840790+0000","last_clean":"2024-03-26T13:59:27.840790+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:27.840790+0000","last_undegraded":"2024-03-26T13:59:27.840790+0000","last_fullsized":"2024-03-26T13:59:27.840790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841011+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:27.841011+0000","last_peered":"2024-03-26T13:59:27.841011+0000","last_clean":"2024-03-26T13:59:27.841011+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:27.841011+0000","last_undegraded":"2024-03-26T13:59:27.841011+0000","last_fullsized":"2024-03-26T13:59:27.841011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.246505+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:50.246505+0000","last_peered":"2024-03-26T13:59:50.246505+0000","last_clean":"2024-03-26T13:59:50.246505+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:50.246505+0000","last_undegraded":"2024-03-26T13:59:50.246505+0000","last_fullsized":"2024-03-26T13:59:50.246505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234557+0000","last_change":"2024-03-26T13:59:23.314700+0000","last_active":"2024-03-26T13:59:30.234557+0000","last_peered":"2024-03-26T13:59:30.234557+0000","last_clean":"2024-03-26T13:59:30.234557+0000","last_became_active":"2024-03-26T13:59:23.314372+0000","last_became_peered":"2024-03-26T13:59:23.314372+0000","last_unstale":"2024-03-26T13:59:30.234557+0000","last_undegraded":"2024-03-26T13:59:30.234557+0000","last_fullsized":"2024-03-26T13:59:30.234557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:32.631858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469875+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:29.469875+0000","last_peered":"2024-03-26T13:59:29.469875+0000","last_clean":"2024-03-26T13:59:29.469875+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:29.469875+0000","last_undegraded":"2024-03-26T13:59:29.469875+0000","last_fullsized":"2024-03-26T13:59:29.469875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231159+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:30.231159+0000","last_peered":"2024-03-26T13:59:30.231159+0000","last_clean":"2024-03-26T13:59:30.231159+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:30.231159+0000","last_undegraded":"2024-03-26T13:59:30.231159+0000","last_fullsized":"2024-03-26T13:59:30.231159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782231+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:28.782231+0000","last_peered":"2024-03-26T13:59:28.782231+0000","last_clean":"2024-03-26T13:59:28.782231+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:28.782231+0000","last_undegraded":"2024-03-26T13:59:28.782231+0000","last_fullsized":"2024-03-26T13:59:28.782231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839740+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:27.839740+0000","last_peered":"2024-03-26T13:59:27.839740+0000","last_clean":"2024-03-26T13:59:27.839740+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:27.839740+0000","last_undegraded":"2024-03-26T13:59:27.839740+0000","last_fullsized":"2024-03-26T13:59:27.839740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840640+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:27.840640+0000","last_peered":"2024-03-26T13:59:27.840640+0000","last_clean":"2024-03-26T13:59:27.840640+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:27.840640+0000","last_undegraded":"2024-03-26T13:59:27.840640+0000","last_fullsized":"2024-03-26T13:59:27.840640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781589+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:28.781589+0000","last_peered":"2024-03-26T13:59:28.781589+0000","last_clean":"2024-03-26T13:59:28.781589+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:28.781589+0000","last_undegraded":"2024-03-26T13:59:28.781589+0000","last_fullsized":"2024-03-26T13:59:28.781589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840495+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:27.840495+0000","last_peered":"2024-03-26T13:59:27.840495+0000","last_clean":"2024-03-26T13:59:27.840495+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:27.840495+0000","last_undegraded":"2024-03-26T13:59:27.840495+0000","last_fullsized":"2024-03-26T13:59:27.840495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841110+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:27.841110+0000","last_peered":"2024-03-26T13:59:27.841110+0000","last_clean":"2024-03-26T13:59:27.841110+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:27.841110+0000","last_undegraded":"2024-03-26T13:59:27.841110+0000","last_fullsized":"2024-03-26T13:59:27.841110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.263864+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:50.263864+0000","last_peered":"2024-03-26T13:59:50.263864+0000","last_clean":"2024-03-26T13:59:50.263864+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:50.263864+0000","last_undegraded":"2024-03-26T13:59:50.263864+0000","last_fullsized":"2024-03-26T13:59:50.263864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841168+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:27.841168+0000","last_peered":"2024-03-26T13:59:27.841168+0000","last_clean":"2024-03-26T13:59:27.841168+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:27.841168+0000","last_undegraded":"2024-03-26T13:59:27.841168+0000","last_fullsized":"2024-03-26T13:59:27.841168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314334+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:53.314334+0000","last_peered":"2024-03-26T13:59:53.314334+0000","last_clean":"2024-03-26T13:59:53.314334+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:53.314334+0000","last_undegraded":"2024-03-26T13:59:53.314334+0000","last_fullsized":"2024-03-26T13:59:53.314334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839877+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:27.839877+0000","last_peered":"2024-03-26T13:59:27.839877+0000","last_clean":"2024-03-26T13:59:27.839877+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:27.839877+0000","last_undegraded":"2024-03-26T13:59:27.839877+0000","last_fullsized":"2024-03-26T13:59:27.839877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842500+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:27.842500+0000","last_peered":"2024-03-26T13:59:27.842500+0000","last_clean":"2024-03-26T13:59:27.842500+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:27.842500+0000","last_undegraded":"2024-03-26T13:59:27.842500+0000","last_fullsized":"2024-03-26T13:59:27.842500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.838817+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:27.838817+0000","last_peered":"2024-03-26T13:59:27.838817+0000","last_clean":"2024-03-26T13:59:27.838817+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:27.838817+0000","last_undegraded":"2024-03-26T13:59:27.838817+0000","last_fullsized":"2024-03-26T13:59:27.838817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264433+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:49.264433+0000","last_peered":"2024-03-26T13:59:49.264433+0000","last_clean":"2024-03-26T13:59:49.264433+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:49.264433+0000","last_undegraded":"2024-03-26T13:59:49.264433+0000","last_fullsized":"2024-03-26T13:59:49.264433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233597+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:30.233597+0000","last_peered":"2024-03-26T13:59:30.233597+0000","last_clean":"2024-03-26T13:59:30.233597+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:30.233597+0000","last_undegraded":"2024-03-26T13:59:30.233597+0000","last_fullsized":"2024-03-26T13:59:30.233597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246587+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:48.246587+0000","last_peered":"2024-03-26T13:59:48.246587+0000","last_clean":"2024-03-26T13:59:48.246587+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:48.246587+0000","last_undegraded":"2024-03-26T13:59:48.246587+0000","last_fullsized":"2024-03-26T13:59:48.246587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839930+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:27.839930+0000","last_peered":"2024-03-26T13:59:27.839930+0000","last_clean":"2024-03-26T13:59:27.839930+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:27.839930+0000","last_undegraded":"2024-03-26T13:59:27.839930+0000","last_fullsized":"2024-03-26T13:59:27.839930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324348+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:28.324348+0000","last_peered":"2024-03-26T13:59:28.324348+0000","last_clean":"2024-03-26T13:59:28.324348+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:28.324348+0000","last_undegraded":"2024-03-26T13:59:28.324348+0000","last_fullsized":"2024-03-26T13:59:28.324348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.260579+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:49.260579+0000","last_peered":"2024-03-26T13:59:49.260579+0000","last_clean":"2024-03-26T13:59:49.260579+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:49.260579+0000","last_undegraded":"2024-03-26T13:59:49.260579+0000","last_fullsized":"2024-03-26T13:59:49.260579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325372+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:28.325372+0000","last_peered":"2024-03-26T13:59:28.325372+0000","last_clean":"2024-03-26T13:59:28.325372+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:28.325372+0000","last_undegraded":"2024-03-26T13:59:28.325372+0000","last_fullsized":"2024-03-26T13:59:28.325372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316202+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:53.316202+0000","last_peered":"2024-03-26T13:59:53.316202+0000","last_clean":"2024-03-26T13:59:53.316202+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:53.316202+0000","last_undegraded":"2024-03-26T13:59:53.316202+0000","last_fullsized":"2024-03-26T13:59:53.316202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315881+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:53.315881+0000","last_peered":"2024-03-26T13:59:53.315881+0000","last_clean":"2024-03-26T13:59:53.315881+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:53.315881+0000","last_undegraded":"2024-03-26T13:59:53.315881+0000","last_fullsized":"2024-03-26T13:59:53.315881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364591+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:29.364591+0000","last_peered":"2024-03-26T13:59:29.364591+0000","last_clean":"2024-03-26T13:59:29.364591+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:29.364591+0000","last_undegraded":"2024-03-26T13:59:29.364591+0000","last_fullsized":"2024-03-26T13:59:29.364591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314755+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:53.314755+0000","last_peered":"2024-03-26T13:59:53.314755+0000","last_clean":"2024-03-26T13:59:53.314755+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:53.314755+0000","last_undegraded":"2024-03-26T13:59:53.314755+0000","last_fullsized":"2024-03-26T13:59:53.314755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.249700+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:48.249700+0000","last_peered":"2024-03-26T13:59:48.249700+0000","last_clean":"2024-03-26T13:59:48.249700+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:48.249700+0000","last_undegraded":"2024-03-26T13:59:48.249700+0000","last_fullsized":"2024-03-26T13:59:48.249700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469234+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:29.469234+0000","last_peered":"2024-03-26T13:59:29.469234+0000","last_clean":"2024-03-26T13:59:29.469234+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:29.469234+0000","last_undegraded":"2024-03-26T13:59:29.469234+0000","last_fullsized":"2024-03-26T13:59:29.469234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231634+0000","last_change":"2024-03-26T13:59:27.332943+0000","last_active":"2024-03-26T13:59:30.231634+0000","last_peered":"2024-03-26T13:59:30.231634+0000","last_clean":"2024-03-26T13:59:30.231634+0000","last_became_active":"2024-03-26T13:59:27.331677+0000","last_became_peered":"2024-03-26T13:59:27.331677+0000","last_unstale":"2024-03-26T13:59:30.231634+0000","last_undegraded":"2024-03-26T13:59:30.231634+0000","last_fullsized":"2024-03-26T13:59:30.231634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:19.990935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246514+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:48.246514+0000","last_peered":"2024-03-26T13:59:48.246514+0000","last_clean":"2024-03-26T13:59:48.246514+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:48.246514+0000","last_undegraded":"2024-03-26T13:59:48.246514+0000","last_fullsized":"2024-03-26T13:59:48.246514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232606+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:30.232606+0000","last_peered":"2024-03-26T13:59:30.232606+0000","last_clean":"2024-03-26T13:59:30.232606+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:30.232606+0000","last_undegraded":"2024-03-26T13:59:30.232606+0000","last_fullsized":"2024-03-26T13:59:30.232606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:52.319197+0000","last_change":"2024-03-26T13:59:53.314318+0000","last_active":"2024-03-26T13:59:53.314318+0000","last_peered":"2024-03-26T13:59:52.319197+0000","last_clean":"2024-03-26T13:59:52.319197+0000","last_became_active":"2024-03-26T13:59:27.326077+0000","last_became_peered":"2024-03-26T13:59:27.326077+0000","last_unstale":"2024-03-26T13:59:52.319197+0000","last_undegraded":"2024-03-26T13:59:52.319197+0000","last_fullsized":"2024-03-26T13:59:52.319197+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7,8,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317082+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:53.317082+0000","last_peered":"2024-03-26T13:59:53.317082+0000","last_clean":"2024-03-26T13:59:53.317082+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:53.317082+0000","last_undegraded":"2024-03-26T13:59:53.317082+0000","last_fullsized":"2024-03-26T13:59:53.317082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325092+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:28.325092+0000","last_peered":"2024-03-26T13:59:28.325092+0000","last_clean":"2024-03-26T13:59:28.325092+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:28.325092+0000","last_undegraded":"2024-03-26T13:59:28.325092+0000","last_fullsized":"2024-03-26T13:59:28.325092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.313859+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:53.313859+0000","last_peered":"2024-03-26T13:59:53.313859+0000","last_clean":"2024-03-26T13:59:53.313859+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:53.313859+0000","last_undegraded":"2024-03-26T13:59:53.313859+0000","last_fullsized":"2024-03-26T13:59:53.313859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.252376+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:48.252376+0000","last_peered":"2024-03-26T13:59:48.252376+0000","last_clean":"2024-03-26T13:59:48.252376+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:48.252376+0000","last_undegraded":"2024-03-26T13:59:48.252376+0000","last_fullsized":"2024-03-26T13:59:48.252376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840788+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:27.840788+0000","last_peered":"2024-03-26T13:59:27.840788+0000","last_clean":"2024-03-26T13:59:27.840788+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:27.840788+0000","last_undegraded":"2024-03-26T13:59:27.840788+0000","last_fullsized":"2024-03-26T13:59:27.840788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348985+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:29.348985+0000","last_peered":"2024-03-26T13:59:29.348985+0000","last_clean":"2024-03-26T13:59:29.348985+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:29.348985+0000","last_undegraded":"2024-03-26T13:59:29.348985+0000","last_fullsized":"2024-03-26T13:59:29.348985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319280+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:53.319280+0000","last_peered":"2024-03-26T13:59:53.319280+0000","last_clean":"2024-03-26T13:59:53.319280+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:53.319280+0000","last_undegraded":"2024-03-26T13:59:53.319280+0000","last_fullsized":"2024-03-26T13:59:53.319280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841263+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:27.841263+0000","last_peered":"2024-03-26T13:59:27.841263+0000","last_clean":"2024-03-26T13:59:27.841263+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:27.841263+0000","last_undegraded":"2024-03-26T13:59:27.841263+0000","last_fullsized":"2024-03-26T13:59:27.841263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363505+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:29.363505+0000","last_peered":"2024-03-26T13:59:29.363505+0000","last_clean":"2024-03-26T13:59:29.363505+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:29.363505+0000","last_undegraded":"2024-03-26T13:59:29.363505+0000","last_fullsized":"2024-03-26T13:59:29.363505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842212+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:27.842212+0000","last_peered":"2024-03-26T13:59:27.842212+0000","last_clean":"2024-03-26T13:59:27.842212+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:27.842212+0000","last_undegraded":"2024-03-26T13:59:27.842212+0000","last_fullsized":"2024-03-26T13:59:27.842212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841024+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:27.841024+0000","last_peered":"2024-03-26T13:59:27.841024+0000","last_clean":"2024-03-26T13:59:27.841024+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:27.841024+0000","last_undegraded":"2024-03-26T13:59:27.841024+0000","last_fullsized":"2024-03-26T13:59:27.841024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840326+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:27.840326+0000","last_peered":"2024-03-26T13:59:27.840326+0000","last_clean":"2024-03-26T13:59:27.840326+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:27.840326+0000","last_undegraded":"2024-03-26T13:59:27.840326+0000","last_fullsized":"2024-03-26T13:59:27.840326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840337+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:27.840337+0000","last_peered":"2024-03-26T13:59:27.840337+0000","last_clean":"2024-03-26T13:59:27.840337+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:27.840337+0000","last_undegraded":"2024-03-26T13:59:27.840337+0000","last_fullsized":"2024-03-26T13:59:27.840337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841677+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:27.841677+0000","last_peered":"2024-03-26T13:59:27.841677+0000","last_clean":"2024-03-26T13:59:27.841677+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:27.841677+0000","last_undegraded":"2024-03-26T13:59:27.841677+0000","last_fullsized":"2024-03-26T13:59:27.841677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233480+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:30.233480+0000","last_peered":"2024-03-26T13:59:30.233480+0000","last_clean":"2024-03-26T13:59:30.233480+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:30.233480+0000","last_undegraded":"2024-03-26T13:59:30.233480+0000","last_fullsized":"2024-03-26T13:59:30.233480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316709+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:53.316709+0000","last_peered":"2024-03-26T13:59:53.316709+0000","last_clean":"2024-03-26T13:59:53.316709+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:53.316709+0000","last_undegraded":"2024-03-26T13:59:53.316709+0000","last_fullsized":"2024-03-26T13:59:53.316709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316722+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:53.316722+0000","last_peered":"2024-03-26T13:59:53.316722+0000","last_clean":"2024-03-26T13:59:53.316722+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:53.316722+0000","last_undegraded":"2024-03-26T13:59:53.316722+0000","last_fullsized":"2024-03-26T13:59:53.316722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363485+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:29.363485+0000","last_peered":"2024-03-26T13:59:29.363485+0000","last_clean":"2024-03-26T13:59:29.363485+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:29.363485+0000","last_undegraded":"2024-03-26T13:59:29.363485+0000","last_fullsized":"2024-03-26T13:59:29.363485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840283+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:27.840283+0000","last_peered":"2024-03-26T13:59:27.840283+0000","last_clean":"2024-03-26T13:59:27.840283+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:27.840283+0000","last_undegraded":"2024-03-26T13:59:27.840283+0000","last_fullsized":"2024-03-26T13:59:27.840283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234703+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:30.234703+0000","last_peered":"2024-03-26T13:59:30.234703+0000","last_clean":"2024-03-26T13:59:30.234703+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:30.234703+0000","last_undegraded":"2024-03-26T13:59:30.234703+0000","last_fullsized":"2024-03-26T13:59:30.234703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319745+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:53.319745+0000","last_peered":"2024-03-26T13:59:53.319745+0000","last_clean":"2024-03-26T13:59:53.319745+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:53.319745+0000","last_undegraded":"2024-03-26T13:59:53.319745+0000","last_fullsized":"2024-03-26T13:59:53.319745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325560+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:28.325560+0000","last_peered":"2024-03-26T13:59:28.325560+0000","last_clean":"2024-03-26T13:59:28.325560+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:28.325560+0000","last_undegraded":"2024-03-26T13:59:28.325560+0000","last_fullsized":"2024-03-26T13:59:28.325560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840768+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:27.840768+0000","last_peered":"2024-03-26T13:59:27.840768+0000","last_clean":"2024-03-26T13:59:27.840768+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:27.840768+0000","last_undegraded":"2024-03-26T13:59:27.840768+0000","last_fullsized":"2024-03-26T13:59:27.840768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324700+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:28.324700+0000","last_peered":"2024-03-26T13:59:28.324700+0000","last_clean":"2024-03-26T13:59:28.324700+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:28.324700+0000","last_undegraded":"2024-03-26T13:59:28.324700+0000","last_fullsized":"2024-03-26T13:59:28.324700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348523+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:29.348523+0000","last_peered":"2024-03-26T13:59:29.348523+0000","last_clean":"2024-03-26T13:59:29.348523+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:29.348523+0000","last_undegraded":"2024-03-26T13:59:29.348523+0000","last_fullsized":"2024-03-26T13:59:29.348523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839530+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:27.839530+0000","last_peered":"2024-03-26T13:59:27.839530+0000","last_clean":"2024-03-26T13:59:27.839530+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:27.839530+0000","last_undegraded":"2024-03-26T13:59:27.839530+0000","last_fullsized":"2024-03-26T13:59:27.839530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234803+0000","last_change":"2024-03-26T13:59:23.308422+0000","last_active":"2024-03-26T13:59:30.234803+0000","last_peered":"2024-03-26T13:59:30.234803+0000","last_clean":"2024-03-26T13:59:30.234803+0000","last_became_active":"2024-03-26T13:59:23.308011+0000","last_became_peered":"2024-03-26T13:59:23.308011+0000","last_unstale":"2024-03-26T13:59:30.234803+0000","last_undegraded":"2024-03-26T13:59:30.234803+0000","last_fullsized":"2024-03-26T13:59:30.234803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:08.894340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14443,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":133,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":112630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1083045,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481084,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":481004,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1072817,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":512012,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":538070,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485397,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1076206,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":33731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T13:59:55.196 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-03-26T13:59:55.196 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 15, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-03-26T13:59:30.236338+0000', 'last_change': '2024-03-26T13:59:53.317881+0000', 'last_active': '2024-03-26T13:59:53.317881+0000', 'last_peered': '2024-03-26T13:59:30.236338+0000', 'last_clean': '2024-03-26T13:59:30.236338+0000', 'last_became_active': '2024-03-26T13:59:27.323585+0000', 'last_became_peered': '2024-03-26T13:59:27.323585+0000', 'last_unstale': '2024-03-26T13:59:30.236338+0000', 'last_undegraded': '2024-03-26T13:59:30.236338+0000', 'last_fullsized': '2024-03-26T13:59:30.236338+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 89, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_clean_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 8, 7, 11], 'acting': [2, 8, 7, 11], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 7, 8, 11], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-26T13:59:55.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-03-26T13:59:55.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 15, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-03-26T13:59:30.234348+0000', 'last_change': '2024-03-26T13:59:53.316731+0000', 'last_active': '2024-03-26T13:59:53.316731+0000', 'last_peered': '2024-03-26T13:59:30.234348+0000', 'last_clean': '2024-03-26T13:59:30.234348+0000', 'last_became_active': '2024-03-26T13:59:27.324762+0000', 'last_became_peered': '2024-03-26T13:59:27.324762+0000', 'last_unstale': '2024-03-26T13:59:30.234348+0000', 'last_undegraded': '2024-03-26T13:59:30.234348+0000', 'last_fullsized': '2024-03-26T13:59:30.234348+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 89, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_clean_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6, 8, 5], 'acting': [2, 6, 8, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 6, 8, 9], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-26T13:59:55.197 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2024-03-26T13:59:55.197 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "90'1", 'reported_seq': 16, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-03-26T13:59:52.319197+0000', 'last_change': '2024-03-26T13:59:53.314318+0000', 'last_active': '2024-03-26T13:59:53.314318+0000', 'last_peered': '2024-03-26T13:59:52.319197+0000', 'last_clean': '2024-03-26T13:59:52.319197+0000', 'last_became_active': '2024-03-26T13:59:27.326077+0000', 'last_became_peered': '2024-03-26T13:59:27.326077+0000', 'last_unstale': '2024-03-26T13:59:52.319197+0000', 'last_undegraded': '2024-03-26T13:59:52.319197+0000', 'last_fullsized': '2024-03-26T13:59:52.319197+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 89, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_clean_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 117, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1, 'num_write_kb': 1, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 10, 7, 8], 'acting': [4, 10, 7, 8], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7, 8, 9, 10], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-26T13:59:55.197 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-26T13:59:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.453+0000 7f7450200640 1 -- 172.21.15.137:0/280187463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f744810dc90 msgr2=0x7f7448110080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.453+0000 7f7450200640 1 --2- 172.21.15.137:0/280187463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f744810dc90 0x7f7448110080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7440009cd0 tx=0x7f744002c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 -- 172.21.15.137:0/280187463 shutdown_connections 2024-03-26T13:59:55.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 --2- 172.21.15.137:0/280187463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f744810dc90 0x7f7448110080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 --2- 172.21.15.137:0/280187463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448074100 0x7f7448074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 --2- 172.21.15.137:0/280187463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f74480756e0 0x7f7448073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 -- 172.21.15.137:0/280187463 >> 172.21.15.137:0/280187463 conn(0x7f74480fcc50 msgr2=0x7f74480ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 -- 172.21.15.137:0/280187463 shutdown_connections 2024-03-26T13:59:55.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.454+0000 7f7450200640 1 -- 172.21.15.137:0/280187463 wait complete. 2024-03-26T13:59:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 Processor -- start 2024-03-26T13:59:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 -- start start 2024-03-26T13:59:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448074100 0x7f744819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 0x7f744819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f744810dc90 0x7f74481a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74481137e0 con 0x7f74480756e0 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448113660 con 0x7f744810dc90 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f7450200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448113960 con 0x7f7448074100 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f744ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 0x7f744819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.455+0000 7f744ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 0x7f744819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36462/0 (socket says 172.21.15.137:36462) 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 -- 172.21.15.137:0/3220191744 learned_addr learned my addr 172.21.15.137:0/3220191744 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744e200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f744810dc90 0x7f74481a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744d800640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448074100 0x7f744819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 -- 172.21.15.137:0/3220191744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448074100 msgr2=0x7f744819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448074100 0x7f744819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 -- 172.21.15.137:0/3220191744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f744810dc90 msgr2=0x7f74481a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f744810dc90 0x7f74481a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 -- 172.21.15.137:0/3220191744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74481a5af0 con 0x7f74480756e0 2024-03-26T13:59:55.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f744ce00640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 0x7f744819e410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f743400c970 tx=0x7f743400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434007bb0 con 0x7f74480756e0 2024-03-26T13:59:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.457+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74481a5de0 con 0x7f74480756e0 2024-03-26T13:59:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.456+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7434007d50 con 0x7f74480756e0 2024-03-26T13:59:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.457+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743400f450 con 0x7f74480756e0 2024-03-26T13:59:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.457+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74481a6250 con 0x7f74480756e0 2024-03-26T13:59:55.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.458+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7448103e60 con 0x7f74480756e0 2024-03-26T13:59:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.460+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f7434004f40 con 0x7f74480756e0 2024-03-26T13:59:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.462+0000 7f7446000640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f74300a2890 0x7f74300a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.462+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f74340cece0 con 0x7f74480756e0 2024-03-26T13:59:55.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.462+0000 7f744d800640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f74300a2890 0x7f74300a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.463+0000 7f744d800640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f74300a2890 0x7f74300a4d50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f743c009b60 tx=0x7f743c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:55.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.466+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7434014030 con 0x7f74480756e0 2024-03-26T13:59:55.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:55 smithi137 ceph-mon[31539]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T13:59:55.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:55 smithi137 ceph-mon[31539]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:55.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:55 smithi137 ceph-mon[31539]: pgmap v383: 129 pgs: 3 peering, 126 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-03-26T13:59:55.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:55 smithi137 ceph-mon[31539]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:55.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:55 smithi179 ceph-mon[35472]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T13:59:55.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:55 smithi179 ceph-mon[35472]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:55 smithi179 ceph-mon[35472]: pgmap v383: 129 pgs: 3 peering, 126 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-03-26T13:59:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:55 smithi179 ceph-mon[35472]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:55.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:55 smithi159 ceph-mon[35501]: osdmap e92: 12 total, 12 up, 12 in 2024-03-26T13:59:55.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:55 smithi159 ceph-mon[35501]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:55.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:55 smithi159 ceph-mon[35501]: pgmap v383: 129 pgs: 3 peering, 126 active+clean; 591 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-03-26T13:59:55.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:55 smithi159 ceph-mon[35501]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:55.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.737+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f744802d070 con 0x7f74480756e0 2024-03-26T13:59:55.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.738+0000 7f7446000640 1 -- 172.21.15.137:0/3220191744 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2639 (secure 0 0 0) 0x7f743409b860 con 0x7f74480756e0 2024-03-26T13:59:55.735 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:55.735 INFO:teuthology.orchestra.run.smithi137.stdout:{"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":523,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":12,"num_up_osds":12,"osd_up_since":1711461495,"num_in_osds":12,"osd_in_since":1711461470,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":126},{"state_name":"peering","count":3}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":605045,"bytes_used":369672192,"bytes_avail":1151570755584,"bytes_total":1151940427776,"inactive_pgs_ratio":0.023255813866853714,"read_bytes_sec":3582,"write_bytes_sec":4477,"read_op_per_sec":3,"write_op_per_sec":10},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"c","status":"up:standby-replay","gid":24460},{"filesystem_id":1,"rank":1,"name":"i","status":"up:standby-replay","gid":24481},{"filesystem_id":1,"rank":0,"name":"k","status":"up:standby-replay","gid":34394},{"filesystem_id":1,"rank":2,"name":"e","status":"up:active","gid":34424},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":34451},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":34496}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.137:8443/"}},"servicemap":{"epoch":24,"modified":"2024-03-26T13:59:55.084709+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f74300a2890 msgr2=0x7f74300a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f74300a2890 0x7f74300a4d50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f743c009b60 tx=0x7f743c005f90 comp rx=0 tx=0).stop 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 msgr2=0x7f744819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 0x7f744819e410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f743400c970 tx=0x7f743400ce30 comp rx=0 tx=0).stop 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 shutdown_connections 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f74300a2890 0x7f74300a4d50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f744810dc90 0x7f74481a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f74480756e0 0x7f744819e410 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 --2- 172.21.15.137:0/3220191744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7448074100 0x7f744819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 >> 172.21.15.137:0/3220191744 conn(0x7f74480fcc50 msgr2=0x7f7448101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 shutdown_connections 2024-03-26T13:59:55.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.741+0000 7f7450200640 1 -- 172.21.15.137:0/3220191744 wait complete. 2024-03-26T13:59:55.756 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-26T13:59:55.756 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 -- 172.21.15.137:0/3953468376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 msgr2=0x7f3b0c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3953468376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c103d00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3afc009ca0 tx=0x7f3afc02c100 comp rx=0 tx=0).stop 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 -- 172.21.15.137:0/3953468376 shutdown_connections 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3953468376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 0x7f3b0c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3953468376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3953468376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c103d00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 -- 172.21.15.137:0/3953468376 >> 172.21.15.137:0/3953468376 conn(0x7f3b0c0fee90 msgr2=0x7f3b0c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 -- 172.21.15.137:0/3953468376 shutdown_connections 2024-03-26T13:59:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.978+0000 7f3b11e00640 1 -- 172.21.15.137:0/3953468376 wait complete. 2024-03-26T13:59:55.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.979+0000 7f3b11e00640 1 Processor -- start 2024-03-26T13:59:55.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.979+0000 7f3b11e00640 1 -- start start 2024-03-26T13:59:55.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b11e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b11e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b11e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b11e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10f400 con 0x7f3b0c1038e0 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b11e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10f280 con 0x7f3b0c105490 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b11e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c10f580 con 0x7f3b0c104ad0 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36484/0 (socket says 172.21.15.137:36484) 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 -- 172.21.15.137:0/3701609469 learned_addr learned my addr 172.21.15.137:0/3701609469 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0b400640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0a000640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 -- 172.21.15.137:0/3701609469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0c104ad0 msgr2=0x7f3b0c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 -- 172.21.15.137:0/3701609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 msgr2=0x7f3b0c1a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.980+0000 7f3b0aa00640 1 -- 172.21.15.137:0/3701609469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c1a1740 con 0x7f3b0c1038e0 2024-03-26T13:59:55.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b0b400640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b0aa00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199b20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3afc032380 tx=0x7f3afc035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc039040 con 0x7f3b0c1038e0 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3afc044070 con 0x7f3b0c1038e0 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c1a19d0 con 0x7f3b0c1038e0 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0045a0 con 0x7f3b0c1038e0 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.981+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1a1ec0 con 0x7f3b0c1038e0 2024-03-26T13:59:55.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.982+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c10acd0 con 0x7f3b0c1038e0 2024-03-26T13:59:55.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.984+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3afc003970 con 0x7f3b0c1038e0 2024-03-26T13:59:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.986+0000 7f3b10a00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3af00a2890 0x7f3af00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.986+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f3afc0f15f0 con 0x7f3b0c1038e0 2024-03-26T13:59:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.986+0000 7f3b0a000640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3af00a2890 0x7f3af00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:55.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.987+0000 7f3b0a000640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3af00a2890 0x7f3af00a4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005fd0 tx=0x7f3af4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:55.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:55.990+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3afc0be2e0 con 0x7f3b0c1038e0 2024-03-26T13:59:56.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.186+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b0c061ae0 con 0x7f3af00a2890 2024-03-26T13:59:56.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.215+0000 7f3b10a00640 1 -- 172.21.15.137:0/3701609469 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340636 (secure 0 0 0) 0x7f3b0c061ae0 con 0x7f3af00a2890 2024-03-26T13:59:56.215 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:56.218 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3af00a2890 msgr2=0x7f3af00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3af00a2890 0x7f3af00a4d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3af4005fd0 tx=0x7f3af4005850 comp rx=0 tx=0).stop 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 msgr2=0x7f3b0c199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199b20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3afc032380 tx=0x7f3afc035ce0 comp rx=0 tx=0).stop 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 shutdown_connections 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3af00a2890 0x7f3af00a4d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:56.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b0c105490 0x7f3b0c1a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:56.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b0c104ad0 0x7f3b0c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:56.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 --2- 172.21.15.137:0/3701609469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b0c1038e0 0x7f3b0c199b20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:56.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 >> 172.21.15.137:0/3701609469 conn(0x7f3b0c0fee90 msgr2=0x7f3b0c107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:56.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 shutdown_connections 2024-03-26T13:59:56.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:56.223+0000 7f3b11e00640 1 -- 172.21.15.137:0/3701609469 wait complete. 2024-03-26T13:59:56.238 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-03-26T13:59:55.084637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605045,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":158,"num_read_kb":137,"num_write":316,"num_write_kb":2992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361008,"kb_used_data":11856,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580816,"statfs":{"total":1151940427776,"available":1151570755584,"internally_reserved":0,"allocated":12140544,"data_stored":7653133,"data_compressed":283341,"data_compressed_allocated":380928,"data_compressed_original":2260992,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10037,"num_objects":40,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":86,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003855"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235192+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:30.235192+0000","last_peered":"2024-03-26T13:59:30.235192+0000","last_clean":"2024-03-26T13:59:30.235192+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:30.235192+0000","last_undegraded":"2024-03-26T13:59:30.235192+0000","last_fullsized":"2024-03-26T13:59:30.235192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364396+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:29.364396+0000","last_peered":"2024-03-26T13:59:29.364396+0000","last_clean":"2024-03-26T13:59:29.364396+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:29.364396+0000","last_undegraded":"2024-03-26T13:59:29.364396+0000","last_fullsized":"2024-03-26T13:59:29.364396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364118+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:29.364118+0000","last_peered":"2024-03-26T13:59:29.364118+0000","last_clean":"2024-03-26T13:59:29.364118+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:29.364118+0000","last_undegraded":"2024-03-26T13:59:29.364118+0000","last_fullsized":"2024-03-26T13:59:29.364118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315175+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:53.315175+0000","last_peered":"2024-03-26T13:59:53.315175+0000","last_clean":"2024-03-26T13:59:53.315175+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:53.315175+0000","last_undegraded":"2024-03-26T13:59:53.315175+0000","last_fullsized":"2024-03-26T13:59:53.315175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469440+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:29.469440+0000","last_peered":"2024-03-26T13:59:29.469440+0000","last_clean":"2024-03-26T13:59:29.469440+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:29.469440+0000","last_undegraded":"2024-03-26T13:59:29.469440+0000","last_fullsized":"2024-03-26T13:59:29.469440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781472+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:28.781472+0000","last_peered":"2024-03-26T13:59:28.781472+0000","last_clean":"2024-03-26T13:59:28.781472+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:28.781472+0000","last_undegraded":"2024-03-26T13:59:28.781472+0000","last_fullsized":"2024-03-26T13:59:28.781472+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839948+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:27.839948+0000","last_peered":"2024-03-26T13:59:27.839948+0000","last_clean":"2024-03-26T13:59:27.839948+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:27.839948+0000","last_undegraded":"2024-03-26T13:59:27.839948+0000","last_fullsized":"2024-03-26T13:59:27.839948+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349872+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:29.349872+0000","last_peered":"2024-03-26T13:59:29.349872+0000","last_clean":"2024-03-26T13:59:29.349872+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:29.349872+0000","last_undegraded":"2024-03-26T13:59:29.349872+0000","last_fullsized":"2024-03-26T13:59:29.349872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782777+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:28.782777+0000","last_peered":"2024-03-26T13:59:28.782777+0000","last_clean":"2024-03-26T13:59:28.782777+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:28.782777+0000","last_undegraded":"2024-03-26T13:59:28.782777+0000","last_fullsized":"2024-03-26T13:59:28.782777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841218+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:27.841218+0000","last_peered":"2024-03-26T13:59:27.841218+0000","last_clean":"2024-03-26T13:59:27.841218+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:27.841218+0000","last_undegraded":"2024-03-26T13:59:27.841218+0000","last_fullsized":"2024-03-26T13:59:27.841218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315062+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:53.315062+0000","last_peered":"2024-03-26T13:59:53.315062+0000","last_clean":"2024-03-26T13:59:53.315062+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:53.315062+0000","last_undegraded":"2024-03-26T13:59:53.315062+0000","last_fullsized":"2024-03-26T13:59:53.315062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839733+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:27.839733+0000","last_peered":"2024-03-26T13:59:27.839733+0000","last_clean":"2024-03-26T13:59:27.839733+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:27.839733+0000","last_undegraded":"2024-03-26T13:59:27.839733+0000","last_fullsized":"2024-03-26T13:59:27.839733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363825+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:29.363825+0000","last_peered":"2024-03-26T13:59:29.363825+0000","last_clean":"2024-03-26T13:59:29.363825+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:29.363825+0000","last_undegraded":"2024-03-26T13:59:29.363825+0000","last_fullsized":"2024-03-26T13:59:29.363825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363266+0000","last_change":"2024-03-26T13:59:23.312717+0000","last_active":"2024-03-26T13:59:29.363266+0000","last_peered":"2024-03-26T13:59:29.363266+0000","last_clean":"2024-03-26T13:59:29.363266+0000","last_became_active":"2024-03-26T13:59:23.312122+0000","last_became_peered":"2024-03-26T13:59:23.312122+0000","last_unstale":"2024-03-26T13:59:29.363266+0000","last_undegraded":"2024-03-26T13:59:29.363266+0000","last_fullsized":"2024-03-26T13:59:29.363266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:41:54.487004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348001+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:29.348001+0000","last_peered":"2024-03-26T13:59:29.348001+0000","last_clean":"2024-03-26T13:59:29.348001+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:29.348001+0000","last_undegraded":"2024-03-26T13:59:29.348001+0000","last_fullsized":"2024-03-26T13:59:29.348001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234897+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:30.234897+0000","last_peered":"2024-03-26T13:59:30.234897+0000","last_clean":"2024-03-26T13:59:30.234897+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:30.234897+0000","last_undegraded":"2024-03-26T13:59:30.234897+0000","last_fullsized":"2024-03-26T13:59:30.234897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231510+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:30.231510+0000","last_peered":"2024-03-26T13:59:30.231510+0000","last_clean":"2024-03-26T13:59:30.231510+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:30.231510+0000","last_undegraded":"2024-03-26T13:59:30.231510+0000","last_fullsized":"2024-03-26T13:59:30.231510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319222+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:53.319222+0000","last_peered":"2024-03-26T13:59:53.319222+0000","last_clean":"2024-03-26T13:59:53.319222+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:53.319222+0000","last_undegraded":"2024-03-26T13:59:53.319222+0000","last_fullsized":"2024-03-26T13:59:53.319222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317072+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:53.317072+0000","last_peered":"2024-03-26T13:59:53.317072+0000","last_clean":"2024-03-26T13:59:53.317072+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:53.317072+0000","last_undegraded":"2024-03-26T13:59:53.317072+0000","last_fullsized":"2024-03-26T13:59:53.317072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264793+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:49.264793+0000","last_peered":"2024-03-26T13:59:49.264793+0000","last_clean":"2024-03-26T13:59:49.264793+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:49.264793+0000","last_undegraded":"2024-03-26T13:59:49.264793+0000","last_fullsized":"2024-03-26T13:59:49.264793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325746+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:28.325746+0000","last_peered":"2024-03-26T13:59:28.325746+0000","last_clean":"2024-03-26T13:59:28.325746+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:28.325746+0000","last_undegraded":"2024-03-26T13:59:28.325746+0000","last_fullsized":"2024-03-26T13:59:28.325746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.313767+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:53.313767+0000","last_peered":"2024-03-26T13:59:53.313767+0000","last_clean":"2024-03-26T13:59:53.313767+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:53.313767+0000","last_undegraded":"2024-03-26T13:59:53.313767+0000","last_fullsized":"2024-03-26T13:59:53.313767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.326299+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:28.326299+0000","last_peered":"2024-03-26T13:59:28.326299+0000","last_clean":"2024-03-26T13:59:28.326299+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:28.326299+0000","last_undegraded":"2024-03-26T13:59:28.326299+0000","last_fullsized":"2024-03-26T13:59:28.326299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364222+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:29.364222+0000","last_peered":"2024-03-26T13:59:29.364222+0000","last_clean":"2024-03-26T13:59:29.364222+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:29.364222+0000","last_undegraded":"2024-03-26T13:59:29.364222+0000","last_fullsized":"2024-03-26T13:59:29.364222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314244+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:53.314244+0000","last_peered":"2024-03-26T13:59:53.314244+0000","last_clean":"2024-03-26T13:59:53.314244+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:53.314244+0000","last_undegraded":"2024-03-26T13:59:53.314244+0000","last_fullsized":"2024-03-26T13:59:53.314244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264016+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:49.264016+0000","last_peered":"2024-03-26T13:59:49.264016+0000","last_clean":"2024-03-26T13:59:49.264016+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:49.264016+0000","last_undegraded":"2024-03-26T13:59:49.264016+0000","last_fullsized":"2024-03-26T13:59:49.264016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349048+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:29.349048+0000","last_peered":"2024-03-26T13:59:29.349048+0000","last_clean":"2024-03-26T13:59:29.349048+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:29.349048+0000","last_undegraded":"2024-03-26T13:59:29.349048+0000","last_fullsized":"2024-03-26T13:59:29.349048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314137+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:53.314137+0000","last_peered":"2024-03-26T13:59:53.314137+0000","last_clean":"2024-03-26T13:59:53.314137+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:53.314137+0000","last_undegraded":"2024-03-26T13:59:53.314137+0000","last_fullsized":"2024-03-26T13:59:53.314137+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469895+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:29.469895+0000","last_peered":"2024-03-26T13:59:29.469895+0000","last_clean":"2024-03-26T13:59:29.469895+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:29.469895+0000","last_undegraded":"2024-03-26T13:59:29.469895+0000","last_fullsized":"2024-03-26T13:59:29.469895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841812+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:27.841812+0000","last_peered":"2024-03-26T13:59:27.841812+0000","last_clean":"2024-03-26T13:59:27.841812+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:27.841812+0000","last_undegraded":"2024-03-26T13:59:27.841812+0000","last_fullsized":"2024-03-26T13:59:27.841812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.236338+0000","last_change":"2024-03-26T13:59:53.317881+0000","last_active":"2024-03-26T13:59:53.317881+0000","last_peered":"2024-03-26T13:59:30.236338+0000","last_clean":"2024-03-26T13:59:30.236338+0000","last_became_active":"2024-03-26T13:59:27.323585+0000","last_became_peered":"2024-03-26T13:59:27.323585+0000","last_unstale":"2024-03-26T13:59:30.236338+0000","last_undegraded":"2024-03-26T13:59:30.236338+0000","last_fullsized":"2024-03-26T13:59:30.236338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7,8,11],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.265450+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:50.265450+0000","last_peered":"2024-03-26T13:59:50.265450+0000","last_clean":"2024-03-26T13:59:50.265450+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:50.265450+0000","last_undegraded":"2024-03-26T13:59:50.265450+0000","last_fullsized":"2024-03-26T13:59:50.265450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469799+0000","last_change":"2024-03-26T13:59:23.299805+0000","last_active":"2024-03-26T13:59:29.469799+0000","last_peered":"2024-03-26T13:59:29.469799+0000","last_clean":"2024-03-26T13:59:29.469799+0000","last_became_active":"2024-03-26T13:59:23.295723+0000","last_became_peered":"2024-03-26T13:59:23.295723+0000","last_unstale":"2024-03-26T13:59:29.469799+0000","last_undegraded":"2024-03-26T13:59:29.469799+0000","last_fullsized":"2024-03-26T13:59:29.469799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:25:05.112905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.234348+0000","last_change":"2024-03-26T13:59:53.316731+0000","last_active":"2024-03-26T13:59:53.316731+0000","last_peered":"2024-03-26T13:59:30.234348+0000","last_clean":"2024-03-26T13:59:30.234348+0000","last_became_active":"2024-03-26T13:59:27.324762+0000","last_became_peered":"2024-03-26T13:59:27.324762+0000","last_unstale":"2024-03-26T13:59:30.234348+0000","last_undegraded":"2024-03-26T13:59:30.234348+0000","last_fullsized":"2024-03-26T13:59:30.234348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.261450+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:49.261450+0000","last_peered":"2024-03-26T13:59:49.261450+0000","last_clean":"2024-03-26T13:59:49.261450+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:49.261450+0000","last_undegraded":"2024-03-26T13:59:49.261450+0000","last_fullsized":"2024-03-26T13:59:49.261450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841919+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:27.841919+0000","last_peered":"2024-03-26T13:59:27.841919+0000","last_clean":"2024-03-26T13:59:27.841919+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:27.841919+0000","last_undegraded":"2024-03-26T13:59:27.841919+0000","last_fullsized":"2024-03-26T13:59:27.841919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.235034+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:30.235034+0000","last_peered":"2024-03-26T13:59:30.235034+0000","last_clean":"2024-03-26T13:59:30.235034+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:30.235034+0000","last_undegraded":"2024-03-26T13:59:30.235034+0000","last_fullsized":"2024-03-26T13:59:30.235034+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264064+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:49.264064+0000","last_peered":"2024-03-26T13:59:49.264064+0000","last_clean":"2024-03-26T13:59:49.264064+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:49.264064+0000","last_undegraded":"2024-03-26T13:59:49.264064+0000","last_fullsized":"2024-03-26T13:59:49.264064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324313+0000","last_change":"2024-03-26T13:59:23.298324+0000","last_active":"2024-03-26T13:59:28.324313+0000","last_peered":"2024-03-26T13:59:28.324313+0000","last_clean":"2024-03-26T13:59:28.324313+0000","last_became_active":"2024-03-26T13:59:23.297577+0000","last_became_peered":"2024-03-26T13:59:23.297577+0000","last_unstale":"2024-03-26T13:59:28.324313+0000","last_undegraded":"2024-03-26T13:59:28.324313+0000","last_fullsized":"2024-03-26T13:59:28.324313+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:24.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364431+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:29.364431+0000","last_peered":"2024-03-26T13:59:29.364431+0000","last_clean":"2024-03-26T13:59:29.364431+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:29.364431+0000","last_undegraded":"2024-03-26T13:59:29.364431+0000","last_fullsized":"2024-03-26T13:59:29.364431+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840981+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:27.840981+0000","last_peered":"2024-03-26T13:59:27.840981+0000","last_clean":"2024-03-26T13:59:27.840981+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:27.840981+0000","last_undegraded":"2024-03-26T13:59:27.840981+0000","last_fullsized":"2024-03-26T13:59:27.840981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230691+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:30.230691+0000","last_peered":"2024-03-26T13:59:30.230691+0000","last_clean":"2024-03-26T13:59:30.230691+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:30.230691+0000","last_undegraded":"2024-03-26T13:59:30.230691+0000","last_fullsized":"2024-03-26T13:59:30.230691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324666+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:28.324666+0000","last_peered":"2024-03-26T13:59:28.324666+0000","last_clean":"2024-03-26T13:59:28.324666+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:28.324666+0000","last_undegraded":"2024-03-26T13:59:28.324666+0000","last_fullsized":"2024-03-26T13:59:28.324666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.250279+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:48.250279+0000","last_peered":"2024-03-26T13:59:48.250279+0000","last_clean":"2024-03-26T13:59:48.250279+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:48.250279+0000","last_undegraded":"2024-03-26T13:59:48.250279+0000","last_fullsized":"2024-03-26T13:59:48.250279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349771+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:29.349771+0000","last_peered":"2024-03-26T13:59:29.349771+0000","last_clean":"2024-03-26T13:59:29.349771+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:29.349771+0000","last_undegraded":"2024-03-26T13:59:29.349771+0000","last_fullsized":"2024-03-26T13:59:29.349771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231033+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:30.231033+0000","last_peered":"2024-03-26T13:59:30.231033+0000","last_clean":"2024-03-26T13:59:30.231033+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:30.231033+0000","last_undegraded":"2024-03-26T13:59:30.231033+0000","last_fullsized":"2024-03-26T13:59:30.231033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.243150+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:48.243150+0000","last_peered":"2024-03-26T13:59:48.243150+0000","last_clean":"2024-03-26T13:59:48.243150+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:48.243150+0000","last_undegraded":"2024-03-26T13:59:48.243150+0000","last_fullsized":"2024-03-26T13:59:48.243150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364498+0000","last_change":"2024-03-26T13:59:23.314458+0000","last_active":"2024-03-26T13:59:29.364498+0000","last_peered":"2024-03-26T13:59:29.364498+0000","last_clean":"2024-03-26T13:59:29.364498+0000","last_became_active":"2024-03-26T13:59:23.314060+0000","last_became_peered":"2024-03-26T13:59:23.314060+0000","last_unstale":"2024-03-26T13:59:29.364498+0000","last_undegraded":"2024-03-26T13:59:29.364498+0000","last_fullsized":"2024-03-26T13:59:29.364498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:11:40.434518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314347+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:53.314347+0000","last_peered":"2024-03-26T13:59:53.314347+0000","last_clean":"2024-03-26T13:59:53.314347+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:53.314347+0000","last_undegraded":"2024-03-26T13:59:53.314347+0000","last_fullsized":"2024-03-26T13:59:53.314347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.242523+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:48.242523+0000","last_peered":"2024-03-26T13:59:48.242523+0000","last_clean":"2024-03-26T13:59:48.242523+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:48.242523+0000","last_undegraded":"2024-03-26T13:59:48.242523+0000","last_fullsized":"2024-03-26T13:59:48.242523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.349320+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:29.349320+0000","last_peered":"2024-03-26T13:59:29.349320+0000","last_clean":"2024-03-26T13:59:29.349320+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:29.349320+0000","last_undegraded":"2024-03-26T13:59:29.349320+0000","last_fullsized":"2024-03-26T13:59:29.349320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.230980+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:30.230980+0000","last_peered":"2024-03-26T13:59:30.230980+0000","last_clean":"2024-03-26T13:59:30.230980+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:30.230980+0000","last_undegraded":"2024-03-26T13:59:30.230980+0000","last_fullsized":"2024-03-26T13:59:30.230980+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.748837+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:50.748837+0000","last_peered":"2024-03-26T13:59:50.748837+0000","last_clean":"2024-03-26T13:59:50.748837+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:50.748837+0000","last_undegraded":"2024-03-26T13:59:50.748837+0000","last_fullsized":"2024-03-26T13:59:50.748837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10442,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":16,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.470191+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:29.470191+0000","last_peered":"2024-03-26T13:59:29.470191+0000","last_clean":"2024-03-26T13:59:29.470191+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:29.470191+0000","last_undegraded":"2024-03-26T13:59:29.470191+0000","last_fullsized":"2024-03-26T13:59:29.470191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842554+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:27.842554+0000","last_peered":"2024-03-26T13:59:27.842554+0000","last_clean":"2024-03-26T13:59:27.842554+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:27.842554+0000","last_undegraded":"2024-03-26T13:59:27.842554+0000","last_fullsized":"2024-03-26T13:59:27.842554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.318377+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:53.318377+0000","last_peered":"2024-03-26T13:59:53.318377+0000","last_clean":"2024-03-26T13:59:53.318377+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:53.318377+0000","last_undegraded":"2024-03-26T13:59:53.318377+0000","last_fullsized":"2024-03-26T13:59:53.318377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839344+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:27.839344+0000","last_peered":"2024-03-26T13:59:27.839344+0000","last_clean":"2024-03-26T13:59:27.839344+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:27.839344+0000","last_undegraded":"2024-03-26T13:59:27.839344+0000","last_fullsized":"2024-03-26T13:59:27.839344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233995+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:30.233995+0000","last_peered":"2024-03-26T13:59:30.233995+0000","last_clean":"2024-03-26T13:59:30.233995+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:30.233995+0000","last_undegraded":"2024-03-26T13:59:30.233995+0000","last_fullsized":"2024-03-26T13:59:30.233995+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.248560+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:48.248560+0000","last_peered":"2024-03-26T13:59:48.248560+0000","last_clean":"2024-03-26T13:59:48.248560+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:48.248560+0000","last_undegraded":"2024-03-26T13:59:48.248560+0000","last_fullsized":"2024-03-26T13:59:48.248560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840635+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:27.840635+0000","last_peered":"2024-03-26T13:59:27.840635+0000","last_clean":"2024-03-26T13:59:27.840635+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:27.840635+0000","last_undegraded":"2024-03-26T13:59:27.840635+0000","last_fullsized":"2024-03-26T13:59:27.840635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231524+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:30.231524+0000","last_peered":"2024-03-26T13:59:30.231524+0000","last_clean":"2024-03-26T13:59:30.231524+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:30.231524+0000","last_undegraded":"2024-03-26T13:59:30.231524+0000","last_fullsized":"2024-03-26T13:59:30.231524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316181+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:53.316181+0000","last_peered":"2024-03-26T13:59:53.316181+0000","last_clean":"2024-03-26T13:59:53.316181+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:53.316181+0000","last_undegraded":"2024-03-26T13:59:53.316181+0000","last_fullsized":"2024-03-26T13:59:53.316181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840102+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:27.840102+0000","last_peered":"2024-03-26T13:59:27.840102+0000","last_clean":"2024-03-26T13:59:27.840102+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:27.840102+0000","last_undegraded":"2024-03-26T13:59:27.840102+0000","last_fullsized":"2024-03-26T13:59:27.840102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840430+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:27.840430+0000","last_peered":"2024-03-26T13:59:27.840430+0000","last_clean":"2024-03-26T13:59:27.840430+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:27.840430+0000","last_undegraded":"2024-03-26T13:59:27.840430+0000","last_fullsized":"2024-03-26T13:59:27.840430+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.266019+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:49.266019+0000","last_peered":"2024-03-26T13:59:49.266019+0000","last_clean":"2024-03-26T13:59:49.266019+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:49.266019+0000","last_undegraded":"2024-03-26T13:59:49.266019+0000","last_fullsized":"2024-03-26T13:59:49.266019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840790+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:27.840790+0000","last_peered":"2024-03-26T13:59:27.840790+0000","last_clean":"2024-03-26T13:59:27.840790+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:27.840790+0000","last_undegraded":"2024-03-26T13:59:27.840790+0000","last_fullsized":"2024-03-26T13:59:27.840790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841011+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:27.841011+0000","last_peered":"2024-03-26T13:59:27.841011+0000","last_clean":"2024-03-26T13:59:27.841011+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:27.841011+0000","last_undegraded":"2024-03-26T13:59:27.841011+0000","last_fullsized":"2024-03-26T13:59:27.841011+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.246505+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:50.246505+0000","last_peered":"2024-03-26T13:59:50.246505+0000","last_clean":"2024-03-26T13:59:50.246505+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:50.246505+0000","last_undegraded":"2024-03-26T13:59:50.246505+0000","last_fullsized":"2024-03-26T13:59:50.246505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234557+0000","last_change":"2024-03-26T13:59:23.314700+0000","last_active":"2024-03-26T13:59:30.234557+0000","last_peered":"2024-03-26T13:59:30.234557+0000","last_clean":"2024-03-26T13:59:30.234557+0000","last_became_active":"2024-03-26T13:59:23.314372+0000","last_became_peered":"2024-03-26T13:59:23.314372+0000","last_unstale":"2024-03-26T13:59:30.234557+0000","last_undegraded":"2024-03-26T13:59:30.234557+0000","last_fullsized":"2024-03-26T13:59:30.234557+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:33:32.631858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469875+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:29.469875+0000","last_peered":"2024-03-26T13:59:29.469875+0000","last_clean":"2024-03-26T13:59:29.469875+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:29.469875+0000","last_undegraded":"2024-03-26T13:59:29.469875+0000","last_fullsized":"2024-03-26T13:59:29.469875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231159+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:30.231159+0000","last_peered":"2024-03-26T13:59:30.231159+0000","last_clean":"2024-03-26T13:59:30.231159+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:30.231159+0000","last_undegraded":"2024-03-26T13:59:30.231159+0000","last_fullsized":"2024-03-26T13:59:30.231159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.782231+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:28.782231+0000","last_peered":"2024-03-26T13:59:28.782231+0000","last_clean":"2024-03-26T13:59:28.782231+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:28.782231+0000","last_undegraded":"2024-03-26T13:59:28.782231+0000","last_fullsized":"2024-03-26T13:59:28.782231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839740+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:27.839740+0000","last_peered":"2024-03-26T13:59:27.839740+0000","last_clean":"2024-03-26T13:59:27.839740+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:27.839740+0000","last_undegraded":"2024-03-26T13:59:27.839740+0000","last_fullsized":"2024-03-26T13:59:27.839740+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840640+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:27.840640+0000","last_peered":"2024-03-26T13:59:27.840640+0000","last_clean":"2024-03-26T13:59:27.840640+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:27.840640+0000","last_undegraded":"2024-03-26T13:59:27.840640+0000","last_fullsized":"2024-03-26T13:59:27.840640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.781589+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:28.781589+0000","last_peered":"2024-03-26T13:59:28.781589+0000","last_clean":"2024-03-26T13:59:28.781589+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:28.781589+0000","last_undegraded":"2024-03-26T13:59:28.781589+0000","last_fullsized":"2024-03-26T13:59:28.781589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840495+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:27.840495+0000","last_peered":"2024-03-26T13:59:27.840495+0000","last_clean":"2024-03-26T13:59:27.840495+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:27.840495+0000","last_undegraded":"2024-03-26T13:59:27.840495+0000","last_fullsized":"2024-03-26T13:59:27.840495+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841110+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:27.841110+0000","last_peered":"2024-03-26T13:59:27.841110+0000","last_clean":"2024-03-26T13:59:27.841110+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:27.841110+0000","last_undegraded":"2024-03-26T13:59:27.841110+0000","last_fullsized":"2024-03-26T13:59:27.841110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:50.263864+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:50.263864+0000","last_peered":"2024-03-26T13:59:50.263864+0000","last_clean":"2024-03-26T13:59:50.263864+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:50.263864+0000","last_undegraded":"2024-03-26T13:59:50.263864+0000","last_fullsized":"2024-03-26T13:59:50.263864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841168+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:27.841168+0000","last_peered":"2024-03-26T13:59:27.841168+0000","last_clean":"2024-03-26T13:59:27.841168+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:27.841168+0000","last_undegraded":"2024-03-26T13:59:27.841168+0000","last_fullsized":"2024-03-26T13:59:27.841168+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324582+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:28.324582+0000","last_peered":"2024-03-26T13:59:28.324582+0000","last_clean":"2024-03-26T13:59:28.324582+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:28.324582+0000","last_undegraded":"2024-03-26T13:59:28.324582+0000","last_fullsized":"2024-03-26T13:59:28.324582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314334+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:53.314334+0000","last_peered":"2024-03-26T13:59:53.314334+0000","last_clean":"2024-03-26T13:59:53.314334+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:53.314334+0000","last_undegraded":"2024-03-26T13:59:53.314334+0000","last_fullsized":"2024-03-26T13:59:53.314334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839877+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:27.839877+0000","last_peered":"2024-03-26T13:59:27.839877+0000","last_clean":"2024-03-26T13:59:27.839877+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:27.839877+0000","last_undegraded":"2024-03-26T13:59:27.839877+0000","last_fullsized":"2024-03-26T13:59:27.839877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842500+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:27.842500+0000","last_peered":"2024-03-26T13:59:27.842500+0000","last_clean":"2024-03-26T13:59:27.842500+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:27.842500+0000","last_undegraded":"2024-03-26T13:59:27.842500+0000","last_fullsized":"2024-03-26T13:59:27.842500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.838817+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:27.838817+0000","last_peered":"2024-03-26T13:59:27.838817+0000","last_clean":"2024-03-26T13:59:27.838817+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:27.838817+0000","last_undegraded":"2024-03-26T13:59:27.838817+0000","last_fullsized":"2024-03-26T13:59:27.838817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.264433+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:49.264433+0000","last_peered":"2024-03-26T13:59:49.264433+0000","last_clean":"2024-03-26T13:59:49.264433+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:49.264433+0000","last_undegraded":"2024-03-26T13:59:49.264433+0000","last_fullsized":"2024-03-26T13:59:49.264433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233597+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:30.233597+0000","last_peered":"2024-03-26T13:59:30.233597+0000","last_clean":"2024-03-26T13:59:30.233597+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:30.233597+0000","last_undegraded":"2024-03-26T13:59:30.233597+0000","last_fullsized":"2024-03-26T13:59:30.233597+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246587+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:48.246587+0000","last_peered":"2024-03-26T13:59:48.246587+0000","last_clean":"2024-03-26T13:59:48.246587+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:48.246587+0000","last_undegraded":"2024-03-26T13:59:48.246587+0000","last_fullsized":"2024-03-26T13:59:48.246587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839930+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:27.839930+0000","last_peered":"2024-03-26T13:59:27.839930+0000","last_clean":"2024-03-26T13:59:27.839930+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:27.839930+0000","last_undegraded":"2024-03-26T13:59:27.839930+0000","last_fullsized":"2024-03-26T13:59:27.839930+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324348+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:28.324348+0000","last_peered":"2024-03-26T13:59:28.324348+0000","last_clean":"2024-03-26T13:59:28.324348+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:28.324348+0000","last_undegraded":"2024-03-26T13:59:28.324348+0000","last_fullsized":"2024-03-26T13:59:28.324348+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:49.260579+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:49.260579+0000","last_peered":"2024-03-26T13:59:49.260579+0000","last_clean":"2024-03-26T13:59:49.260579+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:49.260579+0000","last_undegraded":"2024-03-26T13:59:49.260579+0000","last_fullsized":"2024-03-26T13:59:49.260579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325372+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:28.325372+0000","last_peered":"2024-03-26T13:59:28.325372+0000","last_clean":"2024-03-26T13:59:28.325372+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:28.325372+0000","last_undegraded":"2024-03-26T13:59:28.325372+0000","last_fullsized":"2024-03-26T13:59:28.325372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316202+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:53.316202+0000","last_peered":"2024-03-26T13:59:53.316202+0000","last_clean":"2024-03-26T13:59:53.316202+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:53.316202+0000","last_undegraded":"2024-03-26T13:59:53.316202+0000","last_fullsized":"2024-03-26T13:59:53.316202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.315881+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:53.315881+0000","last_peered":"2024-03-26T13:59:53.315881+0000","last_clean":"2024-03-26T13:59:53.315881+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:53.315881+0000","last_undegraded":"2024-03-26T13:59:53.315881+0000","last_fullsized":"2024-03-26T13:59:53.315881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.364591+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:29.364591+0000","last_peered":"2024-03-26T13:59:29.364591+0000","last_clean":"2024-03-26T13:59:29.364591+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:29.364591+0000","last_undegraded":"2024-03-26T13:59:29.364591+0000","last_fullsized":"2024-03-26T13:59:29.364591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.314755+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:53.314755+0000","last_peered":"2024-03-26T13:59:53.314755+0000","last_clean":"2024-03-26T13:59:53.314755+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:53.314755+0000","last_undegraded":"2024-03-26T13:59:53.314755+0000","last_fullsized":"2024-03-26T13:59:53.314755+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.249700+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:48.249700+0000","last_peered":"2024-03-26T13:59:48.249700+0000","last_clean":"2024-03-26T13:59:48.249700+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:48.249700+0000","last_undegraded":"2024-03-26T13:59:48.249700+0000","last_fullsized":"2024-03-26T13:59:48.249700+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.469234+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:29.469234+0000","last_peered":"2024-03-26T13:59:29.469234+0000","last_clean":"2024-03-26T13:59:29.469234+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:29.469234+0000","last_undegraded":"2024-03-26T13:59:29.469234+0000","last_fullsized":"2024-03-26T13:59:29.469234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.231634+0000","last_change":"2024-03-26T13:59:27.332943+0000","last_active":"2024-03-26T13:59:30.231634+0000","last_peered":"2024-03-26T13:59:30.231634+0000","last_clean":"2024-03-26T13:59:30.231634+0000","last_became_active":"2024-03-26T13:59:27.331677+0000","last_became_peered":"2024-03-26T13:59:27.331677+0000","last_unstale":"2024-03-26T13:59:30.231634+0000","last_undegraded":"2024-03-26T13:59:30.231634+0000","last_fullsized":"2024-03-26T13:59:30.231634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:36:19.990935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.246514+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:48.246514+0000","last_peered":"2024-03-26T13:59:48.246514+0000","last_clean":"2024-03-26T13:59:48.246514+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:48.246514+0000","last_undegraded":"2024-03-26T13:59:48.246514+0000","last_fullsized":"2024-03-26T13:59:48.246514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.232606+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:30.232606+0000","last_peered":"2024-03-26T13:59:30.232606+0000","last_clean":"2024-03-26T13:59:30.232606+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:30.232606+0000","last_undegraded":"2024-03-26T13:59:30.232606+0000","last_fullsized":"2024-03-26T13:59:30.232606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":16,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:52.319197+0000","last_change":"2024-03-26T13:59:53.314318+0000","last_active":"2024-03-26T13:59:53.314318+0000","last_peered":"2024-03-26T13:59:52.319197+0000","last_clean":"2024-03-26T13:59:52.319197+0000","last_became_active":"2024-03-26T13:59:27.326077+0000","last_became_peered":"2024-03-26T13:59:27.326077+0000","last_unstale":"2024-03-26T13:59:52.319197+0000","last_undegraded":"2024-03-26T13:59:52.319197+0000","last_fullsized":"2024-03-26T13:59:52.319197+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7,8,9,10],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317082+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:53.317082+0000","last_peered":"2024-03-26T13:59:53.317082+0000","last_clean":"2024-03-26T13:59:53.317082+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:53.317082+0000","last_undegraded":"2024-03-26T13:59:53.317082+0000","last_fullsized":"2024-03-26T13:59:53.317082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325092+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:28.325092+0000","last_peered":"2024-03-26T13:59:28.325092+0000","last_clean":"2024-03-26T13:59:28.325092+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:28.325092+0000","last_undegraded":"2024-03-26T13:59:28.325092+0000","last_fullsized":"2024-03-26T13:59:28.325092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.313859+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:53.313859+0000","last_peered":"2024-03-26T13:59:53.313859+0000","last_clean":"2024-03-26T13:59:53.313859+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:53.313859+0000","last_undegraded":"2024-03-26T13:59:53.313859+0000","last_fullsized":"2024-03-26T13:59:53.313859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:48.252376+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:48.252376+0000","last_peered":"2024-03-26T13:59:48.252376+0000","last_clean":"2024-03-26T13:59:48.252376+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:48.252376+0000","last_undegraded":"2024-03-26T13:59:48.252376+0000","last_fullsized":"2024-03-26T13:59:48.252376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840788+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:27.840788+0000","last_peered":"2024-03-26T13:59:27.840788+0000","last_clean":"2024-03-26T13:59:27.840788+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:27.840788+0000","last_undegraded":"2024-03-26T13:59:27.840788+0000","last_fullsized":"2024-03-26T13:59:27.840788+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348985+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:29.348985+0000","last_peered":"2024-03-26T13:59:29.348985+0000","last_clean":"2024-03-26T13:59:29.348985+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:29.348985+0000","last_undegraded":"2024-03-26T13:59:29.348985+0000","last_fullsized":"2024-03-26T13:59:29.348985+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319280+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:53.319280+0000","last_peered":"2024-03-26T13:59:53.319280+0000","last_clean":"2024-03-26T13:59:53.319280+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:53.319280+0000","last_undegraded":"2024-03-26T13:59:53.319280+0000","last_fullsized":"2024-03-26T13:59:53.319280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841263+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:27.841263+0000","last_peered":"2024-03-26T13:59:27.841263+0000","last_clean":"2024-03-26T13:59:27.841263+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:27.841263+0000","last_undegraded":"2024-03-26T13:59:27.841263+0000","last_fullsized":"2024-03-26T13:59:27.841263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363505+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:29.363505+0000","last_peered":"2024-03-26T13:59:29.363505+0000","last_clean":"2024-03-26T13:59:29.363505+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:29.363505+0000","last_undegraded":"2024-03-26T13:59:29.363505+0000","last_fullsized":"2024-03-26T13:59:29.363505+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.842212+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:27.842212+0000","last_peered":"2024-03-26T13:59:27.842212+0000","last_clean":"2024-03-26T13:59:27.842212+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:27.842212+0000","last_undegraded":"2024-03-26T13:59:27.842212+0000","last_fullsized":"2024-03-26T13:59:27.842212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841024+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:27.841024+0000","last_peered":"2024-03-26T13:59:27.841024+0000","last_clean":"2024-03-26T13:59:27.841024+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:27.841024+0000","last_undegraded":"2024-03-26T13:59:27.841024+0000","last_fullsized":"2024-03-26T13:59:27.841024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840326+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:27.840326+0000","last_peered":"2024-03-26T13:59:27.840326+0000","last_clean":"2024-03-26T13:59:27.840326+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:27.840326+0000","last_undegraded":"2024-03-26T13:59:27.840326+0000","last_fullsized":"2024-03-26T13:59:27.840326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840337+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:27.840337+0000","last_peered":"2024-03-26T13:59:27.840337+0000","last_clean":"2024-03-26T13:59:27.840337+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:27.840337+0000","last_undegraded":"2024-03-26T13:59:27.840337+0000","last_fullsized":"2024-03-26T13:59:27.840337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.841677+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:27.841677+0000","last_peered":"2024-03-26T13:59:27.841677+0000","last_clean":"2024-03-26T13:59:27.841677+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:27.841677+0000","last_undegraded":"2024-03-26T13:59:27.841677+0000","last_fullsized":"2024-03-26T13:59:27.841677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.233480+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:30.233480+0000","last_peered":"2024-03-26T13:59:30.233480+0000","last_clean":"2024-03-26T13:59:30.233480+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:30.233480+0000","last_undegraded":"2024-03-26T13:59:30.233480+0000","last_fullsized":"2024-03-26T13:59:30.233480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316709+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:53.316709+0000","last_peered":"2024-03-26T13:59:53.316709+0000","last_clean":"2024-03-26T13:59:53.316709+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:53.316709+0000","last_undegraded":"2024-03-26T13:59:53.316709+0000","last_fullsized":"2024-03-26T13:59:53.316709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316722+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:53.316722+0000","last_peered":"2024-03-26T13:59:53.316722+0000","last_clean":"2024-03-26T13:59:53.316722+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:53.316722+0000","last_undegraded":"2024-03-26T13:59:53.316722+0000","last_fullsized":"2024-03-26T13:59:53.316722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.363485+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:29.363485+0000","last_peered":"2024-03-26T13:59:29.363485+0000","last_clean":"2024-03-26T13:59:29.363485+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:29.363485+0000","last_undegraded":"2024-03-26T13:59:29.363485+0000","last_fullsized":"2024-03-26T13:59:29.363485+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840283+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:27.840283+0000","last_peered":"2024-03-26T13:59:27.840283+0000","last_clean":"2024-03-26T13:59:27.840283+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:27.840283+0000","last_undegraded":"2024-03-26T13:59:27.840283+0000","last_fullsized":"2024-03-26T13:59:27.840283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234703+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:30.234703+0000","last_peered":"2024-03-26T13:59:30.234703+0000","last_clean":"2024-03-26T13:59:30.234703+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:30.234703+0000","last_undegraded":"2024-03-26T13:59:30.234703+0000","last_fullsized":"2024-03-26T13:59:30.234703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319745+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:53.319745+0000","last_peered":"2024-03-26T13:59:53.319745+0000","last_clean":"2024-03-26T13:59:53.319745+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:53.319745+0000","last_undegraded":"2024-03-26T13:59:53.319745+0000","last_fullsized":"2024-03-26T13:59:53.319745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.325560+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:28.325560+0000","last_peered":"2024-03-26T13:59:28.325560+0000","last_clean":"2024-03-26T13:59:28.325560+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:28.325560+0000","last_undegraded":"2024-03-26T13:59:28.325560+0000","last_fullsized":"2024-03-26T13:59:28.325560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.840768+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:27.840768+0000","last_peered":"2024-03-26T13:59:27.840768+0000","last_clean":"2024-03-26T13:59:27.840768+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:27.840768+0000","last_undegraded":"2024-03-26T13:59:27.840768+0000","last_fullsized":"2024-03-26T13:59:27.840768+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:28.324700+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:28.324700+0000","last_peered":"2024-03-26T13:59:28.324700+0000","last_clean":"2024-03-26T13:59:28.324700+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:28.324700+0000","last_undegraded":"2024-03-26T13:59:28.324700+0000","last_fullsized":"2024-03-26T13:59:28.324700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:29.348523+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:29.348523+0000","last_peered":"2024-03-26T13:59:29.348523+0000","last_clean":"2024-03-26T13:59:29.348523+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:29.348523+0000","last_undegraded":"2024-03-26T13:59:29.348523+0000","last_fullsized":"2024-03-26T13:59:29.348523+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:27.839530+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:27.839530+0000","last_peered":"2024-03-26T13:59:27.839530+0000","last_clean":"2024-03-26T13:59:27.839530+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:27.839530+0000","last_undegraded":"2024-03-26T13:59:27.839530+0000","last_fullsized":"2024-03-26T13:59:27.839530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-26T13:59:30.234803+0000","last_change":"2024-03-26T13:59:23.308422+0000","last_active":"2024-03-26T13:59:30.234803+0000","last_peered":"2024-03-26T13:59:30.234803+0000","last_clean":"2024-03-26T13:59:30.234803+0000","last_became_active":"2024-03-26T13:59:23.308011+0000","last_became_peered":"2024-03-26T13:59:23.308011+0000","last_unstale":"2024-03-26T13:59:30.234803+0000","last_undegraded":"2024-03-26T13:59:30.234803+0000","last_fullsized":"2024-03-26T13:59:30.234803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:08.894340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14443,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":32,"num_read_kb":28,"num_write":133,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":339968,"data_stored":112630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480848,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1083045,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":481084,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037802,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":481004,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299441,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1072817,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":480356,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":512012,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":481744,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":538070,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":480550,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":485397,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":1076206,"data_compressed":50353,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":33731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":12411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T13:59:56.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3220191744' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-26T13:59:56.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:56 smithi137 ceph-mon[31539]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:56.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3220191744' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-26T13:59:56.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:56 smithi179 ceph-mon[35472]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:56.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3220191744' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-26T13:59:56.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:56 smithi159 ceph-mon[35501]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T13:59:57.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 13:59:57 smithi179 ceph-mon[35472]: pgmap v384: 129 pgs: 3 peering, 126 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 18 op/s 2024-03-26T13:59:57.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 13:59:57 smithi159 ceph-mon[35501]: pgmap v384: 129 pgs: 3 peering, 126 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 18 op/s 2024-03-26T13:59:57.781 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 13:59:57 smithi137 ceph-mon[31539]: pgmap v384: 129 pgs: 3 peering, 126 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 18 op/s 2024-03-26T13:59:59.244 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T13:59:59.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.470+0000 7f2899a00640 1 -- 172.21.15.137:0/1235043007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2894076060 msgr2=0x7f2894076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.470+0000 7f2899a00640 1 --2- 172.21.15.137:0/1235043007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2894076060 0x7f2894076480 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2884009ca0 tx=0x7f288402c0f0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 -- 172.21.15.137:0/1235043007 shutdown_connections 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 --2- 172.21.15.137:0/1235043007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894077c40 0x7f28940784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 --2- 172.21.15.137:0/1235043007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28940772a0 0x7f2894077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 --2- 172.21.15.137:0/1235043007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2894076060 0x7f2894076480 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 -- 172.21.15.137:0/1235043007 >> 172.21.15.137:0/1235043007 conn(0x7f28940fee50 msgr2=0x7f2894101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 -- 172.21.15.137:0/1235043007 shutdown_connections 2024-03-26T13:59:59.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.471+0000 7f2899a00640 1 -- 172.21.15.137:0/1235043007 wait complete. 2024-03-26T13:59:59.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 Processor -- start 2024-03-26T13:59:59.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 -- start start 2024-03-26T13:59:59.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894076060 0x7f289419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 0x7f289419e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894077c40 0x7f28941a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894113820 con 0x7f28940772a0 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28941136a0 con 0x7f2894076060 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2899a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28941139a0 con 0x7f2894077c40 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2892000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 0x7f289419e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2892000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 0x7f289419e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54730/0 (socket says 172.21.15.137:54730) 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.472+0000 7f2892000640 1 -- 172.21.15.137:0/4173393074 learned_addr learned my addr 172.21.15.137:0/4173393074 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894076060 0x7f289419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2893400640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894077c40 0x7f28941a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892000640 1 -- 172.21.15.137:0/4173393074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894077c40 msgr2=0x7f28941a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892000640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894077c40 0x7f28941a5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892000640 1 -- 172.21.15.137:0/4173393074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894076060 msgr2=0x7f289419df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892000640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894076060 0x7f289419df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892000640 1 -- 172.21.15.137:0/4173393074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28941a5b80 con 0x7f28940772a0 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894076060 0x7f289419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2892000640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 0x7f289419e4a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d770 tx=0x7f287c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c017070 con 0x7f28940772a0 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28941a5e70 con 0x7f28940772a0 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f287c0054d0 con 0x7f28940772a0 2024-03-26T13:59:59.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.473+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c004cf0 con 0x7f28940772a0 2024-03-26T13:59:59.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.474+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28941a6330 con 0x7f28940772a0 2024-03-26T13:59:59.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.475+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28941104e0 con 0x7f28940772a0 2024-03-26T13:59:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.477+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f287c004e90 con 0x7f28940772a0 2024-03-26T13:59:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.478+0000 7f288b400640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28780a2910 0x7f28780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.478+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f287c0cea10 con 0x7f28940772a0 2024-03-26T13:59:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.478+0000 7f2892a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28780a2910 0x7f28780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.479+0000 7f2892a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28780a2910 0x7f28780a4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2884037040 tx=0x7f2884000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.483+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f287c09b590 con 0x7f28940772a0 2024-03-26T13:59:59.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.678+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2894061ae0 con 0x7f28780a2910 2024-03-26T13:59:59.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.706+0000 7f288b400640 1 -- 172.21.15.137:0/4173393074 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340631 (secure 0 0 0) 0x7f2894061ae0 con 0x7f28780a2910 2024-03-26T13:59:59.710 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T13:59:59.712 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.713+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28780a2910 msgr2=0x7f28780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.713+0000 7f2899a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28780a2910 0x7f28780a4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2884037040 tx=0x7f2884000f40 comp rx=0 tx=0).stop 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.713+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 msgr2=0x7f289419e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.713+0000 7f2899a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 0x7f289419e4a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d770 tx=0x7f287c00dc30 comp rx=0 tx=0).stop 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 shutdown_connections 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28780a2910 0x7f28780a4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2894077c40 0x7f28941a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28940772a0 0x7f289419e4a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 --2- 172.21.15.137:0/4173393074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2894076060 0x7f289419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 >> 172.21.15.137:0/4173393074 conn(0x7f28940fee50 msgr2=0x7f2894100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 shutdown_connections 2024-03-26T13:59:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.714+0000 7f2899a00640 1 -- 172.21.15.137:0/4173393074 wait complete. 2024-03-26T13:59:59.731 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-03-26T13:59:59.086451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651057,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":191,"num_read_kb":205,"num_write":333,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361428,"kb_used_data":12212,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580396,"statfs":{"total":1151940427776,"available":1151570325504,"internally_reserved":0,"allocated":12505088,"data_stored":7956571,"data_compressed":349077,"data_compressed_allocated":471040,"data_compressed_original":2441216,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56049,"num_objects":36,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":65,"num_read_kb":96,"num_write":94,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334990+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:54.334990+0000","last_peered":"2024-03-26T13:59:54.334990+0000","last_clean":"2024-03-26T13:59:54.334990+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:54.334990+0000","last_undegraded":"2024-03-26T13:59:54.334990+0000","last_fullsized":"2024-03-26T13:59:54.334990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330109+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:54.330109+0000","last_peered":"2024-03-26T13:59:54.330109+0000","last_clean":"2024-03-26T13:59:54.330109+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:54.330109+0000","last_undegraded":"2024-03-26T13:59:54.330109+0000","last_fullsized":"2024-03-26T13:59:54.330109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327603+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:54.327603+0000","last_peered":"2024-03-26T13:59:54.327603+0000","last_clean":"2024-03-26T13:59:54.327603+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:54.327603+0000","last_undegraded":"2024-03-26T13:59:54.327603+0000","last_fullsized":"2024-03-26T13:59:54.327603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332907+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:54.332907+0000","last_peered":"2024-03-26T13:59:54.332907+0000","last_clean":"2024-03-26T13:59:54.332907+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:54.332907+0000","last_undegraded":"2024-03-26T13:59:54.332907+0000","last_fullsized":"2024-03-26T13:59:54.332907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703579+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:54.703579+0000","last_peered":"2024-03-26T13:59:54.703579+0000","last_clean":"2024-03-26T13:59:54.703579+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:54.703579+0000","last_undegraded":"2024-03-26T13:59:54.703579+0000","last_fullsized":"2024-03-26T13:59:54.703579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329612+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:54.329612+0000","last_peered":"2024-03-26T13:59:54.329612+0000","last_clean":"2024-03-26T13:59:54.329612+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:54.329612+0000","last_undegraded":"2024-03-26T13:59:54.329612+0000","last_fullsized":"2024-03-26T13:59:54.329612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755315+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:54.755315+0000","last_peered":"2024-03-26T13:59:54.755315+0000","last_clean":"2024-03-26T13:59:54.755315+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:54.755315+0000","last_undegraded":"2024-03-26T13:59:54.755315+0000","last_fullsized":"2024-03-26T13:59:54.755315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754310+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:54.754310+0000","last_peered":"2024-03-26T13:59:54.754310+0000","last_clean":"2024-03-26T13:59:54.754310+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:54.754310+0000","last_undegraded":"2024-03-26T13:59:54.754310+0000","last_fullsized":"2024-03-26T13:59:54.754310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704481+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:54.704481+0000","last_peered":"2024-03-26T13:59:54.704481+0000","last_clean":"2024-03-26T13:59:54.704481+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:54.704481+0000","last_undegraded":"2024-03-26T13:59:54.704481+0000","last_fullsized":"2024-03-26T13:59:54.704481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330340+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:54.330340+0000","last_peered":"2024-03-26T13:59:54.330340+0000","last_clean":"2024-03-26T13:59:54.330340+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:54.330340+0000","last_undegraded":"2024-03-26T13:59:54.330340+0000","last_fullsized":"2024-03-26T13:59:54.330340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327473+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:54.327473+0000","last_peered":"2024-03-26T13:59:54.327473+0000","last_clean":"2024-03-26T13:59:54.327473+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:54.327473+0000","last_undegraded":"2024-03-26T13:59:54.327473+0000","last_fullsized":"2024-03-26T13:59:54.327473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757043+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:54.757043+0000","last_peered":"2024-03-26T13:59:54.757043+0000","last_clean":"2024-03-26T13:59:54.757043+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:54.757043+0000","last_undegraded":"2024-03-26T13:59:54.757043+0000","last_fullsized":"2024-03-26T13:59:54.757043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331805+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:54.331805+0000","last_peered":"2024-03-26T13:59:54.331805+0000","last_clean":"2024-03-26T13:59:54.331805+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:54.331805+0000","last_undegraded":"2024-03-26T13:59:54.331805+0000","last_fullsized":"2024-03-26T13:59:54.331805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.344831+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T13:59:54.344831+0000","last_peered":"2024-03-26T13:59:54.344831+0000","last_clean":"2024-03-26T13:59:54.344831+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T13:59:54.344831+0000","last_undegraded":"2024-03-26T13:59:54.344831+0000","last_fullsized":"2024-03-26T13:59:54.344831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753840+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:54.753840+0000","last_peered":"2024-03-26T13:59:54.753840+0000","last_clean":"2024-03-26T13:59:54.753840+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:54.753840+0000","last_undegraded":"2024-03-26T13:59:54.753840+0000","last_fullsized":"2024-03-26T13:59:54.753840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334663+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:54.334663+0000","last_peered":"2024-03-26T13:59:54.334663+0000","last_clean":"2024-03-26T13:59:54.334663+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:54.334663+0000","last_undegraded":"2024-03-26T13:59:54.334663+0000","last_fullsized":"2024-03-26T13:59:54.334663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330566+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:54.330566+0000","last_peered":"2024-03-26T13:59:54.330566+0000","last_clean":"2024-03-26T13:59:54.330566+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:54.330566+0000","last_undegraded":"2024-03-26T13:59:54.330566+0000","last_fullsized":"2024-03-26T13:59:54.330566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319222+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:53.319222+0000","last_peered":"2024-03-26T13:59:53.319222+0000","last_clean":"2024-03-26T13:59:53.319222+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:53.319222+0000","last_undegraded":"2024-03-26T13:59:53.319222+0000","last_fullsized":"2024-03-26T13:59:53.319222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317072+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:53.317072+0000","last_peered":"2024-03-26T13:59:53.317072+0000","last_clean":"2024-03-26T13:59:53.317072+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:53.317072+0000","last_undegraded":"2024-03-26T13:59:53.317072+0000","last_fullsized":"2024-03-26T13:59:53.317072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330916+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:54.330916+0000","last_peered":"2024-03-26T13:59:54.330916+0000","last_clean":"2024-03-26T13:59:54.330916+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:54.330916+0000","last_undegraded":"2024-03-26T13:59:54.330916+0000","last_fullsized":"2024-03-26T13:59:54.330916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757563+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:54.757563+0000","last_peered":"2024-03-26T13:59:54.757563+0000","last_clean":"2024-03-26T13:59:54.757563+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:54.757563+0000","last_undegraded":"2024-03-26T13:59:54.757563+0000","last_fullsized":"2024-03-26T13:59:54.757563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.325949+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:54.325949+0000","last_peered":"2024-03-26T13:59:54.325949+0000","last_clean":"2024-03-26T13:59:54.325949+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:54.325949+0000","last_undegraded":"2024-03-26T13:59:54.325949+0000","last_fullsized":"2024-03-26T13:59:54.325949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757809+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:54.757809+0000","last_peered":"2024-03-26T13:59:54.757809+0000","last_clean":"2024-03-26T13:59:54.757809+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:54.757809+0000","last_undegraded":"2024-03-26T13:59:54.757809+0000","last_fullsized":"2024-03-26T13:59:54.757809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330337+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:54.330337+0000","last_peered":"2024-03-26T13:59:54.330337+0000","last_clean":"2024-03-26T13:59:54.330337+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:54.330337+0000","last_undegraded":"2024-03-26T13:59:54.330337+0000","last_fullsized":"2024-03-26T13:59:54.330337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326515+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:54.326515+0000","last_peered":"2024-03-26T13:59:54.326515+0000","last_clean":"2024-03-26T13:59:54.326515+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:54.326515+0000","last_undegraded":"2024-03-26T13:59:54.326515+0000","last_fullsized":"2024-03-26T13:59:54.326515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339658+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:54.339658+0000","last_peered":"2024-03-26T13:59:54.339658+0000","last_clean":"2024-03-26T13:59:54.339658+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:54.339658+0000","last_undegraded":"2024-03-26T13:59:54.339658+0000","last_fullsized":"2024-03-26T13:59:54.339658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754197+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:54.754197+0000","last_peered":"2024-03-26T13:59:54.754197+0000","last_clean":"2024-03-26T13:59:54.754197+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:54.754197+0000","last_undegraded":"2024-03-26T13:59:54.754197+0000","last_fullsized":"2024-03-26T13:59:54.754197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326200+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:54.326200+0000","last_peered":"2024-03-26T13:59:54.326200+0000","last_clean":"2024-03-26T13:59:54.326200+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:54.326200+0000","last_undegraded":"2024-03-26T13:59:54.326200+0000","last_fullsized":"2024-03-26T13:59:54.326200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.342115+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:54.342115+0000","last_peered":"2024-03-26T13:59:54.342115+0000","last_clean":"2024-03-26T13:59:54.342115+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:54.342115+0000","last_undegraded":"2024-03-26T13:59:54.342115+0000","last_fullsized":"2024-03-26T13:59:54.342115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761061+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:54.761061+0000","last_peered":"2024-03-26T13:59:54.761061+0000","last_clean":"2024-03-26T13:59:54.761061+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:54.761061+0000","last_undegraded":"2024-03-26T13:59:54.761061+0000","last_fullsized":"2024-03-26T13:59:54.761061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.236338+0000","last_change":"2024-03-26T13:59:53.317881+0000","last_active":"2024-03-26T13:59:53.317881+0000","last_peered":"2024-03-26T13:59:30.236338+0000","last_clean":"2024-03-26T13:59:30.236338+0000","last_became_active":"2024-03-26T13:59:27.323585+0000","last_became_peered":"2024-03-26T13:59:27.323585+0000","last_unstale":"2024-03-26T13:59:30.236338+0000","last_undegraded":"2024-03-26T13:59:30.236338+0000","last_fullsized":"2024-03-26T13:59:30.236338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7,8,11],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.279834+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:55.279834+0000","last_peered":"2024-03-26T13:59:55.279834+0000","last_clean":"2024-03-26T13:59:55.279834+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:55.279834+0000","last_undegraded":"2024-03-26T13:59:55.279834+0000","last_fullsized":"2024-03-26T13:59:55.279834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346323+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T13:59:54.346323+0000","last_peered":"2024-03-26T13:59:54.346323+0000","last_clean":"2024-03-26T13:59:54.346323+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T13:59:54.346323+0000","last_undegraded":"2024-03-26T13:59:54.346323+0000","last_fullsized":"2024-03-26T13:59:54.346323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.234348+0000","last_change":"2024-03-26T13:59:53.316731+0000","last_active":"2024-03-26T13:59:53.316731+0000","last_peered":"2024-03-26T13:59:30.234348+0000","last_clean":"2024-03-26T13:59:30.234348+0000","last_became_active":"2024-03-26T13:59:27.324762+0000","last_became_peered":"2024-03-26T13:59:27.324762+0000","last_unstale":"2024-03-26T13:59:30.234348+0000","last_undegraded":"2024-03-26T13:59:30.234348+0000","last_fullsized":"2024-03-26T13:59:30.234348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703422+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:54.703422+0000","last_peered":"2024-03-26T13:59:54.703422+0000","last_clean":"2024-03-26T13:59:54.703422+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:54.703422+0000","last_undegraded":"2024-03-26T13:59:54.703422+0000","last_fullsized":"2024-03-26T13:59:54.703422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758242+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:54.758242+0000","last_peered":"2024-03-26T13:59:54.758242+0000","last_clean":"2024-03-26T13:59:54.758242+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:54.758242+0000","last_undegraded":"2024-03-26T13:59:54.758242+0000","last_fullsized":"2024-03-26T13:59:54.758242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334258+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:54.334258+0000","last_peered":"2024-03-26T13:59:54.334258+0000","last_clean":"2024-03-26T13:59:54.334258+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:54.334258+0000","last_undegraded":"2024-03-26T13:59:54.334258+0000","last_fullsized":"2024-03-26T13:59:54.334258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755967+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:54.755967+0000","last_peered":"2024-03-26T13:59:54.755967+0000","last_clean":"2024-03-26T13:59:54.755967+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:54.755967+0000","last_undegraded":"2024-03-26T13:59:54.755967+0000","last_fullsized":"2024-03-26T13:59:54.755967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.759517+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T13:59:54.759517+0000","last_peered":"2024-03-26T13:59:54.759517+0000","last_clean":"2024-03-26T13:59:54.759517+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T13:59:54.759517+0000","last_undegraded":"2024-03-26T13:59:54.759517+0000","last_fullsized":"2024-03-26T13:59:54.759517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329961+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:54.329961+0000","last_peered":"2024-03-26T13:59:54.329961+0000","last_clean":"2024-03-26T13:59:54.329961+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:54.329961+0000","last_undegraded":"2024-03-26T13:59:54.329961+0000","last_fullsized":"2024-03-26T13:59:54.329961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757020+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:54.757020+0000","last_peered":"2024-03-26T13:59:54.757020+0000","last_clean":"2024-03-26T13:59:54.757020+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:54.757020+0000","last_undegraded":"2024-03-26T13:59:54.757020+0000","last_fullsized":"2024-03-26T13:59:54.757020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327822+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:54.327822+0000","last_peered":"2024-03-26T13:59:54.327822+0000","last_clean":"2024-03-26T13:59:54.327822+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:54.327822+0000","last_undegraded":"2024-03-26T13:59:54.327822+0000","last_fullsized":"2024-03-26T13:59:54.327822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755747+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:54.755747+0000","last_peered":"2024-03-26T13:59:54.755747+0000","last_clean":"2024-03-26T13:59:54.755747+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:54.755747+0000","last_undegraded":"2024-03-26T13:59:54.755747+0000","last_fullsized":"2024-03-26T13:59:54.755747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704508+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:54.704508+0000","last_peered":"2024-03-26T13:59:54.704508+0000","last_clean":"2024-03-26T13:59:54.704508+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:54.704508+0000","last_undegraded":"2024-03-26T13:59:54.704508+0000","last_fullsized":"2024-03-26T13:59:54.704508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754499+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:54.754499+0000","last_peered":"2024-03-26T13:59:54.754499+0000","last_clean":"2024-03-26T13:59:54.754499+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:54.754499+0000","last_undegraded":"2024-03-26T13:59:54.754499+0000","last_fullsized":"2024-03-26T13:59:54.754499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327602+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:54.327602+0000","last_peered":"2024-03-26T13:59:54.327602+0000","last_clean":"2024-03-26T13:59:54.327602+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:54.327602+0000","last_undegraded":"2024-03-26T13:59:54.327602+0000","last_fullsized":"2024-03-26T13:59:54.327602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330446+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:54.330446+0000","last_peered":"2024-03-26T13:59:54.330446+0000","last_clean":"2024-03-26T13:59:54.330446+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:54.330446+0000","last_undegraded":"2024-03-26T13:59:54.330446+0000","last_fullsized":"2024-03-26T13:59:54.330446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761347+0000","last_change":"2024-03-26T13:59:54.761347+0000","last_active":"2024-03-26T13:59:54.761347+0000","last_peered":"2024-03-26T13:59:54.761347+0000","last_clean":"2024-03-26T13:59:54.761347+0000","last_became_active":"2024-03-26T13:59:54.760966+0000","last_became_peered":"2024-03-26T13:59:54.760966+0000","last_unstale":"2024-03-26T13:59:54.761347+0000","last_undegraded":"2024-03-26T13:59:54.761347+0000","last_fullsized":"2024-03-26T13:59:54.761347+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:27:13.241398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326627+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:54.326627+0000","last_peered":"2024-03-26T13:59:54.326627+0000","last_clean":"2024-03-26T13:59:54.326627+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:54.326627+0000","last_undegraded":"2024-03-26T13:59:54.326627+0000","last_fullsized":"2024-03-26T13:59:54.326627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.760687+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:54.760687+0000","last_peered":"2024-03-26T13:59:54.760687+0000","last_clean":"2024-03-26T13:59:54.760687+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:54.760687+0000","last_undegraded":"2024-03-26T13:59:54.760687+0000","last_fullsized":"2024-03-26T13:59:54.760687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754283+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:54.754283+0000","last_peered":"2024-03-26T13:59:54.754283+0000","last_clean":"2024-03-26T13:59:54.754283+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:54.754283+0000","last_undegraded":"2024-03-26T13:59:54.754283+0000","last_fullsized":"2024-03-26T13:59:54.754283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.328197+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:54.328197+0000","last_peered":"2024-03-26T13:59:54.328197+0000","last_clean":"2024-03-26T13:59:54.328197+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:54.328197+0000","last_undegraded":"2024-03-26T13:59:54.328197+0000","last_fullsized":"2024-03-26T13:59:54.328197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'26","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.283705+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:55.283705+0000","last_peered":"2024-03-26T13:59:55.283705+0000","last_clean":"2024-03-26T13:59:55.283705+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:55.283705+0000","last_undegraded":"2024-03-26T13:59:55.283705+0000","last_fullsized":"2024-03-26T13:59:55.283705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56454,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":64,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.340153+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:54.340153+0000","last_peered":"2024-03-26T13:59:54.340153+0000","last_clean":"2024-03-26T13:59:54.340153+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:54.340153+0000","last_undegraded":"2024-03-26T13:59:54.340153+0000","last_fullsized":"2024-03-26T13:59:54.340153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761598+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:54.761598+0000","last_peered":"2024-03-26T13:59:54.761598+0000","last_clean":"2024-03-26T13:59:54.761598+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:54.761598+0000","last_undegraded":"2024-03-26T13:59:54.761598+0000","last_fullsized":"2024-03-26T13:59:54.761598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.318377+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:53.318377+0000","last_peered":"2024-03-26T13:59:53.318377+0000","last_clean":"2024-03-26T13:59:53.318377+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:53.318377+0000","last_undegraded":"2024-03-26T13:59:53.318377+0000","last_fullsized":"2024-03-26T13:59:53.318377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756449+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:54.756449+0000","last_peered":"2024-03-26T13:59:54.756449+0000","last_clean":"2024-03-26T13:59:54.756449+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:54.756449+0000","last_undegraded":"2024-03-26T13:59:54.756449+0000","last_fullsized":"2024-03-26T13:59:54.756449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332076+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:54.332076+0000","last_peered":"2024-03-26T13:59:54.332076+0000","last_clean":"2024-03-26T13:59:54.332076+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:54.332076+0000","last_undegraded":"2024-03-26T13:59:54.332076+0000","last_fullsized":"2024-03-26T13:59:54.332076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757435+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:54.757435+0000","last_peered":"2024-03-26T13:59:54.757435+0000","last_clean":"2024-03-26T13:59:54.757435+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:54.757435+0000","last_undegraded":"2024-03-26T13:59:54.757435+0000","last_fullsized":"2024-03-26T13:59:54.757435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330136+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:54.330136+0000","last_peered":"2024-03-26T13:59:54.330136+0000","last_clean":"2024-03-26T13:59:54.330136+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:54.330136+0000","last_undegraded":"2024-03-26T13:59:54.330136+0000","last_fullsized":"2024-03-26T13:59:54.330136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330531+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:54.330531+0000","last_peered":"2024-03-26T13:59:54.330531+0000","last_clean":"2024-03-26T13:59:54.330531+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:54.330531+0000","last_undegraded":"2024-03-26T13:59:54.330531+0000","last_fullsized":"2024-03-26T13:59:54.330531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:57.287720+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:57.287720+0000","last_peered":"2024-03-26T13:59:57.287720+0000","last_clean":"2024-03-26T13:59:57.287720+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:57.287720+0000","last_undegraded":"2024-03-26T13:59:57.287720+0000","last_fullsized":"2024-03-26T13:59:57.287720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329932+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:54.329932+0000","last_peered":"2024-03-26T13:59:54.329932+0000","last_clean":"2024-03-26T13:59:54.329932+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:54.329932+0000","last_undegraded":"2024-03-26T13:59:54.329932+0000","last_fullsized":"2024-03-26T13:59:54.329932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758226+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:54.758226+0000","last_peered":"2024-03-26T13:59:54.758226+0000","last_clean":"2024-03-26T13:59:54.758226+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:54.758226+0000","last_undegraded":"2024-03-26T13:59:54.758226+0000","last_fullsized":"2024-03-26T13:59:54.758226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330561+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:54.330561+0000","last_peered":"2024-03-26T13:59:54.330561+0000","last_clean":"2024-03-26T13:59:54.330561+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:54.330561+0000","last_undegraded":"2024-03-26T13:59:54.330561+0000","last_fullsized":"2024-03-26T13:59:54.330561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756514+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:54.756514+0000","last_peered":"2024-03-26T13:59:54.756514+0000","last_clean":"2024-03-26T13:59:54.756514+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:54.756514+0000","last_undegraded":"2024-03-26T13:59:54.756514+0000","last_fullsized":"2024-03-26T13:59:54.756514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756211+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:54.756211+0000","last_peered":"2024-03-26T13:59:54.756211+0000","last_clean":"2024-03-26T13:59:54.756211+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:54.756211+0000","last_undegraded":"2024-03-26T13:59:54.756211+0000","last_fullsized":"2024-03-26T13:59:54.756211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.341535+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:54.341535+0000","last_peered":"2024-03-26T13:59:54.341535+0000","last_clean":"2024-03-26T13:59:54.341535+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:54.341535+0000","last_undegraded":"2024-03-26T13:59:54.341535+0000","last_fullsized":"2024-03-26T13:59:54.341535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761409+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T13:59:54.761409+0000","last_peered":"2024-03-26T13:59:54.761409+0000","last_clean":"2024-03-26T13:59:54.761409+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T13:59:54.761409+0000","last_undegraded":"2024-03-26T13:59:54.761409+0000","last_fullsized":"2024-03-26T13:59:54.761409+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334043+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:54.334043+0000","last_peered":"2024-03-26T13:59:54.334043+0000","last_clean":"2024-03-26T13:59:54.334043+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:54.334043+0000","last_undegraded":"2024-03-26T13:59:54.334043+0000","last_fullsized":"2024-03-26T13:59:54.334043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329309+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:54.329309+0000","last_peered":"2024-03-26T13:59:54.329309+0000","last_clean":"2024-03-26T13:59:54.329309+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:54.329309+0000","last_undegraded":"2024-03-26T13:59:54.329309+0000","last_fullsized":"2024-03-26T13:59:54.329309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703899+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:54.703899+0000","last_peered":"2024-03-26T13:59:54.703899+0000","last_clean":"2024-03-26T13:59:54.703899+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:54.703899+0000","last_undegraded":"2024-03-26T13:59:54.703899+0000","last_fullsized":"2024-03-26T13:59:54.703899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757312+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:54.757312+0000","last_peered":"2024-03-26T13:59:54.757312+0000","last_clean":"2024-03-26T13:59:54.757312+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:54.757312+0000","last_undegraded":"2024-03-26T13:59:54.757312+0000","last_fullsized":"2024-03-26T13:59:54.757312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329838+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:54.329838+0000","last_peered":"2024-03-26T13:59:54.329838+0000","last_clean":"2024-03-26T13:59:54.329838+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:54.329838+0000","last_undegraded":"2024-03-26T13:59:54.329838+0000","last_fullsized":"2024-03-26T13:59:54.329838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703337+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:54.703337+0000","last_peered":"2024-03-26T13:59:54.703337+0000","last_clean":"2024-03-26T13:59:54.703337+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:54.703337+0000","last_undegraded":"2024-03-26T13:59:54.703337+0000","last_fullsized":"2024-03-26T13:59:54.703337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758035+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:54.758035+0000","last_peered":"2024-03-26T13:59:54.758035+0000","last_clean":"2024-03-26T13:59:54.758035+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:54.758035+0000","last_undegraded":"2024-03-26T13:59:54.758035+0000","last_fullsized":"2024-03-26T13:59:54.758035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757574+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:54.757574+0000","last_peered":"2024-03-26T13:59:54.757574+0000","last_clean":"2024-03-26T13:59:54.757574+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:54.757574+0000","last_undegraded":"2024-03-26T13:59:54.757574+0000","last_fullsized":"2024-03-26T13:59:54.757574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.281085+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:55.281085+0000","last_peered":"2024-03-26T13:59:55.281085+0000","last_clean":"2024-03-26T13:59:55.281085+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:55.281085+0000","last_undegraded":"2024-03-26T13:59:55.281085+0000","last_fullsized":"2024-03-26T13:59:55.281085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330006+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:54.330006+0000","last_peered":"2024-03-26T13:59:54.330006+0000","last_clean":"2024-03-26T13:59:54.330006+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:54.330006+0000","last_undegraded":"2024-03-26T13:59:54.330006+0000","last_fullsized":"2024-03-26T13:59:54.330006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756304+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:54.756304+0000","last_peered":"2024-03-26T13:59:54.756304+0000","last_clean":"2024-03-26T13:59:54.756304+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:54.756304+0000","last_undegraded":"2024-03-26T13:59:54.756304+0000","last_fullsized":"2024-03-26T13:59:54.756304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326508+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:54.326508+0000","last_peered":"2024-03-26T13:59:54.326508+0000","last_clean":"2024-03-26T13:59:54.326508+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:54.326508+0000","last_undegraded":"2024-03-26T13:59:54.326508+0000","last_fullsized":"2024-03-26T13:59:54.326508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329667+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:54.329667+0000","last_peered":"2024-03-26T13:59:54.329667+0000","last_clean":"2024-03-26T13:59:54.329667+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:54.329667+0000","last_undegraded":"2024-03-26T13:59:54.329667+0000","last_fullsized":"2024-03-26T13:59:54.329667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.764260+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:54.764260+0000","last_peered":"2024-03-26T13:59:54.764260+0000","last_clean":"2024-03-26T13:59:54.764260+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:54.764260+0000","last_undegraded":"2024-03-26T13:59:54.764260+0000","last_fullsized":"2024-03-26T13:59:54.764260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756478+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:54.756478+0000","last_peered":"2024-03-26T13:59:54.756478+0000","last_clean":"2024-03-26T13:59:54.756478+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:54.756478+0000","last_undegraded":"2024-03-26T13:59:54.756478+0000","last_fullsized":"2024-03-26T13:59:54.756478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329794+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:54.329794+0000","last_peered":"2024-03-26T13:59:54.329794+0000","last_clean":"2024-03-26T13:59:54.329794+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:54.329794+0000","last_undegraded":"2024-03-26T13:59:54.329794+0000","last_fullsized":"2024-03-26T13:59:54.329794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331714+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:54.331714+0000","last_peered":"2024-03-26T13:59:54.331714+0000","last_clean":"2024-03-26T13:59:54.331714+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:54.331714+0000","last_undegraded":"2024-03-26T13:59:54.331714+0000","last_fullsized":"2024-03-26T13:59:54.331714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339975+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:54.339975+0000","last_peered":"2024-03-26T13:59:54.339975+0000","last_clean":"2024-03-26T13:59:54.339975+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:54.339975+0000","last_undegraded":"2024-03-26T13:59:54.339975+0000","last_fullsized":"2024-03-26T13:59:54.339975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757504+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:54.757504+0000","last_peered":"2024-03-26T13:59:54.757504+0000","last_clean":"2024-03-26T13:59:54.757504+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:54.757504+0000","last_undegraded":"2024-03-26T13:59:54.757504+0000","last_fullsized":"2024-03-26T13:59:54.757504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755154+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:54.755154+0000","last_peered":"2024-03-26T13:59:54.755154+0000","last_clean":"2024-03-26T13:59:54.755154+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:54.755154+0000","last_undegraded":"2024-03-26T13:59:54.755154+0000","last_fullsized":"2024-03-26T13:59:54.755154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331659+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:54.331659+0000","last_peered":"2024-03-26T13:59:54.331659+0000","last_clean":"2024-03-26T13:59:54.331659+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:54.331659+0000","last_undegraded":"2024-03-26T13:59:54.331659+0000","last_fullsized":"2024-03-26T13:59:54.331659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755545+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:54.755545+0000","last_peered":"2024-03-26T13:59:54.755545+0000","last_clean":"2024-03-26T13:59:54.755545+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:54.755545+0000","last_undegraded":"2024-03-26T13:59:54.755545+0000","last_fullsized":"2024-03-26T13:59:54.755545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316202+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:53.316202+0000","last_peered":"2024-03-26T13:59:53.316202+0000","last_clean":"2024-03-26T13:59:53.316202+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:53.316202+0000","last_undegraded":"2024-03-26T13:59:53.316202+0000","last_fullsized":"2024-03-26T13:59:53.316202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327246+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:54.327246+0000","last_peered":"2024-03-26T13:59:54.327246+0000","last_clean":"2024-03-26T13:59:54.327246+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:54.327246+0000","last_undegraded":"2024-03-26T13:59:54.327246+0000","last_fullsized":"2024-03-26T13:59:54.327246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330244+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:54.330244+0000","last_peered":"2024-03-26T13:59:54.330244+0000","last_clean":"2024-03-26T13:59:54.330244+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:54.330244+0000","last_undegraded":"2024-03-26T13:59:54.330244+0000","last_fullsized":"2024-03-26T13:59:54.330244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326884+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:54.326884+0000","last_peered":"2024-03-26T13:59:54.326884+0000","last_clean":"2024-03-26T13:59:54.326884+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:54.326884+0000","last_undegraded":"2024-03-26T13:59:54.326884+0000","last_fullsized":"2024-03-26T13:59:54.326884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753877+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:54.753877+0000","last_peered":"2024-03-26T13:59:54.753877+0000","last_clean":"2024-03-26T13:59:54.753877+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:54.753877+0000","last_undegraded":"2024-03-26T13:59:54.753877+0000","last_fullsized":"2024-03-26T13:59:54.753877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331172+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:54.331172+0000","last_peered":"2024-03-26T13:59:54.331172+0000","last_clean":"2024-03-26T13:59:54.331172+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:54.331172+0000","last_undegraded":"2024-03-26T13:59:54.331172+0000","last_fullsized":"2024-03-26T13:59:54.331172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.768353+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T13:59:54.768353+0000","last_peered":"2024-03-26T13:59:54.768353+0000","last_clean":"2024-03-26T13:59:54.768353+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T13:59:54.768353+0000","last_undegraded":"2024-03-26T13:59:54.768353+0000","last_fullsized":"2024-03-26T13:59:54.768353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755911+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:54.755911+0000","last_peered":"2024-03-26T13:59:54.755911+0000","last_clean":"2024-03-26T13:59:54.755911+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:54.755911+0000","last_undegraded":"2024-03-26T13:59:54.755911+0000","last_fullsized":"2024-03-26T13:59:54.755911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331265+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:54.331265+0000","last_peered":"2024-03-26T13:59:54.331265+0000","last_clean":"2024-03-26T13:59:54.331265+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:54.331265+0000","last_undegraded":"2024-03-26T13:59:54.331265+0000","last_fullsized":"2024-03-26T13:59:54.331265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.714081+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T13:59:54.714081+0000","last_peered":"2024-03-26T13:59:54.714081+0000","last_clean":"2024-03-26T13:59:54.714081+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T13:59:54.714081+0000","last_undegraded":"2024-03-26T13:59:54.714081+0000","last_fullsized":"2024-03-26T13:59:54.714081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317082+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:53.317082+0000","last_peered":"2024-03-26T13:59:53.317082+0000","last_clean":"2024-03-26T13:59:53.317082+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:53.317082+0000","last_undegraded":"2024-03-26T13:59:53.317082+0000","last_fullsized":"2024-03-26T13:59:53.317082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756671+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:54.756671+0000","last_peered":"2024-03-26T13:59:54.756671+0000","last_clean":"2024-03-26T13:59:54.756671+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:54.756671+0000","last_undegraded":"2024-03-26T13:59:54.756671+0000","last_fullsized":"2024-03-26T13:59:54.756671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326864+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:54.326864+0000","last_peered":"2024-03-26T13:59:54.326864+0000","last_clean":"2024-03-26T13:59:54.326864+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:54.326864+0000","last_undegraded":"2024-03-26T13:59:54.326864+0000","last_fullsized":"2024-03-26T13:59:54.326864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333249+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:54.333249+0000","last_peered":"2024-03-26T13:59:54.333249+0000","last_clean":"2024-03-26T13:59:54.333249+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:54.333249+0000","last_undegraded":"2024-03-26T13:59:54.333249+0000","last_fullsized":"2024-03-26T13:59:54.333249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329708+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:54.329708+0000","last_peered":"2024-03-26T13:59:54.329708+0000","last_clean":"2024-03-26T13:59:54.329708+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:54.329708+0000","last_undegraded":"2024-03-26T13:59:54.329708+0000","last_fullsized":"2024-03-26T13:59:54.329708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753959+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:54.753959+0000","last_peered":"2024-03-26T13:59:54.753959+0000","last_clean":"2024-03-26T13:59:54.753959+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:54.753959+0000","last_undegraded":"2024-03-26T13:59:54.753959+0000","last_fullsized":"2024-03-26T13:59:54.753959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319280+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:53.319280+0000","last_peered":"2024-03-26T13:59:53.319280+0000","last_clean":"2024-03-26T13:59:53.319280+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:53.319280+0000","last_undegraded":"2024-03-26T13:59:53.319280+0000","last_fullsized":"2024-03-26T13:59:53.319280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330394+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:54.330394+0000","last_peered":"2024-03-26T13:59:54.330394+0000","last_clean":"2024-03-26T13:59:54.330394+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:54.330394+0000","last_undegraded":"2024-03-26T13:59:54.330394+0000","last_fullsized":"2024-03-26T13:59:54.330394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761247+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:54.761247+0000","last_peered":"2024-03-26T13:59:54.761247+0000","last_clean":"2024-03-26T13:59:54.761247+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:54.761247+0000","last_undegraded":"2024-03-26T13:59:54.761247+0000","last_fullsized":"2024-03-26T13:59:54.761247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330119+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:54.330119+0000","last_peered":"2024-03-26T13:59:54.330119+0000","last_clean":"2024-03-26T13:59:54.330119+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:54.330119+0000","last_undegraded":"2024-03-26T13:59:54.330119+0000","last_fullsized":"2024-03-26T13:59:54.330119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756124+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:54.756124+0000","last_peered":"2024-03-26T13:59:54.756124+0000","last_clean":"2024-03-26T13:59:54.756124+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:54.756124+0000","last_undegraded":"2024-03-26T13:59:54.756124+0000","last_fullsized":"2024-03-26T13:59:54.756124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758106+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:54.758106+0000","last_peered":"2024-03-26T13:59:54.758106+0000","last_clean":"2024-03-26T13:59:54.758106+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:54.758106+0000","last_undegraded":"2024-03-26T13:59:54.758106+0000","last_fullsized":"2024-03-26T13:59:54.758106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330912+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:54.330912+0000","last_peered":"2024-03-26T13:59:54.330912+0000","last_clean":"2024-03-26T13:59:54.330912+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:54.330912+0000","last_undegraded":"2024-03-26T13:59:54.330912+0000","last_fullsized":"2024-03-26T13:59:54.330912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332068+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:54.332068+0000","last_peered":"2024-03-26T13:59:54.332068+0000","last_clean":"2024-03-26T13:59:54.332068+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:54.332068+0000","last_undegraded":"2024-03-26T13:59:54.332068+0000","last_fullsized":"2024-03-26T13:59:54.332068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316709+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:53.316709+0000","last_peered":"2024-03-26T13:59:53.316709+0000","last_clean":"2024-03-26T13:59:53.316709+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:53.316709+0000","last_undegraded":"2024-03-26T13:59:53.316709+0000","last_fullsized":"2024-03-26T13:59:53.316709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316722+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:53.316722+0000","last_peered":"2024-03-26T13:59:53.316722+0000","last_clean":"2024-03-26T13:59:53.316722+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:53.316722+0000","last_undegraded":"2024-03-26T13:59:53.316722+0000","last_fullsized":"2024-03-26T13:59:53.316722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332280+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:54.332280+0000","last_peered":"2024-03-26T13:59:54.332280+0000","last_clean":"2024-03-26T13:59:54.332280+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:54.332280+0000","last_undegraded":"2024-03-26T13:59:54.332280+0000","last_fullsized":"2024-03-26T13:59:54.332280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757622+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:54.757622+0000","last_peered":"2024-03-26T13:59:54.757622+0000","last_clean":"2024-03-26T13:59:54.757622+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:54.757622+0000","last_undegraded":"2024-03-26T13:59:54.757622+0000","last_fullsized":"2024-03-26T13:59:54.757622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333876+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:54.333876+0000","last_peered":"2024-03-26T13:59:54.333876+0000","last_clean":"2024-03-26T13:59:54.333876+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:54.333876+0000","last_undegraded":"2024-03-26T13:59:54.333876+0000","last_fullsized":"2024-03-26T13:59:54.333876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319745+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:53.319745+0000","last_peered":"2024-03-26T13:59:53.319745+0000","last_clean":"2024-03-26T13:59:53.319745+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:53.319745+0000","last_undegraded":"2024-03-26T13:59:53.319745+0000","last_fullsized":"2024-03-26T13:59:53.319745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755777+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:54.755777+0000","last_peered":"2024-03-26T13:59:54.755777+0000","last_clean":"2024-03-26T13:59:54.755777+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:54.755777+0000","last_undegraded":"2024-03-26T13:59:54.755777+0000","last_fullsized":"2024-03-26T13:59:54.755777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757431+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:54.757431+0000","last_peered":"2024-03-26T13:59:54.757431+0000","last_clean":"2024-03-26T13:59:54.757431+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:54.757431+0000","last_undegraded":"2024-03-26T13:59:54.757431+0000","last_fullsized":"2024-03-26T13:59:54.757431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756044+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:54.756044+0000","last_peered":"2024-03-26T13:59:54.756044+0000","last_clean":"2024-03-26T13:59:54.756044+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:54.756044+0000","last_undegraded":"2024-03-26T13:59:54.756044+0000","last_fullsized":"2024-03-26T13:59:54.756044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754447+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:54.754447+0000","last_peered":"2024-03-26T13:59:54.754447+0000","last_clean":"2024-03-26T13:59:54.754447+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:54.754447+0000","last_undegraded":"2024-03-26T13:59:54.754447+0000","last_fullsized":"2024-03-26T13:59:54.754447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756832+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:54.756832+0000","last_peered":"2024-03-26T13:59:54.756832+0000","last_clean":"2024-03-26T13:59:54.756832+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:54.756832+0000","last_undegraded":"2024-03-26T13:59:54.756832+0000","last_fullsized":"2024-03-26T13:59:54.756832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346428+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T13:59:54.346428+0000","last_peered":"2024-03-26T13:59:54.346428+0000","last_clean":"2024-03-26T13:59:54.346428+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T13:59:54.346428+0000","last_undegraded":"2024-03-26T13:59:54.346428+0000","last_fullsized":"2024-03-26T13:59:54.346428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60455,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":65,"num_read_kb":96,"num_write":150,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":187289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":503866,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1152075,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":504102,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":504022,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1099931,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":499278,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":555642,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":500666,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":538070,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":499472,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":504319,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":1095128,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T13:59:59.736 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T13:59:59.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 -- 172.21.15.137:0/2181100395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c0104ad0 msgr2=0x7f20c0104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 --2- 172.21.15.137:0/2181100395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c0104ad0 0x7f20c0104f50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009900 tx=0x7f20a802bc30 comp rx=0 tx=0).stop 2024-03-26T13:59:59.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 -- 172.21.15.137:0/2181100395 shutdown_connections 2024-03-26T13:59:59.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 --2- 172.21.15.137:0/2181100395 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c0105490 0x7f20c010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 --2- 172.21.15.137:0/2181100395 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c0104ad0 0x7f20c0104f50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 --2- 172.21.15.137:0/2181100395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c01038e0 0x7f20c0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.990+0000 7f20c6400640 1 -- 172.21.15.137:0/2181100395 >> 172.21.15.137:0/2181100395 conn(0x7f20c00fee50 msgr2=0x7f20c0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T13:59:59.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.991+0000 7f20c6400640 1 -- 172.21.15.137:0/2181100395 shutdown_connections 2024-03-26T13:59:59.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.991+0000 7f20c6400640 1 -- 172.21.15.137:0/2181100395 wait complete. 2024-03-26T13:59:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.991+0000 7f20c6400640 1 Processor -- start 2024-03-26T13:59:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 -- start start 2024-03-26T13:59:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 0x7f20c0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c0104ad0 0x7f20c019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c0105490 0x7f20c01a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c010f460 con 0x7f20c01038e0 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c010f2e0 con 0x7f20c0105490 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20c6400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c010f5e0 con 0x7f20c0104ad0 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 0x7f20c0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 0x7f20c0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54754/0 (socket says 172.21.15.137:54754) 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.992+0000 7f20bf400640 1 -- 172.21.15.137:0/413344636 learned_addr learned my addr 172.21.15.137:0/413344636 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bfe00640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c0105490 0x7f20c01a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20b6a00640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c0104ad0 0x7f20c019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bf400640 1 -- 172.21.15.137:0/413344636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c0104ad0 msgr2=0x7f20c019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bf400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c0104ad0 0x7f20c019a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bf400640 1 -- 172.21.15.137:0/413344636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c0105490 msgr2=0x7f20c01a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bf400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c0105490 0x7f20c01a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bf400640 1 -- 172.21.15.137:0/413344636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c01a1750 con 0x7f20c01038e0 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bfe00640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c0105490 0x7f20c01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T13:59:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bf400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 0x7f20c0199b30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f20b000e9c0 tx=0x7f20b000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000cd80 con 0x7f20c01038e0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c01a1a40 con 0x7f20c01038e0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20b00035f0 con 0x7f20c01038e0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0010690 con 0x7f20c01038e0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.993+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c01a1f90 con 0x7f20c01038e0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.995+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c010aca0 con 0x7f20c01038e0 2024-03-26T13:59:59.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.996+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f20b00037a0 con 0x7f20c01038e0 2024-03-26T13:59:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.998+0000 7f20bcc00640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f20a40a2890 0x7f20a40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T13:59:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.998+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f20b00cf430 con 0x7f20c01038e0 2024-03-26T13:59:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.998+0000 7f20b6a00640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f20a40a2890 0x7f20a40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T13:59:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T13:59:59.999+0000 7f20b6a00640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f20a40a2890 0x7f20a40a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20a80096c0 tx=0x7f20a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T13:59:59.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.002+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f20b009c010 con 0x7f20c01038e0 2024-03-26T14:00:00.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.199+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20c0061ae0 con 0x7f20a40a2890 2024-03-26T14:00:00.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.226+0000 7f20bcc00640 1 -- 172.21.15.137:0/413344636 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340631 (secure 0 0 0) 0x7f20c0061ae0 con 0x7f20a40a2890 2024-03-26T14:00:00.227 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:00.229 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f20a40a2890 msgr2=0x7f20a40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f20a40a2890 0x7f20a40a4d50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f20a80096c0 tx=0x7f20a8005fd0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 msgr2=0x7f20c0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 0x7f20c0199b30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f20b000e9c0 tx=0x7f20b000ee80 comp rx=0 tx=0).stop 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 shutdown_connections 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f20a40a2890 0x7f20a40a4d50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20c0105490 0x7f20c01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c0104ad0 0x7f20c019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 --2- 172.21.15.137:0/413344636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f20c01038e0 0x7f20c0199b30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 >> 172.21.15.137:0/413344636 conn(0x7f20c00fee50 msgr2=0x7f20c0107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:00.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 shutdown_connections 2024-03-26T14:00:00.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.233+0000 7f20c6400640 1 -- 172.21.15.137:0/413344636 wait complete. 2024-03-26T14:00:00.250 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-03-26T13:59:59.086451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651057,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":191,"num_read_kb":205,"num_write":333,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361428,"kb_used_data":12212,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580396,"statfs":{"total":1151940427776,"available":1151570325504,"internally_reserved":0,"allocated":12505088,"data_stored":7956571,"data_compressed":349077,"data_compressed_allocated":471040,"data_compressed_original":2441216,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":56049,"num_objects":36,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":65,"num_read_kb":96,"num_write":94,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003352"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334990+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:54.334990+0000","last_peered":"2024-03-26T13:59:54.334990+0000","last_clean":"2024-03-26T13:59:54.334990+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:54.334990+0000","last_undegraded":"2024-03-26T13:59:54.334990+0000","last_fullsized":"2024-03-26T13:59:54.334990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330109+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:54.330109+0000","last_peered":"2024-03-26T13:59:54.330109+0000","last_clean":"2024-03-26T13:59:54.330109+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:54.330109+0000","last_undegraded":"2024-03-26T13:59:54.330109+0000","last_fullsized":"2024-03-26T13:59:54.330109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327603+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:54.327603+0000","last_peered":"2024-03-26T13:59:54.327603+0000","last_clean":"2024-03-26T13:59:54.327603+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:54.327603+0000","last_undegraded":"2024-03-26T13:59:54.327603+0000","last_fullsized":"2024-03-26T13:59:54.327603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332907+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:54.332907+0000","last_peered":"2024-03-26T13:59:54.332907+0000","last_clean":"2024-03-26T13:59:54.332907+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:54.332907+0000","last_undegraded":"2024-03-26T13:59:54.332907+0000","last_fullsized":"2024-03-26T13:59:54.332907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703579+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:54.703579+0000","last_peered":"2024-03-26T13:59:54.703579+0000","last_clean":"2024-03-26T13:59:54.703579+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:54.703579+0000","last_undegraded":"2024-03-26T13:59:54.703579+0000","last_fullsized":"2024-03-26T13:59:54.703579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329612+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:54.329612+0000","last_peered":"2024-03-26T13:59:54.329612+0000","last_clean":"2024-03-26T13:59:54.329612+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:54.329612+0000","last_undegraded":"2024-03-26T13:59:54.329612+0000","last_fullsized":"2024-03-26T13:59:54.329612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755315+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:54.755315+0000","last_peered":"2024-03-26T13:59:54.755315+0000","last_clean":"2024-03-26T13:59:54.755315+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:54.755315+0000","last_undegraded":"2024-03-26T13:59:54.755315+0000","last_fullsized":"2024-03-26T13:59:54.755315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754310+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:54.754310+0000","last_peered":"2024-03-26T13:59:54.754310+0000","last_clean":"2024-03-26T13:59:54.754310+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:54.754310+0000","last_undegraded":"2024-03-26T13:59:54.754310+0000","last_fullsized":"2024-03-26T13:59:54.754310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704481+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:54.704481+0000","last_peered":"2024-03-26T13:59:54.704481+0000","last_clean":"2024-03-26T13:59:54.704481+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:54.704481+0000","last_undegraded":"2024-03-26T13:59:54.704481+0000","last_fullsized":"2024-03-26T13:59:54.704481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330340+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:54.330340+0000","last_peered":"2024-03-26T13:59:54.330340+0000","last_clean":"2024-03-26T13:59:54.330340+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:54.330340+0000","last_undegraded":"2024-03-26T13:59:54.330340+0000","last_fullsized":"2024-03-26T13:59:54.330340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327473+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:54.327473+0000","last_peered":"2024-03-26T13:59:54.327473+0000","last_clean":"2024-03-26T13:59:54.327473+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:54.327473+0000","last_undegraded":"2024-03-26T13:59:54.327473+0000","last_fullsized":"2024-03-26T13:59:54.327473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757043+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:54.757043+0000","last_peered":"2024-03-26T13:59:54.757043+0000","last_clean":"2024-03-26T13:59:54.757043+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:54.757043+0000","last_undegraded":"2024-03-26T13:59:54.757043+0000","last_fullsized":"2024-03-26T13:59:54.757043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331805+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:54.331805+0000","last_peered":"2024-03-26T13:59:54.331805+0000","last_clean":"2024-03-26T13:59:54.331805+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:54.331805+0000","last_undegraded":"2024-03-26T13:59:54.331805+0000","last_fullsized":"2024-03-26T13:59:54.331805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.344831+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T13:59:54.344831+0000","last_peered":"2024-03-26T13:59:54.344831+0000","last_clean":"2024-03-26T13:59:54.344831+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T13:59:54.344831+0000","last_undegraded":"2024-03-26T13:59:54.344831+0000","last_fullsized":"2024-03-26T13:59:54.344831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753840+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:54.753840+0000","last_peered":"2024-03-26T13:59:54.753840+0000","last_clean":"2024-03-26T13:59:54.753840+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:54.753840+0000","last_undegraded":"2024-03-26T13:59:54.753840+0000","last_fullsized":"2024-03-26T13:59:54.753840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334663+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:54.334663+0000","last_peered":"2024-03-26T13:59:54.334663+0000","last_clean":"2024-03-26T13:59:54.334663+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:54.334663+0000","last_undegraded":"2024-03-26T13:59:54.334663+0000","last_fullsized":"2024-03-26T13:59:54.334663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330566+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:54.330566+0000","last_peered":"2024-03-26T13:59:54.330566+0000","last_clean":"2024-03-26T13:59:54.330566+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:54.330566+0000","last_undegraded":"2024-03-26T13:59:54.330566+0000","last_fullsized":"2024-03-26T13:59:54.330566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319222+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:53.319222+0000","last_peered":"2024-03-26T13:59:53.319222+0000","last_clean":"2024-03-26T13:59:53.319222+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:53.319222+0000","last_undegraded":"2024-03-26T13:59:53.319222+0000","last_fullsized":"2024-03-26T13:59:53.319222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317072+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:53.317072+0000","last_peered":"2024-03-26T13:59:53.317072+0000","last_clean":"2024-03-26T13:59:53.317072+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:53.317072+0000","last_undegraded":"2024-03-26T13:59:53.317072+0000","last_fullsized":"2024-03-26T13:59:53.317072+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330916+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:54.330916+0000","last_peered":"2024-03-26T13:59:54.330916+0000","last_clean":"2024-03-26T13:59:54.330916+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:54.330916+0000","last_undegraded":"2024-03-26T13:59:54.330916+0000","last_fullsized":"2024-03-26T13:59:54.330916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757563+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:54.757563+0000","last_peered":"2024-03-26T13:59:54.757563+0000","last_clean":"2024-03-26T13:59:54.757563+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:54.757563+0000","last_undegraded":"2024-03-26T13:59:54.757563+0000","last_fullsized":"2024-03-26T13:59:54.757563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.325949+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:54.325949+0000","last_peered":"2024-03-26T13:59:54.325949+0000","last_clean":"2024-03-26T13:59:54.325949+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:54.325949+0000","last_undegraded":"2024-03-26T13:59:54.325949+0000","last_fullsized":"2024-03-26T13:59:54.325949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757809+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:54.757809+0000","last_peered":"2024-03-26T13:59:54.757809+0000","last_clean":"2024-03-26T13:59:54.757809+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:54.757809+0000","last_undegraded":"2024-03-26T13:59:54.757809+0000","last_fullsized":"2024-03-26T13:59:54.757809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330337+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:54.330337+0000","last_peered":"2024-03-26T13:59:54.330337+0000","last_clean":"2024-03-26T13:59:54.330337+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:54.330337+0000","last_undegraded":"2024-03-26T13:59:54.330337+0000","last_fullsized":"2024-03-26T13:59:54.330337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326515+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:54.326515+0000","last_peered":"2024-03-26T13:59:54.326515+0000","last_clean":"2024-03-26T13:59:54.326515+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:54.326515+0000","last_undegraded":"2024-03-26T13:59:54.326515+0000","last_fullsized":"2024-03-26T13:59:54.326515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339658+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:54.339658+0000","last_peered":"2024-03-26T13:59:54.339658+0000","last_clean":"2024-03-26T13:59:54.339658+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:54.339658+0000","last_undegraded":"2024-03-26T13:59:54.339658+0000","last_fullsized":"2024-03-26T13:59:54.339658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754197+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:54.754197+0000","last_peered":"2024-03-26T13:59:54.754197+0000","last_clean":"2024-03-26T13:59:54.754197+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:54.754197+0000","last_undegraded":"2024-03-26T13:59:54.754197+0000","last_fullsized":"2024-03-26T13:59:54.754197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326200+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:54.326200+0000","last_peered":"2024-03-26T13:59:54.326200+0000","last_clean":"2024-03-26T13:59:54.326200+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:54.326200+0000","last_undegraded":"2024-03-26T13:59:54.326200+0000","last_fullsized":"2024-03-26T13:59:54.326200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.342115+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:54.342115+0000","last_peered":"2024-03-26T13:59:54.342115+0000","last_clean":"2024-03-26T13:59:54.342115+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:54.342115+0000","last_undegraded":"2024-03-26T13:59:54.342115+0000","last_fullsized":"2024-03-26T13:59:54.342115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761061+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:54.761061+0000","last_peered":"2024-03-26T13:59:54.761061+0000","last_clean":"2024-03-26T13:59:54.761061+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:54.761061+0000","last_undegraded":"2024-03-26T13:59:54.761061+0000","last_fullsized":"2024-03-26T13:59:54.761061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.236338+0000","last_change":"2024-03-26T13:59:53.317881+0000","last_active":"2024-03-26T13:59:53.317881+0000","last_peered":"2024-03-26T13:59:30.236338+0000","last_clean":"2024-03-26T13:59:30.236338+0000","last_became_active":"2024-03-26T13:59:27.323585+0000","last_became_peered":"2024-03-26T13:59:27.323585+0000","last_unstale":"2024-03-26T13:59:30.236338+0000","last_undegraded":"2024-03-26T13:59:30.236338+0000","last_fullsized":"2024-03-26T13:59:30.236338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7,8,11],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.279834+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:55.279834+0000","last_peered":"2024-03-26T13:59:55.279834+0000","last_clean":"2024-03-26T13:59:55.279834+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:55.279834+0000","last_undegraded":"2024-03-26T13:59:55.279834+0000","last_fullsized":"2024-03-26T13:59:55.279834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346323+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T13:59:54.346323+0000","last_peered":"2024-03-26T13:59:54.346323+0000","last_clean":"2024-03-26T13:59:54.346323+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T13:59:54.346323+0000","last_undegraded":"2024-03-26T13:59:54.346323+0000","last_fullsized":"2024-03-26T13:59:54.346323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"peering","last_fresh":"2024-03-26T13:59:30.234348+0000","last_change":"2024-03-26T13:59:53.316731+0000","last_active":"2024-03-26T13:59:53.316731+0000","last_peered":"2024-03-26T13:59:30.234348+0000","last_clean":"2024-03-26T13:59:30.234348+0000","last_became_active":"2024-03-26T13:59:27.324762+0000","last_became_peered":"2024-03-26T13:59:27.324762+0000","last_unstale":"2024-03-26T13:59:30.234348+0000","last_undegraded":"2024-03-26T13:59:30.234348+0000","last_fullsized":"2024-03-26T13:59:30.234348+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703422+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:54.703422+0000","last_peered":"2024-03-26T13:59:54.703422+0000","last_clean":"2024-03-26T13:59:54.703422+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:54.703422+0000","last_undegraded":"2024-03-26T13:59:54.703422+0000","last_fullsized":"2024-03-26T13:59:54.703422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758242+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:54.758242+0000","last_peered":"2024-03-26T13:59:54.758242+0000","last_clean":"2024-03-26T13:59:54.758242+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:54.758242+0000","last_undegraded":"2024-03-26T13:59:54.758242+0000","last_fullsized":"2024-03-26T13:59:54.758242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334258+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:54.334258+0000","last_peered":"2024-03-26T13:59:54.334258+0000","last_clean":"2024-03-26T13:59:54.334258+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:54.334258+0000","last_undegraded":"2024-03-26T13:59:54.334258+0000","last_fullsized":"2024-03-26T13:59:54.334258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755967+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:54.755967+0000","last_peered":"2024-03-26T13:59:54.755967+0000","last_clean":"2024-03-26T13:59:54.755967+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:54.755967+0000","last_undegraded":"2024-03-26T13:59:54.755967+0000","last_fullsized":"2024-03-26T13:59:54.755967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.759517+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T13:59:54.759517+0000","last_peered":"2024-03-26T13:59:54.759517+0000","last_clean":"2024-03-26T13:59:54.759517+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T13:59:54.759517+0000","last_undegraded":"2024-03-26T13:59:54.759517+0000","last_fullsized":"2024-03-26T13:59:54.759517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329961+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:54.329961+0000","last_peered":"2024-03-26T13:59:54.329961+0000","last_clean":"2024-03-26T13:59:54.329961+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:54.329961+0000","last_undegraded":"2024-03-26T13:59:54.329961+0000","last_fullsized":"2024-03-26T13:59:54.329961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757020+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:54.757020+0000","last_peered":"2024-03-26T13:59:54.757020+0000","last_clean":"2024-03-26T13:59:54.757020+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:54.757020+0000","last_undegraded":"2024-03-26T13:59:54.757020+0000","last_fullsized":"2024-03-26T13:59:54.757020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327822+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:54.327822+0000","last_peered":"2024-03-26T13:59:54.327822+0000","last_clean":"2024-03-26T13:59:54.327822+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:54.327822+0000","last_undegraded":"2024-03-26T13:59:54.327822+0000","last_fullsized":"2024-03-26T13:59:54.327822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755747+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:54.755747+0000","last_peered":"2024-03-26T13:59:54.755747+0000","last_clean":"2024-03-26T13:59:54.755747+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:54.755747+0000","last_undegraded":"2024-03-26T13:59:54.755747+0000","last_fullsized":"2024-03-26T13:59:54.755747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704508+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:54.704508+0000","last_peered":"2024-03-26T13:59:54.704508+0000","last_clean":"2024-03-26T13:59:54.704508+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:54.704508+0000","last_undegraded":"2024-03-26T13:59:54.704508+0000","last_fullsized":"2024-03-26T13:59:54.704508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754499+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:54.754499+0000","last_peered":"2024-03-26T13:59:54.754499+0000","last_clean":"2024-03-26T13:59:54.754499+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:54.754499+0000","last_undegraded":"2024-03-26T13:59:54.754499+0000","last_fullsized":"2024-03-26T13:59:54.754499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327602+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:54.327602+0000","last_peered":"2024-03-26T13:59:54.327602+0000","last_clean":"2024-03-26T13:59:54.327602+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:54.327602+0000","last_undegraded":"2024-03-26T13:59:54.327602+0000","last_fullsized":"2024-03-26T13:59:54.327602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330446+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:54.330446+0000","last_peered":"2024-03-26T13:59:54.330446+0000","last_clean":"2024-03-26T13:59:54.330446+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:54.330446+0000","last_undegraded":"2024-03-26T13:59:54.330446+0000","last_fullsized":"2024-03-26T13:59:54.330446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761347+0000","last_change":"2024-03-26T13:59:54.761347+0000","last_active":"2024-03-26T13:59:54.761347+0000","last_peered":"2024-03-26T13:59:54.761347+0000","last_clean":"2024-03-26T13:59:54.761347+0000","last_became_active":"2024-03-26T13:59:54.760966+0000","last_became_peered":"2024-03-26T13:59:54.760966+0000","last_unstale":"2024-03-26T13:59:54.761347+0000","last_undegraded":"2024-03-26T13:59:54.761347+0000","last_fullsized":"2024-03-26T13:59:54.761347+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:27:13.241398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326627+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:54.326627+0000","last_peered":"2024-03-26T13:59:54.326627+0000","last_clean":"2024-03-26T13:59:54.326627+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:54.326627+0000","last_undegraded":"2024-03-26T13:59:54.326627+0000","last_fullsized":"2024-03-26T13:59:54.326627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.760687+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:54.760687+0000","last_peered":"2024-03-26T13:59:54.760687+0000","last_clean":"2024-03-26T13:59:54.760687+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:54.760687+0000","last_undegraded":"2024-03-26T13:59:54.760687+0000","last_fullsized":"2024-03-26T13:59:54.760687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754283+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:54.754283+0000","last_peered":"2024-03-26T13:59:54.754283+0000","last_clean":"2024-03-26T13:59:54.754283+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:54.754283+0000","last_undegraded":"2024-03-26T13:59:54.754283+0000","last_fullsized":"2024-03-26T13:59:54.754283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.328197+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:54.328197+0000","last_peered":"2024-03-26T13:59:54.328197+0000","last_clean":"2024-03-26T13:59:54.328197+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:54.328197+0000","last_undegraded":"2024-03-26T13:59:54.328197+0000","last_fullsized":"2024-03-26T13:59:54.328197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'26","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.283705+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:55.283705+0000","last_peered":"2024-03-26T13:59:55.283705+0000","last_clean":"2024-03-26T13:59:55.283705+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:55.283705+0000","last_undegraded":"2024-03-26T13:59:55.283705+0000","last_fullsized":"2024-03-26T13:59:55.283705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56454,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":64,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.340153+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:54.340153+0000","last_peered":"2024-03-26T13:59:54.340153+0000","last_clean":"2024-03-26T13:59:54.340153+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:54.340153+0000","last_undegraded":"2024-03-26T13:59:54.340153+0000","last_fullsized":"2024-03-26T13:59:54.340153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761598+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:54.761598+0000","last_peered":"2024-03-26T13:59:54.761598+0000","last_clean":"2024-03-26T13:59:54.761598+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:54.761598+0000","last_undegraded":"2024-03-26T13:59:54.761598+0000","last_fullsized":"2024-03-26T13:59:54.761598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.318377+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:53.318377+0000","last_peered":"2024-03-26T13:59:53.318377+0000","last_clean":"2024-03-26T13:59:53.318377+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:53.318377+0000","last_undegraded":"2024-03-26T13:59:53.318377+0000","last_fullsized":"2024-03-26T13:59:53.318377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756449+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:54.756449+0000","last_peered":"2024-03-26T13:59:54.756449+0000","last_clean":"2024-03-26T13:59:54.756449+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:54.756449+0000","last_undegraded":"2024-03-26T13:59:54.756449+0000","last_fullsized":"2024-03-26T13:59:54.756449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332076+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:54.332076+0000","last_peered":"2024-03-26T13:59:54.332076+0000","last_clean":"2024-03-26T13:59:54.332076+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:54.332076+0000","last_undegraded":"2024-03-26T13:59:54.332076+0000","last_fullsized":"2024-03-26T13:59:54.332076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757435+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:54.757435+0000","last_peered":"2024-03-26T13:59:54.757435+0000","last_clean":"2024-03-26T13:59:54.757435+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:54.757435+0000","last_undegraded":"2024-03-26T13:59:54.757435+0000","last_fullsized":"2024-03-26T13:59:54.757435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330136+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:54.330136+0000","last_peered":"2024-03-26T13:59:54.330136+0000","last_clean":"2024-03-26T13:59:54.330136+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:54.330136+0000","last_undegraded":"2024-03-26T13:59:54.330136+0000","last_fullsized":"2024-03-26T13:59:54.330136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330531+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:54.330531+0000","last_peered":"2024-03-26T13:59:54.330531+0000","last_clean":"2024-03-26T13:59:54.330531+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:54.330531+0000","last_undegraded":"2024-03-26T13:59:54.330531+0000","last_fullsized":"2024-03-26T13:59:54.330531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:57.287720+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:57.287720+0000","last_peered":"2024-03-26T13:59:57.287720+0000","last_clean":"2024-03-26T13:59:57.287720+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:57.287720+0000","last_undegraded":"2024-03-26T13:59:57.287720+0000","last_fullsized":"2024-03-26T13:59:57.287720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329932+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:54.329932+0000","last_peered":"2024-03-26T13:59:54.329932+0000","last_clean":"2024-03-26T13:59:54.329932+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:54.329932+0000","last_undegraded":"2024-03-26T13:59:54.329932+0000","last_fullsized":"2024-03-26T13:59:54.329932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758226+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:54.758226+0000","last_peered":"2024-03-26T13:59:54.758226+0000","last_clean":"2024-03-26T13:59:54.758226+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:54.758226+0000","last_undegraded":"2024-03-26T13:59:54.758226+0000","last_fullsized":"2024-03-26T13:59:54.758226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330561+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:54.330561+0000","last_peered":"2024-03-26T13:59:54.330561+0000","last_clean":"2024-03-26T13:59:54.330561+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:54.330561+0000","last_undegraded":"2024-03-26T13:59:54.330561+0000","last_fullsized":"2024-03-26T13:59:54.330561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756514+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:54.756514+0000","last_peered":"2024-03-26T13:59:54.756514+0000","last_clean":"2024-03-26T13:59:54.756514+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:54.756514+0000","last_undegraded":"2024-03-26T13:59:54.756514+0000","last_fullsized":"2024-03-26T13:59:54.756514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756211+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:54.756211+0000","last_peered":"2024-03-26T13:59:54.756211+0000","last_clean":"2024-03-26T13:59:54.756211+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:54.756211+0000","last_undegraded":"2024-03-26T13:59:54.756211+0000","last_fullsized":"2024-03-26T13:59:54.756211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.341535+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:54.341535+0000","last_peered":"2024-03-26T13:59:54.341535+0000","last_clean":"2024-03-26T13:59:54.341535+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:54.341535+0000","last_undegraded":"2024-03-26T13:59:54.341535+0000","last_fullsized":"2024-03-26T13:59:54.341535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761409+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T13:59:54.761409+0000","last_peered":"2024-03-26T13:59:54.761409+0000","last_clean":"2024-03-26T13:59:54.761409+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T13:59:54.761409+0000","last_undegraded":"2024-03-26T13:59:54.761409+0000","last_fullsized":"2024-03-26T13:59:54.761409+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334043+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:54.334043+0000","last_peered":"2024-03-26T13:59:54.334043+0000","last_clean":"2024-03-26T13:59:54.334043+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:54.334043+0000","last_undegraded":"2024-03-26T13:59:54.334043+0000","last_fullsized":"2024-03-26T13:59:54.334043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329309+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:54.329309+0000","last_peered":"2024-03-26T13:59:54.329309+0000","last_clean":"2024-03-26T13:59:54.329309+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:54.329309+0000","last_undegraded":"2024-03-26T13:59:54.329309+0000","last_fullsized":"2024-03-26T13:59:54.329309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703899+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:54.703899+0000","last_peered":"2024-03-26T13:59:54.703899+0000","last_clean":"2024-03-26T13:59:54.703899+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:54.703899+0000","last_undegraded":"2024-03-26T13:59:54.703899+0000","last_fullsized":"2024-03-26T13:59:54.703899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757312+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:54.757312+0000","last_peered":"2024-03-26T13:59:54.757312+0000","last_clean":"2024-03-26T13:59:54.757312+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:54.757312+0000","last_undegraded":"2024-03-26T13:59:54.757312+0000","last_fullsized":"2024-03-26T13:59:54.757312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329838+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:54.329838+0000","last_peered":"2024-03-26T13:59:54.329838+0000","last_clean":"2024-03-26T13:59:54.329838+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:54.329838+0000","last_undegraded":"2024-03-26T13:59:54.329838+0000","last_fullsized":"2024-03-26T13:59:54.329838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703337+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:54.703337+0000","last_peered":"2024-03-26T13:59:54.703337+0000","last_clean":"2024-03-26T13:59:54.703337+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:54.703337+0000","last_undegraded":"2024-03-26T13:59:54.703337+0000","last_fullsized":"2024-03-26T13:59:54.703337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758035+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:54.758035+0000","last_peered":"2024-03-26T13:59:54.758035+0000","last_clean":"2024-03-26T13:59:54.758035+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:54.758035+0000","last_undegraded":"2024-03-26T13:59:54.758035+0000","last_fullsized":"2024-03-26T13:59:54.758035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757574+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:54.757574+0000","last_peered":"2024-03-26T13:59:54.757574+0000","last_clean":"2024-03-26T13:59:54.757574+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:54.757574+0000","last_undegraded":"2024-03-26T13:59:54.757574+0000","last_fullsized":"2024-03-26T13:59:54.757574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.281085+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:55.281085+0000","last_peered":"2024-03-26T13:59:55.281085+0000","last_clean":"2024-03-26T13:59:55.281085+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:55.281085+0000","last_undegraded":"2024-03-26T13:59:55.281085+0000","last_fullsized":"2024-03-26T13:59:55.281085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330006+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:54.330006+0000","last_peered":"2024-03-26T13:59:54.330006+0000","last_clean":"2024-03-26T13:59:54.330006+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:54.330006+0000","last_undegraded":"2024-03-26T13:59:54.330006+0000","last_fullsized":"2024-03-26T13:59:54.330006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756304+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:54.756304+0000","last_peered":"2024-03-26T13:59:54.756304+0000","last_clean":"2024-03-26T13:59:54.756304+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:54.756304+0000","last_undegraded":"2024-03-26T13:59:54.756304+0000","last_fullsized":"2024-03-26T13:59:54.756304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326508+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:54.326508+0000","last_peered":"2024-03-26T13:59:54.326508+0000","last_clean":"2024-03-26T13:59:54.326508+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:54.326508+0000","last_undegraded":"2024-03-26T13:59:54.326508+0000","last_fullsized":"2024-03-26T13:59:54.326508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329667+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:54.329667+0000","last_peered":"2024-03-26T13:59:54.329667+0000","last_clean":"2024-03-26T13:59:54.329667+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:54.329667+0000","last_undegraded":"2024-03-26T13:59:54.329667+0000","last_fullsized":"2024-03-26T13:59:54.329667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.764260+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:54.764260+0000","last_peered":"2024-03-26T13:59:54.764260+0000","last_clean":"2024-03-26T13:59:54.764260+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:54.764260+0000","last_undegraded":"2024-03-26T13:59:54.764260+0000","last_fullsized":"2024-03-26T13:59:54.764260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756478+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:54.756478+0000","last_peered":"2024-03-26T13:59:54.756478+0000","last_clean":"2024-03-26T13:59:54.756478+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:54.756478+0000","last_undegraded":"2024-03-26T13:59:54.756478+0000","last_fullsized":"2024-03-26T13:59:54.756478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329794+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:54.329794+0000","last_peered":"2024-03-26T13:59:54.329794+0000","last_clean":"2024-03-26T13:59:54.329794+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:54.329794+0000","last_undegraded":"2024-03-26T13:59:54.329794+0000","last_fullsized":"2024-03-26T13:59:54.329794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331714+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:54.331714+0000","last_peered":"2024-03-26T13:59:54.331714+0000","last_clean":"2024-03-26T13:59:54.331714+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:54.331714+0000","last_undegraded":"2024-03-26T13:59:54.331714+0000","last_fullsized":"2024-03-26T13:59:54.331714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339975+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:54.339975+0000","last_peered":"2024-03-26T13:59:54.339975+0000","last_clean":"2024-03-26T13:59:54.339975+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:54.339975+0000","last_undegraded":"2024-03-26T13:59:54.339975+0000","last_fullsized":"2024-03-26T13:59:54.339975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757504+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:54.757504+0000","last_peered":"2024-03-26T13:59:54.757504+0000","last_clean":"2024-03-26T13:59:54.757504+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:54.757504+0000","last_undegraded":"2024-03-26T13:59:54.757504+0000","last_fullsized":"2024-03-26T13:59:54.757504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755154+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:54.755154+0000","last_peered":"2024-03-26T13:59:54.755154+0000","last_clean":"2024-03-26T13:59:54.755154+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:54.755154+0000","last_undegraded":"2024-03-26T13:59:54.755154+0000","last_fullsized":"2024-03-26T13:59:54.755154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331659+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:54.331659+0000","last_peered":"2024-03-26T13:59:54.331659+0000","last_clean":"2024-03-26T13:59:54.331659+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:54.331659+0000","last_undegraded":"2024-03-26T13:59:54.331659+0000","last_fullsized":"2024-03-26T13:59:54.331659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755545+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:54.755545+0000","last_peered":"2024-03-26T13:59:54.755545+0000","last_clean":"2024-03-26T13:59:54.755545+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:54.755545+0000","last_undegraded":"2024-03-26T13:59:54.755545+0000","last_fullsized":"2024-03-26T13:59:54.755545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316202+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:53.316202+0000","last_peered":"2024-03-26T13:59:53.316202+0000","last_clean":"2024-03-26T13:59:53.316202+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:53.316202+0000","last_undegraded":"2024-03-26T13:59:53.316202+0000","last_fullsized":"2024-03-26T13:59:53.316202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327246+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:54.327246+0000","last_peered":"2024-03-26T13:59:54.327246+0000","last_clean":"2024-03-26T13:59:54.327246+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:54.327246+0000","last_undegraded":"2024-03-26T13:59:54.327246+0000","last_fullsized":"2024-03-26T13:59:54.327246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330244+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:54.330244+0000","last_peered":"2024-03-26T13:59:54.330244+0000","last_clean":"2024-03-26T13:59:54.330244+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:54.330244+0000","last_undegraded":"2024-03-26T13:59:54.330244+0000","last_fullsized":"2024-03-26T13:59:54.330244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326884+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:54.326884+0000","last_peered":"2024-03-26T13:59:54.326884+0000","last_clean":"2024-03-26T13:59:54.326884+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:54.326884+0000","last_undegraded":"2024-03-26T13:59:54.326884+0000","last_fullsized":"2024-03-26T13:59:54.326884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753877+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:54.753877+0000","last_peered":"2024-03-26T13:59:54.753877+0000","last_clean":"2024-03-26T13:59:54.753877+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:54.753877+0000","last_undegraded":"2024-03-26T13:59:54.753877+0000","last_fullsized":"2024-03-26T13:59:54.753877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331172+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:54.331172+0000","last_peered":"2024-03-26T13:59:54.331172+0000","last_clean":"2024-03-26T13:59:54.331172+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:54.331172+0000","last_undegraded":"2024-03-26T13:59:54.331172+0000","last_fullsized":"2024-03-26T13:59:54.331172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.768353+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T13:59:54.768353+0000","last_peered":"2024-03-26T13:59:54.768353+0000","last_clean":"2024-03-26T13:59:54.768353+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T13:59:54.768353+0000","last_undegraded":"2024-03-26T13:59:54.768353+0000","last_fullsized":"2024-03-26T13:59:54.768353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755911+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:54.755911+0000","last_peered":"2024-03-26T13:59:54.755911+0000","last_clean":"2024-03-26T13:59:54.755911+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:54.755911+0000","last_undegraded":"2024-03-26T13:59:54.755911+0000","last_fullsized":"2024-03-26T13:59:54.755911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331265+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:54.331265+0000","last_peered":"2024-03-26T13:59:54.331265+0000","last_clean":"2024-03-26T13:59:54.331265+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:54.331265+0000","last_undegraded":"2024-03-26T13:59:54.331265+0000","last_fullsized":"2024-03-26T13:59:54.331265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.714081+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T13:59:54.714081+0000","last_peered":"2024-03-26T13:59:54.714081+0000","last_clean":"2024-03-26T13:59:54.714081+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T13:59:54.714081+0000","last_undegraded":"2024-03-26T13:59:54.714081+0000","last_fullsized":"2024-03-26T13:59:54.714081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.317082+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:53.317082+0000","last_peered":"2024-03-26T13:59:53.317082+0000","last_clean":"2024-03-26T13:59:53.317082+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:53.317082+0000","last_undegraded":"2024-03-26T13:59:53.317082+0000","last_fullsized":"2024-03-26T13:59:53.317082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756671+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:54.756671+0000","last_peered":"2024-03-26T13:59:54.756671+0000","last_clean":"2024-03-26T13:59:54.756671+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:54.756671+0000","last_undegraded":"2024-03-26T13:59:54.756671+0000","last_fullsized":"2024-03-26T13:59:54.756671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326864+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:54.326864+0000","last_peered":"2024-03-26T13:59:54.326864+0000","last_clean":"2024-03-26T13:59:54.326864+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:54.326864+0000","last_undegraded":"2024-03-26T13:59:54.326864+0000","last_fullsized":"2024-03-26T13:59:54.326864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333249+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:54.333249+0000","last_peered":"2024-03-26T13:59:54.333249+0000","last_clean":"2024-03-26T13:59:54.333249+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:54.333249+0000","last_undegraded":"2024-03-26T13:59:54.333249+0000","last_fullsized":"2024-03-26T13:59:54.333249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329708+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:54.329708+0000","last_peered":"2024-03-26T13:59:54.329708+0000","last_clean":"2024-03-26T13:59:54.329708+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:54.329708+0000","last_undegraded":"2024-03-26T13:59:54.329708+0000","last_fullsized":"2024-03-26T13:59:54.329708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753959+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:54.753959+0000","last_peered":"2024-03-26T13:59:54.753959+0000","last_clean":"2024-03-26T13:59:54.753959+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:54.753959+0000","last_undegraded":"2024-03-26T13:59:54.753959+0000","last_fullsized":"2024-03-26T13:59:54.753959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319280+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:53.319280+0000","last_peered":"2024-03-26T13:59:53.319280+0000","last_clean":"2024-03-26T13:59:53.319280+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:53.319280+0000","last_undegraded":"2024-03-26T13:59:53.319280+0000","last_fullsized":"2024-03-26T13:59:53.319280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330394+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:54.330394+0000","last_peered":"2024-03-26T13:59:54.330394+0000","last_clean":"2024-03-26T13:59:54.330394+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:54.330394+0000","last_undegraded":"2024-03-26T13:59:54.330394+0000","last_fullsized":"2024-03-26T13:59:54.330394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761247+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:54.761247+0000","last_peered":"2024-03-26T13:59:54.761247+0000","last_clean":"2024-03-26T13:59:54.761247+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:54.761247+0000","last_undegraded":"2024-03-26T13:59:54.761247+0000","last_fullsized":"2024-03-26T13:59:54.761247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330119+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:54.330119+0000","last_peered":"2024-03-26T13:59:54.330119+0000","last_clean":"2024-03-26T13:59:54.330119+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:54.330119+0000","last_undegraded":"2024-03-26T13:59:54.330119+0000","last_fullsized":"2024-03-26T13:59:54.330119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756124+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:54.756124+0000","last_peered":"2024-03-26T13:59:54.756124+0000","last_clean":"2024-03-26T13:59:54.756124+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:54.756124+0000","last_undegraded":"2024-03-26T13:59:54.756124+0000","last_fullsized":"2024-03-26T13:59:54.756124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758106+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:54.758106+0000","last_peered":"2024-03-26T13:59:54.758106+0000","last_clean":"2024-03-26T13:59:54.758106+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:54.758106+0000","last_undegraded":"2024-03-26T13:59:54.758106+0000","last_fullsized":"2024-03-26T13:59:54.758106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330912+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:54.330912+0000","last_peered":"2024-03-26T13:59:54.330912+0000","last_clean":"2024-03-26T13:59:54.330912+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:54.330912+0000","last_undegraded":"2024-03-26T13:59:54.330912+0000","last_fullsized":"2024-03-26T13:59:54.330912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332068+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:54.332068+0000","last_peered":"2024-03-26T13:59:54.332068+0000","last_clean":"2024-03-26T13:59:54.332068+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:54.332068+0000","last_undegraded":"2024-03-26T13:59:54.332068+0000","last_fullsized":"2024-03-26T13:59:54.332068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316709+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:53.316709+0000","last_peered":"2024-03-26T13:59:53.316709+0000","last_clean":"2024-03-26T13:59:53.316709+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:53.316709+0000","last_undegraded":"2024-03-26T13:59:53.316709+0000","last_fullsized":"2024-03-26T13:59:53.316709+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.316722+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:53.316722+0000","last_peered":"2024-03-26T13:59:53.316722+0000","last_clean":"2024-03-26T13:59:53.316722+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:53.316722+0000","last_undegraded":"2024-03-26T13:59:53.316722+0000","last_fullsized":"2024-03-26T13:59:53.316722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332280+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:54.332280+0000","last_peered":"2024-03-26T13:59:54.332280+0000","last_clean":"2024-03-26T13:59:54.332280+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:54.332280+0000","last_undegraded":"2024-03-26T13:59:54.332280+0000","last_fullsized":"2024-03-26T13:59:54.332280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757622+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:54.757622+0000","last_peered":"2024-03-26T13:59:54.757622+0000","last_clean":"2024-03-26T13:59:54.757622+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:54.757622+0000","last_undegraded":"2024-03-26T13:59:54.757622+0000","last_fullsized":"2024-03-26T13:59:54.757622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333876+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:54.333876+0000","last_peered":"2024-03-26T13:59:54.333876+0000","last_clean":"2024-03-26T13:59:54.333876+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:54.333876+0000","last_undegraded":"2024-03-26T13:59:54.333876+0000","last_fullsized":"2024-03-26T13:59:54.333876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-26T13:59:53.319745+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:53.319745+0000","last_peered":"2024-03-26T13:59:53.319745+0000","last_clean":"2024-03-26T13:59:53.319745+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:53.319745+0000","last_undegraded":"2024-03-26T13:59:53.319745+0000","last_fullsized":"2024-03-26T13:59:53.319745+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755777+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:54.755777+0000","last_peered":"2024-03-26T13:59:54.755777+0000","last_clean":"2024-03-26T13:59:54.755777+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:54.755777+0000","last_undegraded":"2024-03-26T13:59:54.755777+0000","last_fullsized":"2024-03-26T13:59:54.755777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757431+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:54.757431+0000","last_peered":"2024-03-26T13:59:54.757431+0000","last_clean":"2024-03-26T13:59:54.757431+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:54.757431+0000","last_undegraded":"2024-03-26T13:59:54.757431+0000","last_fullsized":"2024-03-26T13:59:54.757431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756044+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:54.756044+0000","last_peered":"2024-03-26T13:59:54.756044+0000","last_clean":"2024-03-26T13:59:54.756044+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:54.756044+0000","last_undegraded":"2024-03-26T13:59:54.756044+0000","last_fullsized":"2024-03-26T13:59:54.756044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754447+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:54.754447+0000","last_peered":"2024-03-26T13:59:54.754447+0000","last_clean":"2024-03-26T13:59:54.754447+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:54.754447+0000","last_undegraded":"2024-03-26T13:59:54.754447+0000","last_fullsized":"2024-03-26T13:59:54.754447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756832+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:54.756832+0000","last_peered":"2024-03-26T13:59:54.756832+0000","last_clean":"2024-03-26T13:59:54.756832+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:54.756832+0000","last_undegraded":"2024-03-26T13:59:54.756832+0000","last_fullsized":"2024-03-26T13:59:54.756832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346428+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T13:59:54.346428+0000","last_peered":"2024-03-26T13:59:54.346428+0000","last_clean":"2024-03-26T13:59:54.346428+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T13:59:54.346428+0000","last_undegraded":"2024-03-26T13:59:54.346428+0000","last_fullsized":"2024-03-26T13:59:54.346428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60455,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":65,"num_read_kb":96,"num_write":150,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":187289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":503866,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1152075,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":504102,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":504022,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1099931,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":499278,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":555642,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":500666,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":538070,"data_compressed":14698,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":499472,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":504319,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":1095128,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:00:00.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2024-03-26T14:00:00.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 15, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-03-26T13:59:30.236338+0000', 'last_change': '2024-03-26T13:59:53.317881+0000', 'last_active': '2024-03-26T13:59:53.317881+0000', 'last_peered': '2024-03-26T13:59:30.236338+0000', 'last_clean': '2024-03-26T13:59:30.236338+0000', 'last_became_active': '2024-03-26T13:59:27.323585+0000', 'last_became_peered': '2024-03-26T13:59:27.323585+0000', 'last_unstale': '2024-03-26T13:59:30.236338+0000', 'last_undegraded': '2024-03-26T13:59:30.236338+0000', 'last_fullsized': '2024-03-26T13:59:30.236338+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 89, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_clean_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 8, 7, 11], 'acting': [2, 8, 7, 11], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 7, 8, 11], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-26T14:00:00.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2024-03-26T14:00:00.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 15, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2024-03-26T13:59:30.234348+0000', 'last_change': '2024-03-26T13:59:53.316731+0000', 'last_active': '2024-03-26T13:59:53.316731+0000', 'last_peered': '2024-03-26T13:59:30.234348+0000', 'last_clean': '2024-03-26T13:59:30.234348+0000', 'last_became_active': '2024-03-26T13:59:27.324762+0000', 'last_became_peered': '2024-03-26T13:59:27.324762+0000', 'last_unstale': '2024-03-26T13:59:30.234348+0000', 'last_undegraded': '2024-03-26T13:59:30.234348+0000', 'last_fullsized': '2024-03-26T13:59:30.234348+0000', 'mapping_epoch': 91, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 89, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'last_clean_scrub_stamp': '2024-03-26T13:59:26.279934+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6, 8, 5], 'acting': [2, 6, 8, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5, 6, 8, 9], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-26T14:00:00.256 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-26T14:00:00.396 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:00 smithi137 ceph-mon[31539]: pgmap v385: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T14:00:00.396 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:00 smithi137 ceph-mon[31539]: overall HEALTH_OK 2024-03-26T14:00:00.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:00 smithi179 ceph-mon[35472]: pgmap v385: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T14:00:00.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:00 smithi179 ceph-mon[35472]: overall HEALTH_OK 2024-03-26T14:00:00.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:00 smithi159 ceph-mon[35501]: pgmap v385: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 19 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T14:00:00.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:00 smithi159 ceph-mon[35501]: overall HEALTH_OK 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 -- 172.21.15.137:0/2516840112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41052a0 msgr2=0x7f01f410bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 --2- 172.21.15.137:0/2516840112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41052a0 0x7f01f410bb30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009ca0 tx=0x7f01ec02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 -- 172.21.15.137:0/2516840112 shutdown_connections 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 --2- 172.21.15.137:0/2516840112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f41048e0 0x7f01f4104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 --2- 172.21.15.137:0/2516840112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41052a0 0x7f01f410bb30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 --2- 172.21.15.137:0/2516840112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01f41036f0 0x7f01f4103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 -- 172.21.15.137:0/2516840112 >> 172.21.15.137:0/2516840112 conn(0x7f01f40fec80 msgr2=0x7f01f41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 -- 172.21.15.137:0/2516840112 shutdown_connections 2024-03-26T14:00:00.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.512+0000 7f01f9800640 1 -- 172.21.15.137:0/2516840112 wait complete. 2024-03-26T14:00:00.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.513+0000 7f01f9800640 1 Processor -- start 2024-03-26T14:00:00.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.513+0000 7f01f9800640 1 -- start start 2024-03-26T14:00:00.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01f41036f0 0x7f01f41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:00.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 0x7f01f4199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:00.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f41052a0 0x7f01f41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:00.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f9800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f410f200 con 0x7f01f41048e0 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f410f080 con 0x7f01f41052a0 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f9800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f410f380 con 0x7f01f41036f0 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 0x7f01f4199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 0x7f01f4199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54782/0 (socket says 172.21.15.137:54782) 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 -- 172.21.15.137:0/922311962 learned_addr learned my addr 172.21.15.137:0/922311962 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3e00640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01f41036f0 0x7f01f41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f8e00640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f41052a0 0x7f01f41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 -- 172.21.15.137:0/922311962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01f41036f0 msgr2=0x7f01f41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01f41036f0 0x7f01f41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 -- 172.21.15.137:0/922311962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f41052a0 msgr2=0x7f01f41a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f41052a0 0x7f01f41a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.514+0000 7f01f3400640 1 -- 172.21.15.137:0/922311962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f41a14c0 con 0x7f01f41048e0 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.515+0000 7f01f3400640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 0x7f01f4199de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d950 tx=0x7f01e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.515+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0017070 con 0x7f01f41048e0 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.515+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01e0009c30 con 0x7f01f41048e0 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.515+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000f620 con 0x7f01f41048e0 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.515+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f41a17b0 con 0x7f01f41048e0 2024-03-26T14:00:00.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.515+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f41a1cb0 con 0x7f01f41048e0 2024-03-26T14:00:00.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.516+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f410aa90 con 0x7f01f41048e0 2024-03-26T14:00:00.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.519+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f01e00109a0 con 0x7f01f41048e0 2024-03-26T14:00:00.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.520+0000 7f01f0c00640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01d80a2910 0x7f01d80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:00.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.520+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f01e00cec30 con 0x7f01f41048e0 2024-03-26T14:00:00.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.520+0000 7f01f3e00640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01d80a2910 0x7f01d80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:00.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.521+0000 7f01f3e00640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01d80a2910 0x7f01d80a4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f01e4005db0 tx=0x7f01e4005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:00.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.525+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01e0009de0 con 0x7f01f41048e0 2024-03-26T14:00:00.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.796+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f01f402d070 con 0x7f01f41048e0 2024-03-26T14:00:00.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.797+0000 7f01f0c00640 1 -- 172.21.15.137:0/922311962 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2805 (secure 0 0 0) 0x7f01e009b7b0 con 0x7f01f41048e0 2024-03-26T14:00:00.794 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:00.794 INFO:teuthology.orchestra.run.smithi137.stdout:{"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":528,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":12,"num_up_osds":12,"osd_up_since":1711461495,"num_in_osds":12,"osd_in_since":1711461470,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":651057,"bytes_used":370102272,"bytes_avail":1151570325504,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":14,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"read_bytes_sec":12282,"write_bytes_sec":10491,"read_op_per_sec":8,"write_op_per_sec":11},"fsmap":{"epoch":23,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"c","status":"up:standby-replay","gid":24460},{"filesystem_id":1,"rank":1,"name":"i","status":"up:standby-replay","gid":24481},{"filesystem_id":1,"rank":0,"name":"k","status":"up:standby-replay","gid":34394},{"filesystem_id":1,"rank":2,"name":"e","status":"up:active","gid":34424},{"filesystem_id":1,"rank":1,"name":"h","status":"up:active","gid":34451},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":34496}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.137:8443/"}},"servicemap":{"epoch":24,"modified":"2024-03-26T13:59:55.084709+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01d80a2910 msgr2=0x7f01d80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01d80a2910 0x7f01d80a4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f01e4005db0 tx=0x7f01e4005d40 comp rx=0 tx=0).stop 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 msgr2=0x7f01f4199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 0x7f01f4199de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d950 tx=0x7f01e000de10 comp rx=0 tx=0).stop 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 shutdown_connections 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01d80a2910 0x7f01d80a4dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01f41052a0 0x7f01f41a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01f41048e0 0x7f01f4199de0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 --2- 172.21.15.137:0/922311962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01f41036f0 0x7f01f41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 >> 172.21.15.137:0/922311962 conn(0x7f01f40fec80 msgr2=0x7f01f4107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:00.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 shutdown_connections 2024-03-26T14:00:00.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:00.800+0000 7f01f9800640 1 -- 172.21.15.137:0/922311962 wait complete. 2024-03-26T14:00:00.815 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2024-03-26T14:00:00.815 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T14:00:01.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 -- 172.21.15.137:0/226509631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc640772a0 msgr2=0x7fbc64077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 --2- 172.21.15.137:0/226509631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc640772a0 0x7fbc64077700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009900 tx=0x7fbc4c02bc40 comp rx=0 tx=0).stop 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 -- 172.21.15.137:0/226509631 shutdown_connections 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 --2- 172.21.15.137:0/226509631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64077c40 0x7fbc640784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 --2- 172.21.15.137:0/226509631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc640772a0 0x7fbc64077700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 --2- 172.21.15.137:0/226509631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc64076060 0x7fbc64076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 -- 172.21.15.137:0/226509631 >> 172.21.15.137:0/226509631 conn(0x7fbc640fee50 msgr2=0x7fbc64101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 -- 172.21.15.137:0/226509631 shutdown_connections 2024-03-26T14:00:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.036+0000 7fbc69a00640 1 -- 172.21.15.137:0/226509631 wait complete. 2024-03-26T14:00:01.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 Processor -- start 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 -- start start 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64076060 0x7fbc6419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc640772a0 0x7fbc6419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 0x7fbc641a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64113820 con 0x7fbc64077c40 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc641136a0 con 0x7fbc640772a0 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc69a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc641139a0 con 0x7fbc64076060 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc63400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 0x7fbc641a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc63400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 0x7fbc641a5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54788/0 (socket says 172.21.15.137:54788) 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.038+0000 7fbc63400640 1 -- 172.21.15.137:0/286814858 learned_addr learned my addr 172.21.15.137:0/286814858 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc5be00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc640772a0 0x7fbc6419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc62a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64076060 0x7fbc6419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:01.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc63400640 1 -- 172.21.15.137:0/286814858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64076060 msgr2=0x7fbc6419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc63400640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64076060 0x7fbc6419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc63400640 1 -- 172.21.15.137:0/286814858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc640772a0 msgr2=0x7fbc6419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc63400640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc640772a0 0x7fbc6419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc63400640 1 -- 172.21.15.137:0/286814858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc641a5ac0 con 0x7fbc64077c40 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc62a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64076060 0x7fbc6419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc63400640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 0x7fbc641a5450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cce0 tx=0x7fbc5c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.039+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c00f450 con 0x7fbc64077c40 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.040+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc641a5db0 con 0x7fbc64077c40 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.040+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbc5c00faa0 con 0x7fbc64077c40 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.040+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c018b70 con 0x7fbc64077c40 2024-03-26T14:00:01.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.040+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc641a6270 con 0x7fbc64077c40 2024-03-26T14:00:01.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.041+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64110400 con 0x7fbc64077c40 2024-03-26T14:00:01.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.044+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fbc5c018d10 con 0x7fbc64077c40 2024-03-26T14:00:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.045+0000 7fbc5b400640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbc480a2910 0x7fbc480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.045+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fbc5c0cf380 con 0x7fbc64077c40 2024-03-26T14:00:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.045+0000 7fbc62a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbc480a2910 0x7fbc480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.046+0000 7fbc62a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbc480a2910 0x7fbc480a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009b40 tx=0x7fbc54005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:01.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.049+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbc5c014030 con 0x7fbc64077c40 2024-03-26T14:00:01.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.243+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc64061ae0 con 0x7fbc480a2910 2024-03-26T14:00:01.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.272+0000 7fbc5b400640 1 -- 172.21.15.137:0/286814858 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340696 (secure 0 0 0) 0x7fbc64061ae0 con 0x7fbc480a2910 2024-03-26T14:00:01.273 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:01.275 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbc480a2910 msgr2=0x7fbc480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbc480a2910 0x7fbc480a4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009b40 tx=0x7fbc54005e50 comp rx=0 tx=0).stop 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 msgr2=0x7fbc641a5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 0x7fbc641a5450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c00cce0 tx=0x7fbc5c00a620 comp rx=0 tx=0).stop 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 shutdown_connections 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbc480a2910 0x7fbc480a4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc640772a0 0x7fbc6419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbc64077c40 0x7fbc641a5450 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 --2- 172.21.15.137:0/286814858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbc64076060 0x7fbc6419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:01.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.279+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 >> 172.21.15.137:0/286814858 conn(0x7fbc640fee50 msgr2=0x7fbc64100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:01.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.280+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 shutdown_connections 2024-03-26T14:00:01.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:01.280+0000 7fbc69a00640 1 -- 172.21.15.137:0/286814858 wait complete. 2024-03-26T14:00:01.296 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-03-26T14:00:01.087518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651057,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":191,"num_read_kb":205,"num_write":333,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361464,"kb_used_data":12244,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580360,"statfs":{"total":1151940427776,"available":1151570288640,"internally_reserved":0,"allocated":12537856,"data_stored":7981417,"data_compressed":355053,"data_compressed_allocated":479232,"data_compressed_original":2457600,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47724,"num_objects":8,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":46,"num_read_kb":77,"num_write":30,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003744"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334990+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:54.334990+0000","last_peered":"2024-03-26T13:59:54.334990+0000","last_clean":"2024-03-26T13:59:54.334990+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:54.334990+0000","last_undegraded":"2024-03-26T13:59:54.334990+0000","last_fullsized":"2024-03-26T13:59:54.334990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330109+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:54.330109+0000","last_peered":"2024-03-26T13:59:54.330109+0000","last_clean":"2024-03-26T13:59:54.330109+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:54.330109+0000","last_undegraded":"2024-03-26T13:59:54.330109+0000","last_fullsized":"2024-03-26T13:59:54.330109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327603+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:54.327603+0000","last_peered":"2024-03-26T13:59:54.327603+0000","last_clean":"2024-03-26T13:59:54.327603+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:54.327603+0000","last_undegraded":"2024-03-26T13:59:54.327603+0000","last_fullsized":"2024-03-26T13:59:54.327603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332907+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:54.332907+0000","last_peered":"2024-03-26T13:59:54.332907+0000","last_clean":"2024-03-26T13:59:54.332907+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:54.332907+0000","last_undegraded":"2024-03-26T13:59:54.332907+0000","last_fullsized":"2024-03-26T13:59:54.332907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703579+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:54.703579+0000","last_peered":"2024-03-26T13:59:54.703579+0000","last_clean":"2024-03-26T13:59:54.703579+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:54.703579+0000","last_undegraded":"2024-03-26T13:59:54.703579+0000","last_fullsized":"2024-03-26T13:59:54.703579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329612+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:54.329612+0000","last_peered":"2024-03-26T13:59:54.329612+0000","last_clean":"2024-03-26T13:59:54.329612+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:54.329612+0000","last_undegraded":"2024-03-26T13:59:54.329612+0000","last_fullsized":"2024-03-26T13:59:54.329612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755315+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:54.755315+0000","last_peered":"2024-03-26T13:59:54.755315+0000","last_clean":"2024-03-26T13:59:54.755315+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:54.755315+0000","last_undegraded":"2024-03-26T13:59:54.755315+0000","last_fullsized":"2024-03-26T13:59:54.755315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754310+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:54.754310+0000","last_peered":"2024-03-26T13:59:54.754310+0000","last_clean":"2024-03-26T13:59:54.754310+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:54.754310+0000","last_undegraded":"2024-03-26T13:59:54.754310+0000","last_fullsized":"2024-03-26T13:59:54.754310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704481+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:54.704481+0000","last_peered":"2024-03-26T13:59:54.704481+0000","last_clean":"2024-03-26T13:59:54.704481+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:54.704481+0000","last_undegraded":"2024-03-26T13:59:54.704481+0000","last_fullsized":"2024-03-26T13:59:54.704481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330340+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:54.330340+0000","last_peered":"2024-03-26T13:59:54.330340+0000","last_clean":"2024-03-26T13:59:54.330340+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:54.330340+0000","last_undegraded":"2024-03-26T13:59:54.330340+0000","last_fullsized":"2024-03-26T13:59:54.330340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327473+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:54.327473+0000","last_peered":"2024-03-26T13:59:54.327473+0000","last_clean":"2024-03-26T13:59:54.327473+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:54.327473+0000","last_undegraded":"2024-03-26T13:59:54.327473+0000","last_fullsized":"2024-03-26T13:59:54.327473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757043+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:54.757043+0000","last_peered":"2024-03-26T13:59:54.757043+0000","last_clean":"2024-03-26T13:59:54.757043+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:54.757043+0000","last_undegraded":"2024-03-26T13:59:54.757043+0000","last_fullsized":"2024-03-26T13:59:54.757043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331805+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:54.331805+0000","last_peered":"2024-03-26T13:59:54.331805+0000","last_clean":"2024-03-26T13:59:54.331805+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:54.331805+0000","last_undegraded":"2024-03-26T13:59:54.331805+0000","last_fullsized":"2024-03-26T13:59:54.331805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.344831+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T13:59:54.344831+0000","last_peered":"2024-03-26T13:59:54.344831+0000","last_clean":"2024-03-26T13:59:54.344831+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T13:59:54.344831+0000","last_undegraded":"2024-03-26T13:59:54.344831+0000","last_fullsized":"2024-03-26T13:59:54.344831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753840+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:54.753840+0000","last_peered":"2024-03-26T13:59:54.753840+0000","last_clean":"2024-03-26T13:59:54.753840+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:54.753840+0000","last_undegraded":"2024-03-26T13:59:54.753840+0000","last_fullsized":"2024-03-26T13:59:54.753840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334663+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:54.334663+0000","last_peered":"2024-03-26T13:59:54.334663+0000","last_clean":"2024-03-26T13:59:54.334663+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:54.334663+0000","last_undegraded":"2024-03-26T13:59:54.334663+0000","last_fullsized":"2024-03-26T13:59:54.334663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330566+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:54.330566+0000","last_peered":"2024-03-26T13:59:54.330566+0000","last_clean":"2024-03-26T13:59:54.330566+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:54.330566+0000","last_undegraded":"2024-03-26T13:59:54.330566+0000","last_fullsized":"2024-03-26T13:59:54.330566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339099+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:54.339099+0000","last_peered":"2024-03-26T13:59:54.339099+0000","last_clean":"2024-03-26T13:59:54.339099+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:54.339099+0000","last_undegraded":"2024-03-26T13:59:54.339099+0000","last_fullsized":"2024-03-26T13:59:54.339099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.340163+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:54.340163+0000","last_peered":"2024-03-26T13:59:54.340163+0000","last_clean":"2024-03-26T13:59:54.340163+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:54.340163+0000","last_undegraded":"2024-03-26T13:59:54.340163+0000","last_fullsized":"2024-03-26T13:59:54.340163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330916+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:54.330916+0000","last_peered":"2024-03-26T13:59:54.330916+0000","last_clean":"2024-03-26T13:59:54.330916+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:54.330916+0000","last_undegraded":"2024-03-26T13:59:54.330916+0000","last_fullsized":"2024-03-26T13:59:54.330916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757563+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:54.757563+0000","last_peered":"2024-03-26T13:59:54.757563+0000","last_clean":"2024-03-26T13:59:54.757563+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:54.757563+0000","last_undegraded":"2024-03-26T13:59:54.757563+0000","last_fullsized":"2024-03-26T13:59:54.757563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.325949+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:54.325949+0000","last_peered":"2024-03-26T13:59:54.325949+0000","last_clean":"2024-03-26T13:59:54.325949+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:54.325949+0000","last_undegraded":"2024-03-26T13:59:54.325949+0000","last_fullsized":"2024-03-26T13:59:54.325949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757809+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:54.757809+0000","last_peered":"2024-03-26T13:59:54.757809+0000","last_clean":"2024-03-26T13:59:54.757809+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:54.757809+0000","last_undegraded":"2024-03-26T13:59:54.757809+0000","last_fullsized":"2024-03-26T13:59:54.757809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330337+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:54.330337+0000","last_peered":"2024-03-26T13:59:54.330337+0000","last_clean":"2024-03-26T13:59:54.330337+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:54.330337+0000","last_undegraded":"2024-03-26T13:59:54.330337+0000","last_fullsized":"2024-03-26T13:59:54.330337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326515+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:54.326515+0000","last_peered":"2024-03-26T13:59:54.326515+0000","last_clean":"2024-03-26T13:59:54.326515+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:54.326515+0000","last_undegraded":"2024-03-26T13:59:54.326515+0000","last_fullsized":"2024-03-26T13:59:54.326515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339658+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:54.339658+0000","last_peered":"2024-03-26T13:59:54.339658+0000","last_clean":"2024-03-26T13:59:54.339658+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:54.339658+0000","last_undegraded":"2024-03-26T13:59:54.339658+0000","last_fullsized":"2024-03-26T13:59:54.339658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754197+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:54.754197+0000","last_peered":"2024-03-26T13:59:54.754197+0000","last_clean":"2024-03-26T13:59:54.754197+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:54.754197+0000","last_undegraded":"2024-03-26T13:59:54.754197+0000","last_fullsized":"2024-03-26T13:59:54.754197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326200+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:54.326200+0000","last_peered":"2024-03-26T13:59:54.326200+0000","last_clean":"2024-03-26T13:59:54.326200+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:54.326200+0000","last_undegraded":"2024-03-26T13:59:54.326200+0000","last_fullsized":"2024-03-26T13:59:54.326200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.342115+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:54.342115+0000","last_peered":"2024-03-26T13:59:54.342115+0000","last_clean":"2024-03-26T13:59:54.342115+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:54.342115+0000","last_undegraded":"2024-03-26T13:59:54.342115+0000","last_fullsized":"2024-03-26T13:59:54.342115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761061+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:54.761061+0000","last_peered":"2024-03-26T13:59:54.761061+0000","last_clean":"2024-03-26T13:59:54.761061+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:54.761061+0000","last_undegraded":"2024-03-26T13:59:54.761061+0000","last_fullsized":"2024-03-26T13:59:54.761061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.348257+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T13:59:54.348257+0000","last_peered":"2024-03-26T13:59:54.348257+0000","last_clean":"2024-03-26T13:59:54.348257+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T13:59:54.348257+0000","last_undegraded":"2024-03-26T13:59:54.348257+0000","last_fullsized":"2024-03-26T13:59:54.348257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.279834+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T13:59:55.279834+0000","last_peered":"2024-03-26T13:59:55.279834+0000","last_clean":"2024-03-26T13:59:55.279834+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T13:59:55.279834+0000","last_undegraded":"2024-03-26T13:59:55.279834+0000","last_fullsized":"2024-03-26T13:59:55.279834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346323+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T13:59:54.346323+0000","last_peered":"2024-03-26T13:59:54.346323+0000","last_clean":"2024-03-26T13:59:54.346323+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T13:59:54.346323+0000","last_undegraded":"2024-03-26T13:59:54.346323+0000","last_fullsized":"2024-03-26T13:59:54.346323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.763269+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T13:59:54.763269+0000","last_peered":"2024-03-26T13:59:54.763269+0000","last_clean":"2024-03-26T13:59:54.763269+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T13:59:54.763269+0000","last_undegraded":"2024-03-26T13:59:54.763269+0000","last_fullsized":"2024-03-26T13:59:54.763269+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703422+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:54.703422+0000","last_peered":"2024-03-26T13:59:54.703422+0000","last_clean":"2024-03-26T13:59:54.703422+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:54.703422+0000","last_undegraded":"2024-03-26T13:59:54.703422+0000","last_fullsized":"2024-03-26T13:59:54.703422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758242+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:54.758242+0000","last_peered":"2024-03-26T13:59:54.758242+0000","last_clean":"2024-03-26T13:59:54.758242+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:54.758242+0000","last_undegraded":"2024-03-26T13:59:54.758242+0000","last_fullsized":"2024-03-26T13:59:54.758242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334258+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:54.334258+0000","last_peered":"2024-03-26T13:59:54.334258+0000","last_clean":"2024-03-26T13:59:54.334258+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:54.334258+0000","last_undegraded":"2024-03-26T13:59:54.334258+0000","last_fullsized":"2024-03-26T13:59:54.334258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755967+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:54.755967+0000","last_peered":"2024-03-26T13:59:54.755967+0000","last_clean":"2024-03-26T13:59:54.755967+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:54.755967+0000","last_undegraded":"2024-03-26T13:59:54.755967+0000","last_fullsized":"2024-03-26T13:59:54.755967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.759517+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T13:59:54.759517+0000","last_peered":"2024-03-26T13:59:54.759517+0000","last_clean":"2024-03-26T13:59:54.759517+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T13:59:54.759517+0000","last_undegraded":"2024-03-26T13:59:54.759517+0000","last_fullsized":"2024-03-26T13:59:54.759517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329961+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:54.329961+0000","last_peered":"2024-03-26T13:59:54.329961+0000","last_clean":"2024-03-26T13:59:54.329961+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:54.329961+0000","last_undegraded":"2024-03-26T13:59:54.329961+0000","last_fullsized":"2024-03-26T13:59:54.329961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757020+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:54.757020+0000","last_peered":"2024-03-26T13:59:54.757020+0000","last_clean":"2024-03-26T13:59:54.757020+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:54.757020+0000","last_undegraded":"2024-03-26T13:59:54.757020+0000","last_fullsized":"2024-03-26T13:59:54.757020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327822+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:54.327822+0000","last_peered":"2024-03-26T13:59:54.327822+0000","last_clean":"2024-03-26T13:59:54.327822+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:54.327822+0000","last_undegraded":"2024-03-26T13:59:54.327822+0000","last_fullsized":"2024-03-26T13:59:54.327822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755747+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:54.755747+0000","last_peered":"2024-03-26T13:59:54.755747+0000","last_clean":"2024-03-26T13:59:54.755747+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:54.755747+0000","last_undegraded":"2024-03-26T13:59:54.755747+0000","last_fullsized":"2024-03-26T13:59:54.755747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704508+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:54.704508+0000","last_peered":"2024-03-26T13:59:54.704508+0000","last_clean":"2024-03-26T13:59:54.704508+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:54.704508+0000","last_undegraded":"2024-03-26T13:59:54.704508+0000","last_fullsized":"2024-03-26T13:59:54.704508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754499+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:54.754499+0000","last_peered":"2024-03-26T13:59:54.754499+0000","last_clean":"2024-03-26T13:59:54.754499+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:54.754499+0000","last_undegraded":"2024-03-26T13:59:54.754499+0000","last_fullsized":"2024-03-26T13:59:54.754499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327602+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:54.327602+0000","last_peered":"2024-03-26T13:59:54.327602+0000","last_clean":"2024-03-26T13:59:54.327602+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:54.327602+0000","last_undegraded":"2024-03-26T13:59:54.327602+0000","last_fullsized":"2024-03-26T13:59:54.327602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330446+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:54.330446+0000","last_peered":"2024-03-26T13:59:54.330446+0000","last_clean":"2024-03-26T13:59:54.330446+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:54.330446+0000","last_undegraded":"2024-03-26T13:59:54.330446+0000","last_fullsized":"2024-03-26T13:59:54.330446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761347+0000","last_change":"2024-03-26T13:59:54.761347+0000","last_active":"2024-03-26T13:59:54.761347+0000","last_peered":"2024-03-26T13:59:54.761347+0000","last_clean":"2024-03-26T13:59:54.761347+0000","last_became_active":"2024-03-26T13:59:54.760966+0000","last_became_peered":"2024-03-26T13:59:54.760966+0000","last_unstale":"2024-03-26T13:59:54.761347+0000","last_undegraded":"2024-03-26T13:59:54.761347+0000","last_fullsized":"2024-03-26T13:59:54.761347+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:27:13.241398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326627+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:54.326627+0000","last_peered":"2024-03-26T13:59:54.326627+0000","last_clean":"2024-03-26T13:59:54.326627+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:54.326627+0000","last_undegraded":"2024-03-26T13:59:54.326627+0000","last_fullsized":"2024-03-26T13:59:54.326627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.760687+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:54.760687+0000","last_peered":"2024-03-26T13:59:54.760687+0000","last_clean":"2024-03-26T13:59:54.760687+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:54.760687+0000","last_undegraded":"2024-03-26T13:59:54.760687+0000","last_fullsized":"2024-03-26T13:59:54.760687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754283+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:54.754283+0000","last_peered":"2024-03-26T13:59:54.754283+0000","last_clean":"2024-03-26T13:59:54.754283+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:54.754283+0000","last_undegraded":"2024-03-26T13:59:54.754283+0000","last_fullsized":"2024-03-26T13:59:54.754283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.328197+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:54.328197+0000","last_peered":"2024-03-26T13:59:54.328197+0000","last_clean":"2024-03-26T13:59:54.328197+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:54.328197+0000","last_undegraded":"2024-03-26T13:59:54.328197+0000","last_fullsized":"2024-03-26T13:59:54.328197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'26","reported_seq":70,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.283705+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T13:59:55.283705+0000","last_peered":"2024-03-26T13:59:55.283705+0000","last_clean":"2024-03-26T13:59:55.283705+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T13:59:55.283705+0000","last_undegraded":"2024-03-26T13:59:55.283705+0000","last_fullsized":"2024-03-26T13:59:55.283705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56454,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":64,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.340153+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:54.340153+0000","last_peered":"2024-03-26T13:59:54.340153+0000","last_clean":"2024-03-26T13:59:54.340153+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:54.340153+0000","last_undegraded":"2024-03-26T13:59:54.340153+0000","last_fullsized":"2024-03-26T13:59:54.340153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761598+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:54.761598+0000","last_peered":"2024-03-26T13:59:54.761598+0000","last_clean":"2024-03-26T13:59:54.761598+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:54.761598+0000","last_undegraded":"2024-03-26T13:59:54.761598+0000","last_fullsized":"2024-03-26T13:59:54.761598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.336926+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:54.336926+0000","last_peered":"2024-03-26T13:59:54.336926+0000","last_clean":"2024-03-26T13:59:54.336926+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:54.336926+0000","last_undegraded":"2024-03-26T13:59:54.336926+0000","last_fullsized":"2024-03-26T13:59:54.336926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756449+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:54.756449+0000","last_peered":"2024-03-26T13:59:54.756449+0000","last_clean":"2024-03-26T13:59:54.756449+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:54.756449+0000","last_undegraded":"2024-03-26T13:59:54.756449+0000","last_fullsized":"2024-03-26T13:59:54.756449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332076+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:54.332076+0000","last_peered":"2024-03-26T13:59:54.332076+0000","last_clean":"2024-03-26T13:59:54.332076+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:54.332076+0000","last_undegraded":"2024-03-26T13:59:54.332076+0000","last_fullsized":"2024-03-26T13:59:54.332076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757435+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:54.757435+0000","last_peered":"2024-03-26T13:59:54.757435+0000","last_clean":"2024-03-26T13:59:54.757435+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:54.757435+0000","last_undegraded":"2024-03-26T13:59:54.757435+0000","last_fullsized":"2024-03-26T13:59:54.757435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330136+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:54.330136+0000","last_peered":"2024-03-26T13:59:54.330136+0000","last_clean":"2024-03-26T13:59:54.330136+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:54.330136+0000","last_undegraded":"2024-03-26T13:59:54.330136+0000","last_fullsized":"2024-03-26T13:59:54.330136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330531+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:54.330531+0000","last_peered":"2024-03-26T13:59:54.330531+0000","last_clean":"2024-03-26T13:59:54.330531+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:54.330531+0000","last_undegraded":"2024-03-26T13:59:54.330531+0000","last_fullsized":"2024-03-26T13:59:54.330531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:57.287720+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:57.287720+0000","last_peered":"2024-03-26T13:59:57.287720+0000","last_clean":"2024-03-26T13:59:57.287720+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:57.287720+0000","last_undegraded":"2024-03-26T13:59:57.287720+0000","last_fullsized":"2024-03-26T13:59:57.287720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329932+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:54.329932+0000","last_peered":"2024-03-26T13:59:54.329932+0000","last_clean":"2024-03-26T13:59:54.329932+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:54.329932+0000","last_undegraded":"2024-03-26T13:59:54.329932+0000","last_fullsized":"2024-03-26T13:59:54.329932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758226+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:54.758226+0000","last_peered":"2024-03-26T13:59:54.758226+0000","last_clean":"2024-03-26T13:59:54.758226+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:54.758226+0000","last_undegraded":"2024-03-26T13:59:54.758226+0000","last_fullsized":"2024-03-26T13:59:54.758226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330561+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:54.330561+0000","last_peered":"2024-03-26T13:59:54.330561+0000","last_clean":"2024-03-26T13:59:54.330561+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:54.330561+0000","last_undegraded":"2024-03-26T13:59:54.330561+0000","last_fullsized":"2024-03-26T13:59:54.330561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756514+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:54.756514+0000","last_peered":"2024-03-26T13:59:54.756514+0000","last_clean":"2024-03-26T13:59:54.756514+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:54.756514+0000","last_undegraded":"2024-03-26T13:59:54.756514+0000","last_fullsized":"2024-03-26T13:59:54.756514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756211+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:54.756211+0000","last_peered":"2024-03-26T13:59:54.756211+0000","last_clean":"2024-03-26T13:59:54.756211+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:54.756211+0000","last_undegraded":"2024-03-26T13:59:54.756211+0000","last_fullsized":"2024-03-26T13:59:54.756211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.341535+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:54.341535+0000","last_peered":"2024-03-26T13:59:54.341535+0000","last_clean":"2024-03-26T13:59:54.341535+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:54.341535+0000","last_undegraded":"2024-03-26T13:59:54.341535+0000","last_fullsized":"2024-03-26T13:59:54.341535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761409+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T13:59:54.761409+0000","last_peered":"2024-03-26T13:59:54.761409+0000","last_clean":"2024-03-26T13:59:54.761409+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T13:59:54.761409+0000","last_undegraded":"2024-03-26T13:59:54.761409+0000","last_fullsized":"2024-03-26T13:59:54.761409+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334043+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:54.334043+0000","last_peered":"2024-03-26T13:59:54.334043+0000","last_clean":"2024-03-26T13:59:54.334043+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:54.334043+0000","last_undegraded":"2024-03-26T13:59:54.334043+0000","last_fullsized":"2024-03-26T13:59:54.334043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329309+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:54.329309+0000","last_peered":"2024-03-26T13:59:54.329309+0000","last_clean":"2024-03-26T13:59:54.329309+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:54.329309+0000","last_undegraded":"2024-03-26T13:59:54.329309+0000","last_fullsized":"2024-03-26T13:59:54.329309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703899+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:54.703899+0000","last_peered":"2024-03-26T13:59:54.703899+0000","last_clean":"2024-03-26T13:59:54.703899+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:54.703899+0000","last_undegraded":"2024-03-26T13:59:54.703899+0000","last_fullsized":"2024-03-26T13:59:54.703899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757312+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:54.757312+0000","last_peered":"2024-03-26T13:59:54.757312+0000","last_clean":"2024-03-26T13:59:54.757312+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:54.757312+0000","last_undegraded":"2024-03-26T13:59:54.757312+0000","last_fullsized":"2024-03-26T13:59:54.757312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329838+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:54.329838+0000","last_peered":"2024-03-26T13:59:54.329838+0000","last_clean":"2024-03-26T13:59:54.329838+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:54.329838+0000","last_undegraded":"2024-03-26T13:59:54.329838+0000","last_fullsized":"2024-03-26T13:59:54.329838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703337+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:54.703337+0000","last_peered":"2024-03-26T13:59:54.703337+0000","last_clean":"2024-03-26T13:59:54.703337+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:54.703337+0000","last_undegraded":"2024-03-26T13:59:54.703337+0000","last_fullsized":"2024-03-26T13:59:54.703337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758035+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:54.758035+0000","last_peered":"2024-03-26T13:59:54.758035+0000","last_clean":"2024-03-26T13:59:54.758035+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:54.758035+0000","last_undegraded":"2024-03-26T13:59:54.758035+0000","last_fullsized":"2024-03-26T13:59:54.758035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757574+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:54.757574+0000","last_peered":"2024-03-26T13:59:54.757574+0000","last_clean":"2024-03-26T13:59:54.757574+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:54.757574+0000","last_undegraded":"2024-03-26T13:59:54.757574+0000","last_fullsized":"2024-03-26T13:59:54.757574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:55.281085+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T13:59:55.281085+0000","last_peered":"2024-03-26T13:59:55.281085+0000","last_clean":"2024-03-26T13:59:55.281085+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T13:59:55.281085+0000","last_undegraded":"2024-03-26T13:59:55.281085+0000","last_fullsized":"2024-03-26T13:59:55.281085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330006+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:54.330006+0000","last_peered":"2024-03-26T13:59:54.330006+0000","last_clean":"2024-03-26T13:59:54.330006+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:54.330006+0000","last_undegraded":"2024-03-26T13:59:54.330006+0000","last_fullsized":"2024-03-26T13:59:54.330006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756304+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:54.756304+0000","last_peered":"2024-03-26T13:59:54.756304+0000","last_clean":"2024-03-26T13:59:54.756304+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:54.756304+0000","last_undegraded":"2024-03-26T13:59:54.756304+0000","last_fullsized":"2024-03-26T13:59:54.756304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326508+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:54.326508+0000","last_peered":"2024-03-26T13:59:54.326508+0000","last_clean":"2024-03-26T13:59:54.326508+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:54.326508+0000","last_undegraded":"2024-03-26T13:59:54.326508+0000","last_fullsized":"2024-03-26T13:59:54.326508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329667+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:54.329667+0000","last_peered":"2024-03-26T13:59:54.329667+0000","last_clean":"2024-03-26T13:59:54.329667+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:54.329667+0000","last_undegraded":"2024-03-26T13:59:54.329667+0000","last_fullsized":"2024-03-26T13:59:54.329667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.764260+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:54.764260+0000","last_peered":"2024-03-26T13:59:54.764260+0000","last_clean":"2024-03-26T13:59:54.764260+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:54.764260+0000","last_undegraded":"2024-03-26T13:59:54.764260+0000","last_fullsized":"2024-03-26T13:59:54.764260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756478+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:54.756478+0000","last_peered":"2024-03-26T13:59:54.756478+0000","last_clean":"2024-03-26T13:59:54.756478+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:54.756478+0000","last_undegraded":"2024-03-26T13:59:54.756478+0000","last_fullsized":"2024-03-26T13:59:54.756478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329794+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:54.329794+0000","last_peered":"2024-03-26T13:59:54.329794+0000","last_clean":"2024-03-26T13:59:54.329794+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:54.329794+0000","last_undegraded":"2024-03-26T13:59:54.329794+0000","last_fullsized":"2024-03-26T13:59:54.329794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331714+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:54.331714+0000","last_peered":"2024-03-26T13:59:54.331714+0000","last_clean":"2024-03-26T13:59:54.331714+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:54.331714+0000","last_undegraded":"2024-03-26T13:59:54.331714+0000","last_fullsized":"2024-03-26T13:59:54.331714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339975+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:54.339975+0000","last_peered":"2024-03-26T13:59:54.339975+0000","last_clean":"2024-03-26T13:59:54.339975+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:54.339975+0000","last_undegraded":"2024-03-26T13:59:54.339975+0000","last_fullsized":"2024-03-26T13:59:54.339975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757504+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:54.757504+0000","last_peered":"2024-03-26T13:59:54.757504+0000","last_clean":"2024-03-26T13:59:54.757504+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:54.757504+0000","last_undegraded":"2024-03-26T13:59:54.757504+0000","last_fullsized":"2024-03-26T13:59:54.757504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755154+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:54.755154+0000","last_peered":"2024-03-26T13:59:54.755154+0000","last_clean":"2024-03-26T13:59:54.755154+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:54.755154+0000","last_undegraded":"2024-03-26T13:59:54.755154+0000","last_fullsized":"2024-03-26T13:59:54.755154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331659+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:54.331659+0000","last_peered":"2024-03-26T13:59:54.331659+0000","last_clean":"2024-03-26T13:59:54.331659+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:54.331659+0000","last_undegraded":"2024-03-26T13:59:54.331659+0000","last_fullsized":"2024-03-26T13:59:54.331659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755545+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:54.755545+0000","last_peered":"2024-03-26T13:59:54.755545+0000","last_clean":"2024-03-26T13:59:54.755545+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:54.755545+0000","last_undegraded":"2024-03-26T13:59:54.755545+0000","last_fullsized":"2024-03-26T13:59:54.755545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.337460+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:54.337460+0000","last_peered":"2024-03-26T13:59:54.337460+0000","last_clean":"2024-03-26T13:59:54.337460+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:54.337460+0000","last_undegraded":"2024-03-26T13:59:54.337460+0000","last_fullsized":"2024-03-26T13:59:54.337460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327246+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:54.327246+0000","last_peered":"2024-03-26T13:59:54.327246+0000","last_clean":"2024-03-26T13:59:54.327246+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:54.327246+0000","last_undegraded":"2024-03-26T13:59:54.327246+0000","last_fullsized":"2024-03-26T13:59:54.327246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330244+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:54.330244+0000","last_peered":"2024-03-26T13:59:54.330244+0000","last_clean":"2024-03-26T13:59:54.330244+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:54.330244+0000","last_undegraded":"2024-03-26T13:59:54.330244+0000","last_fullsized":"2024-03-26T13:59:54.330244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326884+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:54.326884+0000","last_peered":"2024-03-26T13:59:54.326884+0000","last_clean":"2024-03-26T13:59:54.326884+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:54.326884+0000","last_undegraded":"2024-03-26T13:59:54.326884+0000","last_fullsized":"2024-03-26T13:59:54.326884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753877+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:54.753877+0000","last_peered":"2024-03-26T13:59:54.753877+0000","last_clean":"2024-03-26T13:59:54.753877+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:54.753877+0000","last_undegraded":"2024-03-26T13:59:54.753877+0000","last_fullsized":"2024-03-26T13:59:54.753877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331172+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:54.331172+0000","last_peered":"2024-03-26T13:59:54.331172+0000","last_clean":"2024-03-26T13:59:54.331172+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:54.331172+0000","last_undegraded":"2024-03-26T13:59:54.331172+0000","last_fullsized":"2024-03-26T13:59:54.331172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.768353+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T13:59:54.768353+0000","last_peered":"2024-03-26T13:59:54.768353+0000","last_clean":"2024-03-26T13:59:54.768353+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T13:59:54.768353+0000","last_undegraded":"2024-03-26T13:59:54.768353+0000","last_fullsized":"2024-03-26T13:59:54.768353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755911+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:54.755911+0000","last_peered":"2024-03-26T13:59:54.755911+0000","last_clean":"2024-03-26T13:59:54.755911+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:54.755911+0000","last_undegraded":"2024-03-26T13:59:54.755911+0000","last_fullsized":"2024-03-26T13:59:54.755911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331265+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:54.331265+0000","last_peered":"2024-03-26T13:59:54.331265+0000","last_clean":"2024-03-26T13:59:54.331265+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:54.331265+0000","last_undegraded":"2024-03-26T13:59:54.331265+0000","last_fullsized":"2024-03-26T13:59:54.331265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.714081+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T13:59:54.714081+0000","last_peered":"2024-03-26T13:59:54.714081+0000","last_clean":"2024-03-26T13:59:54.714081+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T13:59:54.714081+0000","last_undegraded":"2024-03-26T13:59:54.714081+0000","last_fullsized":"2024-03-26T13:59:54.714081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339742+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:54.339742+0000","last_peered":"2024-03-26T13:59:54.339742+0000","last_clean":"2024-03-26T13:59:54.339742+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:54.339742+0000","last_undegraded":"2024-03-26T13:59:54.339742+0000","last_fullsized":"2024-03-26T13:59:54.339742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756671+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:54.756671+0000","last_peered":"2024-03-26T13:59:54.756671+0000","last_clean":"2024-03-26T13:59:54.756671+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:54.756671+0000","last_undegraded":"2024-03-26T13:59:54.756671+0000","last_fullsized":"2024-03-26T13:59:54.756671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326864+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:54.326864+0000","last_peered":"2024-03-26T13:59:54.326864+0000","last_clean":"2024-03-26T13:59:54.326864+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:54.326864+0000","last_undegraded":"2024-03-26T13:59:54.326864+0000","last_fullsized":"2024-03-26T13:59:54.326864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333249+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:54.333249+0000","last_peered":"2024-03-26T13:59:54.333249+0000","last_clean":"2024-03-26T13:59:54.333249+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:54.333249+0000","last_undegraded":"2024-03-26T13:59:54.333249+0000","last_fullsized":"2024-03-26T13:59:54.333249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329708+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:54.329708+0000","last_peered":"2024-03-26T13:59:54.329708+0000","last_clean":"2024-03-26T13:59:54.329708+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:54.329708+0000","last_undegraded":"2024-03-26T13:59:54.329708+0000","last_fullsized":"2024-03-26T13:59:54.329708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753959+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:54.753959+0000","last_peered":"2024-03-26T13:59:54.753959+0000","last_clean":"2024-03-26T13:59:54.753959+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:54.753959+0000","last_undegraded":"2024-03-26T13:59:54.753959+0000","last_fullsized":"2024-03-26T13:59:54.753959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.338291+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:54.338291+0000","last_peered":"2024-03-26T13:59:54.338291+0000","last_clean":"2024-03-26T13:59:54.338291+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:54.338291+0000","last_undegraded":"2024-03-26T13:59:54.338291+0000","last_fullsized":"2024-03-26T13:59:54.338291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330394+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:54.330394+0000","last_peered":"2024-03-26T13:59:54.330394+0000","last_clean":"2024-03-26T13:59:54.330394+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:54.330394+0000","last_undegraded":"2024-03-26T13:59:54.330394+0000","last_fullsized":"2024-03-26T13:59:54.330394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761247+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:54.761247+0000","last_peered":"2024-03-26T13:59:54.761247+0000","last_clean":"2024-03-26T13:59:54.761247+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:54.761247+0000","last_undegraded":"2024-03-26T13:59:54.761247+0000","last_fullsized":"2024-03-26T13:59:54.761247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330119+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:54.330119+0000","last_peered":"2024-03-26T13:59:54.330119+0000","last_clean":"2024-03-26T13:59:54.330119+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:54.330119+0000","last_undegraded":"2024-03-26T13:59:54.330119+0000","last_fullsized":"2024-03-26T13:59:54.330119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756124+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:54.756124+0000","last_peered":"2024-03-26T13:59:54.756124+0000","last_clean":"2024-03-26T13:59:54.756124+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:54.756124+0000","last_undegraded":"2024-03-26T13:59:54.756124+0000","last_fullsized":"2024-03-26T13:59:54.756124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758106+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:54.758106+0000","last_peered":"2024-03-26T13:59:54.758106+0000","last_clean":"2024-03-26T13:59:54.758106+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:54.758106+0000","last_undegraded":"2024-03-26T13:59:54.758106+0000","last_fullsized":"2024-03-26T13:59:54.758106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330912+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:54.330912+0000","last_peered":"2024-03-26T13:59:54.330912+0000","last_clean":"2024-03-26T13:59:54.330912+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:54.330912+0000","last_undegraded":"2024-03-26T13:59:54.330912+0000","last_fullsized":"2024-03-26T13:59:54.330912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332068+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:54.332068+0000","last_peered":"2024-03-26T13:59:54.332068+0000","last_clean":"2024-03-26T13:59:54.332068+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:54.332068+0000","last_undegraded":"2024-03-26T13:59:54.332068+0000","last_fullsized":"2024-03-26T13:59:54.332068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.338037+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:54.338037+0000","last_peered":"2024-03-26T13:59:54.338037+0000","last_clean":"2024-03-26T13:59:54.338037+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:54.338037+0000","last_undegraded":"2024-03-26T13:59:54.338037+0000","last_fullsized":"2024-03-26T13:59:54.338037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339394+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:54.339394+0000","last_peered":"2024-03-26T13:59:54.339394+0000","last_clean":"2024-03-26T13:59:54.339394+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:54.339394+0000","last_undegraded":"2024-03-26T13:59:54.339394+0000","last_fullsized":"2024-03-26T13:59:54.339394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332280+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:54.332280+0000","last_peered":"2024-03-26T13:59:54.332280+0000","last_clean":"2024-03-26T13:59:54.332280+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:54.332280+0000","last_undegraded":"2024-03-26T13:59:54.332280+0000","last_fullsized":"2024-03-26T13:59:54.332280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757622+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:54.757622+0000","last_peered":"2024-03-26T13:59:54.757622+0000","last_clean":"2024-03-26T13:59:54.757622+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:54.757622+0000","last_undegraded":"2024-03-26T13:59:54.757622+0000","last_fullsized":"2024-03-26T13:59:54.757622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333876+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:54.333876+0000","last_peered":"2024-03-26T13:59:54.333876+0000","last_clean":"2024-03-26T13:59:54.333876+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:54.333876+0000","last_undegraded":"2024-03-26T13:59:54.333876+0000","last_fullsized":"2024-03-26T13:59:54.333876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.337727+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:54.337727+0000","last_peered":"2024-03-26T13:59:54.337727+0000","last_clean":"2024-03-26T13:59:54.337727+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:54.337727+0000","last_undegraded":"2024-03-26T13:59:54.337727+0000","last_fullsized":"2024-03-26T13:59:54.337727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755777+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:54.755777+0000","last_peered":"2024-03-26T13:59:54.755777+0000","last_clean":"2024-03-26T13:59:54.755777+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:54.755777+0000","last_undegraded":"2024-03-26T13:59:54.755777+0000","last_fullsized":"2024-03-26T13:59:54.755777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757431+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:54.757431+0000","last_peered":"2024-03-26T13:59:54.757431+0000","last_clean":"2024-03-26T13:59:54.757431+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:54.757431+0000","last_undegraded":"2024-03-26T13:59:54.757431+0000","last_fullsized":"2024-03-26T13:59:54.757431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756044+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:54.756044+0000","last_peered":"2024-03-26T13:59:54.756044+0000","last_clean":"2024-03-26T13:59:54.756044+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:54.756044+0000","last_undegraded":"2024-03-26T13:59:54.756044+0000","last_fullsized":"2024-03-26T13:59:54.756044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754447+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:54.754447+0000","last_peered":"2024-03-26T13:59:54.754447+0000","last_clean":"2024-03-26T13:59:54.754447+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:54.754447+0000","last_undegraded":"2024-03-26T13:59:54.754447+0000","last_fullsized":"2024-03-26T13:59:54.754447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756832+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:54.756832+0000","last_peered":"2024-03-26T13:59:54.756832+0000","last_clean":"2024-03-26T13:59:54.756832+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:54.756832+0000","last_undegraded":"2024-03-26T13:59:54.756832+0000","last_fullsized":"2024-03-26T13:59:54.756832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346428+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T13:59:54.346428+0000","last_peered":"2024-03-26T13:59:54.346428+0000","last_clean":"2024-03-26T13:59:54.346428+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T13:59:54.346428+0000","last_undegraded":"2024-03-26T13:59:54.346428+0000","last_fullsized":"2024-03-26T13:59:54.346428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60455,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":65,"num_read_kb":96,"num_write":150,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":193213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":503866,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":1152075,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":504102,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1019904,"data_stored":504022,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1099931,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":499278,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1060864,"data_stored":555642,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":500666,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":562916,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149774,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":499472,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":504319,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":1095128,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:00:01.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:01 smithi179 ceph-mon[35472]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:01.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:01 smithi179 ceph-mon[35472]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:01.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:01 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/922311962' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-26T14:00:01.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:01 smithi159 ceph-mon[35501]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:01.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:01 smithi159 ceph-mon[35501]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:01.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:01 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/922311962' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-26T14:00:01.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:01 smithi137 ceph-mon[31539]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:01.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:01 smithi137 ceph-mon[31539]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:01.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:01 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/922311962' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-03-26T14:00:02.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:02 smithi179 ceph-mon[35472]: pgmap v386: 129 pgs: 129 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T14:00:02.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:02 smithi179 ceph-mon[35472]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:02 smithi159 ceph-mon[35501]: pgmap v386: 129 pgs: 129 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T14:00:02.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:02 smithi159 ceph-mon[35501]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:02.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:02 smithi137 ceph-mon[31539]: pgmap v386: 129 pgs: 129 active+clean; 636 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 7.2 KiB/s wr, 9 op/s; 14 B/s, 0 objects/s recovering 2024-03-26T14:00:02.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:02 smithi137 ceph-mon[31539]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:04.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T14:00:04.393 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:04 smithi137 ceph-mon[31539]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 11 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T14:00:04.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:04 smithi179 ceph-mon[35472]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 11 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T14:00:04.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:04 smithi159 ceph-mon[35501]: pgmap v387: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 11 op/s; 11 B/s, 0 objects/s recovering 2024-03-26T14:00:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 -- 172.21.15.137:0/252202370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f0103270 msgr2=0x7fc5f0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/252202370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f0103270 0x7fc5f0109b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009900 tx=0x7fc5e802bc40 comp rx=0 tx=0).stop 2024-03-26T14:00:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 -- 172.21.15.137:0/252202370 shutdown_connections 2024-03-26T14:00:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/252202370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f01028b0 0x7fc5f0102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/252202370 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f0103270 0x7fc5f0109b00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/252202370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f01016c0 0x7fc5f0101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 -- 172.21.15.137:0/252202370 >> 172.21.15.137:0/252202370 conn(0x7fc5f00fcc50 msgr2=0x7fc5f00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 -- 172.21.15.137:0/252202370 shutdown_connections 2024-03-26T14:00:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.532+0000 7fc5f7c00640 1 -- 172.21.15.137:0/252202370 wait complete. 2024-03-26T14:00:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.533+0000 7fc5f7c00640 1 Processor -- start 2024-03-26T14:00:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.533+0000 7fc5f7c00640 1 -- start start 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.533+0000 7fc5f7c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f01016c0 0x7fc5f0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f7c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 0x7fc5f0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f7c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f0103270 0x7fc5f01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f7c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010d240 con 0x7fc5f01028b0 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f7c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010d0c0 con 0x7fc5f0103270 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f7c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010d3c0 con 0x7fc5f01016c0 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 0x7fc5f0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 0x7fc5f0199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54820/0 (socket says 172.21.15.137:54820) 2024-03-26T14:00:04.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 -- 172.21.15.137:0/1140293077 learned_addr learned my addr 172.21.15.137:0/1140293077 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f5c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f0103270 0x7fc5f01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5f5200640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f01016c0 0x7fc5f0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 -- 172.21.15.137:0/1140293077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f01016c0 msgr2=0x7fc5f0199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f01016c0 0x7fc5f0199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 -- 172.21.15.137:0/1140293077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f0103270 msgr2=0x7fc5f01a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f0103270 0x7fc5f01a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.534+0000 7fc5efe00640 1 -- 172.21.15.137:0/1140293077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f01a16b0 con 0x7fc5f01028b0 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.535+0000 7fc5efe00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 0x7fc5f0199fd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800eab0 tx=0x7fc5d800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.535+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800ce40 con 0x7fc5f01028b0 2024-03-26T14:00:04.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.535+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc5d8011920 con 0x7fc5f01028b0 2024-03-26T14:00:04.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.535+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8010a00 con 0x7fc5f01028b0 2024-03-26T14:00:04.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.535+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f01a19a0 con 0x7fc5f01028b0 2024-03-26T14:00:04.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.535+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f01a1ea0 con 0x7fc5f01028b0 2024-03-26T14:00:04.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.536+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f01a2130 con 0x7fc5f01028b0 2024-03-26T14:00:04.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.538+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fc5d8011ad0 con 0x7fc5f01028b0 2024-03-26T14:00:04.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.539+0000 7fc5ed600640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5d00a2910 0x7fc5d00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:04.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.540+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fc5d80cf5a0 con 0x7fc5f01028b0 2024-03-26T14:00:04.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.540+0000 7fc5f5200640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5d00a2910 0x7fc5d00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:04.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.540+0000 7fc5f5200640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5d00a2910 0x7fc5d00a4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00058c0 tx=0x7fc5e000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:04.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.544+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc5d8016030 con 0x7fc5f01028b0 2024-03-26T14:00:04.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.735+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5f01a2300 con 0x7fc5d00a2910 2024-03-26T14:00:04.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.762+0000 7fc5ed600640 1 -- 172.21.15.137:0/1140293077 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340669 (secure 0 0 0) 0x7fc5f01a2300 con 0x7fc5d00a2910 2024-03-26T14:00:04.764 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:04.766 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:00:04.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.769+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5d00a2910 msgr2=0x7fc5d00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.769+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5d00a2910 0x7fc5d00a4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00058c0 tx=0x7fc5e000a4c0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.769+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 msgr2=0x7fc5f0199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.769+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 0x7fc5f0199fd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800eab0 tx=0x7fc5d800ef70 comp rx=0 tx=0).stop 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 shutdown_connections 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5d00a2910 0x7fc5d00a4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f0103270 0x7fc5f01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc5f01028b0 0x7fc5f0199fd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 --2- 172.21.15.137:0/1140293077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5f01016c0 0x7fc5f0199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 >> 172.21.15.137:0/1140293077 conn(0x7fc5f00fcc50 msgr2=0x7fc5f00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 shutdown_connections 2024-03-26T14:00:04.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:04.770+0000 7fc5f7c00640 1 -- 172.21.15.137:0/1140293077 wait complete. 2024-03-26T14:00:04.786 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-03-26T14:00:03.088341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656981,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":217,"num_read_kb":226,"num_write":342,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361460,"kb_used_data":12260,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580364,"statfs":{"total":1151940427776,"available":1151570292736,"internally_reserved":0,"allocated":12554240,"data_stored":7993265,"data_compressed":355053,"data_compressed_allocated":479232,"data_compressed_original":2457600,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53648,"num_objects":8,"num_object_clones":0,"num_object_copies":25,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":72,"num_read_kb":98,"num_write":39,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.778699"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334990+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T13:59:54.334990+0000","last_peered":"2024-03-26T13:59:54.334990+0000","last_clean":"2024-03-26T13:59:54.334990+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T13:59:54.334990+0000","last_undegraded":"2024-03-26T13:59:54.334990+0000","last_fullsized":"2024-03-26T13:59:54.334990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330109+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T13:59:54.330109+0000","last_peered":"2024-03-26T13:59:54.330109+0000","last_clean":"2024-03-26T13:59:54.330109+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T13:59:54.330109+0000","last_undegraded":"2024-03-26T13:59:54.330109+0000","last_fullsized":"2024-03-26T13:59:54.330109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327603+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T13:59:54.327603+0000","last_peered":"2024-03-26T13:59:54.327603+0000","last_clean":"2024-03-26T13:59:54.327603+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T13:59:54.327603+0000","last_undegraded":"2024-03-26T13:59:54.327603+0000","last_fullsized":"2024-03-26T13:59:54.327603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332907+0000","last_change":"2024-03-26T13:59:23.313957+0000","last_active":"2024-03-26T13:59:54.332907+0000","last_peered":"2024-03-26T13:59:54.332907+0000","last_clean":"2024-03-26T13:59:54.332907+0000","last_became_active":"2024-03-26T13:59:23.313612+0000","last_became_peered":"2024-03-26T13:59:23.313612+0000","last_unstale":"2024-03-26T13:59:54.332907+0000","last_undegraded":"2024-03-26T13:59:54.332907+0000","last_fullsized":"2024-03-26T13:59:54.332907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:37:52.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703579+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T13:59:54.703579+0000","last_peered":"2024-03-26T13:59:54.703579+0000","last_clean":"2024-03-26T13:59:54.703579+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T13:59:54.703579+0000","last_undegraded":"2024-03-26T13:59:54.703579+0000","last_fullsized":"2024-03-26T13:59:54.703579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329612+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T13:59:54.329612+0000","last_peered":"2024-03-26T13:59:54.329612+0000","last_clean":"2024-03-26T13:59:54.329612+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T13:59:54.329612+0000","last_undegraded":"2024-03-26T13:59:54.329612+0000","last_fullsized":"2024-03-26T13:59:54.329612+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755315+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T13:59:54.755315+0000","last_peered":"2024-03-26T13:59:54.755315+0000","last_clean":"2024-03-26T13:59:54.755315+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T13:59:54.755315+0000","last_undegraded":"2024-03-26T13:59:54.755315+0000","last_fullsized":"2024-03-26T13:59:54.755315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754310+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T13:59:54.754310+0000","last_peered":"2024-03-26T13:59:54.754310+0000","last_clean":"2024-03-26T13:59:54.754310+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T13:59:54.754310+0000","last_undegraded":"2024-03-26T13:59:54.754310+0000","last_fullsized":"2024-03-26T13:59:54.754310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.704481+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T13:59:54.704481+0000","last_peered":"2024-03-26T13:59:54.704481+0000","last_clean":"2024-03-26T13:59:54.704481+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T13:59:54.704481+0000","last_undegraded":"2024-03-26T13:59:54.704481+0000","last_fullsized":"2024-03-26T13:59:54.704481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330340+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T13:59:54.330340+0000","last_peered":"2024-03-26T13:59:54.330340+0000","last_clean":"2024-03-26T13:59:54.330340+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T13:59:54.330340+0000","last_undegraded":"2024-03-26T13:59:54.330340+0000","last_fullsized":"2024-03-26T13:59:54.330340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327473+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T13:59:54.327473+0000","last_peered":"2024-03-26T13:59:54.327473+0000","last_clean":"2024-03-26T13:59:54.327473+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T13:59:54.327473+0000","last_undegraded":"2024-03-26T13:59:54.327473+0000","last_fullsized":"2024-03-26T13:59:54.327473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757043+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T13:59:54.757043+0000","last_peered":"2024-03-26T13:59:54.757043+0000","last_clean":"2024-03-26T13:59:54.757043+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T13:59:54.757043+0000","last_undegraded":"2024-03-26T13:59:54.757043+0000","last_fullsized":"2024-03-26T13:59:54.757043+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331805+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T13:59:54.331805+0000","last_peered":"2024-03-26T13:59:54.331805+0000","last_clean":"2024-03-26T13:59:54.331805+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T13:59:54.331805+0000","last_undegraded":"2024-03-26T13:59:54.331805+0000","last_fullsized":"2024-03-26T13:59:54.331805+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.344831+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T13:59:54.344831+0000","last_peered":"2024-03-26T13:59:54.344831+0000","last_clean":"2024-03-26T13:59:54.344831+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T13:59:54.344831+0000","last_undegraded":"2024-03-26T13:59:54.344831+0000","last_fullsized":"2024-03-26T13:59:54.344831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753840+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T13:59:54.753840+0000","last_peered":"2024-03-26T13:59:54.753840+0000","last_clean":"2024-03-26T13:59:54.753840+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T13:59:54.753840+0000","last_undegraded":"2024-03-26T13:59:54.753840+0000","last_fullsized":"2024-03-26T13:59:54.753840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334663+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T13:59:54.334663+0000","last_peered":"2024-03-26T13:59:54.334663+0000","last_clean":"2024-03-26T13:59:54.334663+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T13:59:54.334663+0000","last_undegraded":"2024-03-26T13:59:54.334663+0000","last_fullsized":"2024-03-26T13:59:54.334663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330566+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T13:59:54.330566+0000","last_peered":"2024-03-26T13:59:54.330566+0000","last_clean":"2024-03-26T13:59:54.330566+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T13:59:54.330566+0000","last_undegraded":"2024-03-26T13:59:54.330566+0000","last_fullsized":"2024-03-26T13:59:54.330566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339099+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T13:59:54.339099+0000","last_peered":"2024-03-26T13:59:54.339099+0000","last_clean":"2024-03-26T13:59:54.339099+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T13:59:54.339099+0000","last_undegraded":"2024-03-26T13:59:54.339099+0000","last_fullsized":"2024-03-26T13:59:54.339099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.340163+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T13:59:54.340163+0000","last_peered":"2024-03-26T13:59:54.340163+0000","last_clean":"2024-03-26T13:59:54.340163+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T13:59:54.340163+0000","last_undegraded":"2024-03-26T13:59:54.340163+0000","last_fullsized":"2024-03-26T13:59:54.340163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330916+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T13:59:54.330916+0000","last_peered":"2024-03-26T13:59:54.330916+0000","last_clean":"2024-03-26T13:59:54.330916+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T13:59:54.330916+0000","last_undegraded":"2024-03-26T13:59:54.330916+0000","last_fullsized":"2024-03-26T13:59:54.330916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757563+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T13:59:54.757563+0000","last_peered":"2024-03-26T13:59:54.757563+0000","last_clean":"2024-03-26T13:59:54.757563+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T13:59:54.757563+0000","last_undegraded":"2024-03-26T13:59:54.757563+0000","last_fullsized":"2024-03-26T13:59:54.757563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.325949+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T13:59:54.325949+0000","last_peered":"2024-03-26T13:59:54.325949+0000","last_clean":"2024-03-26T13:59:54.325949+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T13:59:54.325949+0000","last_undegraded":"2024-03-26T13:59:54.325949+0000","last_fullsized":"2024-03-26T13:59:54.325949+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757809+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T13:59:54.757809+0000","last_peered":"2024-03-26T13:59:54.757809+0000","last_clean":"2024-03-26T13:59:54.757809+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T13:59:54.757809+0000","last_undegraded":"2024-03-26T13:59:54.757809+0000","last_fullsized":"2024-03-26T13:59:54.757809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330337+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T13:59:54.330337+0000","last_peered":"2024-03-26T13:59:54.330337+0000","last_clean":"2024-03-26T13:59:54.330337+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T13:59:54.330337+0000","last_undegraded":"2024-03-26T13:59:54.330337+0000","last_fullsized":"2024-03-26T13:59:54.330337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326515+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T13:59:54.326515+0000","last_peered":"2024-03-26T13:59:54.326515+0000","last_clean":"2024-03-26T13:59:54.326515+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T13:59:54.326515+0000","last_undegraded":"2024-03-26T13:59:54.326515+0000","last_fullsized":"2024-03-26T13:59:54.326515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339658+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T13:59:54.339658+0000","last_peered":"2024-03-26T13:59:54.339658+0000","last_clean":"2024-03-26T13:59:54.339658+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T13:59:54.339658+0000","last_undegraded":"2024-03-26T13:59:54.339658+0000","last_fullsized":"2024-03-26T13:59:54.339658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754197+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T13:59:54.754197+0000","last_peered":"2024-03-26T13:59:54.754197+0000","last_clean":"2024-03-26T13:59:54.754197+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T13:59:54.754197+0000","last_undegraded":"2024-03-26T13:59:54.754197+0000","last_fullsized":"2024-03-26T13:59:54.754197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326200+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T13:59:54.326200+0000","last_peered":"2024-03-26T13:59:54.326200+0000","last_clean":"2024-03-26T13:59:54.326200+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T13:59:54.326200+0000","last_undegraded":"2024-03-26T13:59:54.326200+0000","last_fullsized":"2024-03-26T13:59:54.326200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.342115+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T13:59:54.342115+0000","last_peered":"2024-03-26T13:59:54.342115+0000","last_clean":"2024-03-26T13:59:54.342115+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T13:59:54.342115+0000","last_undegraded":"2024-03-26T13:59:54.342115+0000","last_fullsized":"2024-03-26T13:59:54.342115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761061+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T13:59:54.761061+0000","last_peered":"2024-03-26T13:59:54.761061+0000","last_clean":"2024-03-26T13:59:54.761061+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T13:59:54.761061+0000","last_undegraded":"2024-03-26T13:59:54.761061+0000","last_fullsized":"2024-03-26T13:59:54.761061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.348257+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T13:59:54.348257+0000","last_peered":"2024-03-26T13:59:54.348257+0000","last_clean":"2024-03-26T13:59:54.348257+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T13:59:54.348257+0000","last_undegraded":"2024-03-26T13:59:54.348257+0000","last_fullsized":"2024-03-26T13:59:54.348257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T14:00:00.293439+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T14:00:00.293439+0000","last_peered":"2024-03-26T14:00:00.293439+0000","last_clean":"2024-03-26T14:00:00.293439+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T14:00:00.293439+0000","last_undegraded":"2024-03-26T14:00:00.293439+0000","last_fullsized":"2024-03-26T14:00:00.293439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346323+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T13:59:54.346323+0000","last_peered":"2024-03-26T13:59:54.346323+0000","last_clean":"2024-03-26T13:59:54.346323+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T13:59:54.346323+0000","last_undegraded":"2024-03-26T13:59:54.346323+0000","last_fullsized":"2024-03-26T13:59:54.346323+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.763269+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T13:59:54.763269+0000","last_peered":"2024-03-26T13:59:54.763269+0000","last_clean":"2024-03-26T13:59:54.763269+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T13:59:54.763269+0000","last_undegraded":"2024-03-26T13:59:54.763269+0000","last_fullsized":"2024-03-26T13:59:54.763269+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703422+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T13:59:54.703422+0000","last_peered":"2024-03-26T13:59:54.703422+0000","last_clean":"2024-03-26T13:59:54.703422+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T13:59:54.703422+0000","last_undegraded":"2024-03-26T13:59:54.703422+0000","last_fullsized":"2024-03-26T13:59:54.703422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758242+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T13:59:54.758242+0000","last_peered":"2024-03-26T13:59:54.758242+0000","last_clean":"2024-03-26T13:59:54.758242+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T13:59:54.758242+0000","last_undegraded":"2024-03-26T13:59:54.758242+0000","last_fullsized":"2024-03-26T13:59:54.758242+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334258+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T13:59:54.334258+0000","last_peered":"2024-03-26T13:59:54.334258+0000","last_clean":"2024-03-26T13:59:54.334258+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T13:59:54.334258+0000","last_undegraded":"2024-03-26T13:59:54.334258+0000","last_fullsized":"2024-03-26T13:59:54.334258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755967+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T13:59:54.755967+0000","last_peered":"2024-03-26T13:59:54.755967+0000","last_clean":"2024-03-26T13:59:54.755967+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T13:59:54.755967+0000","last_undegraded":"2024-03-26T13:59:54.755967+0000","last_fullsized":"2024-03-26T13:59:54.755967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.759517+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T13:59:54.759517+0000","last_peered":"2024-03-26T13:59:54.759517+0000","last_clean":"2024-03-26T13:59:54.759517+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T13:59:54.759517+0000","last_undegraded":"2024-03-26T13:59:54.759517+0000","last_fullsized":"2024-03-26T13:59:54.759517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329961+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T13:59:54.329961+0000","last_peered":"2024-03-26T13:59:54.329961+0000","last_clean":"2024-03-26T13:59:54.329961+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T13:59:54.329961+0000","last_undegraded":"2024-03-26T13:59:54.329961+0000","last_fullsized":"2024-03-26T13:59:54.329961+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757020+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T13:59:54.757020+0000","last_peered":"2024-03-26T13:59:54.757020+0000","last_clean":"2024-03-26T13:59:54.757020+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T13:59:54.757020+0000","last_undegraded":"2024-03-26T13:59:54.757020+0000","last_fullsized":"2024-03-26T13:59:54.757020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327822+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T13:59:54.327822+0000","last_peered":"2024-03-26T13:59:54.327822+0000","last_clean":"2024-03-26T13:59:54.327822+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T13:59:54.327822+0000","last_undegraded":"2024-03-26T13:59:54.327822+0000","last_fullsized":"2024-03-26T13:59:54.327822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755747+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T13:59:54.755747+0000","last_peered":"2024-03-26T13:59:54.755747+0000","last_clean":"2024-03-26T13:59:54.755747+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T13:59:54.755747+0000","last_undegraded":"2024-03-26T13:59:54.755747+0000","last_fullsized":"2024-03-26T13:59:54.755747+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:59.288831+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T13:59:59.288831+0000","last_peered":"2024-03-26T13:59:59.288831+0000","last_clean":"2024-03-26T13:59:59.288831+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T13:59:59.288831+0000","last_undegraded":"2024-03-26T13:59:59.288831+0000","last_fullsized":"2024-03-26T13:59:59.288831+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754499+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T13:59:54.754499+0000","last_peered":"2024-03-26T13:59:54.754499+0000","last_clean":"2024-03-26T13:59:54.754499+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T13:59:54.754499+0000","last_undegraded":"2024-03-26T13:59:54.754499+0000","last_fullsized":"2024-03-26T13:59:54.754499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327602+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T13:59:54.327602+0000","last_peered":"2024-03-26T13:59:54.327602+0000","last_clean":"2024-03-26T13:59:54.327602+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T13:59:54.327602+0000","last_undegraded":"2024-03-26T13:59:54.327602+0000","last_fullsized":"2024-03-26T13:59:54.327602+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330446+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T13:59:54.330446+0000","last_peered":"2024-03-26T13:59:54.330446+0000","last_clean":"2024-03-26T13:59:54.330446+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T13:59:54.330446+0000","last_undegraded":"2024-03-26T13:59:54.330446+0000","last_fullsized":"2024-03-26T13:59:54.330446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761347+0000","last_change":"2024-03-26T13:59:54.761347+0000","last_active":"2024-03-26T13:59:54.761347+0000","last_peered":"2024-03-26T13:59:54.761347+0000","last_clean":"2024-03-26T13:59:54.761347+0000","last_became_active":"2024-03-26T13:59:54.760966+0000","last_became_peered":"2024-03-26T13:59:54.760966+0000","last_unstale":"2024-03-26T13:59:54.761347+0000","last_undegraded":"2024-03-26T13:59:54.761347+0000","last_fullsized":"2024-03-26T13:59:54.761347+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:27:13.241398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326627+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T13:59:54.326627+0000","last_peered":"2024-03-26T13:59:54.326627+0000","last_clean":"2024-03-26T13:59:54.326627+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T13:59:54.326627+0000","last_undegraded":"2024-03-26T13:59:54.326627+0000","last_fullsized":"2024-03-26T13:59:54.326627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.760687+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T13:59:54.760687+0000","last_peered":"2024-03-26T13:59:54.760687+0000","last_clean":"2024-03-26T13:59:54.760687+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T13:59:54.760687+0000","last_undegraded":"2024-03-26T13:59:54.760687+0000","last_fullsized":"2024-03-26T13:59:54.760687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754283+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T13:59:54.754283+0000","last_peered":"2024-03-26T13:59:54.754283+0000","last_clean":"2024-03-26T13:59:54.754283+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T13:59:54.754283+0000","last_undegraded":"2024-03-26T13:59:54.754283+0000","last_fullsized":"2024-03-26T13:59:54.754283+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.328197+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T13:59:54.328197+0000","last_peered":"2024-03-26T13:59:54.328197+0000","last_clean":"2024-03-26T13:59:54.328197+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T13:59:54.328197+0000","last_undegraded":"2024-03-26T13:59:54.328197+0000","last_fullsized":"2024-03-26T13:59:54.328197+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'31","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T14:00:00.295668+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T14:00:00.295668+0000","last_peered":"2024-03-26T14:00:00.295668+0000","last_clean":"2024-03-26T14:00:00.295668+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T14:00:00.295668+0000","last_undegraded":"2024-03-26T14:00:00.295668+0000","last_fullsized":"2024-03-26T14:00:00.295668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62378,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":75,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.340153+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T13:59:54.340153+0000","last_peered":"2024-03-26T13:59:54.340153+0000","last_clean":"2024-03-26T13:59:54.340153+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T13:59:54.340153+0000","last_undegraded":"2024-03-26T13:59:54.340153+0000","last_fullsized":"2024-03-26T13:59:54.340153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761598+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T13:59:54.761598+0000","last_peered":"2024-03-26T13:59:54.761598+0000","last_clean":"2024-03-26T13:59:54.761598+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T13:59:54.761598+0000","last_undegraded":"2024-03-26T13:59:54.761598+0000","last_fullsized":"2024-03-26T13:59:54.761598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.336926+0000","last_change":"2024-03-26T13:59:22.281532+0000","last_active":"2024-03-26T13:59:54.336926+0000","last_peered":"2024-03-26T13:59:54.336926+0000","last_clean":"2024-03-26T13:59:54.336926+0000","last_became_active":"2024-03-26T13:59:22.280763+0000","last_became_peered":"2024-03-26T13:59:22.280763+0000","last_unstale":"2024-03-26T13:59:54.336926+0000","last_undegraded":"2024-03-26T13:59:54.336926+0000","last_fullsized":"2024-03-26T13:59:54.336926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:07:20.640800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756449+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T13:59:54.756449+0000","last_peered":"2024-03-26T13:59:54.756449+0000","last_clean":"2024-03-26T13:59:54.756449+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T13:59:54.756449+0000","last_undegraded":"2024-03-26T13:59:54.756449+0000","last_fullsized":"2024-03-26T13:59:54.756449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332076+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T13:59:54.332076+0000","last_peered":"2024-03-26T13:59:54.332076+0000","last_clean":"2024-03-26T13:59:54.332076+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T13:59:54.332076+0000","last_undegraded":"2024-03-26T13:59:54.332076+0000","last_fullsized":"2024-03-26T13:59:54.332076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757435+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T13:59:54.757435+0000","last_peered":"2024-03-26T13:59:54.757435+0000","last_clean":"2024-03-26T13:59:54.757435+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T13:59:54.757435+0000","last_undegraded":"2024-03-26T13:59:54.757435+0000","last_fullsized":"2024-03-26T13:59:54.757435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330136+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T13:59:54.330136+0000","last_peered":"2024-03-26T13:59:54.330136+0000","last_clean":"2024-03-26T13:59:54.330136+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T13:59:54.330136+0000","last_undegraded":"2024-03-26T13:59:54.330136+0000","last_fullsized":"2024-03-26T13:59:54.330136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330531+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T13:59:54.330531+0000","last_peered":"2024-03-26T13:59:54.330531+0000","last_clean":"2024-03-26T13:59:54.330531+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T13:59:54.330531+0000","last_undegraded":"2024-03-26T13:59:54.330531+0000","last_fullsized":"2024-03-26T13:59:54.330531+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:57.287720+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T13:59:57.287720+0000","last_peered":"2024-03-26T13:59:57.287720+0000","last_clean":"2024-03-26T13:59:57.287720+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T13:59:57.287720+0000","last_undegraded":"2024-03-26T13:59:57.287720+0000","last_fullsized":"2024-03-26T13:59:57.287720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329932+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T13:59:54.329932+0000","last_peered":"2024-03-26T13:59:54.329932+0000","last_clean":"2024-03-26T13:59:54.329932+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T13:59:54.329932+0000","last_undegraded":"2024-03-26T13:59:54.329932+0000","last_fullsized":"2024-03-26T13:59:54.329932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758226+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T13:59:54.758226+0000","last_peered":"2024-03-26T13:59:54.758226+0000","last_clean":"2024-03-26T13:59:54.758226+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T13:59:54.758226+0000","last_undegraded":"2024-03-26T13:59:54.758226+0000","last_fullsized":"2024-03-26T13:59:54.758226+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330561+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T13:59:54.330561+0000","last_peered":"2024-03-26T13:59:54.330561+0000","last_clean":"2024-03-26T13:59:54.330561+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T13:59:54.330561+0000","last_undegraded":"2024-03-26T13:59:54.330561+0000","last_fullsized":"2024-03-26T13:59:54.330561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756514+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T13:59:54.756514+0000","last_peered":"2024-03-26T13:59:54.756514+0000","last_clean":"2024-03-26T13:59:54.756514+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T13:59:54.756514+0000","last_undegraded":"2024-03-26T13:59:54.756514+0000","last_fullsized":"2024-03-26T13:59:54.756514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756211+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T13:59:54.756211+0000","last_peered":"2024-03-26T13:59:54.756211+0000","last_clean":"2024-03-26T13:59:54.756211+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T13:59:54.756211+0000","last_undegraded":"2024-03-26T13:59:54.756211+0000","last_fullsized":"2024-03-26T13:59:54.756211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.341535+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T13:59:54.341535+0000","last_peered":"2024-03-26T13:59:54.341535+0000","last_clean":"2024-03-26T13:59:54.341535+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T13:59:54.341535+0000","last_undegraded":"2024-03-26T13:59:54.341535+0000","last_fullsized":"2024-03-26T13:59:54.341535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761409+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T13:59:54.761409+0000","last_peered":"2024-03-26T13:59:54.761409+0000","last_clean":"2024-03-26T13:59:54.761409+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T13:59:54.761409+0000","last_undegraded":"2024-03-26T13:59:54.761409+0000","last_fullsized":"2024-03-26T13:59:54.761409+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.334043+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T13:59:54.334043+0000","last_peered":"2024-03-26T13:59:54.334043+0000","last_clean":"2024-03-26T13:59:54.334043+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T13:59:54.334043+0000","last_undegraded":"2024-03-26T13:59:54.334043+0000","last_fullsized":"2024-03-26T13:59:54.334043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329309+0000","last_change":"2024-03-26T13:59:22.279783+0000","last_active":"2024-03-26T13:59:54.329309+0000","last_peered":"2024-03-26T13:59:54.329309+0000","last_clean":"2024-03-26T13:59:54.329309+0000","last_became_active":"2024-03-26T13:59:22.279377+0000","last_became_peered":"2024-03-26T13:59:22.279377+0000","last_unstale":"2024-03-26T13:59:54.329309+0000","last_undegraded":"2024-03-26T13:59:54.329309+0000","last_fullsized":"2024-03-26T13:59:54.329309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:55:48.429324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703899+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T13:59:54.703899+0000","last_peered":"2024-03-26T13:59:54.703899+0000","last_clean":"2024-03-26T13:59:54.703899+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T13:59:54.703899+0000","last_undegraded":"2024-03-26T13:59:54.703899+0000","last_fullsized":"2024-03-26T13:59:54.703899+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757312+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T13:59:54.757312+0000","last_peered":"2024-03-26T13:59:54.757312+0000","last_clean":"2024-03-26T13:59:54.757312+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T13:59:54.757312+0000","last_undegraded":"2024-03-26T13:59:54.757312+0000","last_fullsized":"2024-03-26T13:59:54.757312+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329838+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T13:59:54.329838+0000","last_peered":"2024-03-26T13:59:54.329838+0000","last_clean":"2024-03-26T13:59:54.329838+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T13:59:54.329838+0000","last_undegraded":"2024-03-26T13:59:54.329838+0000","last_fullsized":"2024-03-26T13:59:54.329838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.703337+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T13:59:54.703337+0000","last_peered":"2024-03-26T13:59:54.703337+0000","last_clean":"2024-03-26T13:59:54.703337+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T13:59:54.703337+0000","last_undegraded":"2024-03-26T13:59:54.703337+0000","last_fullsized":"2024-03-26T13:59:54.703337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758035+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T13:59:54.758035+0000","last_peered":"2024-03-26T13:59:54.758035+0000","last_clean":"2024-03-26T13:59:54.758035+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T13:59:54.758035+0000","last_undegraded":"2024-03-26T13:59:54.758035+0000","last_fullsized":"2024-03-26T13:59:54.758035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757574+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T13:59:54.757574+0000","last_peered":"2024-03-26T13:59:54.757574+0000","last_clean":"2024-03-26T13:59:54.757574+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T13:59:54.757574+0000","last_undegraded":"2024-03-26T13:59:54.757574+0000","last_fullsized":"2024-03-26T13:59:54.757574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T14:00:00.294431+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T14:00:00.294431+0000","last_peered":"2024-03-26T14:00:00.294431+0000","last_clean":"2024-03-26T14:00:00.294431+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T14:00:00.294431+0000","last_undegraded":"2024-03-26T14:00:00.294431+0000","last_fullsized":"2024-03-26T14:00:00.294431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330006+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T13:59:54.330006+0000","last_peered":"2024-03-26T13:59:54.330006+0000","last_clean":"2024-03-26T13:59:54.330006+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T13:59:54.330006+0000","last_undegraded":"2024-03-26T13:59:54.330006+0000","last_fullsized":"2024-03-26T13:59:54.330006+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756304+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T13:59:54.756304+0000","last_peered":"2024-03-26T13:59:54.756304+0000","last_clean":"2024-03-26T13:59:54.756304+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T13:59:54.756304+0000","last_undegraded":"2024-03-26T13:59:54.756304+0000","last_fullsized":"2024-03-26T13:59:54.756304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326508+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T13:59:54.326508+0000","last_peered":"2024-03-26T13:59:54.326508+0000","last_clean":"2024-03-26T13:59:54.326508+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T13:59:54.326508+0000","last_undegraded":"2024-03-26T13:59:54.326508+0000","last_fullsized":"2024-03-26T13:59:54.326508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":352,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329716+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T13:59:54.329716+0000","last_peered":"2024-03-26T13:59:54.329716+0000","last_clean":"2024-03-26T13:59:54.329716+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T13:59:54.329716+0000","last_undegraded":"2024-03-26T13:59:54.329716+0000","last_fullsized":"2024-03-26T13:59:54.329716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.764260+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T13:59:54.764260+0000","last_peered":"2024-03-26T13:59:54.764260+0000","last_clean":"2024-03-26T13:59:54.764260+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T13:59:54.764260+0000","last_undegraded":"2024-03-26T13:59:54.764260+0000","last_fullsized":"2024-03-26T13:59:54.764260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756478+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T13:59:54.756478+0000","last_peered":"2024-03-26T13:59:54.756478+0000","last_clean":"2024-03-26T13:59:54.756478+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T13:59:54.756478+0000","last_undegraded":"2024-03-26T13:59:54.756478+0000","last_fullsized":"2024-03-26T13:59:54.756478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329794+0000","last_change":"2024-03-26T13:59:22.283900+0000","last_active":"2024-03-26T13:59:54.329794+0000","last_peered":"2024-03-26T13:59:54.329794+0000","last_clean":"2024-03-26T13:59:54.329794+0000","last_became_active":"2024-03-26T13:59:22.282813+0000","last_became_peered":"2024-03-26T13:59:22.282813+0000","last_unstale":"2024-03-26T13:59:54.329794+0000","last_undegraded":"2024-03-26T13:59:54.329794+0000","last_fullsized":"2024-03-26T13:59:54.329794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:02:55.536620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331714+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T13:59:54.331714+0000","last_peered":"2024-03-26T13:59:54.331714+0000","last_clean":"2024-03-26T13:59:54.331714+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T13:59:54.331714+0000","last_undegraded":"2024-03-26T13:59:54.331714+0000","last_fullsized":"2024-03-26T13:59:54.331714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339975+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T13:59:54.339975+0000","last_peered":"2024-03-26T13:59:54.339975+0000","last_clean":"2024-03-26T13:59:54.339975+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T13:59:54.339975+0000","last_undegraded":"2024-03-26T13:59:54.339975+0000","last_fullsized":"2024-03-26T13:59:54.339975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757504+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T13:59:54.757504+0000","last_peered":"2024-03-26T13:59:54.757504+0000","last_clean":"2024-03-26T13:59:54.757504+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T13:59:54.757504+0000","last_undegraded":"2024-03-26T13:59:54.757504+0000","last_fullsized":"2024-03-26T13:59:54.757504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755154+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T13:59:54.755154+0000","last_peered":"2024-03-26T13:59:54.755154+0000","last_clean":"2024-03-26T13:59:54.755154+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T13:59:54.755154+0000","last_undegraded":"2024-03-26T13:59:54.755154+0000","last_fullsized":"2024-03-26T13:59:54.755154+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:57.329180+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T13:59:57.329180+0000","last_peered":"2024-03-26T13:59:57.329180+0000","last_clean":"2024-03-26T13:59:57.329180+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T13:59:57.329180+0000","last_undegraded":"2024-03-26T13:59:57.329180+0000","last_fullsized":"2024-03-26T13:59:57.329180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755545+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T13:59:54.755545+0000","last_peered":"2024-03-26T13:59:54.755545+0000","last_clean":"2024-03-26T13:59:54.755545+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T13:59:54.755545+0000","last_undegraded":"2024-03-26T13:59:54.755545+0000","last_fullsized":"2024-03-26T13:59:54.755545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.337460+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T13:59:54.337460+0000","last_peered":"2024-03-26T13:59:54.337460+0000","last_clean":"2024-03-26T13:59:54.337460+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T13:59:54.337460+0000","last_undegraded":"2024-03-26T13:59:54.337460+0000","last_fullsized":"2024-03-26T13:59:54.337460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.327246+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T13:59:54.327246+0000","last_peered":"2024-03-26T13:59:54.327246+0000","last_clean":"2024-03-26T13:59:54.327246+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T13:59:54.327246+0000","last_undegraded":"2024-03-26T13:59:54.327246+0000","last_fullsized":"2024-03-26T13:59:54.327246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330244+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T13:59:54.330244+0000","last_peered":"2024-03-26T13:59:54.330244+0000","last_clean":"2024-03-26T13:59:54.330244+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T13:59:54.330244+0000","last_undegraded":"2024-03-26T13:59:54.330244+0000","last_fullsized":"2024-03-26T13:59:54.330244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326884+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T13:59:54.326884+0000","last_peered":"2024-03-26T13:59:54.326884+0000","last_clean":"2024-03-26T13:59:54.326884+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T13:59:54.326884+0000","last_undegraded":"2024-03-26T13:59:54.326884+0000","last_fullsized":"2024-03-26T13:59:54.326884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753877+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T13:59:54.753877+0000","last_peered":"2024-03-26T13:59:54.753877+0000","last_clean":"2024-03-26T13:59:54.753877+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T13:59:54.753877+0000","last_undegraded":"2024-03-26T13:59:54.753877+0000","last_fullsized":"2024-03-26T13:59:54.753877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331172+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T13:59:54.331172+0000","last_peered":"2024-03-26T13:59:54.331172+0000","last_clean":"2024-03-26T13:59:54.331172+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T13:59:54.331172+0000","last_undegraded":"2024-03-26T13:59:54.331172+0000","last_fullsized":"2024-03-26T13:59:54.331172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.768353+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T13:59:54.768353+0000","last_peered":"2024-03-26T13:59:54.768353+0000","last_clean":"2024-03-26T13:59:54.768353+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T13:59:54.768353+0000","last_undegraded":"2024-03-26T13:59:54.768353+0000","last_fullsized":"2024-03-26T13:59:54.768353+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755911+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T13:59:54.755911+0000","last_peered":"2024-03-26T13:59:54.755911+0000","last_clean":"2024-03-26T13:59:54.755911+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T13:59:54.755911+0000","last_undegraded":"2024-03-26T13:59:54.755911+0000","last_fullsized":"2024-03-26T13:59:54.755911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331265+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T13:59:54.331265+0000","last_peered":"2024-03-26T13:59:54.331265+0000","last_clean":"2024-03-26T13:59:54.331265+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T13:59:54.331265+0000","last_undegraded":"2024-03-26T13:59:54.331265+0000","last_fullsized":"2024-03-26T13:59:54.331265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.714081+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T13:59:54.714081+0000","last_peered":"2024-03-26T13:59:54.714081+0000","last_clean":"2024-03-26T13:59:54.714081+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T13:59:54.714081+0000","last_undegraded":"2024-03-26T13:59:54.714081+0000","last_fullsized":"2024-03-26T13:59:54.714081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339742+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T13:59:54.339742+0000","last_peered":"2024-03-26T13:59:54.339742+0000","last_clean":"2024-03-26T13:59:54.339742+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T13:59:54.339742+0000","last_undegraded":"2024-03-26T13:59:54.339742+0000","last_fullsized":"2024-03-26T13:59:54.339742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756671+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T13:59:54.756671+0000","last_peered":"2024-03-26T13:59:54.756671+0000","last_clean":"2024-03-26T13:59:54.756671+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T13:59:54.756671+0000","last_undegraded":"2024-03-26T13:59:54.756671+0000","last_fullsized":"2024-03-26T13:59:54.756671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.326864+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T13:59:54.326864+0000","last_peered":"2024-03-26T13:59:54.326864+0000","last_clean":"2024-03-26T13:59:54.326864+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T13:59:54.326864+0000","last_undegraded":"2024-03-26T13:59:54.326864+0000","last_fullsized":"2024-03-26T13:59:54.326864+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333249+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T13:59:54.333249+0000","last_peered":"2024-03-26T13:59:54.333249+0000","last_clean":"2024-03-26T13:59:54.333249+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T13:59:54.333249+0000","last_undegraded":"2024-03-26T13:59:54.333249+0000","last_fullsized":"2024-03-26T13:59:54.333249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.329708+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T13:59:54.329708+0000","last_peered":"2024-03-26T13:59:54.329708+0000","last_clean":"2024-03-26T13:59:54.329708+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T13:59:54.329708+0000","last_undegraded":"2024-03-26T13:59:54.329708+0000","last_fullsized":"2024-03-26T13:59:54.329708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.753959+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T13:59:54.753959+0000","last_peered":"2024-03-26T13:59:54.753959+0000","last_clean":"2024-03-26T13:59:54.753959+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T13:59:54.753959+0000","last_undegraded":"2024-03-26T13:59:54.753959+0000","last_fullsized":"2024-03-26T13:59:54.753959+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.338291+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T13:59:54.338291+0000","last_peered":"2024-03-26T13:59:54.338291+0000","last_clean":"2024-03-26T13:59:54.338291+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T13:59:54.338291+0000","last_undegraded":"2024-03-26T13:59:54.338291+0000","last_fullsized":"2024-03-26T13:59:54.338291+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330394+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T13:59:54.330394+0000","last_peered":"2024-03-26T13:59:54.330394+0000","last_clean":"2024-03-26T13:59:54.330394+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T13:59:54.330394+0000","last_undegraded":"2024-03-26T13:59:54.330394+0000","last_fullsized":"2024-03-26T13:59:54.330394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.331478+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T13:59:54.331478+0000","last_peered":"2024-03-26T13:59:54.331478+0000","last_clean":"2024-03-26T13:59:54.331478+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T13:59:54.331478+0000","last_undegraded":"2024-03-26T13:59:54.331478+0000","last_fullsized":"2024-03-26T13:59:54.331478+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.761247+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T13:59:54.761247+0000","last_peered":"2024-03-26T13:59:54.761247+0000","last_clean":"2024-03-26T13:59:54.761247+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T13:59:54.761247+0000","last_undegraded":"2024-03-26T13:59:54.761247+0000","last_fullsized":"2024-03-26T13:59:54.761247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330119+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T13:59:54.330119+0000","last_peered":"2024-03-26T13:59:54.330119+0000","last_clean":"2024-03-26T13:59:54.330119+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T13:59:54.330119+0000","last_undegraded":"2024-03-26T13:59:54.330119+0000","last_fullsized":"2024-03-26T13:59:54.330119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756124+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T13:59:54.756124+0000","last_peered":"2024-03-26T13:59:54.756124+0000","last_clean":"2024-03-26T13:59:54.756124+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T13:59:54.756124+0000","last_undegraded":"2024-03-26T13:59:54.756124+0000","last_fullsized":"2024-03-26T13:59:54.756124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.758106+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T13:59:54.758106+0000","last_peered":"2024-03-26T13:59:54.758106+0000","last_clean":"2024-03-26T13:59:54.758106+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T13:59:54.758106+0000","last_undegraded":"2024-03-26T13:59:54.758106+0000","last_fullsized":"2024-03-26T13:59:54.758106+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.330912+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T13:59:54.330912+0000","last_peered":"2024-03-26T13:59:54.330912+0000","last_clean":"2024-03-26T13:59:54.330912+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T13:59:54.330912+0000","last_undegraded":"2024-03-26T13:59:54.330912+0000","last_fullsized":"2024-03-26T13:59:54.330912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332068+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T13:59:54.332068+0000","last_peered":"2024-03-26T13:59:54.332068+0000","last_clean":"2024-03-26T13:59:54.332068+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T13:59:54.332068+0000","last_undegraded":"2024-03-26T13:59:54.332068+0000","last_fullsized":"2024-03-26T13:59:54.332068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.338037+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T13:59:54.338037+0000","last_peered":"2024-03-26T13:59:54.338037+0000","last_clean":"2024-03-26T13:59:54.338037+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T13:59:54.338037+0000","last_undegraded":"2024-03-26T13:59:54.338037+0000","last_fullsized":"2024-03-26T13:59:54.338037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.339394+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T13:59:54.339394+0000","last_peered":"2024-03-26T13:59:54.339394+0000","last_clean":"2024-03-26T13:59:54.339394+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T13:59:54.339394+0000","last_undegraded":"2024-03-26T13:59:54.339394+0000","last_fullsized":"2024-03-26T13:59:54.339394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.332280+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T13:59:54.332280+0000","last_peered":"2024-03-26T13:59:54.332280+0000","last_clean":"2024-03-26T13:59:54.332280+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T13:59:54.332280+0000","last_undegraded":"2024-03-26T13:59:54.332280+0000","last_fullsized":"2024-03-26T13:59:54.332280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757622+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T13:59:54.757622+0000","last_peered":"2024-03-26T13:59:54.757622+0000","last_clean":"2024-03-26T13:59:54.757622+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T13:59:54.757622+0000","last_undegraded":"2024-03-26T13:59:54.757622+0000","last_fullsized":"2024-03-26T13:59:54.757622+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.333876+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T13:59:54.333876+0000","last_peered":"2024-03-26T13:59:54.333876+0000","last_clean":"2024-03-26T13:59:54.333876+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T13:59:54.333876+0000","last_undegraded":"2024-03-26T13:59:54.333876+0000","last_fullsized":"2024-03-26T13:59:54.333876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.337727+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T13:59:54.337727+0000","last_peered":"2024-03-26T13:59:54.337727+0000","last_clean":"2024-03-26T13:59:54.337727+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T13:59:54.337727+0000","last_undegraded":"2024-03-26T13:59:54.337727+0000","last_fullsized":"2024-03-26T13:59:54.337727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.755777+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T13:59:54.755777+0000","last_peered":"2024-03-26T13:59:54.755777+0000","last_clean":"2024-03-26T13:59:54.755777+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T13:59:54.755777+0000","last_undegraded":"2024-03-26T13:59:54.755777+0000","last_fullsized":"2024-03-26T13:59:54.755777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.757431+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T13:59:54.757431+0000","last_peered":"2024-03-26T13:59:54.757431+0000","last_clean":"2024-03-26T13:59:54.757431+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T13:59:54.757431+0000","last_undegraded":"2024-03-26T13:59:54.757431+0000","last_fullsized":"2024-03-26T13:59:54.757431+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756044+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T13:59:54.756044+0000","last_peered":"2024-03-26T13:59:54.756044+0000","last_clean":"2024-03-26T13:59:54.756044+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T13:59:54.756044+0000","last_undegraded":"2024-03-26T13:59:54.756044+0000","last_fullsized":"2024-03-26T13:59:54.756044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.754447+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T13:59:54.754447+0000","last_peered":"2024-03-26T13:59:54.754447+0000","last_clean":"2024-03-26T13:59:54.754447+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T13:59:54.754447+0000","last_undegraded":"2024-03-26T13:59:54.754447+0000","last_fullsized":"2024-03-26T13:59:54.754447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.756832+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T13:59:54.756832+0000","last_peered":"2024-03-26T13:59:54.756832+0000","last_clean":"2024-03-26T13:59:54.756832+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T13:59:54.756832+0000","last_undegraded":"2024-03-26T13:59:54.756832+0000","last_fullsized":"2024-03-26T13:59:54.756832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-26T13:59:54.346428+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T13:59:54.346428+0000","last_peered":"2024-03-26T13:59:54.346428+0000","last_clean":"2024-03-26T13:59:54.346428+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T13:59:54.346428+0000","last_undegraded":"2024-03-26T13:59:54.346428+0000","last_fullsized":"2024-03-26T13:59:54.346428+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66379,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":91,"num_read_kb":117,"num_write":159,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":199137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1003520,"data_stored":503866,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1155072,"data_stored":1157999,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776165,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":504102,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":504022,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":1099931,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":499278,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1069056,"data_stored":561566,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":500666,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":562916,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149775,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":499472,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888136,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":504319,"data_compressed":20674,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1077248,"data_stored":1095128,"data_compressed":56329,"data_compressed_allocated":73728,"data_compressed_original":647168,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:00:04.791 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-26T14:00:04.792 INFO:tasks.cephadm:Setting up client nodes... 2024-03-26T14:00:04.792 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-26T14:00:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:06.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:06 smithi179 ceph-mon[35472]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:06.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:06 smithi179 ceph-mon[35472]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.8 KiB/s wr, 8 op/s; 10 B/s, 0 objects/s recovering 2024-03-26T14:00:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:06 smithi159 ceph-mon[35501]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:06 smithi159 ceph-mon[35501]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.8 KiB/s wr, 8 op/s; 10 B/s, 0 objects/s recovering 2024-03-26T14:00:06.453 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:06 smithi137 ceph-mon[31539]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:00:06.453 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:06 smithi137 ceph-mon[31539]: pgmap v388: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.8 KiB/s wr, 8 op/s; 10 B/s, 0 objects/s recovering 2024-03-26T14:00:07.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 -- 172.21.15.137:0/4149647265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 msgr2=0x7ffae4109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:07.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 --2- 172.21.15.137:0/4149647265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae4109e70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffad00099c0 tx=0x7ffad0009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 -- 172.21.15.137:0/4149647265 shutdown_connections 2024-03-26T14:00:07.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 --2- 172.21.15.137:0/4149647265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffae410a3b0 0x7ffae4110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 --2- 172.21.15.137:0/4149647265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae4108810 0x7ffae4108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 --2- 172.21.15.137:0/4149647265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae4109e70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 -- 172.21.15.137:0/4149647265 >> 172.21.15.137:0/4149647265 conn(0x7ffae40777e0 msgr2=0x7ffae4079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:07.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.636+0000 7ffaea600700 1 -- 172.21.15.137:0/4149647265 shutdown_connections 2024-03-26T14:00:07.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.637+0000 7ffaea600700 1 -- 172.21.15.137:0/4149647265 wait complete. 2024-03-26T14:00:07.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.638+0000 7ffaea600700 1 Processor -- start 2024-03-26T14:00:07.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.638+0000 7ffaea600700 1 -- start start 2024-03-26T14:00:07.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffaea600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae4108810 0x7ffae419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:07.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffaea600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:07.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffaea600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffae410a3b0 0x7ffae41a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:07.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffaea600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41142f0 con 0x7ffae4109a10 2024-03-26T14:00:07.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffaea600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4114170 con 0x7ffae410a3b0 2024-03-26T14:00:07.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffaea600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4114470 con 0x7ffae4108810 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffae2a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffae2a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae419ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54844/0 (socket says 172.21.15.137:54844) 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffae2a00700 1 -- 172.21.15.137:0/2956711702 learned_addr learned my addr 172.21.15.137:0/2956711702 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffae3e00700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffae410a3b0 0x7ffae41a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.639+0000 7ffae3400700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae4108810 0x7ffae419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffae2a00700 1 -- 172.21.15.137:0/2956711702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae4108810 msgr2=0x7ffae419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffae2a00700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae4108810 0x7ffae419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffae2a00700 1 -- 172.21.15.137:0/2956711702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffae410a3b0 msgr2=0x7ffae41a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffae2a00700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffae410a3b0 0x7ffae41a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffae2a00700 1 -- 172.21.15.137:0/2956711702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae41a6530 con 0x7ffae4109a10 2024-03-26T14:00:07.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffae2a00700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae419ee10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffad000b580 tx=0x7ffad00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:07.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad0027070 con 0x7ffae4109a10 2024-03-26T14:00:07.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffad0004190 con 0x7ffae4109a10 2024-03-26T14:00:07.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad000f460 con 0x7ffae4109a10 2024-03-26T14:00:07.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.640+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae41a67c0 con 0x7ffae4109a10 2024-03-26T14:00:07.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.641+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41a6bc0 con 0x7ffae4109a10 2024-03-26T14:00:07.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.642+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae40506e0 con 0x7ffae4109a10 2024-03-26T14:00:07.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.644+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7ffad002a070 con 0x7ffae4109a10 2024-03-26T14:00:07.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.645+0000 7ffadbe00700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffac80a2a50 0x7ffac80a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:07.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.645+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7ffad00d0070 con 0x7ffae4109a10 2024-03-26T14:00:07.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.645+0000 7ffae3400700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffac80a2a50 0x7ffac80a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:07.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.646+0000 7ffae3400700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffac80a2a50 0x7ffac80a4f10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffad4005fd0 tx=0x7ffad400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:07.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.650+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffad005e740 con 0x7ffae4109a10 2024-03-26T14:00:07.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.956+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ffae402dc80 con 0x7ffae4109a10 2024-03-26T14:00:07.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.960+0000 7ffadbe00700 1 -- 172.21.15.137:0/2956711702 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) v1 ==== 170+0+59 (secure 0 0 0) 0x7ffad0013070 con 0x7ffae4109a10 2024-03-26T14:00:07.958 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-03-26T14:00:07.958 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQDn1AJmIfUqORAAle6hzbXoKqzCCTk+jqk1KQ== 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffac80a2a50 msgr2=0x7ffac80a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffac80a2a50 0x7ffac80a4f10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffad4005fd0 tx=0x7ffad400a560 comp rx=0 tx=0).stop 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 msgr2=0x7ffae419ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae419ee10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffad000b580 tx=0x7ffad00057c0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 shutdown_connections 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffac80a2a50 0x7ffac80a4f10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffae410a3b0 0x7ffae41a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffae4108810 0x7ffae419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 --2- 172.21.15.137:0/2956711702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffae4109a10 0x7ffae419ee10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 >> 172.21.15.137:0/2956711702 conn(0x7ffae40777e0 msgr2=0x7ffae410cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 shutdown_connections 2024-03-26T14:00:07.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:07.963+0000 7ffaea600700 1 -- 172.21.15.137:0/2956711702 wait complete. 2024-03-26T14:00:08.281 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:08 smithi137 ceph-mon[31539]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-26T14:00:08.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2956711702' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T14:00:08.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2956711702' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T14:00:08.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:08 smithi159 ceph-mon[35501]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-26T14:00:08.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2956711702' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T14:00:08.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2956711702' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T14:00:08.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:08 smithi179 ceph-mon[35472]: pgmap v389: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s; 9 B/s, 0 objects/s recovering 2024-03-26T14:00:08.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2956711702' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-26T14:00:08.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2956711702' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-26T14:00:08.859 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T14:00:08.859 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-26T14:00:08.859 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-26T14:00:09.000 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-26T14:00:09.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-26T14:00:09.000 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph mgr dump --format=json 2024-03-26T14:00:09.482 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:10.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:10 smithi137 ceph-mon[31539]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-03-26T14:00:10.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:10 smithi159 ceph-mon[35501]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-03-26T14:00:10.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:10 smithi179 ceph-mon[35472]: pgmap v390: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s; 9 B/s, 0 objects/s recovering 2024-03-26T14:00:11.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.667+0000 7f01df400700 1 -- 172.21.15.137:0/2310695229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81045e0 msgr2=0x7f01d81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:11.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.667+0000 7f01df400700 1 --2- 172.21.15.137:0/2310695229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81045e0 0x7f01d81049c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0099c0 tx=0x7f01cc009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 -- 172.21.15.137:0/2310695229 shutdown_connections 2024-03-26T14:00:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 --2- 172.21.15.137:0/2310695229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d81096c0 0x7f01d8111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 --2- 172.21.15.137:0/2310695229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81045e0 0x7f01d81049c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 --2- 172.21.15.137:0/2310695229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d8104f90 0x7f01d8108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 -- 172.21.15.137:0/2310695229 >> 172.21.15.137:0/2310695229 conn(0x7f01d80ffe70 msgr2=0x7f01d8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 -- 172.21.15.137:0/2310695229 shutdown_connections 2024-03-26T14:00:11.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.668+0000 7f01df400700 1 -- 172.21.15.137:0/2310695229 wait complete. 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.669+0000 7f01df400700 1 Processor -- start 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.669+0000 7f01df400700 1 -- start start 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01df400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 0x7f01d81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01df400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d8104f90 0x7f01d81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01df400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81096c0 0x7f01d819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01df400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81145d0 con 0x7f01d81096c0 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01df400700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8114450 con 0x7f01d81045e0 2024-03-26T14:00:11.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01df400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8114750 con 0x7f01d8104f90 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01dca00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 0x7f01d81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01dd400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81096c0 0x7f01d819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01dca00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 0x7f01d81a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:43210/0 (socket says 172.21.15.137:43210) 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.670+0000 7f01dca00700 1 -- 172.21.15.137:0/704669595 learned_addr learned my addr 172.21.15.137:0/704669595 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01d7e00700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d8104f90 0x7f01d81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dd400700 1 -- 172.21.15.137:0/704669595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d8104f90 msgr2=0x7f01d81a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dd400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d8104f90 0x7f01d81a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dd400700 1 -- 172.21.15.137:0/704669595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 msgr2=0x7f01d81a0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dd400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 0x7f01d81a0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dd400700 1 -- 172.21.15.137:0/704669595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d819b070 con 0x7f01d81096c0 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01d7e00700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d8104f90 0x7f01d81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dca00700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 0x7f01d81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.671+0000 7f01dd400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81096c0 0x7f01d819a910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01d0007f10 tx=0x7f01d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:11.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.672+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000ddd0 con 0x7f01d81096c0 2024-03-26T14:00:11.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.672+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01d0009e20 con 0x7f01d81096c0 2024-03-26T14:00:11.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.672+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0015400 con 0x7f01d81096c0 2024-03-26T14:00:11.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.672+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d819b360 con 0x7f01d81096c0 2024-03-26T14:00:11.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.672+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81a75d0 con 0x7f01d81096c0 2024-03-26T14:00:11.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.673+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8068e50 con 0x7f01d81096c0 2024-03-26T14:00:11.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.675+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f01d0004ad0 con 0x7f01d81096c0 2024-03-26T14:00:11.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.676+0000 7f01d5600700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01c00a29a0 0x7f01c00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:11.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.676+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f01d00ce840 con 0x7f01d81096c0 2024-03-26T14:00:11.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.677+0000 7f01dca00700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01c00a29a0 0x7f01c00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:11.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.677+0000 7f01dca00700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01c00a29a0 0x7f01c00a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f01cc003760 tx=0x7f01cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:11.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.681+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f01d0010070 con 0x7f01d81096c0 2024-03-26T14:00:11.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.964+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f01d802dbf0 con 0x7f01d81096c0 2024-03-26T14:00:11.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.973+0000 7f01d5600700 1 -- 172.21.15.137:0/704669595 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262669 (secure 0 0 0) 0x7f01d009b420 con 0x7f01d81096c0 2024-03-26T14:00:11.972 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.977+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01c00a29a0 msgr2=0x7f01c00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.977+0000 7f01df400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01c00a29a0 0x7f01c00a4e60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f01cc003760 tx=0x7f01cc005fd0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81096c0 msgr2=0x7f01d819a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81096c0 0x7f01d819a910 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01d0007f10 tx=0x7f01d000d3b0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 shutdown_connections 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01c00a29a0 0x7f01c00a4e60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01d81096c0 0x7f01d819a910 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01d81045e0 0x7f01d81a0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 --2- 172.21.15.137:0/704669595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01d8104f90 0x7f01d81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 >> 172.21.15.137:0/704669595 conn(0x7f01d80ffe70 msgr2=0x7f01d8102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 shutdown_connections 2024-03-26T14:00:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:11.978+0000 7f01df400700 1 -- 172.21.15.137:0/704669595 wait complete. 2024-03-26T14:00:12.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:12 smithi137 ceph-mon[31539]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:00:12.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/704669595' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T14:00:12.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:12 smithi159 ceph-mon[35501]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:00:12.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/704669595' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T14:00:12.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:12 smithi179 ceph-mon[35472]: pgmap v391: 129 pgs: 129 active+clean; 642 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:00:12.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/704669595' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-26T14:00:12.856 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3167395503},{"type":"v1","addr":"172.21.15.137:6801","nonce":3167395503}]},"active_addr":"172.21.15.137:6801/3167395503","active_change":"2024-03-26T13:49:52.910706+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14223,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.137:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":1857852421}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":1371432069}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":406354822}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":1685729166}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.137:0","nonce":3665569811}]}]} 2024-03-26T14:00:12.862 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-26T14:00:12.862 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-26T14:00:12.862 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T14:00:13.310 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:14.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:14 smithi159 ceph-mon[35501]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-26T14:00:14.488 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:14 smithi137 ceph-mon[31539]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-26T14:00:14.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:14 smithi179 ceph-mon[35472]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-03-26T14:00:15.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 -- 172.21.15.137:0/771226001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c1081e0 msgr2=0x7f8a4c1085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:15.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 --2- 172.21.15.137:0/771226001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c1081e0 0x7f8a4c1085c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0099c0 tx=0x7f8a3c009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 -- 172.21.15.137:0/771226001 shutdown_connections 2024-03-26T14:00:15.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 --2- 172.21.15.137:0/771226001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 --2- 172.21.15.137:0/771226001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c1081e0 0x7f8a4c1085c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 --2- 172.21.15.137:0/771226001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a4c069e40 0x7f8a4c06a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 -- 172.21.15.137:0/771226001 >> 172.21.15.137:0/771226001 conn(0x7f8a4c077fc0 msgr2=0x7f8a4c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:15.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 -- 172.21.15.137:0/771226001 shutdown_connections 2024-03-26T14:00:15.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.559+0000 7f8a53200700 1 -- 172.21.15.137:0/771226001 wait complete. 2024-03-26T14:00:15.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.560+0000 7f8a53200700 1 Processor -- start 2024-03-26T14:00:15.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.560+0000 7f8a53200700 1 -- start start 2024-03-26T14:00:15.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a53200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c069e40 0x7f8a4c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a53200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a53200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a4c1081e0 0x7f8a4c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a53200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c1122f0 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a53200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c112170 con 0x7f8a4c069e40 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a53200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c112470 con 0x7f8a4c1081e0 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c19eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54452/0 (socket says 172.21.15.137:54452) 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 -- 172.21.15.137:0/3134427134 learned_addr learned my addr 172.21.15.137:0/3134427134 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:15.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a4be00700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c069e40 0x7f8a4c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a51200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a4c1081e0 0x7f8a4c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 -- 172.21.15.137:0/3134427134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a4c1081e0 msgr2=0x7f8a4c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a4c1081e0 0x7f8a4c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 -- 172.21.15.137:0/3134427134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c069e40 msgr2=0x7f8a4c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c069e40 0x7f8a4c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.561+0000 7f8a43400700 1 -- 172.21.15.137:0/3134427134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c198f10 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a4be00700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c069e40 0x7f8a4c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a43400700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c19eaa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400d980 tx=0x7f8a3400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34017070 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c199200 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a34004830 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c1a54e0 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.562+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340053e0 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.563+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c103890 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.565+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f8a34005580 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.567+0000 7f8a49600700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a300a2920 0x7f8a300a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:15.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.567+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f8a340cecc0 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.567+0000 7f8a4be00700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a300a2920 0x7f8a300a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:15.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.568+0000 7f8a4be00700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a300a2920 0x7f8a300a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0059d0 tx=0x7f8a3c005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:15.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.572+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8a340d5080 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.796+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a4c199b60 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.799+0000 7f8a49600700 1 -- 172.21.15.137:0/3134427134 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24668 (secure 0 0 0) 0x7f8a34020020 con 0x7f8a4c06a7e0 2024-03-26T14:00:15.796 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:15.796 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":92,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:54.308336+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:00:15.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a300a2920 msgr2=0x7f8a300a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a300a2920 0x7f8a300a4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0059d0 tx=0x7f8a3c005940 comp rx=0 tx=0).stop 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 msgr2=0x7f8a4c19eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c19eaa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400d980 tx=0x7f8a3400dd40 comp rx=0 tx=0).stop 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 shutdown_connections 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a300a2920 0x7f8a300a4de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a4c1081e0 0x7f8a4c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a4c069e40 0x7f8a4c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 --2- 172.21.15.137:0/3134427134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a4c06a7e0 0x7f8a4c19eaa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 >> 172.21.15.137:0/3134427134 conn(0x7f8a4c077fc0 msgr2=0x7f8a4c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 shutdown_connections 2024-03-26T14:00:15.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:15.802+0000 7f8a53200700 1 -- 172.21.15.137:0/3134427134 wait complete. 2024-03-26T14:00:16.198 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:16 smithi137 ceph-mon[31539]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-26T14:00:16.198 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:16 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3134427134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:00:16.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:16 smithi159 ceph-mon[35501]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-26T14:00:16.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:16 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3134427134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:00:16.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:16 smithi179 ceph-mon[35472]: pgmap v393: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-26T14:00:16.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:16 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3134427134' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:00:16.759 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-26T14:00:16.759 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd dump --format=json 2024-03-26T14:00:17.217 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:18.453 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:18 smithi137 ceph-mon[31539]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-26T14:00:18.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:18 smithi179 ceph-mon[35472]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-26T14:00:18.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:18 smithi159 ceph-mon[35501]: pgmap v394: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-26T14:00:19.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 -- 172.21.15.137:0/1816898457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1096c0 msgr2=0x7f915c111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 --2- 172.21.15.137:0/1816898457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1096c0 0x7f915c111290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f915400b0d0 tx=0x7f915400b3e0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 -- 172.21.15.137:0/1816898457 shutdown_connections 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 --2- 172.21.15.137:0/1816898457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1096c0 0x7f915c111290 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 --2- 172.21.15.137:0/1816898457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f915c1045e0 0x7f915c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 --2- 172.21.15.137:0/1816898457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c104f90 0x7f915c108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 -- 172.21.15.137:0/1816898457 >> 172.21.15.137:0/1816898457 conn(0x7f915c0ffe90 msgr2=0x7f915c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 -- 172.21.15.137:0/1816898457 shutdown_connections 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.590+0000 7f9161800700 1 -- 172.21.15.137:0/1816898457 wait complete. 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.591+0000 7f9161800700 1 Processor -- start 2024-03-26T14:00:19.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.591+0000 7f9161800700 1 -- start start 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f9161800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 0x7f915c1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f9161800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f915c104f90 0x7f915c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f9161800700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c1096c0 0x7f915c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f9161800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c1145b0 con 0x7f915c1045e0 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f9161800700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c114430 con 0x7f915c1096c0 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f9161800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915c114730 con 0x7f915c104f90 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 0x7f915c1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 0x7f915c1a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54170/0 (socket says 172.21.15.137:54170) 2024-03-26T14:00:19.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 -- 172.21.15.137:0/3094718298 learned_addr learned my addr 172.21.15.137:0/3094718298 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915b400700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c1096c0 0x7f915c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915a000700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f915c104f90 0x7f915c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 -- 172.21.15.137:0/3094718298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f915c104f90 msgr2=0x7f915c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f915c104f90 0x7f915c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 -- 172.21.15.137:0/3094718298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c1096c0 msgr2=0x7f915c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.592+0000 7f915aa00700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c1096c0 0x7f915c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f915aa00700 1 -- 172.21.15.137:0/3094718298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c19b030 con 0x7f915c1045e0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f915b400700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c1096c0 0x7f915c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f915aa00700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 0x7f915c1a0680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c9b0 tx=0x7f914c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c007b00 con 0x7f915c1045e0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f914c007ca0 con 0x7f915c1045e0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915c19b320 con 0x7f915c1045e0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c018730 con 0x7f915c1045e0 2024-03-26T14:00:19.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.593+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915c1a74e0 con 0x7f915c1045e0 2024-03-26T14:00:19.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.595+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f915c105ac0 con 0x7f915c1045e0 2024-03-26T14:00:19.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.597+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f914c00f450 con 0x7f915c1045e0 2024-03-26T14:00:19.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.598+0000 7f9153400700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91400a2920 0x7f91400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:19.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.598+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f914c0cefe0 con 0x7f915c1045e0 2024-03-26T14:00:19.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.598+0000 7f915a000700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91400a2920 0x7f91400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:19.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.599+0000 7f915a000700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91400a2920 0x7f91400a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9148005950 tx=0x7f91480058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:19.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.603+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f914c014030 con 0x7f915c1045e0 2024-03-26T14:00:19.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.823+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f915c02da30 con 0x7f915c1045e0 2024-03-26T14:00:19.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.825+0000 7f9153400700 1 -- 172.21.15.137:0/3094718298 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24668 (secure 0 0 0) 0x7f914c09bbc0 con 0x7f915c1045e0 2024-03-26T14:00:19.823 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:00:19.823 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":92,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T13:59:54.308336+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:00:19.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91400a2920 msgr2=0x7f91400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:19.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91400a2920 0x7f91400a4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9148005950 tx=0x7f91480058e0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 msgr2=0x7f915c1a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:19.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 0x7f915c1a0680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c9b0 tx=0x7f914c00cd70 comp rx=0 tx=0).stop 2024-03-26T14:00:19.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 shutdown_connections 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91400a2920 0x7f91400a4de0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f915c1096c0 0x7f915c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f915c1045e0 0x7f915c1a0680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 --2- 172.21.15.137:0/3094718298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f915c104f90 0x7f915c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 >> 172.21.15.137:0/3094718298 conn(0x7f915c0ffe90 msgr2=0x7f915c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.829+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 shutdown_connections 2024-03-26T14:00:19.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:19.830+0000 7f9161800700 1 -- 172.21.15.137:0/3094718298 wait complete. 2024-03-26T14:00:20.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:20 smithi137 ceph-mon[31539]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-26T14:00:20.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:20 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3094718298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:00:20.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:20 smithi179 ceph-mon[35472]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-26T14:00:20.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:20 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3094718298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:00:20.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:20 smithi159 ceph-mon[35501]: pgmap v395: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-03-26T14:00:20.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:20 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3094718298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:00:20.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-26T14:00:20.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-26T14:00:20.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-26T14:00:20.718 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-26T14:00:20.719 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-26T14:00:20.720 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-26T14:00:20.720 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-26T14:00:20.721 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-26T14:00:20.722 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-26T14:00:20.723 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-26T14:00:20.723 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-26T14:00:20.724 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-26T14:00:21.916 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.916 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.925 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.930 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.932 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.933 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.966 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.973 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.976 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.983 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.990 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:21.994 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:22.404 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:22 smithi137 ceph-mon[31539]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-26T14:00:22.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:22 smithi179 ceph-mon[35472]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-26T14:00:22.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:22 smithi159 ceph-mon[35501]: pgmap v396: 129 pgs: 129 active+clean; 645 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-26T14:00:24.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:24 smithi137 ceph-mon[31539]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:00:24.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:24 smithi179 ceph-mon[35472]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:00:24.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:24 smithi159 ceph-mon[35501]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:00:26.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:26 smithi137 ceph-mon[31539]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:00:26.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:26 smithi179 ceph-mon[35472]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:00:26.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:26 smithi159 ceph-mon[35501]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:00:27.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:00:27.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:27.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:27.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:00:27.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:27.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:27.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:00:27.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:27.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:28.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:28 smithi137 ceph-mon[31539]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-26T14:00:28.617 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:28 smithi159 ceph-mon[35501]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-26T14:00:28.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:28 smithi179 ceph-mon[35472]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-26T14:00:30.449 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-26T14:00:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.450 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-03-26T14:00:30.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:00:31.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:31 smithi137 ceph-mon[31539]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:00:31.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:31 smithi179 ceph-mon[35472]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:00:31.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:31 smithi159 ceph-mon[35501]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:00:31.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 -- 172.21.15.137:0/2002363286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888113360 msgr2=0x7fe888115750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:31.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 --2- 172.21.15.137:0/2002363286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888113360 0x7fe888115750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe88000b0d0 tx=0x7fe88000b3e0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 -- 172.21.15.137:0/2002363286 shutdown_connections 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 --2- 172.21.15.137:0/2002363286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888113360 0x7fe888115750 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 --2- 172.21.15.137:0/2002363286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8880743c0 0x7fe8880747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 --2- 172.21.15.137:0/2002363286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe888074ce0 0x7fe888112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 -- 172.21.15.137:0/2002363286 >> 172.21.15.137:0/2002363286 conn(0x7fe8880792f0 msgr2=0x7fe888079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 -- 172.21.15.137:0/2002363286 shutdown_connections 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.509+0000 7fe88f800700 1 -- 172.21.15.137:0/2002363286 wait complete. 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.510+0000 7fe88f800700 1 Processor -- start 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.510+0000 7fe88f800700 1 -- start start 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88f800700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8880743c0 0x7fe88819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88f800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888074ce0 0x7fe88819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88f800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe888113360 0x7fe8881a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88f800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe888118a60 con 0x7fe888074ce0 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88f800700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881188e0 con 0x7fe8880743c0 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88f800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe888118be0 con 0x7fe888113360 2024-03-26T14:00:31.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88ce00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8880743c0 0x7fe88819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe887e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888074ce0 0x7fe88819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88ce00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8880743c0 0x7fe88819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37142/0 (socket says 172.21.15.137:37142) 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88ce00700 1 -- 172.21.15.137:0/1198031573 learned_addr learned my addr 172.21.15.137:0/1198031573 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.511+0000 7fe88d800700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe888113360 0x7fe8881a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe887e00700 1 -- 172.21.15.137:0/1198031573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe888113360 msgr2=0x7fe8881a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe887e00700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe888113360 0x7fe8881a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe887e00700 1 -- 172.21.15.137:0/1198031573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8880743c0 msgr2=0x7fe88819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe887e00700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8880743c0 0x7fe88819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe887e00700 1 -- 172.21.15.137:0/1198031573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8881a3de0 con 0x7fe888074ce0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe887e00700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888074ce0 0x7fe88819f310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c940 tx=0x7fe87400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874004220 con 0x7fe888074ce0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8881a40d0 con 0x7fe888074ce0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe87400ce30 con 0x7fe888074ce0 2024-03-26T14:00:31.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874004df0 con 0x7fe888074ce0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.512+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8881ab950 con 0x7fe888074ce0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.514+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe888112a30 con 0x7fe888074ce0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.516+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fe874007900 con 0x7fe888074ce0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.517+0000 7fe885600700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8700a2920 0x7fe8700a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.517+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fe8740ceea0 con 0x7fe888074ce0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.517+0000 7fe88ce00700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8700a2920 0x7fe8700a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.517+0000 7fe885600700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] conn(0x7fe8700ada80 0x7fe8700aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.517+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 --> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8700b05f0 con 0x7fe8700ada80 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.517+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8740cf250 con 0x7fe888074ce0 2024-03-26T14:00:31.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.518+0000 7fe88d800700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] conn(0x7fe8700ada80 0x7fe8700aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.518+0000 7fe88ce00700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8700a2920 0x7fe8700a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ad20 tx=0x7fe87c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.518+0000 7fe88d800700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] conn(0x7fe8700ada80 0x7fe8700aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.520+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== osd.3 v2:172.21.15.137:6810/3541368776 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe8700b05f0 con 0x7fe8700ada80 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.537+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 --> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe888075af0 con 0x7fe8700ada80 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe885600700 1 -- 172.21.15.137:0/1198031573 <== osd.3 v2:172.21.15.137:6810/3541368776 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe888075af0 con 0x7fe8700ada80 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] conn(0x7fe8700ada80 msgr2=0x7fe8700aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88f800700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] conn(0x7fe8700ada80 0x7fe8700aff20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8700a2920 msgr2=0x7fe8700a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88f800700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8700a2920 0x7fe8700a4de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ad20 tx=0x7fe87c005f90 comp rx=0 tx=0).stop 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888074ce0 msgr2=0x7fe88819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88f800700 1 --2- 172.21.15.137:0/1198031573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe888074ce0 0x7fe88819f310 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c940 tx=0x7fe87400cc50 comp rx=0 tx=0).stop 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.538+0000 7fe88d800700 1 -- 172.21.15.137:0/1198031573 reap_dead start 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.539+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 shutdown_connections 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.539+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 >> 172.21.15.137:0/1198031573 conn(0x7fe8880792f0 msgr2=0x7fe888112050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.539+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 shutdown_connections 2024-03-26T14:00:31.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:30.539+0000 7fe88f800700 1 -- 172.21.15.137:0/1198031573 wait complete. 2024-03-26T14:00:34.421 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:34 smithi159 ceph-mon[35501]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-26T14:00:34.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:34 smithi179 ceph-mon[35472]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-26T14:00:34.509 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:34 smithi137 ceph-mon[31539]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-03-26T14:00:35.203 INFO:teuthology.orchestra.run.smithi137.stdout:111669149781 2024-03-26T14:00:35.203 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-26T14:00:36.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 -- 172.21.15.137:0/1569116406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c104f90 msgr2=0x7fad1c108fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 --2- 172.21.15.137:0/1569116406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c104f90 0x7fad1c108fe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fad080099c0 tx=0x7fad08009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 -- 172.21.15.137:0/1569116406 shutdown_connections 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 --2- 172.21.15.137:0/1569116406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c1096c0 0x7fad1c111290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 --2- 172.21.15.137:0/1569116406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1045e0 0x7fad1c1049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 --2- 172.21.15.137:0/1569116406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c104f90 0x7fad1c108fe0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 -- 172.21.15.137:0/1569116406 >> 172.21.15.137:0/1569116406 conn(0x7fad1c0ffe90 msgr2=0x7fad1c1022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 -- 172.21.15.137:0/1569116406 shutdown_connections 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.326+0000 7fad23400700 1 -- 172.21.15.137:0/1569116406 wait complete. 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.327+0000 7fad23400700 1 Processor -- start 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.327+0000 7fad23400700 1 -- start start 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.327+0000 7fad23400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c1045e0 0x7fad1c1a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.327+0000 7fad23400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c104f90 0x7fad1c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad23400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1096c0 0x7fad1c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad23400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c114530 con 0x7fad1c1045e0 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad23400700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c1143b0 con 0x7fad1c104f90 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad23400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c1146b0 con 0x7fad1c1096c0 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c104f90 0x7fad1c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c104f90 0x7fad1c1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37196/0 (socket says 172.21.15.137:37196) 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 -- 172.21.15.137:0/2636906583 learned_addr learned my addr 172.21.15.137:0/2636906583 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad21400700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1096c0 0x7fad1c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad20a00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c1045e0 0x7fad1c1a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 -- 172.21.15.137:0/2636906583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1096c0 msgr2=0x7fad1c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c1096c0 0x7fad1c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 -- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c1045e0 msgr2=0x7fad1c1a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c1045e0 0x7fad1c1a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad1be00700 1 -- 172.21.15.137:0/2636906583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c19b010 con 0x7fad1c104f90 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.328+0000 7fad20a00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad1c1045e0 0x7fad1c1a06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.329+0000 7fad1be00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c104f90 0x7fad1c1a0c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad080045e0 tx=0x7fad080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:36.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.329+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08026070 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.329+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fad0800f460 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.329+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c19b2a0 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.329+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08009e60 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.330+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c1a75c0 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.331+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad1c062920 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.333+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fad08029070 con 0x7fad1c104f90 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.334+0000 7fad19600700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fad040a2920 0x7fad040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.334+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fad080cfa70 con 0x7fad1c104f90 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.334+0000 7fad20a00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fad040a2920 0x7fad040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.334+0000 7fad19600700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] conn(0x7fad040ad950 0x7fad040afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.334+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 --> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad08018e50 con 0x7fad040ad950 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.334+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad080cfe20 con 0x7fad1c104f90 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.335+0000 7fad21400700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] conn(0x7fad040ad950 0x7fad040afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.335+0000 7fad20a00700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fad040a2920 0x7fad040a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad10005fd0 tx=0x7fad10005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.335+0000 7fad21400700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] conn(0x7fad040ad950 0x7fad040afdf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.336+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== osd.9 v2:172.21.15.137:6826/3720717247 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fad08018e50 con 0x7fad040ad950 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.354+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 --> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad1c062da0 con 0x7fad040ad950 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.355+0000 7fad19600700 1 -- 172.21.15.137:0/2636906583 <== osd.9 v2:172.21.15.137:6826/3720717247 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fad1c062da0 con 0x7fad040ad950 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] conn(0x7fad040ad950 msgr2=0x7fad040afdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] conn(0x7fad040ad950 0x7fad040afdf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fad040a2920 msgr2=0x7fad040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fad040a2920 0x7fad040a4de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad10005fd0 tx=0x7fad10005f00 comp rx=0 tx=0).stop 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c104f90 msgr2=0x7fad1c1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 --2- 172.21.15.137:0/2636906583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c104f90 0x7fad1c1a0c30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad080045e0 tx=0x7fad080046c0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad21400700 1 -- 172.21.15.137:0/2636906583 reap_dead start 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 shutdown_connections 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 >> 172.21.15.137:0/2636906583 conn(0x7fad1c0ffe90 msgr2=0x7fad1c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 shutdown_connections 2024-03-26T14:00:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.356+0000 7fad23400700 1 -- 172.21.15.137:0/2636906583 wait complete. 2024-03-26T14:00:36.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:36 smithi179 ceph-mon[35472]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:36.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:36 smithi159 ceph-mon[35501]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:36.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:36 smithi137 ceph-mon[31539]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:36.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 -- 172.21.15.137:0/2151403669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc10eb90 msgr2=0x7fa9fc110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:36.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 --2- 172.21.15.137:0/2151403669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc10eb90 0x7fa9fc110f80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400b0a0 tx=0x7fa9f400b3b0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 -- 172.21.15.137:0/2151403669 shutdown_connections 2024-03-26T14:00:36.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 --2- 172.21.15.137:0/2151403669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc10eb90 0x7fa9fc110f80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 --2- 172.21.15.137:0/2151403669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9fc0743c0 0x7fa9fc0747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 --2- 172.21.15.137:0/2151403669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9fc074ce0 0x7fa9fc10e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 -- 172.21.15.137:0/2151403669 >> 172.21.15.137:0/2151403669 conn(0x7fa9fc0792f0 msgr2=0x7fa9fc079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 -- 172.21.15.137:0/2151403669 shutdown_connections 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.222+0000 7faa00e00700 1 -- 172.21.15.137:0/2151403669 wait complete. 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.223+0000 7faa00e00700 1 Processor -- start 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.223+0000 7faa00e00700 1 -- start start 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.223+0000 7faa00e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9fc0743c0 0x7fa9fc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7faa00e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc074ce0 0x7fa9fc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7faa00e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9fc10eb90 0x7fa9fc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7faa00e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc114650 con 0x7fa9fc074ce0 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7faa00e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc1144d0 con 0x7fa9fc0743c0 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7faa00e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc1147d0 con 0x7fa9fc10eb90 2024-03-26T14:00:36.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc074ce0 0x7fa9fc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc074ce0 0x7fa9fc1a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50020/0 (socket says 172.21.15.137:50020) 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 -- 172.21.15.137:0/4241518522 learned_addr learned my addr 172.21.15.137:0/4241518522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9e00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9fc0743c0 0x7fa9fc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9fa800700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9fc10eb90 0x7fa9fc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 -- 172.21.15.137:0/4241518522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9fc10eb90 msgr2=0x7fa9fc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9fc10eb90 0x7fa9fc19a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 -- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9fc0743c0 msgr2=0x7fa9fc1a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9fc0743c0 0x7fa9fc1a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.224+0000 7fa9f9400700 1 -- 172.21.15.137:0/4241518522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc19b030 con 0x7fa9fc074ce0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.225+0000 7fa9f9400700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc074ce0 0x7fa9fc1a0c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800d980 tx=0x7fa9e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.225+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8017070 con 0x7fa9fc074ce0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.225+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9e8005020 con 0x7fa9fc074ce0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.225+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc19b320 con 0x7fa9fc074ce0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.225+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8009b50 con 0x7fa9fc074ce0 2024-03-26T14:00:37.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.225+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc1a7590 con 0x7fa9fc074ce0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.226+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9fc062860 con 0x7fa9fc074ce0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.228+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa9e8009cf0 con 0x7fa9fc074ce0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.230+0000 7fa9f2a00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa9e00a29a0 0x7fa9e00a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.230+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa9e80cf3e0 con 0x7fa9fc074ce0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.230+0000 7fa9f9e00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa9e00a29a0 0x7fa9e00a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.230+0000 7fa9f2a00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] conn(0x7fa9e00adb00 0x7fa9e00affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.230+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 --> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9e00b0670 con 0x7fa9e00adb00 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.230+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9e80cf790 con 0x7fa9fc074ce0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.231+0000 7fa9fa800700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] conn(0x7fa9e00adb00 0x7fa9e00affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.231+0000 7fa9f9e00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa9e00a29a0 0x7fa9e00a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc101730 tx=0x7fa9ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.231+0000 7fa9fa800700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] conn(0x7fa9e00adb00 0x7fa9e00affa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.233+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== osd.10 v2:172.21.15.159:6824/2917781375 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa9e00b0670 con 0x7fa9e00adb00 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.251+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 --> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9fc062c60 con 0x7fa9e00adb00 2024-03-26T14:00:37.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7fa9f2a00700 1 -- 172.21.15.137:0/4241518522 <== osd.10 v2:172.21.15.159:6824/2917781375 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa9fc062c60 con 0x7fa9e00adb00 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] conn(0x7fa9e00adb00 msgr2=0x7fa9e00affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7faa00e00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] conn(0x7fa9e00adb00 0x7fa9e00affa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa9e00a29a0 msgr2=0x7fa9e00a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7faa00e00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa9e00a29a0 0x7fa9e00a4e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc101730 tx=0x7fa9ec009260 comp rx=0 tx=0).stop 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc074ce0 msgr2=0x7fa9fc1a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7faa00e00700 1 --2- 172.21.15.137:0/4241518522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa9fc074ce0 0x7fa9fc1a0c00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800d980 tx=0x7fa9e800dc90 comp rx=0 tx=0).stop 2024-03-26T14:00:37.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.252+0000 7fa9fa800700 1 -- 172.21.15.137:0/4241518522 reap_dead start 2024-03-26T14:00:37.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.253+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 shutdown_connections 2024-03-26T14:00:37.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.253+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 >> 172.21.15.137:0/4241518522 conn(0x7fa9fc0792f0 msgr2=0x7fa9fc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:37.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.253+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 shutdown_connections 2024-03-26T14:00:37.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.253+0000 7faa00e00700 1 -- 172.21.15.137:0/4241518522 wait complete. 2024-03-26T14:00:38.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 -- 172.21.15.137:0/1408328007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88041087a0 msgr2=0x7f8804108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 --2- 172.21.15.137:0/1408328007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88041087a0 0x7f8804108c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009990 tx=0x7f87f0009ca0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 -- 172.21.15.137:0/1408328007 shutdown_connections 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 --2- 172.21.15.137:0/1408328007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8804109160 0x7f8804115670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 --2- 172.21.15.137:0/1408328007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f880410e7a0 0x7f880410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 --2- 172.21.15.137:0/1408328007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88041087a0 0x7f8804108c20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 -- 172.21.15.137:0/1408328007 >> 172.21.15.137:0/1408328007 conn(0x7f8804075960 msgr2=0x7f8804077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.162+0000 7f880b800700 1 -- 172.21.15.137:0/1408328007 shutdown_connections 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.163+0000 7f880b800700 1 -- 172.21.15.137:0/1408328007 wait complete. 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 Processor -- start 2024-03-26T14:00:38.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 -- start start 2024-03-26T14:00:38.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88041087a0 0x7f880419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8804109160 0x7f880419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f880410e7a0 0x7f88041a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804118a30 con 0x7f8804109160 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88041188b0 con 0x7f880410e7a0 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.164+0000 7f880b800700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8804118bb0 con 0x7f88041087a0 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8804109160 0x7f880419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8804109160 0x7f880419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60760/0 (socket says 172.21.15.137:60760) 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 -- 172.21.15.137:0/1947873982 learned_addr learned my addr 172.21.15.137:0/1947873982 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8808e00700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88041087a0 0x7f880419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8809800700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f880410e7a0 0x7f88041a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 -- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88041087a0 msgr2=0x7f880419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88041087a0 0x7f880419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 -- 172.21.15.137:0/1947873982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f880410e7a0 msgr2=0x7f88041a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f880410e7a0 0x7f88041a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.165+0000 7f8803e00700 1 -- 172.21.15.137:0/1947873982 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88041a3ea0 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.166+0000 7f8803e00700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8804109160 0x7f880419f420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000c00 tx=0x7f87f00057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.166+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0027070 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.166+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88041a4130 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.166+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f87f0022470 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.166+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017650 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.166+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88041aba10 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.167+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8804062920 con 0x7f8804109160 2024-03-26T14:00:38.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.169+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f87f002a070 con 0x7f8804109160 2024-03-26T14:00:38.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.171+0000 7f8801600700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f87ec0a2920 0x7f87ec0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:38.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.171+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f87f00d0280 con 0x7f8804109160 2024-03-26T14:00:38.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.171+0000 7f8808e00700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f87ec0a2920 0x7f87ec0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:38.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.171+0000 7f8801600700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] conn(0x7f87ec0ada80 0x7f87ec0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:38.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.171+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 --> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87ec0b05f0 con 0x7f87ec0ada80 2024-03-26T14:00:38.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.171+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f00d0630 con 0x7f8804109160 2024-03-26T14:00:38.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.172+0000 7f8809800700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] conn(0x7f87ec0ada80 0x7f87ec0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:38.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.172+0000 7f8808e00700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f87ec0a2920 0x7f87ec0a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009be0 tx=0x7f87f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:38.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.173+0000 7f8809800700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] conn(0x7f87ec0ada80 0x7f87ec0aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:38.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.175+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== osd.11 v2:172.21.15.179:6824/1258425145 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f87ec0b05f0 con 0x7f87ec0ada80 2024-03-26T14:00:38.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.194+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 --> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8804062da0 con 0x7f87ec0ada80 2024-03-26T14:00:38.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.195+0000 7f8801600700 1 -- 172.21.15.137:0/1947873982 <== osd.11 v2:172.21.15.179:6824/1258425145 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8804062da0 con 0x7f87ec0ada80 2024-03-26T14:00:38.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.195+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] conn(0x7f87ec0ada80 msgr2=0x7f87ec0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:38.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.195+0000 7f880b800700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] conn(0x7f87ec0ada80 0x7f87ec0aff20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.195+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f87ec0a2920 msgr2=0x7f87ec0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f87ec0a2920 0x7f87ec0a4de0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009be0 tx=0x7f87f8009260 comp rx=0 tx=0).stop 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8804109160 msgr2=0x7f880419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 --2- 172.21.15.137:0/1947873982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8804109160 0x7f880419f420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87f0000c00 tx=0x7f87f00057a0 comp rx=0 tx=0).stop 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f8809800700 1 -- 172.21.15.137:0/1947873982 reap_dead start 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 shutdown_connections 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 >> 172.21.15.137:0/1947873982 conn(0x7f8804075960 msgr2=0x7f8804076080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 shutdown_connections 2024-03-26T14:00:38.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:38.196+0000 7f880b800700 1 -- 172.21.15.137:0/1947873982 wait complete. 2024-03-26T14:00:38.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:38 smithi179 ceph-mon[35472]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:00:38.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:38 smithi159 ceph-mon[35501]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:00:38.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:38 smithi137 ceph-mon[31539]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:00:40.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 -- 172.21.15.137:0/1276892770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e80104fa0 msgr2=0x7f2e80108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:40.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 --2- 172.21.15.137:0/1276892770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e80104fa0 0x7f2e80108ff0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0099c0 tx=0x7f2e6c009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 -- 172.21.15.137:0/1276892770 shutdown_connections 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 --2- 172.21.15.137:0/1276892770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e801096d0 0x7f2e801112a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 --2- 172.21.15.137:0/1276892770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e801045f0 0x7f2e801049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 --2- 172.21.15.137:0/1276892770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e80104fa0 0x7f2e80108ff0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 -- 172.21.15.137:0/1276892770 >> 172.21.15.137:0/1276892770 conn(0x7f2e800ffe60 msgr2=0x7f2e80102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 -- 172.21.15.137:0/1276892770 shutdown_connections 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.050+0000 7f2e86c00700 1 -- 172.21.15.137:0/1276892770 wait complete. 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.051+0000 7f2e86c00700 1 Processor -- start 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.051+0000 7f2e86c00700 1 -- start start 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e86c00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e801045f0 0x7f2e801a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e86c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e80104fa0 0x7f2e801a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:40.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e86c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e801096d0 0x7f2e8019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e86c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80114590 con 0x7f2e801096d0 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e86c00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80114410 con 0x7f2e801045f0 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e86c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80114710 con 0x7f2e80104fa0 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e801096d0 0x7f2e8019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e801096d0 0x7f2e8019a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50010/0 (socket says 172.21.15.137:50010) 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 -- 172.21.15.137:0/2852513782 learned_addr learned my addr 172.21.15.137:0/2852513782 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e7fe00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e801045f0 0x7f2e801a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e7f400700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e80104fa0 0x7f2e801a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 -- 172.21.15.137:0/2852513782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e80104fa0 msgr2=0x7f2e801a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e80104fa0 0x7f2e801a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 -- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e801045f0 msgr2=0x7f2e801a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e801045f0 0x7f2e801a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.052+0000 7f2e84c00700 1 -- 172.21.15.137:0/2852513782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8019b030 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.053+0000 7f2e84c00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e801096d0 0x7f2e8019a8d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2e78007ee0 tx=0x7f2e7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.053+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78004bb0 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.053+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e78004d50 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.053+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8019b320 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.053+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780058e0 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.053+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e801a7590 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.054+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e80062920 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.056+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f2e7801a020 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.058+0000 7f2e7cc00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e640a2920 0x7f2e640a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.058+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f2e780ced30 con 0x7f2e801096d0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.058+0000 7f2e7fe00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e640a2920 0x7f2e640a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.058+0000 7f2e7cc00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] conn(0x7f2e640ada80 0x7f2e640aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:40.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.058+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 --> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e7801ae50 con 0x7f2e640ada80 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.058+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e780cf0e0 con 0x7f2e801096d0 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.059+0000 7f2e7fe00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e640a2920 0x7f2e640a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e80100260 tx=0x7f2e7000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.059+0000 7f2e7f400700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] conn(0x7f2e640ada80 0x7f2e640aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.059+0000 7f2e7f400700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] conn(0x7f2e640ada80 0x7f2e640aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.062+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== osd.7 v2:172.21.15.159:6816/1788160471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2e7801ae50 con 0x7f2e640ada80 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.080+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 --> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e80062da0 con 0x7f2e640ada80 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.080+0000 7f2e7cc00700 1 -- 172.21.15.137:0/2852513782 <== osd.7 v2:172.21.15.159:6816/1788160471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2e80062da0 con 0x7f2e640ada80 2024-03-26T14:00:40.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] conn(0x7f2e640ada80 msgr2=0x7f2e640aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] conn(0x7f2e640ada80 0x7f2e640aff20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e640a2920 msgr2=0x7f2e640a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e640a2920 0x7f2e640a4de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e80100260 tx=0x7f2e7000b410 comp rx=0 tx=0).stop 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e801096d0 msgr2=0x7f2e8019a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 --2- 172.21.15.137:0/2852513782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e801096d0 0x7f2e8019a8d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2e78007ee0 tx=0x7f2e7800d3b0 comp rx=0 tx=0).stop 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e84c00700 1 -- 172.21.15.137:0/2852513782 reap_dead start 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 shutdown_connections 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 >> 172.21.15.137:0/2852513782 conn(0x7f2e800ffe60 msgr2=0x7f2e801021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.081+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 shutdown_connections 2024-03-26T14:00:40.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:36.082+0000 7f2e86c00700 1 -- 172.21.15.137:0/2852513782 wait complete. 2024-03-26T14:00:40.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:40 smithi159 ceph-mon[35501]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:40.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:40 smithi137 ceph-mon[31539]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:40.635 INFO:teuthology.orchestra.run.smithi137.stdout:313532612646 2024-03-26T14:00:40.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-26T14:00:40.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:40 smithi179 ceph-mon[35472]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 -- 172.21.15.137:0/926160290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878107510 msgr2=0x7fb87810f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 --2- 172.21.15.137:0/926160290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878107510 0x7fb87810f0e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b0d0 tx=0x7fb87000b3e0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 -- 172.21.15.137:0/926160290 shutdown_connections 2024-03-26T14:00:41.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 --2- 172.21.15.137:0/926160290 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878107510 0x7fb87810f0e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 --2- 172.21.15.137:0/926160290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb878102440 0x7fb878102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 --2- 172.21.15.137:0/926160290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb878102df0 0x7fb878106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 -- 172.21.15.137:0/926160290 >> 172.21.15.137:0/926160290 conn(0x7fb8780792f0 msgr2=0x7fb878079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:41.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 -- 172.21.15.137:0/926160290 shutdown_connections 2024-03-26T14:00:41.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.967+0000 7fb87d000700 1 -- 172.21.15.137:0/926160290 wait complete. 2024-03-26T14:00:41.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.968+0000 7fb87d000700 1 Processor -- start 2024-03-26T14:00:41.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 -- start start 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb878102440 0x7fb87819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878102df0 0x7fb87819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb878107510 0x7fb8781985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781123c0 con 0x7fb878102df0 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb878112240 con 0x7fb878102440 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb87d000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb878112540 con 0x7fb878107510 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb875400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878102df0 0x7fb87819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb875400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878102df0 0x7fb87819e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60800/0 (socket says 172.21.15.137:60800) 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb875400700 1 -- 172.21.15.137:0/1184699730 learned_addr learned my addr 172.21.15.137:0/1184699730 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.969+0000 7fb875e00700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb878102440 0x7fb87819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb876800700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb878107510 0x7fb8781985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb875400700 1 -- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb878107510 msgr2=0x7fb8781985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb875400700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb878107510 0x7fb8781985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb875400700 1 -- 172.21.15.137:0/1184699730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb878102440 msgr2=0x7fb87819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:41.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb875400700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb878102440 0x7fb87819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb875400700 1 -- 172.21.15.137:0/1184699730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb878198d40 con 0x7fb878102df0 2024-03-26T14:00:41.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.970+0000 7fb875400700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878102df0 0x7fb87819e960 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c9e0 tx=0x7fb86400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:41.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.971+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864007aa0 con 0x7fb878102df0 2024-03-26T14:00:41.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.971+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb878199030 con 0x7fb878102df0 2024-03-26T14:00:41.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.971+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb864007c40 con 0x7fb878102df0 2024-03-26T14:00:41.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.971+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864018660 con 0x7fb878102df0 2024-03-26T14:00:41.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.972+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8781a52f0 con 0x7fb878102df0 2024-03-26T14:00:41.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.972+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb878100af0 con 0x7fb878102df0 2024-03-26T14:00:41.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.974+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb86400f450 con 0x7fb878102df0 2024-03-26T14:00:41.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.976+0000 7fb86ea00700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb85c0a29a0 0x7fb85c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.976+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fb8640cee60 con 0x7fb878102df0 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.976+0000 7fb875e00700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb85c0a29a0 0x7fb85c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.976+0000 7fb86ea00700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] conn(0x7fb85c0adb00 0x7fb85c0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.976+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 --> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb864012ec0 con 0x7fb85c0adb00 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.976+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8640d1520 con 0x7fb878102df0 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.977+0000 7fb875e00700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb85c0a29a0 0x7fb85c0a4e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb868005950 tx=0x7fb86800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:41.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.977+0000 7fb876800700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] conn(0x7fb85c0adb00 0x7fb85c0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:41.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.978+0000 7fb876800700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] conn(0x7fb85c0adb00 0x7fb85c0affa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:41.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.980+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== osd.2 v2:172.21.15.179:6800/395733293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb864012ec0 con 0x7fb85c0adb00 2024-03-26T14:00:41.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.998+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 --> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb878103910 con 0x7fb85c0adb00 2024-03-26T14:00:41.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.999+0000 7fb86ea00700 1 -- 172.21.15.137:0/1184699730 <== osd.2 v2:172.21.15.179:6800/395733293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb878103910 con 0x7fb85c0adb00 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.999+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] conn(0x7fb85c0adb00 msgr2=0x7fb85c0affa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] conn(0x7fb85c0adb00 0x7fb85c0affa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb85c0a29a0 msgr2=0x7fb85c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb85c0a29a0 0x7fb85c0a4e60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb868005950 tx=0x7fb86800b410 comp rx=0 tx=0).stop 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878102df0 msgr2=0x7fb87819e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 --2- 172.21.15.137:0/1184699730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb878102df0 0x7fb87819e960 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c9e0 tx=0x7fb86400ccf0 comp rx=0 tx=0).stop 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb876800700 1 -- 172.21.15.137:0/1184699730 reap_dead start 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 shutdown_connections 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 >> 172.21.15.137:0/1184699730 conn(0x7fb8780792f0 msgr2=0x7fb878100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 shutdown_connections 2024-03-26T14:00:41.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.000+0000 7fb87d000700 1 -- 172.21.15.137:0/1184699730 wait complete. 2024-03-26T14:00:42.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:42 smithi159 ceph-mon[35501]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:00:42.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:42 smithi137 ceph-mon[31539]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:00:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 -- 172.21.15.137:0/3923091596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 msgr2=0x7fae0810f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 --2- 172.21.15.137:0/3923091596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0810f0e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae0000b0d0 tx=0x7fae0000b3e0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 -- 172.21.15.137:0/3923091596 shutdown_connections 2024-03-26T14:00:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 --2- 172.21.15.137:0/3923091596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0810f0e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 --2- 172.21.15.137:0/3923091596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae08102440 0x7fae08102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 --2- 172.21.15.137:0/3923091596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae08102df0 0x7fae08106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 -- 172.21.15.137:0/3923091596 >> 172.21.15.137:0/3923091596 conn(0x7fae080792f0 msgr2=0x7fae08079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 -- 172.21.15.137:0/3923091596 shutdown_connections 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.400+0000 7fae0f600700 1 -- 172.21.15.137:0/3923091596 wait complete. 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.401+0000 7fae0f600700 1 Processor -- start 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 -- start start 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae08102440 0x7fae081a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae08102df0 0x7fae081a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae081123e0 con 0x7fae08107510 2024-03-26T14:00:42.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08112260 con 0x7fae08102440 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0f600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae08112560 con 0x7fae08102df0 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0d600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0d600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60784/0 (socket says 172.21.15.137:60784) 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0d600700 1 -- 172.21.15.137:0/229716050 learned_addr learned my addr 172.21.15.137:0/229716050 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.402+0000 7fae0cc00700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae08102440 0x7fae081a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae07e00700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae08102df0 0x7fae081a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae0d600700 1 -- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae08102df0 msgr2=0x7fae081a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae0d600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae08102df0 0x7fae081a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae0d600700 1 -- 172.21.15.137:0/229716050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae08102440 msgr2=0x7fae081a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae0d600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae08102440 0x7fae081a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae0d600700 1 -- 172.21.15.137:0/229716050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0819b050 con 0x7fae08107510 2024-03-26T14:00:42.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.403+0000 7fae0d600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0819a8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fae00007a20 tx=0x7fae00007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.404+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0001f070 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.404+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae00003c00 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.404+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0819b2e0 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.404+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0001b440 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.404+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae081a7600 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.405+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae081009f0 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.407+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fae00022070 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.408+0000 7fae05600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadf00a2920 0x7fadf00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.408+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fae0000f020 con 0x7fae08107510 2024-03-26T14:00:42.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.408+0000 7fae0cc00700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadf00a2920 0x7fadf00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.409+0000 7fae05600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] conn(0x7fadf00ada80 0x7fadf00aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.409+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 --> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae0000cbc0 con 0x7fadf00ada80 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.409+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae000d0630 con 0x7fae08107510 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.409+0000 7fae07e00700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] conn(0x7fadf00ada80 0x7fadf00aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.409+0000 7fae0cc00700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadf00a2920 0x7fadf00a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae0819ba30 tx=0x7fadfc00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.410+0000 7fae07e00700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] conn(0x7fadf00ada80 0x7fadf00aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.412+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== osd.5 v2:172.21.15.179:6808/3906205051 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fae0000cbc0 con 0x7fadf00ada80 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.429+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 --> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fae08103910 con 0x7fadf00ada80 2024-03-26T14:00:42.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.430+0000 7fae05600700 1 -- 172.21.15.137:0/229716050 <== osd.5 v2:172.21.15.179:6808/3906205051 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fae08103910 con 0x7fadf00ada80 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] conn(0x7fadf00ada80 msgr2=0x7fadf00aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] conn(0x7fadf00ada80 0x7fadf00aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadf00a2920 msgr2=0x7fadf00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadf00a2920 0x7fadf00a4de0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae0819ba30 tx=0x7fadfc00b3c0 comp rx=0 tx=0).stop 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 msgr2=0x7fae0819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 --2- 172.21.15.137:0/229716050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae08107510 0x7fae0819a8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fae00007a20 tx=0x7fae00007a50 comp rx=0 tx=0).stop 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0d600700 1 -- 172.21.15.137:0/229716050 reap_dead start 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 shutdown_connections 2024-03-26T14:00:42.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 >> 172.21.15.137:0/229716050 conn(0x7fae080792f0 msgr2=0x7fae08100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:42.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 shutdown_connections 2024-03-26T14:00:42.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:41.431+0000 7fae0f600700 1 -- 172.21.15.137:0/229716050 wait complete. 2024-03-26T14:00:42.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:42 smithi179 ceph-mon[35472]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:00:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 -- 172.21.15.137:0/1753599560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98107510 msgr2=0x7fcc9810f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 --2- 172.21.15.137:0/1753599560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98107510 0x7fcc9810f0e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b0d0 tx=0x7fcc9000b3e0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 -- 172.21.15.137:0/1753599560 shutdown_connections 2024-03-26T14:00:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 --2- 172.21.15.137:0/1753599560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98107510 0x7fcc9810f0e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 --2- 172.21.15.137:0/1753599560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc98102440 0x7fcc98102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 --2- 172.21.15.137:0/1753599560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc98102df0 0x7fcc98106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 -- 172.21.15.137:0/1753599560 >> 172.21.15.137:0/1753599560 conn(0x7fcc980792f0 msgr2=0x7fcc98079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 -- 172.21.15.137:0/1753599560 shutdown_connections 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.370+0000 7fcca0600700 1 -- 172.21.15.137:0/1753599560 wait complete. 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.371+0000 7fcca0600700 1 Processor -- start 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.371+0000 7fcca0600700 1 -- start start 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcca0600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc98102440 0x7fcc981a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcca0600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98102df0 0x7fcc981a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcca0600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc98107510 0x7fcc9819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcca0600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98112410 con 0x7fcc98102df0 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcca0600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98112290 con 0x7fcc98102440 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcca0600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98112590 con 0x7fcc98107510 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98102df0 0x7fcc981a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98102df0 0x7fcc981a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60834/0 (socket says 172.21.15.137:60834) 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 -- 172.21.15.137:0/575638437 learned_addr learned my addr 172.21.15.137:0/575638437 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9dc00700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc98102440 0x7fcc981a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9e600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc98107510 0x7fcc9819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 -- 172.21.15.137:0/575638437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc98107510 msgr2=0x7fcc9819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc98107510 0x7fcc9819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 -- 172.21.15.137:0/575638437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc98102440 msgr2=0x7fcc981a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc98102440 0x7fcc981a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.372+0000 7fcc9d200700 1 -- 172.21.15.137:0/575638437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9819b080 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcc9e600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc98107510 0x7fcc9819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcc9dc00700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc98102440 0x7fcc981a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcc9d200700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98102df0 0x7fcc981a0c00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c8f0 tx=0x7fcc8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84007b00 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc84007ca0 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9819b370 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84018660 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.373+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc981a7590 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.374+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc9810e740 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.376+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fcc8400f450 con 0x7fcc98102df0 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.377+0000 7fcc96000700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcc800a2920 0x7fcc800a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.378+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fcc840cf1d0 con 0x7fcc98102df0 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.378+0000 7fcc9dc00700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcc800a2920 0x7fcc800a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.378+0000 7fcc96000700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] conn(0x7fcc800ada00 0x7fcc800afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.378+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 --> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc84012ec0 con 0x7fcc800ada00 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.378+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc840cf580 con 0x7fcc98102df0 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.378+0000 7fcc9e600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] conn(0x7fcc800ada00 0x7fcc800afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.379+0000 7fcc9dc00700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcc800a2920 0x7fcc800a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005fd0 tx=0x7fcc8c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.379+0000 7fcc9e600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] conn(0x7fcc800ada00 0x7fcc800afea0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.380+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== osd.6 v2:172.21.15.137:6818/102905149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcc84012ec0 con 0x7fcc800ada00 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.398+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 --> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcc98103910 con 0x7fcc800ada00 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.398+0000 7fcc96000700 1 -- 172.21.15.137:0/575638437 <== osd.6 v2:172.21.15.137:6818/102905149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcc98103910 con 0x7fcc800ada00 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] conn(0x7fcc800ada00 msgr2=0x7fcc800afea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] conn(0x7fcc800ada00 0x7fcc800afea0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcc800a2920 msgr2=0x7fcc800a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcc800a2920 0x7fcc800a4de0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005fd0 tx=0x7fcc8c005f60 comp rx=0 tx=0).stop 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98102df0 msgr2=0x7fcc981a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 --2- 172.21.15.137:0/575638437 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc98102df0 0x7fcc981a0c00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c8f0 tx=0x7fcc8400ccb0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcc9e600700 1 -- 172.21.15.137:0/575638437 reap_dead start 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 shutdown_connections 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 >> 172.21.15.137:0/575638437 conn(0x7fcc980792f0 msgr2=0x7fcc98100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 shutdown_connections 2024-03-26T14:00:43.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:43.399+0000 7fcca0600700 1 -- 172.21.15.137:0/575638437 wait complete. 2024-03-26T14:00:43.811 INFO:teuthology.orchestra.run.smithi137.stdout:223338299450 2024-03-26T14:00:43.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-26T14:00:43.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 -- 172.21.15.137:0/4012523225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c10ec40 msgr2=0x7f298c111030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 --2- 172.21.15.137:0/4012523225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c10ec40 0x7f298c111030 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f298400b0a0 tx=0x7f298400b3b0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 -- 172.21.15.137:0/4012523225 shutdown_connections 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 --2- 172.21.15.137:0/4012523225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c10ec40 0x7f298c111030 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 --2- 172.21.15.137:0/4012523225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f298c069e40 0x7f298c06a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 --2- 172.21.15.137:0/4012523225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f298c06a760 0x7f298c10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.166+0000 7f2994000700 1 -- 172.21.15.137:0/4012523225 >> 172.21.15.137:0/4012523225 conn(0x7f298c077fc0 msgr2=0x7f298c0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.167+0000 7f2994000700 1 -- 172.21.15.137:0/4012523225 shutdown_connections 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.167+0000 7f2994000700 1 -- 172.21.15.137:0/4012523225 wait complete. 2024-03-26T14:00:43.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 Processor -- start 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 -- start start 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f298c069e40 0x7f298c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c06a760 0x7f298c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f298c10ec40 0x7f298c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c114550 con 0x7f298c06a760 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1143d0 con 0x7f298c10ec40 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.169+0000 7f2994000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1146d0 con 0x7f298c069e40 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c06a760 0x7f298c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c06a760 0x7f298c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60818/0 (socket says 172.21.15.137:60818) 2024-03-26T14:00:43.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 -- 172.21.15.137:0/3618779636 learned_addr learned my addr 172.21.15.137:0/3618779636 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2992000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f298c10ec40 0x7f298c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2991600700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f298c069e40 0x7f298c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 -- 172.21.15.137:0/3618779636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f298c069e40 msgr2=0x7f298c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f298c069e40 0x7f298c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 -- 172.21.15.137:0/3618779636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f298c10ec40 msgr2=0x7f298c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f298c10ec40 0x7f298c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.170+0000 7f2990c00700 1 -- 172.21.15.137:0/3618779636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c19b050 con 0x7f298c06a760 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.171+0000 7f2990c00700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c06a760 0x7f298c1a0c20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f297c00d950 tx=0x7f297c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.171+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c016070 con 0x7f298c06a760 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.171+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c19b340 con 0x7f298c06a760 2024-03-26T14:00:43.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.171+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f297c009df0 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.171+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c00f3c0 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.171+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c1a75b0 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.172+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f298c062920 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.175+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f297c019070 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.176+0000 7f298a000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29740a29a0 0x7f29740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.176+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f297c0cefd0 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.176+0000 7f2991600700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29740a29a0 0x7f29740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.176+0000 7f298a000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] conn(0x7f29740ada80 0x7f29740aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.176+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 --> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f297c01ee50 con 0x7f29740ada80 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.176+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f297c0cf380 con 0x7f298c06a760 2024-03-26T14:00:43.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.177+0000 7f2992000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] conn(0x7f29740ada80 0x7f29740aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:43.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.177+0000 7f2991600700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29740a29a0 0x7f29740a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2980005950 tx=0x7f29800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:43.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.177+0000 7f2992000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] conn(0x7f29740ada80 0x7f29740aff20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:43.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.178+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== osd.0 v2:172.21.15.137:6802/4170736921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f297c01ee50 con 0x7f29740ada80 2024-03-26T14:00:43.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.196+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 --> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f298c062d20 con 0x7f29740ada80 2024-03-26T14:00:43.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.196+0000 7f298a000700 1 -- 172.21.15.137:0/3618779636 <== osd.0 v2:172.21.15.137:6802/4170736921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f298c062d20 con 0x7f29740ada80 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] conn(0x7f29740ada80 msgr2=0x7f29740aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] conn(0x7f29740ada80 0x7f29740aff20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29740a29a0 msgr2=0x7f29740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29740a29a0 0x7f29740a4e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2980005950 tx=0x7f29800058e0 comp rx=0 tx=0).stop 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c06a760 msgr2=0x7f298c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 --2- 172.21.15.137:0/3618779636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f298c06a760 0x7f298c1a0c20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f297c00d950 tx=0x7f297c00dc60 comp rx=0 tx=0).stop 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2992000700 1 -- 172.21.15.137:0/3618779636 reap_dead start 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 shutdown_connections 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 >> 172.21.15.137:0/3618779636 conn(0x7f298c077fc0 msgr2=0x7f298c110150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 shutdown_connections 2024-03-26T14:00:43.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:42.197+0000 7f2994000700 1 -- 172.21.15.137:0/3618779636 wait complete. 2024-03-26T14:00:44.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:44 smithi159 ceph-mon[35501]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:00:44.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:44 smithi137 ceph-mon[31539]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:00:44.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:44 smithi179 ceph-mon[35472]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:00:44.778 INFO:teuthology.orchestra.run.smithi137.stdout:292057776173 2024-03-26T14:00:44.778 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-26T14:00:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.521+0000 7fa2dc600700 1 -- 172.21.15.137:0/3123265856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4100f00 msgr2=0x7fa2d4104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.521+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3123265856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4100f00 0x7fa2d4104fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00099c0 tx=0x7fa2c0009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 -- 172.21.15.137:0/3123265856 shutdown_connections 2024-03-26T14:00:45.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3123265856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2d4105520 0x7fa2d41059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3123265856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d41005e0 0x7fa2d41009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3123265856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4100f00 0x7fa2d4104fe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 -- 172.21.15.137:0/3123265856 >> 172.21.15.137:0/3123265856 conn(0x7fa2d4077260 msgr2=0x7fa2d40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 -- 172.21.15.137:0/3123265856 shutdown_connections 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.522+0000 7fa2dc600700 1 -- 172.21.15.137:0/3123265856 wait complete. 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.523+0000 7fa2dc600700 1 Processor -- start 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 -- start start 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d41005e0 0x7fa2d40fc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2d4100f00 0x7fa2d40fcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4105520 0x7fa2d40fd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d4111230 con 0x7fa2d4105520 2024-03-26T14:00:45.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41110b0 con 0x7fa2d41005e0 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.524+0000 7fa2dc600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d41113b0 con 0x7fa2d4100f00 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4105520 0x7fa2d40fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2d9c00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d41005e0 0x7fa2d40fc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4105520 0x7fa2d40fd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60774/0 (socket says 172.21.15.137:60774) 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 -- 172.21.15.137:0/3962433193 learned_addr learned my addr 172.21.15.137:0/3962433193 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2d9200700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2d4100f00 0x7fa2d40fcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 -- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2d4100f00 msgr2=0x7fa2d40fcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2d4100f00 0x7fa2d40fcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 -- 172.21.15.137:0/3962433193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d41005e0 msgr2=0x7fa2d40fc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2d41005e0 0x7fa2d40fc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 -- 172.21.15.137:0/3962433193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d41a6e40 con 0x7fa2d4105520 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.525+0000 7fa2da600700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4105520 0x7fa2d40fd120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ada0 tx=0x7fa2cc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:45.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.526+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017020 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.526+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d41a7130 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.526+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa2cc00ce30 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.526+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc005350 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.526+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d41a7690 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.527+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2d4109cf0 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.529+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa2cc00c980 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.530+0000 7fa2d2000700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2bc0a2920 0x7fa2bc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.531+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fa2cc0cefa0 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.531+0000 7fa2d9c00700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2bc0a2920 0x7fa2bc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.531+0000 7fa2d2000700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] conn(0x7fa2bc0ada80 0x7fa2bc0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.531+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 --> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2cc016e70 con 0x7fa2bc0ada80 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.531+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2cc0cf3f0 con 0x7fa2d4105520 2024-03-26T14:00:45.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.531+0000 7fa2d9c00700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2bc0a2920 0x7fa2bc0a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009c60 tx=0x7fa2c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.532+0000 7fa2d9200700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] conn(0x7fa2bc0ada80 0x7fa2bc0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.532+0000 7fa2d9200700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] conn(0x7fa2bc0ada80 0x7fa2bc0aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.535+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== osd.8 v2:172.21.15.179:6816/4118554699 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa2cc016e70 con 0x7fa2bc0ada80 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.553+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 --> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa2d4101ac0 con 0x7fa2bc0ada80 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2d2000700 1 -- 172.21.15.137:0/3962433193 <== osd.8 v2:172.21.15.179:6816/4118554699 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa2d4101ac0 con 0x7fa2bc0ada80 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] conn(0x7fa2bc0ada80 msgr2=0x7fa2bc0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] conn(0x7fa2bc0ada80 0x7fa2bc0aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:45.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2bc0a2920 msgr2=0x7fa2bc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2bc0a2920 0x7fa2bc0a4de0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009c60 tx=0x7fa2c8009260 comp rx=0 tx=0).stop 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4105520 msgr2=0x7fa2d40fd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.554+0000 7fa2dc600700 1 --2- 172.21.15.137:0/3962433193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2d4105520 0x7fa2d40fd120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ada0 tx=0x7fa2cc00c330 comp rx=0 tx=0).stop 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.555+0000 7fa2da600700 1 -- 172.21.15.137:0/3962433193 reap_dead start 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.555+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 shutdown_connections 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.555+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 >> 172.21.15.137:0/3962433193 conn(0x7fa2d4077260 msgr2=0x7fa2d4109310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.555+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 shutdown_connections 2024-03-26T14:00:45.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:39.555+0000 7fa2dc600700 1 -- 172.21.15.137:0/3962433193 wait complete. 2024-03-26T14:00:46.072 INFO:teuthology.orchestra.run.smithi137.stdout:257698037812 2024-03-26T14:00:46.072 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-26T14:00:46.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:46 smithi179 ceph-mon[35472]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:00:46.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:46 smithi159 ceph-mon[35501]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:00:46.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:46 smithi137 ceph-mon[31539]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:00:46.651 INFO:teuthology.orchestra.run.smithi137.stdout:167503724616 2024-03-26T14:00:46.653 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-26T14:00:46.875 INFO:teuthology.orchestra.run.smithi137.stdout:34359738476 2024-03-26T14:00:46.875 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-26T14:00:47.182 INFO:teuthology.orchestra.run.smithi137.stdout:201863462978 2024-03-26T14:00:47.182 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-26T14:00:47.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 -- 172.21.15.137:0/2913612896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd4104600 msgr2=0x7fcdd41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:47.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 --2- 172.21.15.137:0/2913612896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd4104600 0x7fcdd41049e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80099c0 tx=0x7fcdc8009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 -- 172.21.15.137:0/2913612896 shutdown_connections 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 --2- 172.21.15.137:0/2913612896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdd41096e0 0x7fcdd41112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 --2- 172.21.15.137:0/2913612896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd4104600 0x7fcdd41049e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 --2- 172.21.15.137:0/2913612896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd4104fb0 0x7fcdd4109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.869+0000 7fcddc200700 1 -- 172.21.15.137:0/2913612896 >> 172.21.15.137:0/2913612896 conn(0x7fcdd40ffe70 msgr2=0x7fcdd4102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.870+0000 7fcddc200700 1 -- 172.21.15.137:0/2913612896 shutdown_connections 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.870+0000 7fcddc200700 1 -- 172.21.15.137:0/2913612896 wait complete. 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 Processor -- start 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 -- start start 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdd4104600 0x7fcdd41a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:47.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd4104fb0 0x7fcdd41a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd41096e0 0x7fcdd419a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4114640 con 0x7fcdd41096e0 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41144c0 con 0x7fcdd4104600 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.871+0000 7fcddc200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd41147c0 con 0x7fcdd4104fb0 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd41096e0 0x7fcdd419a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd41096e0 0x7fcdd419a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60858/0 (socket says 172.21.15.137:60858) 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 -- 172.21.15.137:0/1122002652 learned_addr learned my addr 172.21.15.137:0/1122002652 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdd9800700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdd4104600 0x7fcdd41a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdd8e00700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd4104fb0 0x7fcdd41a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 -- 172.21.15.137:0/1122002652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd4104fb0 msgr2=0x7fcdd41a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd4104fb0 0x7fcdd41a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 -- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdd4104600 msgr2=0x7fcdd41a06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdd4104600 0x7fcdd41a06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 -- 172.21.15.137:0/1122002652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd419b060 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdd8e00700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcdd4104fb0 0x7fcdd41a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.872+0000 7fcdda200700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd41096e0 0x7fcdd419a900 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007d70 tx=0x7fcdcc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.873+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc016020 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.873+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcdcc00ad40 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.873+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd419b350 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.873+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0115e0 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.873+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd41a75c0 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.874+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdd4062920 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.876+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fcdcc019020 con 0x7fcdd41096e0 2024-03-26T14:00:47.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.877+0000 7fcdd2000700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcdbc0a2920 0x7fcdbc0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.877+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7fcdcc0ced60 con 0x7fcdd41096e0 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.877+0000 7fcdd9800700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcdbc0a2920 0x7fcdbc0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.878+0000 7fcdd2000700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] conn(0x7fcdbc0ada80 0x7fcdbc0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.878+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 --> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdbc0b05f0 con 0x7fcdbc0ada80 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.878+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdcc0d1420 con 0x7fcdd41096e0 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.878+0000 7fcdd8e00700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] conn(0x7fcdbc0ada80 0x7fcdbc0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.878+0000 7fcdd9800700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcdbc0a2920 0x7fcdbc0a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8000c00 tx=0x7fcdc8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.879+0000 7fcdd8e00700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] conn(0x7fcdbc0ada80 0x7fcdbc0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.881+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== osd.1 v2:172.21.15.159:6800/3508415474 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcdbc0b05f0 con 0x7fcdbc0ada80 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.899+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 --> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdd4062da0 con 0x7fcdbc0ada80 2024-03-26T14:00:47.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcdd2000700 1 -- 172.21.15.137:0/1122002652 <== osd.1 v2:172.21.15.159:6800/3508415474 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcdd4062da0 con 0x7fcdbc0ada80 2024-03-26T14:00:47.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] conn(0x7fcdbc0ada80 msgr2=0x7fcdbc0aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:47.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] conn(0x7fcdbc0ada80 0x7fcdbc0aff20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcdbc0a2920 msgr2=0x7fcdbc0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:47.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcdbc0a2920 0x7fcdbc0a4de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8000c00 tx=0x7fcdc8005fb0 comp rx=0 tx=0).stop 2024-03-26T14:00:47.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd41096e0 msgr2=0x7fcdd419a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:47.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 --2- 172.21.15.137:0/1122002652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcdd41096e0 0x7fcdd419a900 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007d70 tx=0x7fcdcc00a340 comp rx=0 tx=0).stop 2024-03-26T14:00:47.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcdda200700 1 -- 172.21.15.137:0/1122002652 reap_dead start 2024-03-26T14:00:47.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 shutdown_connections 2024-03-26T14:00:47.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.900+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 >> 172.21.15.137:0/1122002652 conn(0x7fcdd40ffe70 msgr2=0x7fcdd4102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:47.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.901+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 shutdown_connections 2024-03-26T14:00:47.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:45.901+0000 7fcddc200700 1 -- 172.21.15.137:0/1122002652 wait complete. 2024-03-26T14:00:47.798 INFO:teuthology.orchestra.run.smithi137.stdout:347892351008 2024-03-26T14:00:47.798 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-26T14:00:48.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:48 smithi159 ceph-mon[35501]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:00:48.451 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:48 smithi137 ceph-mon[31539]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:00:48.670 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:48 smithi179 ceph-mon[35472]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:00:48.972 INFO:teuthology.orchestra.run.smithi137.stdout:77309411423 2024-03-26T14:00:48.972 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-26T14:00:49.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 -- 172.21.15.137:0/4090185969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0109160 msgr2=0x7f42a0115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:49.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 --2- 172.21.15.137:0/4090185969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0109160 0x7f42a0115670 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f429800b0a0 tx=0x7f429800b3b0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 -- 172.21.15.137:0/4090185969 shutdown_connections 2024-03-26T14:00:49.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 --2- 172.21.15.137:0/4090185969 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a0109160 0x7f42a0115670 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 --2- 172.21.15.137:0/4090185969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a010e7a0 0x7f42a010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 --2- 172.21.15.137:0/4090185969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a01087a0 0x7f42a0108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 -- 172.21.15.137:0/4090185969 >> 172.21.15.137:0/4090185969 conn(0x7f42a0075960 msgr2=0x7f42a0077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:49.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 -- 172.21.15.137:0/4090185969 shutdown_connections 2024-03-26T14:00:49.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.879+0000 7f42a5000700 1 -- 172.21.15.137:0/4090185969 wait complete. 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.880+0000 7f42a5000700 1 Processor -- start 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.880+0000 7f42a5000700 1 -- start start 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f42a5000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a01087a0 0x7f42a019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f42a5000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0109160 0x7f42a019f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f42a5000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a010e7a0 0x7f42a01a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f42a5000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0118980 con 0x7f42a010e7a0 2024-03-26T14:00:49.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f42a5000700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0118800 con 0x7f42a0109160 2024-03-26T14:00:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f42a5000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0118b00 con 0x7f42a01087a0 2024-03-26T14:00:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f429e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a010e7a0 0x7f42a01a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f429e800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a010e7a0 0x7f42a01a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35962/0 (socket says 172.21.15.137:35962) 2024-03-26T14:00:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f429e800700 1 -- 172.21.15.137:0/4126234999 learned_addr learned my addr 172.21.15.137:0/4126234999 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f429d400700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0109160 0x7f42a019f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:49.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.881+0000 7f429de00700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a01087a0 0x7f42a019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429e800700 1 -- 172.21.15.137:0/4126234999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a01087a0 msgr2=0x7f42a019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429e800700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a01087a0 0x7f42a019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429e800700 1 -- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0109160 msgr2=0x7f42a019f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429e800700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42a0109160 0x7f42a019f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429e800700 1 -- 172.21.15.137:0/4126234999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a01a3d40 con 0x7f42a010e7a0 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429de00700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42a01087a0 0x7f42a019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f429e800700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a010e7a0 0x7f42a01a3600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f429800b830 tx=0x7f4298004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298021070 con 0x7f42a010e7a0 2024-03-26T14:00:49.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a01a3fd0 con 0x7f42a010e7a0 2024-03-26T14:00:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4298007e30 con 0x7f42a010e7a0 2024-03-26T14:00:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.882+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42980047d0 con 0x7f42a010e7a0 2024-03-26T14:00:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.883+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a01ab8b0 con 0x7f42a010e7a0 2024-03-26T14:00:49.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.884+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42a0076860 con 0x7f42a010e7a0 2024-03-26T14:00:49.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.886+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f4298007980 con 0x7f42a010e7a0 2024-03-26T14:00:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.887+0000 7f4296a00700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f42840a2920 0x7f42840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.887+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9092+0+0 (secure 0 0 0) 0x7f429800f020 con 0x7f42a010e7a0 2024-03-26T14:00:49.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.887+0000 7f429de00700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f42840a2920 0x7f42840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.888+0000 7f4296a00700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] conn(0x7f42840ada80 0x7f42840aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.888+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 --> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42840b05f0 con 0x7f42840ada80 2024-03-26T14:00:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.888+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42980d0ab0 con 0x7f42a010e7a0 2024-03-26T14:00:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.888+0000 7f429de00700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f42840a2920 0x7f42840a4de0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4290005fd0 tx=0x7f429000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.888+0000 7f429d400700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] conn(0x7f42840ada80 0x7f42840aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:49.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.889+0000 7f429d400700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] conn(0x7f42840ada80 0x7f42840aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:49.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.891+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== osd.4 v2:172.21.15.159:6808/2050135218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42840b05f0 con 0x7f42840ada80 2024-03-26T14:00:49.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.908+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 --> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42a0109ee0 con 0x7f42840ada80 2024-03-26T14:00:49.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.909+0000 7f4296a00700 1 -- 172.21.15.137:0/4126234999 <== osd.4 v2:172.21.15.159:6808/2050135218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f42a0109ee0 con 0x7f42840ada80 2024-03-26T14:00:49.906 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:49 smithi137 ceph-mon[31539]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:49.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] conn(0x7f42840ada80 msgr2=0x7f42840aff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:49.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f42a5000700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] conn(0x7f42840ada80 0x7f42840aff20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f42840a2920 msgr2=0x7f42840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:49.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f42a5000700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f42840a2920 0x7f42840a4de0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4290005fd0 tx=0x7f429000a5e0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a010e7a0 msgr2=0x7f42a01a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f42a5000700 1 --2- 172.21.15.137:0/4126234999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f42a010e7a0 0x7f42a01a3600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f429800b830 tx=0x7f4298004ce0 comp rx=0 tx=0).stop 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.910+0000 7f429e800700 1 -- 172.21.15.137:0/4126234999 reap_dead start 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.911+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 shutdown_connections 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.911+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 >> 172.21.15.137:0/4126234999 conn(0x7f42a0075960 msgr2=0x7f42a0075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.911+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 shutdown_connections 2024-03-26T14:00:49.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:49.911+0000 7f42a5000700 1 -- 172.21.15.137:0/4126234999 wait complete. 2024-03-26T14:00:49.920 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:49 smithi179 ceph-mon[35472]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:49.949 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:49 smithi159 ceph-mon[35501]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:50.113 INFO:teuthology.orchestra.run.smithi137.stdout:55834574949 2024-03-26T14:00:50.113 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-26T14:00:50.776 INFO:teuthology.orchestra.run.smithi137.stdout:146028888147 2024-03-26T14:00:50.777 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-26T14:00:50.825 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:50.839 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:50.853 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:50.859 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:52.076 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:52.182 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:52.182 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:52.313 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:52 smithi137 ceph-mon[31539]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:52.420 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:52 smithi179 ceph-mon[35472]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:52.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:52 smithi159 ceph-mon[35501]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:00:53.463 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:54 smithi137 ceph-mon[31539]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:00:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-03-26T14:00:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-26T14:00:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-26T14:00:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-03-26T14:00:54.309 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:54 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-26T14:00:54.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:54 smithi159 ceph-mon[35501]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:00:54.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-03-26T14:00:54.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-26T14:00:54.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-26T14:00:54.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-03-26T14:00:54.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:54 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-26T14:00:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:54 smithi179 ceph-mon[35472]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:00:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-03-26T14:00:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-03-26T14:00:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-03-26T14:00:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]: dispatch 2024-03-26T14:00:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:54 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-26T14:00:55.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 -- 172.21.15.137:0/1868076158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c90102c80 msgr2=0x7f4c90106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:55.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/1868076158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c90102c80 0x7f4c90106cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0099c0 tx=0x7f4c7c009cd0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 -- 172.21.15.137:0/1868076158 shutdown_connections 2024-03-26T14:00:55.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/1868076158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 0x7f4c9010ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/1868076158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c901022d0 0x7f4c901026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/1868076158 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c90102c80 0x7f4c90106cd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 -- 172.21.15.137:0/1868076158 >> 172.21.15.137:0/1868076158 conn(0x7f4c900fdb60 msgr2=0x7f4c900fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:55.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 -- 172.21.15.137:0/1868076158 shutdown_connections 2024-03-26T14:00:55.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.198+0000 7f4c8fe00700 1 -- 172.21.15.137:0/1868076158 wait complete. 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.199+0000 7f4c8fe00700 1 Processor -- start 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 -- start start 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 0x7f4c90074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c90102c80 0x7f4c900749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 0x7f4c90074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90112290 con 0x7f4c901022d0 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90112110 con 0x7f4c901073b0 2024-03-26T14:00:55.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8fe00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90112410 con 0x7f4c90102c80 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 0x7f4c90074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8ea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 0x7f4c90074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35976/0 (socket says 172.21.15.137:35976) 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.200+0000 7f4c8ea00700 1 -- 172.21.15.137:0/3686838296 learned_addr learned my addr 172.21.15.137:0/3686838296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8e000700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c90102c80 0x7f4c900749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8f400700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 0x7f4c90074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8ea00700 1 -- 172.21.15.137:0/3686838296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c90102c80 msgr2=0x7f4c900749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8ea00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c90102c80 0x7f4c900749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8ea00700 1 -- 172.21.15.137:0/3686838296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 msgr2=0x7f4c90074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8ea00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 0x7f4c90074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8ea00700 1 -- 172.21.15.137:0/3686838296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c901a6d60 con 0x7f4c901022d0 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8f400700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 0x7f4c90074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:00:55.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.201+0000 7f4c8ea00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 0x7f4c90074490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000e9c0 tx=0x7f4c8000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:55.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.202+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000cbd0 con 0x7f4c901022d0 2024-03-26T14:00:55.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.202+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4c8000cd70 con 0x7f4c901022d0 2024-03-26T14:00:55.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.202+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c901a6ff0 con 0x7f4c901022d0 2024-03-26T14:00:55.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.202+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800056c0 con 0x7f4c901022d0 2024-03-26T14:00:55.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.202+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c901a7450 con 0x7f4c901022d0 2024-03-26T14:00:55.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.203+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c90068e50 con 0x7f4c901022d0 2024-03-26T14:00:55.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.206+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4c80004750 con 0x7f4c901022d0 2024-03-26T14:00:55.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.209+0000 7f4c87400700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c740a2890 0x7f4c740a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:00:55.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.209+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4c800cf0d0 con 0x7f4c901022d0 2024-03-26T14:00:55.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.210+0000 7f4c8e000700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c740a2890 0x7f4c740a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:00:55.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.211+0000 7f4c8e000700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c740a2890 0x7f4c740a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000c00 tx=0x7f4c7c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:00:55.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.217+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4c8009bd60 con 0x7f4c901022d0 2024-03-26T14:00:55.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-03-26T14:00:55.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-26T14:00:55.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-26T14:00:55.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-03-26T14:00:55.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-26T14:00:55.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T14:00:55.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:55 smithi137 ceph-mon[31539]: mgrmap e18: x(active, since 11m), standbys: y, z 2024-03-26T14:00:55.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.444+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f4c9002da30 con 0x7f4c901022d0 2024-03-26T14:00:55.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.444+0000 7f4c87400700 1 -- 172.21.15.137:0/3686838296 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f4c8009b660 con 0x7f4c901022d0 2024-03-26T14:00:55.441 INFO:teuthology.orchestra.run.smithi137.stdout:313532612649 2024-03-26T14:00:55.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c740a2890 msgr2=0x7f4c740a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:55.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c740a2890 0x7f4c740a4d50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000c00 tx=0x7f4c7c005fb0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 msgr2=0x7f4c90074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 0x7f4c90074490 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000e9c0 tx=0x7f4c8000ecd0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 shutdown_connections 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4c740a2890 0x7f4c740a4d50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c901073b0 0x7f4c90074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4c901022d0 0x7f4c90074490 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 --2- 172.21.15.137:0/3686838296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c90102c80 0x7f4c900749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 >> 172.21.15.137:0/3686838296 conn(0x7f4c900fdb60 msgr2=0x7f4c900ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.447+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 shutdown_connections 2024-03-26T14:00:55.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:00:55.448+0000 7f4c8fe00700 1 -- 172.21.15.137:0/3686838296 wait complete. 2024-03-26T14:00:55.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-03-26T14:00:55.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-26T14:00:55.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-26T14:00:55.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-03-26T14:00:55.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-26T14:00:55.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T14:00:55.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:55 smithi159 ceph-mon[35501]: mgrmap e18: x(active, since 11m), standbys: y, z 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8, 5, 6]}]': finished 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: osdmap e93: 12 total, 12 up, 12 in 2024-03-26T14:00:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:55 smithi179 ceph-mon[35472]: mgrmap e18: x(active, since 11m), standbys: y, z 2024-03-26T14:00:56.508 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:56 smithi137 ceph-mon[31539]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-26T14:00:56.508 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:56 smithi137 ceph-mon[31539]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T14:00:56.508 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3686838296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T14:00:56.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:56 smithi179 ceph-mon[35472]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-26T14:00:56.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:56 smithi179 ceph-mon[35472]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T14:00:56.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3686838296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T14:00:56.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:56 smithi159 ceph-mon[35501]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-03-26T14:00:56.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:56 smithi159 ceph-mon[35501]: osdmap e94: 12 total, 12 up, 12 in 2024-03-26T14:00:56.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3686838296' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-26T14:00:58.473 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:00:58 smithi137 ceph-mon[31539]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:00:58.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:00:58 smithi179 ceph-mon[35472]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:00:58.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:00:58 smithi159 ceph-mon[35501]: pgmap v416: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 7 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:00:58.701 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:58.702 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:00:59.222 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:00.455 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:00 smithi137 ceph-mon[31539]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:00.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:00 smithi179 ceph-mon[35472]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:00.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:00 smithi159 ceph-mon[35501]: pgmap v417: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:02.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:02 smithi137 ceph-mon[31539]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:02.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:02 smithi179 ceph-mon[35472]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:02.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:02 smithi159 ceph-mon[35501]: pgmap v418: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 639 B/s wr, 5 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:04.267 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:04 smithi137 ceph-mon[31539]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 458 B/s wr, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:04.502 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:04 smithi159 ceph-mon[35501]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 458 B/s wr, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:04.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 -- 172.21.15.137:0/3135005365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 msgr2=0x7feec410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:04.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 --2- 172.21.15.137:0/3135005365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec410e650 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7feeac009990 tx=0x7feeac009ca0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 -- 172.21.15.137:0/3135005365 shutdown_connections 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 --2- 172.21.15.137:0/3135005365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec410eb90 0x7feec4110f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 --2- 172.21.15.137:0/3135005365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec40743c0 0x7feec40747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 --2- 172.21.15.137:0/3135005365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec410e650 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 -- 172.21.15.137:0/3135005365 >> 172.21.15.137:0/3135005365 conn(0x7feec40792c0 msgr2=0x7feec40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 -- 172.21.15.137:0/3135005365 shutdown_connections 2024-03-26T14:01:04.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.570+0000 7feecb200700 1 -- 172.21.15.137:0/3135005365 wait complete. 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.571+0000 7feecb200700 1 Processor -- start 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 -- start start 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec40743c0 0x7feec41a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec41a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec410eb90 0x7feec419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4114500 con 0x7feec4074ce0 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4114380 con 0x7feec410eb90 2024-03-26T14:01:04.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feecb200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec4114680 con 0x7feec40743c0 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feebbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec41a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feebbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec41a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37332/0 (socket says 172.21.15.137:37332) 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feebbe00700 1 -- 172.21.15.137:0/1223857187 learned_addr learned my addr 172.21.15.137:0/1223857187 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.572+0000 7feec9200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec410eb90 0x7feec419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feec3e00700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec40743c0 0x7feec41a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feebbe00700 1 -- 172.21.15.137:0/1223857187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec40743c0 msgr2=0x7feec41a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feebbe00700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec40743c0 0x7feec41a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feebbe00700 1 -- 172.21.15.137:0/1223857187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec410eb90 msgr2=0x7feec419a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feebbe00700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec410eb90 0x7feec419a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feebbe00700 1 -- 172.21.15.137:0/1223857187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec419b020 con 0x7feec4074ce0 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feebbe00700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec41a0c00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feeac003ab0 tx=0x7feeac003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac027070 con 0x7feec4074ce0 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeac00bdc0 con 0x7feec4074ce0 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.573+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec419b2b0 con 0x7feec4074ce0 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.574+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac022d40 con 0x7feec4074ce0 2024-03-26T14:01:04.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.574+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec41a7520 con 0x7feec4074ce0 2024-03-26T14:01:04.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.575+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4075a60 con 0x7feec4074ce0 2024-03-26T14:01:04.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.577+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7feeac02a070 con 0x7feec4074ce0 2024-03-26T14:01:04.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.578+0000 7feec1600700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feea80a2890 0x7feea80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:04.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.578+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7feeac0cfbc0 con 0x7feec4074ce0 2024-03-26T14:01:04.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.578+0000 7feec3e00700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feea80a2890 0x7feea80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:04.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.579+0000 7feec3e00700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feea80a2890 0x7feea80a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005fd0 tx=0x7feeb4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:04.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.583+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7feeac05e740 con 0x7feec4074ce0 2024-03-26T14:01:04.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:04 smithi179 ceph-mon[35472]: pgmap v419: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 458 B/s wr, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-03-26T14:01:04.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.819+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7feec402da30 con 0x7feec4074ce0 2024-03-26T14:01:04.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.819+0000 7feec1600700 1 -- 172.21.15.137:0/1223857187 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feeac013070 con 0x7feec4074ce0 2024-03-26T14:01:04.817 INFO:teuthology.orchestra.run.smithi137.stdout:223338299456 2024-03-26T14:01:04.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.822+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feea80a2890 msgr2=0x7feea80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:04.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.822+0000 7feecb200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feea80a2890 0x7feea80a4d50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005fd0 tx=0x7feeb4005f60 comp rx=0 tx=0).stop 2024-03-26T14:01:04.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.822+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 msgr2=0x7feec41a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.822+0000 7feecb200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec41a0c00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feeac003ab0 tx=0x7feeac003ae0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.822+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 shutdown_connections 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.822+0000 7feecb200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feea80a2890 0x7feea80a4d50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.823+0000 7feecb200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feec410eb90 0x7feec419a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.823+0000 7feecb200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feec40743c0 0x7feec41a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.823+0000 7feecb200700 1 --2- 172.21.15.137:0/1223857187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feec4074ce0 0x7feec41a0c00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.823+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 >> 172.21.15.137:0/1223857187 conn(0x7feec40792c0 msgr2=0x7feec40ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.823+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 shutdown_connections 2024-03-26T14:01:04.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:04.823+0000 7feecb200700 1 -- 172.21.15.137:0/1223857187 wait complete. 2024-03-26T14:01:05.411 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:05 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1223857187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T14:01:05.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:05 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1223857187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T14:01:05.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:05 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1223857187' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 -- 172.21.15.137:0/2381382816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e181096c0 msgr2=0x7f8e18111290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 --2- 172.21.15.137:0/2381382816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e181096c0 0x7f8e18111290 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000b0d0 tx=0x7f8e1000b3e0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 -- 172.21.15.137:0/2381382816 shutdown_connections 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 --2- 172.21.15.137:0/2381382816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e181096c0 0x7f8e18111290 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 --2- 172.21.15.137:0/2381382816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e181045e0 0x7f8e181049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 --2- 172.21.15.137:0/2381382816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e18104f90 0x7f8e18108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.575+0000 7f8e1de00700 1 -- 172.21.15.137:0/2381382816 >> 172.21.15.137:0/2381382816 conn(0x7f8e180ffe70 msgr2=0x7f8e18102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.576+0000 7f8e1de00700 1 -- 172.21.15.137:0/2381382816 shutdown_connections 2024-03-26T14:01:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.576+0000 7f8e1de00700 1 -- 172.21.15.137:0/2381382816 wait complete. 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 Processor -- start 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 -- start start 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e181045e0 0x7f8e181a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 0x7f8e181a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e181096c0 0x7f8e1819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181145d0 con 0x7f8e18104f90 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18114450 con 0x7f8e181096c0 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.577+0000 7f8e1de00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18114750 con 0x7f8e181045e0 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 0x7f8e181a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 0x7f8e181a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37358/0 (socket says 172.21.15.137:37358) 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 -- 172.21.15.137:0/76445309 learned_addr learned my addr 172.21.15.137:0/76445309 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e17400700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e181096c0 0x7f8e1819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16a00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e181045e0 0x7f8e181a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 -- 172.21.15.137:0/76445309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e181045e0 msgr2=0x7f8e181a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e181045e0 0x7f8e181a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 -- 172.21.15.137:0/76445309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e181096c0 msgr2=0x7f8e1819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e181096c0 0x7f8e1819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 -- 172.21.15.137:0/76445309 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1819b010 con 0x7f8e18104f90 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.578+0000 7f8e16000700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 0x7f8e181a0bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400e9f0 tx=0x7f8e0400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.579+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400cbd0 con 0x7f8e18104f90 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.579+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1819b300 con 0x7f8e18104f90 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.579+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e0400cd70 con 0x7f8e18104f90 2024-03-26T14:01:05.991 INFO:teuthology.orchestra.run.smithi137.stdout:292057776178 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.579+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04010430 con 0x7f8e18104f90 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.579+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e181a7510 con 0x7f8e18104f90 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.580+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e18105ac0 con 0x7f8e18104f90 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.584+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8e04004750 con 0x7f8e18104f90 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.585+0000 7f8e1ca00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8dfc0a2890 0x7f8dfc0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.585+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8e040cf310 con 0x7f8e18104f90 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.585+0000 7f8e16a00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8dfc0a2890 0x7f8dfc0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.586+0000 7f8e16a00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8dfc0a2890 0x7f8dfc0a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009bf0 tx=0x7f8e08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:05.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.587+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e04053740 con 0x7f8e18104f90 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.826+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f8e1802da30 con 0x7f8e18104f90 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.827+0000 7f8e1ca00700 1 -- 172.21.15.137:0/76445309 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e0409bfa0 con 0x7f8e18104f90 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.829+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8dfc0a2890 msgr2=0x7f8dfc0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.829+0000 7f8e1de00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8dfc0a2890 0x7f8dfc0a4d50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8e08009bf0 tx=0x7f8e08009260 comp rx=0 tx=0).stop 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.829+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 msgr2=0x7f8e181a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.829+0000 7f8e1de00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 0x7f8e181a0bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400e9f0 tx=0x7f8e0400edb0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 shutdown_connections 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8dfc0a2890 0x7f8dfc0a4d50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e181096c0 0x7f8e1819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e181045e0 0x7f8e181a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 --2- 172.21.15.137:0/76445309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e18104f90 0x7f8e181a0bf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 >> 172.21.15.137:0/76445309 conn(0x7f8e180ffe70 msgr2=0x7f8e18100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:05.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 shutdown_connections 2024-03-26T14:01:05.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:05.830+0000 7f8e1de00700 1 -- 172.21.15.137:0/76445309 wait complete. 2024-03-26T14:01:06.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:06 smithi137 ceph-mon[31539]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T14:01:06.301 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:06 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/76445309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T14:01:06.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:06 smithi179 ceph-mon[35472]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T14:01:06.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:06 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/76445309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T14:01:06.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:06 smithi159 ceph-mon[35501]: pgmap v420: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T14:01:06.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/76445309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-26T14:01:08.070 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612646 got 313532612649 for osd.10 2024-03-26T14:01:08.070 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 -- 172.21.15.137:0/2928803997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c580745c0 msgr2=0x7f9c580749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/2928803997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c580745c0 0x7f9c580749a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c009ca0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 -- 172.21.15.137:0/2928803997 shutdown_connections 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/2928803997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c5810e9b0 0x7f9c58110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/2928803997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c580745c0 0x7f9c580749a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/2928803997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c58074ee0 0x7f9c5810e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 -- 172.21.15.137:0/2928803997 >> 172.21.15.137:0/2928803997 conn(0x7f9c580fdc40 msgr2=0x7f9c58100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:08.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 -- 172.21.15.137:0/2928803997 shutdown_connections 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.836+0000 7f9c5fa00700 1 -- 172.21.15.137:0/2928803997 wait complete. 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.837+0000 7f9c5fa00700 1 Processor -- start 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.837+0000 7f9c5fa00700 1 -- start start 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5fa00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c580745c0 0x7f9c581a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5fa00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58074ee0 0x7f9c581a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5fa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 0x7f9c5819a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5fa00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c58114550 con 0x7f9c5810e9b0 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5fa00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c581143d0 con 0x7f9c58074ee0 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5fa00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c581146d0 con 0x7f9c580745c0 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 0x7f9c5819a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 0x7f9c5819a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37394/0 (socket says 172.21.15.137:37394) 2024-03-26T14:01:08.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 -- 172.21.15.137:0/1523856493 learned_addr learned my addr 172.21.15.137:0/1523856493 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c57e00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58074ee0 0x7f9c581a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5d000700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c580745c0 0x7f9c581a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 -- 172.21.15.137:0/1523856493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c580745c0 msgr2=0x7f9c581a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c580745c0 0x7f9c581a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 -- 172.21.15.137:0/1523856493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58074ee0 msgr2=0x7f9c581a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58074ee0 0x7f9c581a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.838+0000 7f9c5da00700 1 -- 172.21.15.137:0/1523856493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5819b050 con 0x7f9c5810e9b0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.839+0000 7f9c5da00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 0x7f9c5819a8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007ee0 tx=0x7f9c5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.839+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000ddd0 con 0x7f9c5810e9b0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.839+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c50004020 con 0x7f9c5810e9b0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.839+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50015660 con 0x7f9c5810e9b0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.839+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5819b340 con 0x7f9c5810e9b0 2024-03-26T14:01:08.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.839+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c581a75b0 con 0x7f9c5810e9b0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.841+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58068e50 con 0x7f9c5810e9b0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.844+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9c500041d0 con 0x7f9c5810e9b0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.845+0000 7f9c55600700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c400a2890 0x7f9c400a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.846+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9c500ce7d0 con 0x7f9c5810e9b0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.846+0000 7f9c5d000700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c400a2890 0x7f9c400a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.847+0000 7f9c5d000700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c400a2890 0x7f9c400a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.850+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9c50010070 con 0x7f9c5810e9b0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.107+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9c5802da30 con 0x7f9c5810e9b0 2024-03-26T14:01:08.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.108+0000 7f9c55600700 1 -- 172.21.15.137:0/1523856493 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c5009b460 con 0x7f9c5810e9b0 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stdout:111669149788 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.110+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c400a2890 msgr2=0x7f9c400a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.110+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c400a2890 0x7f9c400a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c00b540 comp rx=0 tx=0).stop 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.110+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 msgr2=0x7f9c5819a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.110+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 0x7f9c5819a8f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007ee0 tx=0x7f9c5000d3b0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 shutdown_connections 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c400a2890 0x7f9c400a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c5810e9b0 0x7f9c5819a8f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c580745c0 0x7f9c581a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 --2- 172.21.15.137:0/1523856493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58074ee0 0x7f9c581a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:08.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 >> 172.21.15.137:0/1523856493 conn(0x7f9c580fdc40 msgr2=0x7f9c580fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:08.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 shutdown_connections 2024-03-26T14:01:08.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:08.111+0000 7f9c5fa00700 1 -- 172.21.15.137:0/1523856493 wait complete. 2024-03-26T14:01:08.268 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:08 smithi137 ceph-mon[31539]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 343 B/s wr, 7 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T14:01:08.268 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2888222522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T14:01:08.268 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1523856493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T14:01:08.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:08 smithi179 ceph-mon[35472]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 343 B/s wr, 7 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T14:01:08.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2888222522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T14:01:08.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1523856493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T14:01:08.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:08 smithi159 ceph-mon[35501]: pgmap v421: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 343 B/s wr, 7 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-03-26T14:01:08.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2888222522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-26T14:01:08.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1523856493' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-26T14:01:08.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299450 got 223338299456 for osd.7 2024-03-26T14:01:08.785 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:09.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 -- 172.21.15.137:0/2844725167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1022d0 msgr2=0x7f199c1026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 --2- 172.21.15.137:0/2844725167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1022d0 0x7f199c1026b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f198c0099c0 tx=0x7f198c009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 -- 172.21.15.137:0/2844725167 shutdown_connections 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 --2- 172.21.15.137:0/2844725167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f199c1073b0 0x7f199c10ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 --2- 172.21.15.137:0/2844725167 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1022d0 0x7f199c1026b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 --2- 172.21.15.137:0/2844725167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c102c80 0x7f199c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 -- 172.21.15.137:0/2844725167 >> 172.21.15.137:0/2844725167 conn(0x7f199c0fdb60 msgr2=0x7f199c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 -- 172.21.15.137:0/2844725167 shutdown_connections 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.739+0000 7f199b200700 1 -- 172.21.15.137:0/2844725167 wait complete. 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 Processor -- start 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 -- start start 2024-03-26T14:01:09.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1022d0 0x7f199c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f199c102c80 0x7f199c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 0x7f199c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c112240 con 0x7f199c1073b0 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1120c0 con 0x7f199c1022d0 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199b200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1123c0 con 0x7f199c102c80 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.741+0000 7f199a800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 0x7f199c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 0x7f199c19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37374/0 (socket says 172.21.15.137:37374) 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 -- 172.21.15.137:0/2888222522 learned_addr learned my addr 172.21.15.137:0/2888222522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f1999e00700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1022d0 0x7f199c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f1999400700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f199c102c80 0x7f199c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 -- 172.21.15.137:0/2888222522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f199c102c80 msgr2=0x7f199c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f199c102c80 0x7f199c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 -- 172.21.15.137:0/2888222522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1022d0 msgr2=0x7f199c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1022d0 0x7f199c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 -- 172.21.15.137:0/2888222522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c19afc0 con 0x7f199c1073b0 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f1999e00700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1022d0 0x7f199c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.742+0000 7f199a800700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 0x7f199c19a7e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f199400ada0 tx=0x7f199400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.743+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994017020 con 0x7f199c1073b0 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.743+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c19b2b0 con 0x7f199c1073b0 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.743+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f199400ce30 con 0x7f199c1073b0 2024-03-26T14:01:09.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.743+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994005400 con 0x7f199c1073b0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.743+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c1a74c0 con 0x7f199c1073b0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.744+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c1037b0 con 0x7f199c1073b0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.746+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f199400c980 con 0x7f199c1073b0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.748+0000 7f1992a00700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19800a2890 0x7f19800a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.749+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f19940ce600 con 0x7f199c1073b0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.749+0000 7f1999e00700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19800a2890 0x7f19800a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.749+0000 7f1999e00700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19800a2890 0x7f19800a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f198c005b40 tx=0x7f198c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.752+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1994021020 con 0x7f199c1073b0 2024-03-26T14:01:09.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.983+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f199c1a76f0 con 0x7f199c1073b0 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.984+0000 7f1992a00700 1 -- 172.21.15.137:0/2888222522 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f199409b320 con 0x7f199c1073b0 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19800a2890 msgr2=0x7f19800a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19800a2890 0x7f19800a4d50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f198c005b40 tx=0x7f198c005ab0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 msgr2=0x7f199c19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 0x7f199c19a7e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f199400ada0 tx=0x7f199400c330 comp rx=0 tx=0).stop 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 shutdown_connections 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19800a2890 0x7f19800a4d50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f199c1073b0 0x7f199c19a7e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.435 INFO:teuthology.orchestra.run.smithi137.stdout:55834574954 2024-03-26T14:01:09.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c1022d0 0x7f199c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 --2- 172.21.15.137:0/2888222522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f199c102c80 0x7f199c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 >> 172.21.15.137:0/2888222522 conn(0x7f199c0fdb60 msgr2=0x7f199c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:09.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 shutdown_connections 2024-03-26T14:01:09.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:07.987+0000 7f199b200700 1 -- 172.21.15.137:0/2888222522 wait complete. 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 -- 172.21.15.137:0/3290795808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 msgr2=0x7eff64102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 --2- 172.21.15.137:0/3290795808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff64102820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7eff540099c0 tx=0x7eff54009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 -- 172.21.15.137:0/3290795808 shutdown_connections 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 --2- 172.21.15.137:0/3290795808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff64107510 0x7eff6410f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 --2- 172.21.15.137:0/3290795808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff64102820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 --2- 172.21.15.137:0/3290795808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64102df0 0x7eff64106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 -- 172.21.15.137:0/3290795808 >> 172.21.15.137:0/3290795808 conn(0x7eff640792f0 msgr2=0x7eff64079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 -- 172.21.15.137:0/3290795808 shutdown_connections 2024-03-26T14:01:09.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.051+0000 7eff68e00700 1 -- 172.21.15.137:0/3290795808 wait complete. 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.052+0000 7eff68e00700 1 Processor -- start 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.052+0000 7eff68e00700 1 -- start start 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff68e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff641a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff68e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff64102df0 0x7eff641a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff68e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64107510 0x7eff6419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff68e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641123f0 con 0x7eff64102440 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff68e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64112270 con 0x7eff64107510 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff68e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64112570 con 0x7eff64102df0 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff641a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff641a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34690/0 (socket says 172.21.15.137:34690) 2024-03-26T14:01:09.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 -- 172.21.15.137:0/3109180238 learned_addr learned my addr 172.21.15.137:0/3109180238 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff62800700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64107510 0x7eff6419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61400700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff64102df0 0x7eff641a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 -- 172.21.15.137:0/3109180238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff64102df0 msgr2=0x7eff641a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff64102df0 0x7eff641a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 -- 172.21.15.137:0/3109180238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64107510 msgr2=0x7eff6419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.053+0000 7eff61e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64107510 0x7eff6419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff61e00700 1 -- 172.21.15.137:0/3109180238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6419b070 con 0x7eff64102440 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff62800700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64107510 0x7eff6419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff61e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff641a0750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7eff54006010 tx=0x7eff54003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:09.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54027070 con 0x7eff64102440 2024-03-26T14:01:09.532 INFO:teuthology.orchestra.run.smithi137.stdout:201863462983 2024-03-26T14:01:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff54004050 con 0x7eff64102440 2024-03-26T14:01:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54022e10 con 0x7eff64102440 2024-03-26T14:01:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.054+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6419b300 con 0x7eff64102440 2024-03-26T14:01:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.055+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a7620 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.056+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff64068e50 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.059+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7eff5402a070 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.060+0000 7eff5aa00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff480a2910 0x7eff480a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.061+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff540d0160 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.061+0000 7eff61400700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff480a2910 0x7eff480a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.062+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7eff54013070 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.062+0000 7eff61400700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff480a2910 0x7eff480a4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff50005ea0 tx=0x7eff50005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.290+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7eff641a77f0 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.291+0000 7eff5aa00700 1 -- 172.21.15.137:0/3109180238 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff5409cec0 con 0x7eff64102440 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff480a2910 msgr2=0x7eff480a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff480a2910 0x7eff480a4dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7eff50005ea0 tx=0x7eff50005e30 comp rx=0 tx=0).stop 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 msgr2=0x7eff641a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff641a0750 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7eff54006010 tx=0x7eff54003680 comp rx=0 tx=0).stop 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 shutdown_connections 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff480a2910 0x7eff480a4dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff64107510 0x7eff6419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff64102440 0x7eff641a0750 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 --2- 172.21.15.137:0/3109180238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff64102df0 0x7eff641a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 >> 172.21.15.137:0/3109180238 conn(0x7eff640792f0 msgr2=0x7eff641019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 shutdown_connections 2024-03-26T14:01:09.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:09.294+0000 7eff68e00700 1 -- 172.21.15.137:0/3109180238 wait complete. 2024-03-26T14:01:10.043 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:10.043 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:09 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:10.043 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:09 smithi137 ceph-mon[31539]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:10.043 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3109180238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T14:01:10.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:10.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:09 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:10.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:09 smithi179 ceph-mon[35472]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:10.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3109180238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T14:01:10.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:10.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:09 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:10.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:09 smithi159 ceph-mon[35501]: pgmap v422: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:10.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3109180238' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-26T14:01:10.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149781 got 111669149788 for osd.3 2024-03-26T14:01:10.365 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:11.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:01:11.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:01:11.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:10 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:11.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:10 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:01:11.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:10 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:01:11.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:10 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:11.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:10 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:01:11.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:10 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:01:11.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:10 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:01:11.277 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776173 got 292057776178 for osd.9 2024-03-26T14:01:11.277 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:11.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574949 got 55834574954 for osd.1 2024-03-26T14:01:11.600 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:11.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 -- 172.21.15.137:0/687538764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 msgr2=0x7f30c8106e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:11.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 --2- 172.21.15.137:0/687538764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c8106e30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f30b40099c0 tx=0x7f30b4009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:11.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 -- 172.21.15.137:0/687538764 shutdown_connections 2024-03-26T14:01:11.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 --2- 172.21.15.137:0/687538764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30c8107510 0x7f30c810f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:11.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 --2- 172.21.15.137:0/687538764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 0x7f30c8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:11.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 --2- 172.21.15.137:0/687538764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c8106e30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:11.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 -- 172.21.15.137:0/687538764 >> 172.21.15.137:0/687538764 conn(0x7f30c80792f0 msgr2=0x7f30c8079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:11.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 -- 172.21.15.137:0/687538764 shutdown_connections 2024-03-26T14:01:11.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.970+0000 7f30cce00700 1 -- 172.21.15.137:0/687538764 wait complete. 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.971+0000 7f30cce00700 1 Processor -- start 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 -- start start 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 0x7f30c81a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30c8107510 0x7f30c819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8112380 con 0x7f30c8102df0 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8112200 con 0x7f30c8102440 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.972+0000 7f30cce00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8112500 con 0x7f30c8107510 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 0x7f30c81a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 0x7f30c81a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:59690/0 (socket says 172.21.15.137:59690) 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5e00700 1 -- 172.21.15.137:0/1584288229 learned_addr learned my addr 172.21.15.137:0/1584288229 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c6800700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30c8107510 0x7f30c819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5400700 1 -- 172.21.15.137:0/1584288229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30c8107510 msgr2=0x7f30c819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5400700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30c8107510 0x7f30c819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5400700 1 -- 172.21.15.137:0/1584288229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 msgr2=0x7f30c81a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5400700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 0x7f30c81a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.973+0000 7f30c5400700 1 -- 172.21.15.137:0/1584288229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c819afe0 con 0x7f30c8102df0 2024-03-26T14:01:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.974+0000 7f30c5400700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c81a0c00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f30b400b580 tx=0x7f30b40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:11.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.976+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4027070 con 0x7f30c8102df0 2024-03-26T14:01:11.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.977+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c819b270 con 0x7f30c8102df0 2024-03-26T14:01:11.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.977+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30b4003680 con 0x7f30c8102df0 2024-03-26T14:01:11.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.977+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4022b30 con 0x7f30c8102df0 2024-03-26T14:01:11.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.977+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c81a7590 con 0x7f30c8102df0 2024-03-26T14:01:11.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.979+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8068e50 con 0x7f30c8102df0 2024-03-26T14:01:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.980+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f30b402a070 con 0x7f30c8102df0 2024-03-26T14:01:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.982+0000 7f30bea00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f30ac0a2910 0x7f30ac0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.983+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f30b40d0100 con 0x7f30c8102df0 2024-03-26T14:01:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.983+0000 7f30c5e00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f30ac0a2910 0x7f30ac0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.984+0000 7f30c5e00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f30ac0a2910 0x7f30ac0a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30c819b9c0 tx=0x7f30b8007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.986+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f30b4013070 con 0x7f30c8102df0 2024-03-26T14:01:12.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 -- 172.21.15.137:0/572937773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de0113360 msgr2=0x7f5de0115750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 --2- 172.21.15.137:0/572937773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de0113360 0x7f5de0115750 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800b0d0 tx=0x7f5dd800b3e0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 -- 172.21.15.137:0/572937773 shutdown_connections 2024-03-26T14:01:12.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 --2- 172.21.15.137:0/572937773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de0113360 0x7f5de0115750 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 --2- 172.21.15.137:0/572937773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de00743c0 0x7f5de00747a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 --2- 172.21.15.137:0/572937773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5de0074ce0 0x7f5de0112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 -- 172.21.15.137:0/572937773 >> 172.21.15.137:0/572937773 conn(0x7f5de00792f0 msgr2=0x7f5de0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 -- 172.21.15.137:0/572937773 shutdown_connections 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.852+0000 7f5de6600700 1 -- 172.21.15.137:0/572937773 wait complete. 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.854+0000 7f5de6600700 1 Processor -- start 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.854+0000 7f5de6600700 1 -- start start 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5de6600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 0x7f5de019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5de6600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5de0074ce0 0x7f5de019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:12.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5de6600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de0113360 0x7f5de01a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5de6600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de01189f0 con 0x7f5de00743c0 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5de6600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0118870 con 0x7f5de0074ce0 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5de6600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de0118b70 con 0x7f5de0113360 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5ddf400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 0x7f5de019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5ddf400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 0x7f5de019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34706/0 (socket says 172.21.15.137:34706) 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5ddf400700 1 -- 172.21.15.137:0/716587575 learned_addr learned my addr 172.21.15.137:0/716587575 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5ddea00700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5de0074ce0 0x7f5de019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.855+0000 7f5ddfe00700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de0113360 0x7f5de01a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5ddf400700 1 -- 172.21.15.137:0/716587575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de0113360 msgr2=0x7f5de01a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5ddf400700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de0113360 0x7f5de01a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5ddf400700 1 -- 172.21.15.137:0/716587575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5de0074ce0 msgr2=0x7f5de019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5ddf400700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5de0074ce0 0x7f5de019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5ddf400700 1 -- 172.21.15.137:0/716587575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de01a3e50 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5ddf400700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 0x7f5de019ee40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000d980 tx=0x7f5dd000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0016070 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5dd0004830 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00054b0 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.856+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de01a4140 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.857+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de01ab9c0 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.858+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de0068e50 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.861+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5dd00049e0 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.862+0000 7f5dd7e00700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5dc40a2910 0x7f5dc40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.862+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5dd00cf120 con 0x7f5de00743c0 2024-03-26T14:01:12.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.862+0000 7f5ddea00700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5dc40a2910 0x7f5dc40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:12.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.863+0000 7f5ddea00700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5dc40a2910 0x7f5dc40a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5de010fc30 tx=0x7f5dcc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:12.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:11.868+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5dd001f020 con 0x7f5de00743c0 2024-03-26T14:01:12.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.115+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5de01abbf0 con 0x7f5de00743c0 2024-03-26T14:01:12.112 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:11 smithi137 ceph-mon[31539]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:12.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.116+0000 7f5dd7e00700 1 -- 172.21.15.137:0/716587575 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5dd00bb030 con 0x7f5de00743c0 2024-03-26T14:01:12.113 INFO:teuthology.orchestra.run.smithi137.stdout:34359738482 2024-03-26T14:01:12.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5dc40a2910 msgr2=0x7f5dc40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5dc40a2910 0x7f5dc40a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5de010fc30 tx=0x7f5dcc009260 comp rx=0 tx=0).stop 2024-03-26T14:01:12.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 msgr2=0x7f5de019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 0x7f5de019ee40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000d980 tx=0x7f5dd000dc90 comp rx=0 tx=0).stop 2024-03-26T14:01:12.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 shutdown_connections 2024-03-26T14:01:12.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5dc40a2910 0x7f5dc40a4dd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de0113360 0x7f5de01a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5de00743c0 0x7f5de019ee40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 --2- 172.21.15.137:0/716587575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5de0074ce0 0x7f5de019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 >> 172.21.15.137:0/716587575 conn(0x7f5de00792f0 msgr2=0x7f5de0112190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:12.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 shutdown_connections 2024-03-26T14:01:12.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.119+0000 7f5de6600700 1 -- 172.21.15.137:0/716587575 wait complete. 2024-03-26T14:01:12.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:11 smithi179 ceph-mon[35472]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:12.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:11 smithi159 ceph-mon[35501]: pgmap v423: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:12.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.243+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f30c81a7760 con 0x7f30c8102df0 2024-03-26T14:01:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.244+0000 7f30bea00700 1 -- 172.21.15.137:0/1584288229 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f30b409ce20 con 0x7f30c8102df0 2024-03-26T14:01:12.241 INFO:teuthology.orchestra.run.smithi137.stdout:167503724622 2024-03-26T14:01:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f30ac0a2910 msgr2=0x7f30ac0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f30ac0a2910 0x7f30ac0a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f30c819b9c0 tx=0x7f30b8007480 comp rx=0 tx=0).stop 2024-03-26T14:01:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 msgr2=0x7f30c81a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c81a0c00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f30b400b580 tx=0x7f30b40057c0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 shutdown_connections 2024-03-26T14:01:12.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f30ac0a2910 0x7f30ac0a4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30c8107510 0x7f30c819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30c8102440 0x7f30c81a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 --2- 172.21.15.137:0/1584288229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f30c8102df0 0x7f30c81a0c00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:12.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.247+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 >> 172.21.15.137:0/1584288229 conn(0x7f30c80792f0 msgr2=0x7f30c81004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:12.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.248+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 shutdown_connections 2024-03-26T14:01:12.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:12.248+0000 7f30cce00700 1 -- 172.21.15.137:0/1584288229 wait complete. 2024-03-26T14:01:12.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462978 got 201863462983 for osd.6 2024-03-26T14:01:12.899 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:13.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/716587575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T14:01:13.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1584288229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T14:01:13.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/716587575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T14:01:13.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1584288229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T14:01:13.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/716587575' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-26T14:01:13.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1584288229' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-26T14:01:14.043 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:13 smithi137 ceph-mon[31539]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:13 smithi179 ceph-mon[35472]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:14.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:13 smithi159 ceph-mon[35501]: pgmap v424: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:14.483 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738476 got 34359738482 for osd.0 2024-03-26T14:01:14.483 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 -- 172.21.15.137:0/2843923847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81044e0 msgr2=0x7fcef81048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 --2- 172.21.15.137:0/2843923847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81044e0 0x7fcef81048c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcee80099c0 tx=0x7fcee8009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 -- 172.21.15.137:0/2843923847 shutdown_connections 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 --2- 172.21.15.137:0/2843923847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef81095c0 0x7fcef8111190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 --2- 172.21.15.137:0/2843923847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81044e0 0x7fcef81048c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 --2- 172.21.15.137:0/2843923847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef8104e90 0x7fcef8108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 -- 172.21.15.137:0/2843923847 >> 172.21.15.137:0/2843923847 conn(0x7fcef80ffdb0 msgr2=0x7fcef81021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 -- 172.21.15.137:0/2843923847 shutdown_connections 2024-03-26T14:01:14.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.793+0000 7fcefd400700 1 -- 172.21.15.137:0/2843923847 wait complete. 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 Processor -- start 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 -- start start 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef81044e0 0x7fcef81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef8104e90 0x7fcef81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 0x7fcef819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef81144c0 con 0x7fcef81095c0 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8114340 con 0x7fcef8104e90 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.794+0000 7fcefd400700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8114640 con 0x7fcef81044e0 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 0x7fcef819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 0x7fcef819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34750/0 (socket says 172.21.15.137:34750) 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 -- 172.21.15.137:0/3052296100 learned_addr learned my addr 172.21.15.137:0/3052296100 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcef7400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef8104e90 0x7fcef81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcef7e00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef81044e0 0x7fcef81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:14.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 -- 172.21.15.137:0/3052296100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef81044e0 msgr2=0x7fcef81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef81044e0 0x7fcef81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 -- 172.21.15.137:0/3052296100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef8104e90 msgr2=0x7fcef81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef8104e90 0x7fcef81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcefca00700 1 -- 172.21.15.137:0/3052296100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef819afa0 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.795+0000 7fcef7400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef8104e90 0x7fcef81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.796+0000 7fcefca00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 0x7fcef819a7c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcef000ada0 tx=0x7fcef000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.796+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0017020 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.796+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcef000ce30 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.796+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0005460 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.796+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef819b290 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.796+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef81a74a0 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.797+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef81059c0 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.799+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcef000c980 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.800+0000 7fcef4c00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcedc0a2910 0x7fcedc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.800+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcef00ce640 con 0x7fcef81095c0 2024-03-26T14:01:14.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.801+0000 7fcef7e00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcedc0a2910 0x7fcedc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.801+0000 7fcef7e00700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcedc0a2910 0x7fcedc0a4dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009750 tx=0x7fcee8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:13.805+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcef0021020 con 0x7fcef81095c0 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.025+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcef802da30 con 0x7fcef81095c0 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.026+0000 7fcef4c00700 1 -- 172.21.15.137:0/3052296100 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcef009b360 con 0x7fcef81095c0 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stdout:77309411429 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcedc0a2910 msgr2=0x7fcedc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:14.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcedc0a2910 0x7fcedc0a4dd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009750 tx=0x7fcee8005fb0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 msgr2=0x7fcef819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 0x7fcef819a7c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fcef000ada0 tx=0x7fcef000c330 comp rx=0 tx=0).stop 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 shutdown_connections 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcedc0a2910 0x7fcedc0a4dd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcef81095c0 0x7fcef819a7c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcef81044e0 0x7fcef81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 --2- 172.21.15.137:0/3052296100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcef8104e90 0x7fcef81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 >> 172.21.15.137:0/3052296100 conn(0x7fcef80ffdb0 msgr2=0x7fcef8102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 shutdown_connections 2024-03-26T14:01:14.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:14.029+0000 7fcefd400700 1 -- 172.21.15.137:0/3052296100 wait complete. 2024-03-26T14:01:15.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:14 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3052296100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T14:01:15.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:14 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3052296100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T14:01:15.199 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:14 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3052296100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-26T14:01:15.809 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411423 got 77309411429 for osd.2 2024-03-26T14:01:15.809 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:16.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:15 smithi137 ceph-mon[31539]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:16.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 -- 172.21.15.137:0/3489488662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941081e0 msgr2=0x7fdb941085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/3489488662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941081e0 0x7fdb941085c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b0d0 tx=0x7fdb8400b3e0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 -- 172.21.15.137:0/3489488662 shutdown_connections 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/3489488662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb9406a7e0 0x7fdb9410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/3489488662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb941081e0 0x7fdb941085c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/3489488662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb94069e40 0x7fdb9406a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 -- 172.21.15.137:0/3489488662 >> 172.21.15.137:0/3489488662 conn(0x7fdb94077fc0 msgr2=0x7fdb940783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.053+0000 7fdb9aa00700 1 -- 172.21.15.137:0/3489488662 shutdown_connections 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.054+0000 7fdb9aa00700 1 -- 172.21.15.137:0/3489488662 wait complete. 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.054+0000 7fdb9aa00700 1 Processor -- start 2024-03-26T14:01:16.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 -- start start 2024-03-26T14:01:16.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 0x7fdb94101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb9406a7e0 0x7fdb941016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb941081e0 0x7fdb94101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941123a0 con 0x7fdb94069e40 2024-03-26T14:01:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94112220 con 0x7fdb9406a7e0 2024-03-26T14:01:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb9aa00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94112520 con 0x7fdb941081e0 2024-03-26T14:01:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 0x7fdb94101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb93e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 0x7fdb94101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34760/0 (socket says 172.21.15.137:34760) 2024-03-26T14:01:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.055+0000 7fdb93e00700 1 -- 172.21.15.137:0/2556304791 learned_addr learned my addr 172.21.15.137:0/2556304791 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:16.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb8b400700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb9406a7e0 0x7fdb941016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb98a00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb941081e0 0x7fdb94101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb93e00700 1 -- 172.21.15.137:0/2556304791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb941081e0 msgr2=0x7fdb94101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb93e00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb941081e0 0x7fdb94101be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb93e00700 1 -- 172.21.15.137:0/2556304791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb9406a7e0 msgr2=0x7fdb941016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb93e00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb9406a7e0 0x7fdb941016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb93e00700 1 -- 172.21.15.137:0/2556304791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb941a6fb0 con 0x7fdb94069e40 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb93e00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 0x7fdb94101160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009fd0 tx=0x7fdb84007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.056+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84021020 con 0x7fdb94069e40 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.057+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb84004c50 con 0x7fdb94069e40 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.057+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb941a7240 con 0x7fdb94069e40 2024-03-26T14:01:16.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.057+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84005340 con 0x7fdb94069e40 2024-03-26T14:01:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.057+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb941a76f0 con 0x7fdb94069e40 2024-03-26T14:01:16.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.058+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb94068e50 con 0x7fdb94069e40 2024-03-26T14:01:16.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.060+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdb84024020 con 0x7fdb94069e40 2024-03-26T14:01:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.061+0000 7fdb91600700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb780a2890 0x7fdb780a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.061+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdb8400f020 con 0x7fdb94069e40 2024-03-26T14:01:16.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.062+0000 7fdb8b400700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb780a2890 0x7fdb780a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.062+0000 7fdb8b400700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb780a2890 0x7fdb780a4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c005950 tx=0x7fdb7c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:16.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.066+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb8409ce90 con 0x7fdb94069e40 2024-03-26T14:01:16.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:15 smithi159 ceph-mon[35501]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:16.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.293+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fdb9402da30 con 0x7fdb94069e40 2024-03-26T14:01:16.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.294+0000 7fdb91600700 1 -- 172.21.15.137:0/2556304791 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fdb8409c790 con 0x7fdb94069e40 2024-03-26T14:01:16.291 INFO:teuthology.orchestra.run.smithi137.stdout:347892351015 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.296+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb780a2890 msgr2=0x7fdb780a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb780a2890 0x7fdb780a4d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c005950 tx=0x7fdb7c0058e0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 msgr2=0x7fdb94101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 0x7fdb94101160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009fd0 tx=0x7fdb84007ad0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 shutdown_connections 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb780a2890 0x7fdb780a4d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb941081e0 0x7fdb94101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb94069e40 0x7fdb94101160 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 --2- 172.21.15.137:0/2556304791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb9406a7e0 0x7fdb941016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 >> 172.21.15.137:0/2556304791 conn(0x7fdb94077fc0 msgr2=0x7fdb940ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:16.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 shutdown_connections 2024-03-26T14:01:16.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.297+0000 7fdb9aa00700 1 -- 172.21.15.137:0/2556304791 wait complete. 2024-03-26T14:01:16.352 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724616 got 167503724622 for osd.5 2024-03-26T14:01:16.352 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:16.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:15 smithi179 ceph-mon[35472]: pgmap v425: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.580+0000 7fdf57600700 1 -- 172.21.15.137:0/3847840788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf50109690 msgr2=0x7fdf50111260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.580+0000 7fdf57600700 1 --2- 172.21.15.137:0/3847840788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf50109690 0x7fdf50111260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800b0d0 tx=0x7fdf4800b3e0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 -- 172.21.15.137:0/3847840788 shutdown_connections 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 --2- 172.21.15.137:0/3847840788 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf50109690 0x7fdf50111260 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 --2- 172.21.15.137:0/3847840788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf501045b0 0x7fdf50104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 --2- 172.21.15.137:0/3847840788 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf50104f60 0x7fdf50108fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 -- 172.21.15.137:0/3847840788 >> 172.21.15.137:0/3847840788 conn(0x7fdf500ffe60 msgr2=0x7fdf50102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 -- 172.21.15.137:0/3847840788 shutdown_connections 2024-03-26T14:01:16.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.581+0000 7fdf57600700 1 -- 172.21.15.137:0/3847840788 wait complete. 2024-03-26T14:01:16.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.582+0000 7fdf57600700 1 Processor -- start 2024-03-26T14:01:16.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.582+0000 7fdf57600700 1 -- start start 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.582+0000 7fdf57600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 0x7fdf501a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.582+0000 7fdf57600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf50104f60 0x7fdf501a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf57600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf50109690 0x7fdf5019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf57600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf501145a0 con 0x7fdf501045b0 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf57600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50114420 con 0x7fdf50109690 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf57600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50114720 con 0x7fdf50104f60 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 0x7fdf501a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 0x7fdf501a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:34782/0 (socket says 172.21.15.137:34782) 2024-03-26T14:01:16.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 -- 172.21.15.137:0/177582487 learned_addr learned my addr 172.21.15.137:0/177582487 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf55600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf50109690 0x7fdf5019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf4fe00700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf50104f60 0x7fdf501a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 -- 172.21.15.137:0/177582487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf50104f60 msgr2=0x7fdf501a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf50104f60 0x7fdf501a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 -- 172.21.15.137:0/177582487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf50109690 msgr2=0x7fdf5019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf50109690 0x7fdf5019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.583+0000 7fdf54c00700 1 -- 172.21.15.137:0/177582487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5019b0a0 con 0x7fdf501045b0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.584+0000 7fdf54c00700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 0x7fdf501a0730 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e9a0 tx=0x7fdf4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.584+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400cbd0 con 0x7fdf501045b0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.584+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf4400cd70 con 0x7fdf501045b0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.584+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5019b390 con 0x7fdf501045b0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.584+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44018680 con 0x7fdf501045b0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.584+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf501a7600 con 0x7fdf501045b0 2024-03-26T14:01:16.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.585+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50068e50 con 0x7fdf501045b0 2024-03-26T14:01:16.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.587+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdf44010430 con 0x7fdf501045b0 2024-03-26T14:01:16.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.588+0000 7fdf4d600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf380a2890 0x7fdf380a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:16.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.589+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdf440cf080 con 0x7fdf501045b0 2024-03-26T14:01:16.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.589+0000 7fdf4fe00700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf380a2890 0x7fdf380a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:16.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.590+0000 7fdf4fe00700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf380a2890 0x7fdf380a4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf5010ac50 tx=0x7fdf3c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:16.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.593+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf4409bd50 con 0x7fdf501045b0 2024-03-26T14:01:16.730 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.821+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fdf5002da30 con 0x7fdf501045b0 2024-03-26T14:01:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.822+0000 7fdf4d600700 1 -- 172.21.15.137:0/177582487 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf4409b650 con 0x7fdf501045b0 2024-03-26T14:01:16.819 INFO:teuthology.orchestra.run.smithi137.stdout:257698037819 2024-03-26T14:01:16.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.825+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf380a2890 msgr2=0x7fdf380a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.825+0000 7fdf57600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf380a2890 0x7fdf380a4d50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf5010ac50 tx=0x7fdf3c00b590 comp rx=0 tx=0).stop 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.825+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 msgr2=0x7fdf501a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.825+0000 7fdf57600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 0x7fdf501a0730 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400e9a0 tx=0x7fdf4400ecb0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.825+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 shutdown_connections 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf380a2890 0x7fdf380a4d50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf50109690 0x7fdf5019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf501045b0 0x7fdf501a0730 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 --2- 172.21.15.137:0/177582487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf50104f60 0x7fdf501a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 >> 172.21.15.137:0/177582487 conn(0x7fdf500ffe60 msgr2=0x7fdf50102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 shutdown_connections 2024-03-26T14:01:16.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:16.826+0000 7fdf57600700 1 -- 172.21.15.137:0/177582487 wait complete. 2024-03-26T14:01:17.182 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:16 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2556304791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T14:01:17.182 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:16 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/177582487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T14:01:17.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:16 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2556304791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T14:01:17.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:16 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/177582487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T14:01:17.229 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351008 got 347892351015 for osd.11 2024-03-26T14:01:17.230 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:17.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:16 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2556304791' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-26T14:01:17.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:16 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/177582487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-26T14:01:18.141 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037812 got 257698037819 for osd.8 2024-03-26T14:01:18.141 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:18.200 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:17 smithi159 ceph-mon[35501]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:18.222 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:17 smithi137 ceph-mon[31539]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:18.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:17 smithi179 ceph-mon[35472]: pgmap v426: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:20.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 -- 172.21.15.137:0/1883516109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c109160 msgr2=0x7f174c115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:20.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 --2- 172.21.15.137:0/1883516109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c109160 0x7f174c115670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f174400b0a0 tx=0x7f174400b3b0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 -- 172.21.15.137:0/1883516109 shutdown_connections 2024-03-26T14:01:20.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 --2- 172.21.15.137:0/1883516109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c109160 0x7f174c115670 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 --2- 172.21.15.137:0/1883516109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f174c10e7a0 0x7f174c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 --2- 172.21.15.137:0/1883516109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f174c1087a0 0x7f174c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 -- 172.21.15.137:0/1883516109 >> 172.21.15.137:0/1883516109 conn(0x7f174c075960 msgr2=0x7f174c077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:20.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 -- 172.21.15.137:0/1883516109 shutdown_connections 2024-03-26T14:01:20.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.091+0000 7f1751600700 1 -- 172.21.15.137:0/1883516109 wait complete. 2024-03-26T14:01:20.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.092+0000 7f1751600700 1 Processor -- start 2024-03-26T14:01:20.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 -- start start 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 0x7f174c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f174c109160 0x7f174c19f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f174c10e7a0 0x7f174c1a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c118a30 con 0x7f174c1087a0 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c1188b0 con 0x7f174c109160 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f1751600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c118bb0 con 0x7f174c10e7a0 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f174aa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 0x7f174c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f174aa00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 0x7f174c19ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48854/0 (socket says 172.21.15.137:48854) 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.093+0000 7f174aa00700 1 -- 172.21.15.137:0/3912727702 learned_addr learned my addr 172.21.15.137:0/3912727702 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f1743e00700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f174c109160 0x7f174c19f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:20.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174b400700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f174c10e7a0 0x7f174c1a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174aa00700 1 -- 172.21.15.137:0/3912727702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f174c10e7a0 msgr2=0x7f174c1a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174aa00700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f174c10e7a0 0x7f174c1a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174aa00700 1 -- 172.21.15.137:0/3912727702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f174c109160 msgr2=0x7f174c19f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174aa00700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f174c109160 0x7f174c19f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174aa00700 1 -- 172.21.15.137:0/3912727702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c1a3e10 con 0x7f174c1087a0 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.094+0000 7f174aa00700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 0x7f174c19ee00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c920 tx=0x7f173c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.095+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c007b00 con 0x7f174c1087a0 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.095+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f173c004ce0 con 0x7f174c1087a0 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.095+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c005550 con 0x7f174c1087a0 2024-03-26T14:01:20.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.095+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c1a4100 con 0x7f174c1087a0 2024-03-26T14:01:20.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.095+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c1ab980 con 0x7f174c1087a0 2024-03-26T14:01:20.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.096+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c068e50 con 0x7f174c1087a0 2024-03-26T14:01:20.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.098+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f173c004750 con 0x7f174c1087a0 2024-03-26T14:01:20.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.099+0000 7f1743400700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17300a2990 0x7f17300a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:20.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.099+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f173c0cec80 con 0x7f174c1087a0 2024-03-26T14:01:20.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.100+0000 7f1743e00700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17300a2990 0x7f17300a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:20.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.100+0000 7f1743e00700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17300a2990 0x7f17300a4e50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f174c10ddc0 tx=0x7f173400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:20.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.104+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f173c014030 con 0x7f174c1087a0 2024-03-26T14:01:20.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.328+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f174c02da30 con 0x7f174c1087a0 2024-03-26T14:01:20.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.329+0000 7f1743400700 1 -- 172.21.15.137:0/3912727702 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f173c09b910 con 0x7f174c1087a0 2024-03-26T14:01:20.326 INFO:teuthology.orchestra.run.smithi137.stdout:146028888153 2024-03-26T14:01:20.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.331+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17300a2990 msgr2=0x7f17300a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:20.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.331+0000 7f1751600700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17300a2990 0x7f17300a4e50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f174c10ddc0 tx=0x7f173400b410 comp rx=0 tx=0).stop 2024-03-26T14:01:20.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.331+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 msgr2=0x7f174c19ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.331+0000 7f1751600700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 0x7f174c19ee00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f173c00c920 tx=0x7f173c00cc30 comp rx=0 tx=0).stop 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 shutdown_connections 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17300a2990 0x7f17300a4e50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f174c10e7a0 0x7f174c1a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f174c1087a0 0x7f174c19ee00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 --2- 172.21.15.137:0/3912727702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f174c109160 0x7f174c19f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 >> 172.21.15.137:0/3912727702 conn(0x7f174c075960 msgr2=0x7f174c075ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 shutdown_connections 2024-03-26T14:01:20.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:20.332+0000 7f1751600700 1 -- 172.21.15.137:0/3912727702 wait complete. 2024-03-26T14:01:20.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:20 smithi159 ceph-mon[35501]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:20.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:20 smithi137 ceph-mon[31539]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:20.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:20 smithi179 ceph-mon[35472]: pgmap v427: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:21.150 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888147 got 146028888153 for osd.4 2024-03-26T14:01:21.151 DEBUG:teuthology.parallel:result is None 2024-03-26T14:01:21.151 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-26T14:01:21.151 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T14:01:21.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:21 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3912727702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T14:01:21.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:21 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3912727702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T14:01:21.591 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:21.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:21 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3912727702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-26T14:01:22.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:22 smithi159 ceph-mon[35501]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:22.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:22 smithi137 ceph-mon[31539]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:22.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:22 smithi179 ceph-mon[35472]: pgmap v428: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 -- 172.21.15.137:0/64493671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 msgr2=0x7ff23c110f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 --2- 172.21.15.137:0/64493671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c110f10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff23400b0a0 tx=0x7ff23400b3b0 comp rx=0 tx=0).stop 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 -- 172.21.15.137:0/64493671 shutdown_connections 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 --2- 172.21.15.137:0/64493671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c110f10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 --2- 172.21.15.137:0/64493671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff23c1030d0 0x7ff23c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 --2- 172.21.15.137:0/64493671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 0x7ff23c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 -- 172.21.15.137:0/64493671 >> 172.21.15.137:0/64493671 conn(0x7ff23c077690 msgr2=0x7ff23c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 -- 172.21.15.137:0/64493671 shutdown_connections 2024-03-26T14:01:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.800+0000 7ff243a00700 1 -- 172.21.15.137:0/64493671 wait complete. 2024-03-26T14:01:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.801+0000 7ff243a00700 1 Processor -- start 2024-03-26T14:01:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 -- start start 2024-03-26T14:01:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 0x7ff23c1a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff23c1030d0 0x7ff23c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c114610 con 0x7ff23c101310 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c114490 con 0x7ff23c1030d0 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff243a00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23c114790 con 0x7ff23c100780 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff23be00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff23be00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c1a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48876/0 (socket says 172.21.15.137:48876) 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.802+0000 7ff23be00700 1 -- 172.21.15.137:0/2998900997 learned_addr learned my addr 172.21.15.137:0/2998900997 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff241000700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 0x7ff23c1a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff241a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff23c1030d0 0x7ff23c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff23be00700 1 -- 172.21.15.137:0/2998900997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 msgr2=0x7ff23c1a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff23be00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 0x7ff23c1a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff23be00700 1 -- 172.21.15.137:0/2998900997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff23c1030d0 msgr2=0x7ff23c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff23be00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff23c1030d0 0x7ff23c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff23be00700 1 -- 172.21.15.137:0/2998900997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c19b070 con 0x7ff23c101310 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff241000700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 0x7ff23c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.803+0000 7ff23be00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c1a0c90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff22800d980 tx=0x7ff22800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:23.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.804+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228017070 con 0x7ff23c101310 2024-03-26T14:01:23.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.804+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff228005020 con 0x7ff23c101310 2024-03-26T14:01:23.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.804+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228009b50 con 0x7ff23c101310 2024-03-26T14:01:23.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.804+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23c19b360 con 0x7ff23c101310 2024-03-26T14:01:23.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.804+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23c1a7620 con 0x7ff23c101310 2024-03-26T14:01:23.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.805+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23c101790 con 0x7ff23c101310 2024-03-26T14:01:23.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.807+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff228009cf0 con 0x7ff23c101310 2024-03-26T14:01:23.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.809+0000 7ff239600700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2240a2890 0x7ff2240a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:23.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.809+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff2280ced80 con 0x7ff23c101310 2024-03-26T14:01:23.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.810+0000 7ff241000700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2240a2890 0x7ff2240a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:23.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.810+0000 7ff241000700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2240a2890 0x7ff2240a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff230005e10 tx=0x7ff230005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:23.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:23.814+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff228020020 con 0x7ff23c101310 2024-03-26T14:01:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.030+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff23c0625d0 con 0x7ff2240a2890 2024-03-26T14:01:24.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.064+0000 7ff239600700 1 -- 172.21.15.137:0/2998900997 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340760 (secure 0 0 0) 0x7ff23c0625d0 con 0x7ff2240a2890 2024-03-26T14:01:24.070 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:24.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2240a2890 msgr2=0x7ff2240a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:24.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2240a2890 0x7ff2240a4d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff230005e10 tx=0x7ff230005d60 comp rx=0 tx=0).stop 2024-03-26T14:01:24.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 msgr2=0x7ff23c1a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:24.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c1a0c90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff22800d980 tx=0x7ff22800dc90 comp rx=0 tx=0).stop 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 shutdown_connections 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2240a2890 0x7ff2240a4d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff23c1030d0 0x7ff23c19a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff23c100780 0x7ff23c1a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 --2- 172.21.15.137:0/2998900997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff23c101310 0x7ff23c1a0c90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 >> 172.21.15.137:0/2998900997 conn(0x7ff23c077690 msgr2=0x7ff23c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 shutdown_connections 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:24.073+0000 7ff243a00700 1 -- 172.21.15.137:0/2998900997 wait complete. 2024-03-26T14:01:24.074 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:01:24.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:24 smithi159 ceph-mon[35501]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:24.490 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:24 smithi137 ceph-mon[31539]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:24.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:24 smithi179 ceph-mon[35472]: pgmap v429: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:24.900 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-03-26T14:01:23.126408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662587,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":701,"num_read_kb":474,"num_write":353,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361788,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580036,"statfs":{"total":1151940427776,"available":1151569956864,"internally_reserved":0,"allocated":12861440,"data_stored":8238383,"data_compressed":427101,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006061"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222471+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T14:00:55.222471+0000","last_peered":"2024-03-26T14:00:55.222471+0000","last_clean":"2024-03-26T14:00:55.222471+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T14:00:55.222471+0000","last_undegraded":"2024-03-26T14:00:55.222471+0000","last_fullsized":"2024-03-26T14:00:55.222471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212482+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T14:00:55.212482+0000","last_peered":"2024-03-26T14:00:55.212482+0000","last_clean":"2024-03-26T14:00:55.212482+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T14:00:55.212482+0000","last_undegraded":"2024-03-26T14:00:55.212482+0000","last_fullsized":"2024-03-26T14:00:55.212482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210979+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T14:00:55.210979+0000","last_peered":"2024-03-26T14:00:55.210979+0000","last_clean":"2024-03-26T14:00:55.210979+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T14:00:55.210979+0000","last_undegraded":"2024-03-26T14:00:55.210979+0000","last_fullsized":"2024-03-26T14:00:55.210979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209979+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T14:00:55.209979+0000","last_peered":"2024-03-26T14:00:55.209979+0000","last_clean":"2024-03-26T14:00:55.209979+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T14:00:55.209979+0000","last_undegraded":"2024-03-26T14:00:55.209979+0000","last_fullsized":"2024-03-26T14:00:55.209979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221738+0000","last_change":"2024-03-26T14:00:55.221738+0000","last_active":"2024-03-26T14:00:55.221738+0000","last_peered":"2024-03-26T14:00:55.221738+0000","last_clean":"2024-03-26T14:00:55.221738+0000","last_became_active":"2024-03-26T14:00:55.220626+0000","last_became_peered":"2024-03-26T14:00:55.220626+0000","last_unstale":"2024-03-26T14:00:55.221738+0000","last_undegraded":"2024-03-26T14:00:55.221738+0000","last_fullsized":"2024-03-26T14:00:55.221738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:50.208896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217644+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T14:00:55.217644+0000","last_peered":"2024-03-26T14:00:55.217644+0000","last_clean":"2024-03-26T14:00:55.217644+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T14:00:55.217644+0000","last_undegraded":"2024-03-26T14:00:55.217644+0000","last_fullsized":"2024-03-26T14:00:55.217644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213715+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T14:00:55.213715+0000","last_peered":"2024-03-26T14:00:55.213715+0000","last_clean":"2024-03-26T14:00:55.213715+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T14:00:55.213715+0000","last_undegraded":"2024-03-26T14:00:55.213715+0000","last_fullsized":"2024-03-26T14:00:55.213715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206692+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T14:00:55.206692+0000","last_peered":"2024-03-26T14:00:55.206692+0000","last_clean":"2024-03-26T14:00:55.206692+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T14:00:55.206692+0000","last_undegraded":"2024-03-26T14:00:55.206692+0000","last_fullsized":"2024-03-26T14:00:55.206692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434453+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T14:00:55.434453+0000","last_peered":"2024-03-26T14:00:55.434453+0000","last_clean":"2024-03-26T14:00:55.434453+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T14:00:55.434453+0000","last_undegraded":"2024-03-26T14:00:55.434453+0000","last_fullsized":"2024-03-26T14:00:55.434453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218316+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T14:00:55.218316+0000","last_peered":"2024-03-26T14:00:55.218316+0000","last_clean":"2024-03-26T14:00:55.218316+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T14:00:55.218316+0000","last_undegraded":"2024-03-26T14:00:55.218316+0000","last_fullsized":"2024-03-26T14:00:55.218316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214164+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T14:00:55.214164+0000","last_peered":"2024-03-26T14:00:55.214164+0000","last_clean":"2024-03-26T14:00:55.214164+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T14:00:55.214164+0000","last_undegraded":"2024-03-26T14:00:55.214164+0000","last_fullsized":"2024-03-26T14:00:55.214164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209734+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T14:00:55.209734+0000","last_peered":"2024-03-26T14:00:55.209734+0000","last_clean":"2024-03-26T14:00:55.209734+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T14:00:55.209734+0000","last_undegraded":"2024-03-26T14:00:55.209734+0000","last_fullsized":"2024-03-26T14:00:55.209734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217206+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T14:00:55.217206+0000","last_peered":"2024-03-26T14:00:55.217206+0000","last_clean":"2024-03-26T14:00:55.217206+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T14:00:55.217206+0000","last_undegraded":"2024-03-26T14:00:55.217206+0000","last_fullsized":"2024-03-26T14:00:55.217206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210320+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T14:00:55.210320+0000","last_peered":"2024-03-26T14:00:55.210320+0000","last_clean":"2024-03-26T14:00:55.210320+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T14:00:55.210320+0000","last_undegraded":"2024-03-26T14:00:55.210320+0000","last_fullsized":"2024-03-26T14:00:55.210320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210676+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T14:00:55.210676+0000","last_peered":"2024-03-26T14:00:55.210676+0000","last_clean":"2024-03-26T14:00:55.210676+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T14:00:55.210676+0000","last_undegraded":"2024-03-26T14:00:55.210676+0000","last_fullsized":"2024-03-26T14:00:55.210676+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433581+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T14:00:55.433581+0000","last_peered":"2024-03-26T14:00:55.433581+0000","last_clean":"2024-03-26T14:00:55.433581+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T14:00:55.433581+0000","last_undegraded":"2024-03-26T14:00:55.433581+0000","last_fullsized":"2024-03-26T14:00:55.433581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222285+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T14:00:55.222285+0000","last_peered":"2024-03-26T14:00:55.222285+0000","last_clean":"2024-03-26T14:00:55.222285+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T14:00:55.222285+0000","last_undegraded":"2024-03-26T14:00:55.222285+0000","last_fullsized":"2024-03-26T14:00:55.222285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212135+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T14:00:55.212135+0000","last_peered":"2024-03-26T14:00:55.212135+0000","last_clean":"2024-03-26T14:00:55.212135+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T14:00:55.212135+0000","last_undegraded":"2024-03-26T14:00:55.212135+0000","last_fullsized":"2024-03-26T14:00:55.212135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215769+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T14:00:55.215769+0000","last_peered":"2024-03-26T14:00:55.215769+0000","last_clean":"2024-03-26T14:00:55.215769+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T14:00:55.215769+0000","last_undegraded":"2024-03-26T14:00:55.215769+0000","last_fullsized":"2024-03-26T14:00:55.215769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212573+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T14:00:55.212573+0000","last_peered":"2024-03-26T14:00:55.212573+0000","last_clean":"2024-03-26T14:00:55.212573+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T14:00:55.212573+0000","last_undegraded":"2024-03-26T14:00:55.212573+0000","last_fullsized":"2024-03-26T14:00:55.212573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212930+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T14:00:55.212930+0000","last_peered":"2024-03-26T14:00:55.212930+0000","last_clean":"2024-03-26T14:00:55.212930+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T14:00:55.212930+0000","last_undegraded":"2024-03-26T14:00:55.212930+0000","last_fullsized":"2024-03-26T14:00:55.212930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207749+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T14:00:55.207749+0000","last_peered":"2024-03-26T14:00:55.207749+0000","last_clean":"2024-03-26T14:00:55.207749+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T14:00:55.207749+0000","last_undegraded":"2024-03-26T14:00:55.207749+0000","last_fullsized":"2024-03-26T14:00:55.207749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209121+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T14:00:55.209121+0000","last_peered":"2024-03-26T14:00:55.209121+0000","last_clean":"2024-03-26T14:00:55.209121+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T14:00:55.209121+0000","last_undegraded":"2024-03-26T14:00:55.209121+0000","last_fullsized":"2024-03-26T14:00:55.209121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207907+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T14:00:55.207907+0000","last_peered":"2024-03-26T14:00:55.207907+0000","last_clean":"2024-03-26T14:00:55.207907+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T14:00:55.207907+0000","last_undegraded":"2024-03-26T14:00:55.207907+0000","last_fullsized":"2024-03-26T14:00:55.207907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211033+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T14:00:55.211033+0000","last_peered":"2024-03-26T14:00:55.211033+0000","last_clean":"2024-03-26T14:00:55.211033+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T14:00:55.211033+0000","last_undegraded":"2024-03-26T14:00:55.211033+0000","last_fullsized":"2024-03-26T14:00:55.211033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209396+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T14:00:55.209396+0000","last_peered":"2024-03-26T14:00:55.209396+0000","last_clean":"2024-03-26T14:00:55.209396+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T14:00:55.209396+0000","last_undegraded":"2024-03-26T14:00:55.209396+0000","last_fullsized":"2024-03-26T14:00:55.209396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213402+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T14:00:55.213402+0000","last_peered":"2024-03-26T14:00:55.213402+0000","last_clean":"2024-03-26T14:00:55.213402+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T14:00:55.213402+0000","last_undegraded":"2024-03-26T14:00:55.213402+0000","last_fullsized":"2024-03-26T14:00:55.213402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433778+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T14:00:55.433778+0000","last_peered":"2024-03-26T14:00:55.433778+0000","last_clean":"2024-03-26T14:00:55.433778+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T14:00:55.433778+0000","last_undegraded":"2024-03-26T14:00:55.433778+0000","last_fullsized":"2024-03-26T14:00:55.433778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209004+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T14:00:55.209004+0000","last_peered":"2024-03-26T14:00:55.209004+0000","last_clean":"2024-03-26T14:00:55.209004+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T14:00:55.209004+0000","last_undegraded":"2024-03-26T14:00:55.209004+0000","last_fullsized":"2024-03-26T14:00:55.209004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214537+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T14:00:55.214537+0000","last_peered":"2024-03-26T14:00:55.214537+0000","last_clean":"2024-03-26T14:00:55.214537+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T14:00:55.214537+0000","last_undegraded":"2024-03-26T14:00:55.214537+0000","last_fullsized":"2024-03-26T14:00:55.214537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228324+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T14:00:55.228324+0000","last_peered":"2024-03-26T14:00:55.228324+0000","last_clean":"2024-03-26T14:00:55.228324+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T14:00:55.228324+0000","last_undegraded":"2024-03-26T14:00:55.228324+0000","last_fullsized":"2024-03-26T14:00:55.228324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214576+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T14:00:55.214576+0000","last_peered":"2024-03-26T14:00:55.214576+0000","last_clean":"2024-03-26T14:00:55.214576+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T14:00:55.214576+0000","last_undegraded":"2024-03-26T14:00:55.214576+0000","last_fullsized":"2024-03-26T14:00:55.214576+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":130,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:20.494321+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T14:01:20.494321+0000","last_peered":"2024-03-26T14:01:20.494321+0000","last_clean":"2024-03-26T14:01:20.494321+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T14:01:20.494321+0000","last_undegraded":"2024-03-26T14:01:20.494321+0000","last_fullsized":"2024-03-26T14:01:20.494321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214675+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T14:00:55.214675+0000","last_peered":"2024-03-26T14:00:55.214675+0000","last_clean":"2024-03-26T14:00:55.214675+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T14:00:55.214675+0000","last_undegraded":"2024-03-26T14:00:55.214675+0000","last_fullsized":"2024-03-26T14:00:55.214675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216200+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T14:00:55.216200+0000","last_peered":"2024-03-26T14:00:55.216200+0000","last_clean":"2024-03-26T14:00:55.216200+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T14:00:55.216200+0000","last_undegraded":"2024-03-26T14:00:55.216200+0000","last_fullsized":"2024-03-26T14:00:55.216200+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217548+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T14:00:55.217548+0000","last_peered":"2024-03-26T14:00:55.217548+0000","last_clean":"2024-03-26T14:00:55.217548+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T14:00:55.217548+0000","last_undegraded":"2024-03-26T14:00:55.217548+0000","last_fullsized":"2024-03-26T14:00:55.217548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227859+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T14:00:55.227859+0000","last_peered":"2024-03-26T14:00:55.227859+0000","last_clean":"2024-03-26T14:00:55.227859+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T14:00:55.227859+0000","last_undegraded":"2024-03-26T14:00:55.227859+0000","last_fullsized":"2024-03-26T14:00:55.227859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222020+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T14:00:55.222020+0000","last_peered":"2024-03-26T14:00:55.222020+0000","last_clean":"2024-03-26T14:00:55.222020+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T14:00:55.222020+0000","last_undegraded":"2024-03-26T14:00:55.222020+0000","last_fullsized":"2024-03-26T14:00:55.222020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217496+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T14:00:55.217496+0000","last_peered":"2024-03-26T14:00:55.217496+0000","last_clean":"2024-03-26T14:00:55.217496+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T14:00:55.217496+0000","last_undegraded":"2024-03-26T14:00:55.217496+0000","last_fullsized":"2024-03-26T14:00:55.217496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210312+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T14:00:55.210312+0000","last_peered":"2024-03-26T14:00:55.210312+0000","last_clean":"2024-03-26T14:00:55.210312+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T14:00:55.210312+0000","last_undegraded":"2024-03-26T14:00:55.210312+0000","last_fullsized":"2024-03-26T14:00:55.210312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212373+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T14:00:55.212373+0000","last_peered":"2024-03-26T14:00:55.212373+0000","last_clean":"2024-03-26T14:00:55.212373+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T14:00:55.212373+0000","last_undegraded":"2024-03-26T14:00:55.212373+0000","last_fullsized":"2024-03-26T14:00:55.212373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226392+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T14:00:55.226392+0000","last_peered":"2024-03-26T14:00:55.226392+0000","last_clean":"2024-03-26T14:00:55.226392+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T14:00:55.226392+0000","last_undegraded":"2024-03-26T14:00:55.226392+0000","last_fullsized":"2024-03-26T14:00:55.226392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211021+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T14:00:55.211021+0000","last_peered":"2024-03-26T14:00:55.211021+0000","last_clean":"2024-03-26T14:00:55.211021+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T14:00:55.211021+0000","last_undegraded":"2024-03-26T14:00:55.211021+0000","last_fullsized":"2024-03-26T14:00:55.211021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206684+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T14:00:55.206684+0000","last_peered":"2024-03-26T14:00:55.206684+0000","last_clean":"2024-03-26T14:00:55.206684+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T14:00:55.206684+0000","last_undegraded":"2024-03-26T14:00:55.206684+0000","last_fullsized":"2024-03-26T14:00:55.206684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":125,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:19.490924+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T14:01:19.490924+0000","last_peered":"2024-03-26T14:01:19.490924+0000","last_clean":"2024-03-26T14:01:19.490924+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T14:01:19.490924+0000","last_undegraded":"2024-03-26T14:01:19.490924+0000","last_fullsized":"2024-03-26T14:01:19.490924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":91,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434076+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T14:00:55.434076+0000","last_peered":"2024-03-26T14:00:55.434076+0000","last_clean":"2024-03-26T14:00:55.434076+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T14:00:55.434076+0000","last_undegraded":"2024-03-26T14:00:55.434076+0000","last_fullsized":"2024-03-26T14:00:55.434076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211798+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T14:00:55.211798+0000","last_peered":"2024-03-26T14:00:55.211798+0000","last_clean":"2024-03-26T14:00:55.211798+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T14:00:55.211798+0000","last_undegraded":"2024-03-26T14:00:55.211798+0000","last_fullsized":"2024-03-26T14:00:55.211798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213925+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T14:00:55.213925+0000","last_peered":"2024-03-26T14:00:55.213925+0000","last_clean":"2024-03-26T14:00:55.213925+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T14:00:55.213925+0000","last_undegraded":"2024-03-26T14:00:55.213925+0000","last_fullsized":"2024-03-26T14:00:55.213925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224154+0000","last_change":"2024-03-26T14:00:55.224154+0000","last_active":"2024-03-26T14:00:55.224154+0000","last_peered":"2024-03-26T14:00:55.224154+0000","last_clean":"2024-03-26T14:00:55.224154+0000","last_became_active":"2024-03-26T14:00:55.223674+0000","last_became_peered":"2024-03-26T14:00:55.223674+0000","last_unstale":"2024-03-26T14:00:55.224154+0000","last_undegraded":"2024-03-26T14:00:55.224154+0000","last_fullsized":"2024-03-26T14:00:55.224154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:48:31.146317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209641+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T14:00:55.209641+0000","last_peered":"2024-03-26T14:00:55.209641+0000","last_clean":"2024-03-26T14:00:55.209641+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T14:00:55.209641+0000","last_undegraded":"2024-03-26T14:00:55.209641+0000","last_fullsized":"2024-03-26T14:00:55.209641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228031+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T14:00:55.228031+0000","last_peered":"2024-03-26T14:00:55.228031+0000","last_clean":"2024-03-26T14:00:55.228031+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T14:00:55.228031+0000","last_undegraded":"2024-03-26T14:00:55.228031+0000","last_fullsized":"2024-03-26T14:00:55.228031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433934+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T14:00:55.433934+0000","last_peered":"2024-03-26T14:00:55.433934+0000","last_clean":"2024-03-26T14:00:55.433934+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T14:00:55.433934+0000","last_undegraded":"2024-03-26T14:00:55.433934+0000","last_fullsized":"2024-03-26T14:00:55.433934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210744+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T14:00:55.210744+0000","last_peered":"2024-03-26T14:00:55.210744+0000","last_clean":"2024-03-26T14:00:55.210744+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T14:00:55.210744+0000","last_undegraded":"2024-03-26T14:00:55.210744+0000","last_fullsized":"2024-03-26T14:00:55.210744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":260,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:20.477438+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T14:01:20.477438+0000","last_peered":"2024-03-26T14:01:20.477438+0000","last_clean":"2024-03-26T14:01:20.477438+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T14:01:20.477438+0000","last_undegraded":"2024-03-26T14:01:20.477438+0000","last_fullsized":"2024-03-26T14:01:20.477438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67843,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":191,"num_read_kb":162,"num_write":39,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217237+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T14:00:55.217237+0000","last_peered":"2024-03-26T14:00:55.217237+0000","last_clean":"2024-03-26T14:00:55.217237+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T14:00:55.217237+0000","last_undegraded":"2024-03-26T14:00:55.217237+0000","last_fullsized":"2024-03-26T14:00:55.217237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.229178+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T14:00:55.229178+0000","last_peered":"2024-03-26T14:00:55.229178+0000","last_clean":"2024-03-26T14:00:55.229178+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T14:00:55.229178+0000","last_undegraded":"2024-03-26T14:00:55.229178+0000","last_fullsized":"2024-03-26T14:00:55.229178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.240886+0000","last_change":"2024-03-26T14:00:55.239852+0000","last_active":"2024-03-26T14:00:55.240886+0000","last_peered":"2024-03-26T14:00:55.240886+0000","last_clean":"2024-03-26T14:00:55.240886+0000","last_became_active":"2024-03-26T14:00:55.221294+0000","last_became_peered":"2024-03-26T14:00:55.221294+0000","last_unstale":"2024-03-26T14:00:55.240886+0000","last_undegraded":"2024-03-26T14:00:55.240886+0000","last_fullsized":"2024-03-26T14:00:55.240886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:14:53.910832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217230+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T14:00:55.217230+0000","last_peered":"2024-03-26T14:00:55.217230+0000","last_clean":"2024-03-26T14:00:55.217230+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T14:00:55.217230+0000","last_undegraded":"2024-03-26T14:00:55.217230+0000","last_fullsized":"2024-03-26T14:00:55.217230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220506+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T14:00:55.220506+0000","last_peered":"2024-03-26T14:00:55.220506+0000","last_clean":"2024-03-26T14:00:55.220506+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T14:00:55.220506+0000","last_undegraded":"2024-03-26T14:00:55.220506+0000","last_fullsized":"2024-03-26T14:00:55.220506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227792+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T14:00:55.227792+0000","last_peered":"2024-03-26T14:00:55.227792+0000","last_clean":"2024-03-26T14:00:55.227792+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T14:00:55.227792+0000","last_undegraded":"2024-03-26T14:00:55.227792+0000","last_fullsized":"2024-03-26T14:00:55.227792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214282+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T14:00:55.214282+0000","last_peered":"2024-03-26T14:00:55.214282+0000","last_clean":"2024-03-26T14:00:55.214282+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T14:00:55.214282+0000","last_undegraded":"2024-03-26T14:00:55.214282+0000","last_fullsized":"2024-03-26T14:00:55.214282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212753+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T14:00:55.212753+0000","last_peered":"2024-03-26T14:00:55.212753+0000","last_clean":"2024-03-26T14:00:55.212753+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T14:00:55.212753+0000","last_undegraded":"2024-03-26T14:00:55.212753+0000","last_fullsized":"2024-03-26T14:00:55.212753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":129,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:17.512643+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T14:01:17.512643+0000","last_peered":"2024-03-26T14:01:17.512643+0000","last_clean":"2024-03-26T14:01:17.512643+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T14:01:17.512643+0000","last_undegraded":"2024-03-26T14:01:17.512643+0000","last_fullsized":"2024-03-26T14:01:17.512643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213461+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T14:00:55.213461+0000","last_peered":"2024-03-26T14:00:55.213461+0000","last_clean":"2024-03-26T14:00:55.213461+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T14:00:55.213461+0000","last_undegraded":"2024-03-26T14:00:55.213461+0000","last_fullsized":"2024-03-26T14:00:55.213461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217886+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T14:00:55.217886+0000","last_peered":"2024-03-26T14:00:55.217886+0000","last_clean":"2024-03-26T14:00:55.217886+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T14:00:55.217886+0000","last_undegraded":"2024-03-26T14:00:55.217886+0000","last_fullsized":"2024-03-26T14:00:55.217886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212358+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T14:00:55.212358+0000","last_peered":"2024-03-26T14:00:55.212358+0000","last_clean":"2024-03-26T14:00:55.212358+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T14:00:55.212358+0000","last_undegraded":"2024-03-26T14:00:55.212358+0000","last_fullsized":"2024-03-26T14:00:55.212358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226910+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T14:00:55.226910+0000","last_peered":"2024-03-26T14:00:55.226910+0000","last_clean":"2024-03-26T14:00:55.226910+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T14:00:55.226910+0000","last_undegraded":"2024-03-26T14:00:55.226910+0000","last_fullsized":"2024-03-26T14:00:55.226910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226746+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T14:00:55.226746+0000","last_peered":"2024-03-26T14:00:55.226746+0000","last_clean":"2024-03-26T14:00:55.226746+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T14:00:55.226746+0000","last_undegraded":"2024-03-26T14:00:55.226746+0000","last_fullsized":"2024-03-26T14:00:55.226746+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217023+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T14:00:55.217023+0000","last_peered":"2024-03-26T14:00:55.217023+0000","last_clean":"2024-03-26T14:00:55.217023+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T14:00:55.217023+0000","last_undegraded":"2024-03-26T14:00:55.217023+0000","last_fullsized":"2024-03-26T14:00:55.217023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221673+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T14:00:55.221673+0000","last_peered":"2024-03-26T14:00:55.221673+0000","last_clean":"2024-03-26T14:00:55.221673+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T14:00:55.221673+0000","last_undegraded":"2024-03-26T14:00:55.221673+0000","last_fullsized":"2024-03-26T14:00:55.221673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214379+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T14:00:55.214379+0000","last_peered":"2024-03-26T14:00:55.214379+0000","last_clean":"2024-03-26T14:00:55.214379+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T14:00:55.214379+0000","last_undegraded":"2024-03-26T14:00:55.214379+0000","last_fullsized":"2024-03-26T14:00:55.214379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224501+0000","last_change":"2024-03-26T14:00:55.224501+0000","last_active":"2024-03-26T14:00:55.224501+0000","last_peered":"2024-03-26T14:00:55.224501+0000","last_clean":"2024-03-26T14:00:55.224501+0000","last_became_active":"2024-03-26T14:00:55.223843+0000","last_became_peered":"2024-03-26T14:00:55.223843+0000","last_unstale":"2024-03-26T14:00:55.224501+0000","last_undegraded":"2024-03-26T14:00:55.224501+0000","last_fullsized":"2024-03-26T14:00:55.224501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:29:11.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217725+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T14:00:55.217725+0000","last_peered":"2024-03-26T14:00:55.217725+0000","last_clean":"2024-03-26T14:00:55.217725+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T14:00:55.217725+0000","last_undegraded":"2024-03-26T14:00:55.217725+0000","last_fullsized":"2024-03-26T14:00:55.217725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217082+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T14:00:55.217082+0000","last_peered":"2024-03-26T14:00:55.217082+0000","last_clean":"2024-03-26T14:00:55.217082+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T14:00:55.217082+0000","last_undegraded":"2024-03-26T14:00:55.217082+0000","last_fullsized":"2024-03-26T14:00:55.217082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214176+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T14:00:55.214176+0000","last_peered":"2024-03-26T14:00:55.214176+0000","last_clean":"2024-03-26T14:00:55.214176+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T14:00:55.214176+0000","last_undegraded":"2024-03-26T14:00:55.214176+0000","last_fullsized":"2024-03-26T14:00:55.214176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217573+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T14:00:55.217573+0000","last_peered":"2024-03-26T14:00:55.217573+0000","last_clean":"2024-03-26T14:00:55.217573+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T14:00:55.217573+0000","last_undegraded":"2024-03-26T14:00:55.217573+0000","last_fullsized":"2024-03-26T14:00:55.217573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217833+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T14:00:55.217833+0000","last_peered":"2024-03-26T14:00:55.217833+0000","last_clean":"2024-03-26T14:00:55.217833+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T14:00:55.217833+0000","last_undegraded":"2024-03-26T14:00:55.217833+0000","last_fullsized":"2024-03-26T14:00:55.217833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227098+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T14:00:55.227098+0000","last_peered":"2024-03-26T14:00:55.227098+0000","last_clean":"2024-03-26T14:00:55.227098+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T14:00:55.227098+0000","last_undegraded":"2024-03-26T14:00:55.227098+0000","last_fullsized":"2024-03-26T14:00:55.227098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":128,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:20.521083+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T14:01:20.521083+0000","last_peered":"2024-03-26T14:01:20.521083+0000","last_clean":"2024-03-26T14:01:20.521083+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T14:01:20.521083+0000","last_undegraded":"2024-03-26T14:01:20.521083+0000","last_fullsized":"2024-03-26T14:01:20.521083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214001+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T14:00:55.214001+0000","last_peered":"2024-03-26T14:00:55.214001+0000","last_clean":"2024-03-26T14:00:55.214001+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T14:00:55.214001+0000","last_undegraded":"2024-03-26T14:00:55.214001+0000","last_fullsized":"2024-03-26T14:00:55.214001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206949+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T14:00:55.206949+0000","last_peered":"2024-03-26T14:00:55.206949+0000","last_clean":"2024-03-26T14:00:55.206949+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T14:00:55.206949+0000","last_undegraded":"2024-03-26T14:00:55.206949+0000","last_fullsized":"2024-03-26T14:00:55.206949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209531+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T14:00:55.209531+0000","last_peered":"2024-03-26T14:00:55.209531+0000","last_clean":"2024-03-26T14:00:55.209531+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T14:00:55.209531+0000","last_undegraded":"2024-03-26T14:00:55.209531+0000","last_fullsized":"2024-03-26T14:00:55.209531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213309+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T14:00:55.213309+0000","last_peered":"2024-03-26T14:00:55.213309+0000","last_clean":"2024-03-26T14:00:55.213309+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T14:00:55.213309+0000","last_undegraded":"2024-03-26T14:00:55.213309+0000","last_fullsized":"2024-03-26T14:00:55.213309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228357+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T14:00:55.228357+0000","last_peered":"2024-03-26T14:00:55.228357+0000","last_clean":"2024-03-26T14:00:55.228357+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T14:00:55.228357+0000","last_undegraded":"2024-03-26T14:00:55.228357+0000","last_fullsized":"2024-03-26T14:00:55.228357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216629+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T14:00:55.216629+0000","last_peered":"2024-03-26T14:00:55.216629+0000","last_clean":"2024-03-26T14:00:55.216629+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T14:00:55.216629+0000","last_undegraded":"2024-03-26T14:00:55.216629+0000","last_fullsized":"2024-03-26T14:00:55.216629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.439884+0000","last_change":"2024-03-26T14:00:55.439230+0000","last_active":"2024-03-26T14:00:55.439884+0000","last_peered":"2024-03-26T14:00:55.439884+0000","last_clean":"2024-03-26T14:00:55.439884+0000","last_became_active":"2024-03-26T14:00:55.434988+0000","last_became_peered":"2024-03-26T14:00:55.434988+0000","last_unstale":"2024-03-26T14:00:55.439884+0000","last_undegraded":"2024-03-26T14:00:55.439884+0000","last_fullsized":"2024-03-26T14:00:55.439884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:58.291575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222955+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T14:00:55.222955+0000","last_peered":"2024-03-26T14:00:55.222955+0000","last_clean":"2024-03-26T14:00:55.222955+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T14:00:55.222955+0000","last_undegraded":"2024-03-26T14:00:55.222955+0000","last_fullsized":"2024-03-26T14:00:55.222955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216888+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T14:00:55.216888+0000","last_peered":"2024-03-26T14:00:55.216888+0000","last_clean":"2024-03-26T14:00:55.216888+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T14:00:55.216888+0000","last_undegraded":"2024-03-26T14:00:55.216888+0000","last_fullsized":"2024-03-26T14:00:55.216888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217284+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T14:00:55.217284+0000","last_peered":"2024-03-26T14:00:55.217284+0000","last_clean":"2024-03-26T14:00:55.217284+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T14:00:55.217284+0000","last_undegraded":"2024-03-26T14:00:55.217284+0000","last_fullsized":"2024-03-26T14:00:55.217284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206683+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T14:00:55.206683+0000","last_peered":"2024-03-26T14:00:55.206683+0000","last_clean":"2024-03-26T14:00:55.206683+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T14:00:55.206683+0000","last_undegraded":"2024-03-26T14:00:55.206683+0000","last_fullsized":"2024-03-26T14:00:55.206683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212027+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T14:00:55.212027+0000","last_peered":"2024-03-26T14:00:55.212027+0000","last_clean":"2024-03-26T14:00:55.212027+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T14:00:55.212027+0000","last_undegraded":"2024-03-26T14:00:55.212027+0000","last_fullsized":"2024-03-26T14:00:55.212027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206914+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T14:00:55.206914+0000","last_peered":"2024-03-26T14:00:55.206914+0000","last_clean":"2024-03-26T14:00:55.206914+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T14:00:55.206914+0000","last_undegraded":"2024-03-26T14:00:55.206914+0000","last_fullsized":"2024-03-26T14:00:55.206914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211560+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T14:00:55.211560+0000","last_peered":"2024-03-26T14:00:55.211560+0000","last_clean":"2024-03-26T14:00:55.211560+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T14:00:55.211560+0000","last_undegraded":"2024-03-26T14:00:55.211560+0000","last_fullsized":"2024-03-26T14:00:55.211560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210372+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T14:00:55.210372+0000","last_peered":"2024-03-26T14:00:55.210372+0000","last_clean":"2024-03-26T14:00:55.210372+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T14:00:55.210372+0000","last_undegraded":"2024-03-26T14:00:55.210372+0000","last_fullsized":"2024-03-26T14:00:55.210372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212585+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T14:00:55.212585+0000","last_peered":"2024-03-26T14:00:55.212585+0000","last_clean":"2024-03-26T14:00:55.212585+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T14:00:55.212585+0000","last_undegraded":"2024-03-26T14:00:55.212585+0000","last_fullsized":"2024-03-26T14:00:55.212585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210097+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T14:00:55.210097+0000","last_peered":"2024-03-26T14:00:55.210097+0000","last_clean":"2024-03-26T14:00:55.210097+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T14:00:55.210097+0000","last_undegraded":"2024-03-26T14:00:55.210097+0000","last_fullsized":"2024-03-26T14:00:55.210097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433759+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T14:00:55.433759+0000","last_peered":"2024-03-26T14:00:55.433759+0000","last_clean":"2024-03-26T14:00:55.433759+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T14:00:55.433759+0000","last_undegraded":"2024-03-26T14:00:55.433759+0000","last_fullsized":"2024-03-26T14:00:55.433759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213140+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T14:00:55.213140+0000","last_peered":"2024-03-26T14:00:55.213140+0000","last_clean":"2024-03-26T14:00:55.213140+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T14:00:55.213140+0000","last_undegraded":"2024-03-26T14:00:55.213140+0000","last_fullsized":"2024-03-26T14:00:55.213140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216585+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T14:00:55.216585+0000","last_peered":"2024-03-26T14:00:55.216585+0000","last_clean":"2024-03-26T14:00:55.216585+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T14:00:55.216585+0000","last_undegraded":"2024-03-26T14:00:55.216585+0000","last_fullsized":"2024-03-26T14:00:55.216585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206741+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T14:00:55.206741+0000","last_peered":"2024-03-26T14:00:55.206741+0000","last_clean":"2024-03-26T14:00:55.206741+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T14:00:55.206741+0000","last_undegraded":"2024-03-26T14:00:55.206741+0000","last_fullsized":"2024-03-26T14:00:55.206741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215304+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T14:00:55.215304+0000","last_peered":"2024-03-26T14:00:55.215304+0000","last_clean":"2024-03-26T14:00:55.215304+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T14:00:55.215304+0000","last_undegraded":"2024-03-26T14:00:55.215304+0000","last_fullsized":"2024-03-26T14:00:55.215304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209195+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T14:00:55.209195+0000","last_peered":"2024-03-26T14:00:55.209195+0000","last_clean":"2024-03-26T14:00:55.209195+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T14:00:55.209195+0000","last_undegraded":"2024-03-26T14:00:55.209195+0000","last_fullsized":"2024-03-26T14:00:55.209195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213884+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T14:00:55.213884+0000","last_peered":"2024-03-26T14:00:55.213884+0000","last_clean":"2024-03-26T14:00:55.213884+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T14:00:55.213884+0000","last_undegraded":"2024-03-26T14:00:55.213884+0000","last_fullsized":"2024-03-26T14:00:55.213884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207328+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T14:00:55.207328+0000","last_peered":"2024-03-26T14:00:55.207328+0000","last_clean":"2024-03-26T14:00:55.207328+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T14:00:55.207328+0000","last_undegraded":"2024-03-26T14:00:55.207328+0000","last_fullsized":"2024-03-26T14:00:55.207328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209775+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T14:00:55.209775+0000","last_peered":"2024-03-26T14:00:55.209775+0000","last_clean":"2024-03-26T14:00:55.209775+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T14:00:55.209775+0000","last_undegraded":"2024-03-26T14:00:55.209775+0000","last_fullsized":"2024-03-26T14:00:55.209775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213124+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T14:00:55.213124+0000","last_peered":"2024-03-26T14:00:55.213124+0000","last_clean":"2024-03-26T14:00:55.213124+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T14:00:55.213124+0000","last_undegraded":"2024-03-26T14:00:55.213124+0000","last_fullsized":"2024-03-26T14:00:55.213124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214058+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T14:00:55.214058+0000","last_peered":"2024-03-26T14:00:55.214058+0000","last_clean":"2024-03-26T14:00:55.214058+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T14:00:55.214058+0000","last_undegraded":"2024-03-26T14:00:55.214058+0000","last_fullsized":"2024-03-26T14:00:55.214058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.432532+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T14:00:55.432532+0000","last_peered":"2024-03-26T14:00:55.432532+0000","last_clean":"2024-03-26T14:00:55.432532+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T14:00:55.432532+0000","last_undegraded":"2024-03-26T14:00:55.432532+0000","last_fullsized":"2024-03-26T14:00:55.432532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212693+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T14:00:55.212693+0000","last_peered":"2024-03-26T14:00:55.212693+0000","last_clean":"2024-03-26T14:00:55.212693+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T14:00:55.212693+0000","last_undegraded":"2024-03-26T14:00:55.212693+0000","last_fullsized":"2024-03-26T14:00:55.212693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214241+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T14:00:55.214241+0000","last_peered":"2024-03-26T14:00:55.214241+0000","last_clean":"2024-03-26T14:00:55.214241+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T14:00:55.214241+0000","last_undegraded":"2024-03-26T14:00:55.214241+0000","last_fullsized":"2024-03-26T14:00:55.214241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210488+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T14:00:55.210488+0000","last_peered":"2024-03-26T14:00:55.210488+0000","last_clean":"2024-03-26T14:00:55.210488+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T14:00:55.210488+0000","last_undegraded":"2024-03-26T14:00:55.210488+0000","last_fullsized":"2024-03-26T14:00:55.210488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228782+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T14:00:55.228782+0000","last_peered":"2024-03-26T14:00:55.228782+0000","last_clean":"2024-03-26T14:00:55.228782+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T14:00:55.228782+0000","last_undegraded":"2024-03-26T14:00:55.228782+0000","last_fullsized":"2024-03-26T14:00:55.228782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214577+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T14:00:55.214577+0000","last_peered":"2024-03-26T14:00:55.214577+0000","last_clean":"2024-03-26T14:00:55.214577+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T14:00:55.214577+0000","last_undegraded":"2024-03-26T14:00:55.214577+0000","last_fullsized":"2024-03-26T14:00:55.214577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225684+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T14:00:55.225684+0000","last_peered":"2024-03-26T14:00:55.225684+0000","last_clean":"2024-03-26T14:00:55.225684+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T14:00:55.225684+0000","last_undegraded":"2024-03-26T14:00:55.225684+0000","last_fullsized":"2024-03-26T14:00:55.225684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217803+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T14:00:55.217803+0000","last_peered":"2024-03-26T14:00:55.217803+0000","last_clean":"2024-03-26T14:00:55.217803+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T14:00:55.217803+0000","last_undegraded":"2024-03-26T14:00:55.217803+0000","last_fullsized":"2024-03-26T14:00:55.217803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214677+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T14:00:55.214677+0000","last_peered":"2024-03-26T14:00:55.214677+0000","last_clean":"2024-03-26T14:00:55.214677+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T14:00:55.214677+0000","last_undegraded":"2024-03-26T14:00:55.214677+0000","last_fullsized":"2024-03-26T14:00:55.214677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215690+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T14:00:55.215690+0000","last_peered":"2024-03-26T14:00:55.215690+0000","last_clean":"2024-03-26T14:00:55.215690+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T14:00:55.215690+0000","last_undegraded":"2024-03-26T14:00:55.215690+0000","last_fullsized":"2024-03-26T14:00:55.215690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212266+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T14:00:55.212266+0000","last_peered":"2024-03-26T14:00:55.212266+0000","last_clean":"2024-03-26T14:00:55.212266+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T14:00:55.212266+0000","last_undegraded":"2024-03-26T14:00:55.212266+0000","last_fullsized":"2024-03-26T14:00:55.212266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213444+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T14:00:55.213444+0000","last_peered":"2024-03-26T14:00:55.213444+0000","last_clean":"2024-03-26T14:00:55.213444+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T14:00:55.213444+0000","last_undegraded":"2024-03-26T14:00:55.213444+0000","last_fullsized":"2024-03-26T14:00:55.213444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213850+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T14:00:55.213850+0000","last_peered":"2024-03-26T14:00:55.213850+0000","last_clean":"2024-03-26T14:00:55.213850+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T14:00:55.213850+0000","last_undegraded":"2024-03-26T14:00:55.213850+0000","last_fullsized":"2024-03-26T14:00:55.213850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217393+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T14:00:55.217393+0000","last_peered":"2024-03-26T14:00:55.217393+0000","last_clean":"2024-03-26T14:00:55.217393+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T14:00:55.217393+0000","last_undegraded":"2024-03-26T14:00:55.217393+0000","last_fullsized":"2024-03-26T14:00:55.217393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221405+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T14:00:55.221405+0000","last_peered":"2024-03-26T14:00:55.221405+0000","last_clean":"2024-03-26T14:00:55.221405+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T14:00:55.221405+0000","last_undegraded":"2024-03-26T14:00:55.221405+0000","last_fullsized":"2024-03-26T14:00:55.221405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218347+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T14:00:55.218347+0000","last_peered":"2024-03-26T14:00:55.218347+0000","last_clean":"2024-03-26T14:00:55.218347+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T14:00:55.218347+0000","last_undegraded":"2024-03-26T14:00:55.218347+0000","last_fullsized":"2024-03-26T14:00:55.218347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207012+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T14:00:55.207012+0000","last_peered":"2024-03-26T14:00:55.207012+0000","last_clean":"2024-03-26T14:00:55.207012+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T14:00:55.207012+0000","last_undegraded":"2024-03-26T14:00:55.207012+0000","last_fullsized":"2024-03-26T14:00:55.207012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226803+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T14:00:55.226803+0000","last_peered":"2024-03-26T14:00:55.226803+0000","last_clean":"2024-03-26T14:00:55.226803+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T14:00:55.226803+0000","last_undegraded":"2024-03-26T14:00:55.226803+0000","last_fullsized":"2024-03-26T14:00:55.226803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206825+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T14:00:55.206825+0000","last_peered":"2024-03-26T14:00:55.206825+0000","last_clean":"2024-03-26T14:00:55.206825+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T14:00:55.206825+0000","last_undegraded":"2024-03-26T14:00:55.206825+0000","last_fullsized":"2024-03-26T14:00:55.206825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433719+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T14:00:55.433719+0000","last_peered":"2024-03-26T14:00:55.433719+0000","last_clean":"2024-03-26T14:00:55.433719+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T14:00:55.433719+0000","last_undegraded":"2024-03-26T14:00:55.433719+0000","last_fullsized":"2024-03-26T14:00:55.433719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217090+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T14:00:55.217090+0000","last_peered":"2024-03-26T14:00:55.217090+0000","last_clean":"2024-03-26T14:00:55.217090+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T14:00:55.217090+0000","last_undegraded":"2024-03-26T14:00:55.217090+0000","last_fullsized":"2024-03-26T14:00:55.217090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220969+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T14:00:55.220969+0000","last_peered":"2024-03-26T14:00:55.220969+0000","last_clean":"2024-03-26T14:00:55.220969+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T14:00:55.220969+0000","last_undegraded":"2024-03-26T14:00:55.220969+0000","last_fullsized":"2024-03-26T14:00:55.220969+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71985,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":575,"num_read_kb":365,"num_write":170,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":215955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":522925,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1183744,"data_stored":1182630,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776182,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":523127,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037821,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":523047,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299460,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1119097,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462986,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518303,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":586056,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519691,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":587372,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149792,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518497,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888153,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523485,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":1114153,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:01:24.906 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph pg dump --format=json 2024-03-26T14:01:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:25.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:25 smithi137 ceph-mon[31539]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:01:25.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:25 smithi179 ceph-mon[35472]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:01:25.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:25 smithi159 ceph-mon[35501]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:01:26.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:26 smithi137 ceph-mon[31539]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:26.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:26 smithi179 ceph-mon[35472]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:26.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:26 smithi159 ceph-mon[35501]: pgmap v430: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:27.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.599+0000 7fa6f3e00700 1 -- 172.21.15.137:0/536032691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec069e40 msgr2=0x7fa6ec06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:27.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.599+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/536032691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec069e40 0x7fa6ec06a2a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80099c0 tx=0x7fa6d8009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 -- 172.21.15.137:0/536032691 shutdown_connections 2024-03-26T14:01:27.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/536032691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/536032691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 0x7fa6ec1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/536032691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec069e40 0x7fa6ec06a2a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 -- 172.21.15.137:0/536032691 >> 172.21.15.137:0/536032691 conn(0x7fa6ec077fc0 msgr2=0x7fa6ec0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:27.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 -- 172.21.15.137:0/536032691 shutdown_connections 2024-03-26T14:01:27.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.600+0000 7fa6f3e00700 1 -- 172.21.15.137:0/536032691 wait complete. 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 Processor -- start 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 -- start start 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec069e40 0x7fa6ec101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec1016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 0x7fa6ec101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec112340 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1121c0 con 0x7fa6ec069e40 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.601+0000 7fa6f3e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec1124c0 con 0x7fa6ec1081e0 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec1016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48896/0 (socket says 172.21.15.137:48896) 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 -- 172.21.15.137:0/1409809613 learned_addr learned my addr 172.21.15.137:0/1409809613 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:27.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f1400700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec069e40 0x7fa6ec101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f1e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 0x7fa6ec101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 -- 172.21.15.137:0/1409809613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 msgr2=0x7fa6ec101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 0x7fa6ec101be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 -- 172.21.15.137:0/1409809613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec069e40 msgr2=0x7fa6ec101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec069e40 0x7fa6ec101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f0a00700 1 -- 172.21.15.137:0/1409809613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec1a6da0 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f1400700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec069e40 0x7fa6ec101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.602+0000 7fa6f1e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 0x7fa6ec101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.603+0000 7fa6f0a00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec1016a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005b40 tx=0x7fa6d8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:27.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.603+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8027070 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.603+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa6d8003fd0 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.603+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec1a7030 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.603+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8022c60 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.603+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec1a7480 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.604+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec068e50 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.606+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa6d802a070 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.608+0000 7fa6ea000700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6d40a2890 0x7fa6d40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:27.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.608+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa6d8022e00 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.608+0000 7fa6f1400700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6d40a2890 0x7fa6d40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:27.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.609+0000 7fa6f1400700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6d40a2890 0x7fa6d40a4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0005fd0 tx=0x7fa6e0005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:27.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.612+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa6d805e740 con 0x7fa6ec06a7e0 2024-03-26T14:01:27.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.832+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6ec0625d0 con 0x7fa6d40a2890 2024-03-26T14:01:27.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.861+0000 7fa6ea000700 1 -- 172.21.15.137:0/1409809613 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340760 (secure 0 0 0) 0x7fa6ec0625d0 con 0x7fa6d40a2890 2024-03-26T14:01:27.861 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:27.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.870+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6d40a2890 msgr2=0x7fa6d40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:27.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.870+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6d40a2890 0x7fa6d40a4d50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0005fd0 tx=0x7fa6e0005db0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 msgr2=0x7fa6ec1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:27.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec1016a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8005b40 tx=0x7fa6d8005e80 comp rx=0 tx=0).stop 2024-03-26T14:01:27.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 shutdown_connections 2024-03-26T14:01:27.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6d40a2890 0x7fa6d40a4d50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6ec1081e0 0x7fa6ec101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ec069e40 0x7fa6ec101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 --2- 172.21.15.137:0/1409809613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa6ec06a7e0 0x7fa6ec1016a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:27.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 >> 172.21.15.137:0/1409809613 conn(0x7fa6ec077fc0 msgr2=0x7fa6ec0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:27.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 shutdown_connections 2024-03-26T14:01:27.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:27.871+0000 7fa6f3e00700 1 -- 172.21.15.137:0/1409809613 wait complete. 2024-03-26T14:01:27.869 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:01:28.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:28 smithi137 ceph-mon[31539]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:28.657 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-03-26T14:01:27.128167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662587,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":726,"num_read_kb":489,"num_write":353,"num_write_kb":3066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361788,"kb_used_data":12560,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580036,"statfs":{"total":1151940427776,"available":1151569956864,"internally_reserved":0,"allocated":12861440,"data_stored":8238383,"data_compressed":427101,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005890"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222471+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T14:00:55.222471+0000","last_peered":"2024-03-26T14:00:55.222471+0000","last_clean":"2024-03-26T14:00:55.222471+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T14:00:55.222471+0000","last_undegraded":"2024-03-26T14:00:55.222471+0000","last_fullsized":"2024-03-26T14:00:55.222471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212482+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T14:00:55.212482+0000","last_peered":"2024-03-26T14:00:55.212482+0000","last_clean":"2024-03-26T14:00:55.212482+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T14:00:55.212482+0000","last_undegraded":"2024-03-26T14:00:55.212482+0000","last_fullsized":"2024-03-26T14:00:55.212482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210979+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T14:00:55.210979+0000","last_peered":"2024-03-26T14:00:55.210979+0000","last_clean":"2024-03-26T14:00:55.210979+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T14:00:55.210979+0000","last_undegraded":"2024-03-26T14:00:55.210979+0000","last_fullsized":"2024-03-26T14:00:55.210979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210071+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T14:00:55.210071+0000","last_peered":"2024-03-26T14:00:55.210071+0000","last_clean":"2024-03-26T14:00:55.210071+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T14:00:55.210071+0000","last_undegraded":"2024-03-26T14:00:55.210071+0000","last_fullsized":"2024-03-26T14:00:55.210071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221738+0000","last_change":"2024-03-26T14:00:55.221738+0000","last_active":"2024-03-26T14:00:55.221738+0000","last_peered":"2024-03-26T14:00:55.221738+0000","last_clean":"2024-03-26T14:00:55.221738+0000","last_became_active":"2024-03-26T14:00:55.220626+0000","last_became_peered":"2024-03-26T14:00:55.220626+0000","last_unstale":"2024-03-26T14:00:55.221738+0000","last_undegraded":"2024-03-26T14:00:55.221738+0000","last_fullsized":"2024-03-26T14:00:55.221738+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:50.208896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217644+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T14:00:55.217644+0000","last_peered":"2024-03-26T14:00:55.217644+0000","last_clean":"2024-03-26T14:00:55.217644+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T14:00:55.217644+0000","last_undegraded":"2024-03-26T14:00:55.217644+0000","last_fullsized":"2024-03-26T14:00:55.217644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213715+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T14:00:55.213715+0000","last_peered":"2024-03-26T14:00:55.213715+0000","last_clean":"2024-03-26T14:00:55.213715+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T14:00:55.213715+0000","last_undegraded":"2024-03-26T14:00:55.213715+0000","last_fullsized":"2024-03-26T14:00:55.213715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206692+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T14:00:55.206692+0000","last_peered":"2024-03-26T14:00:55.206692+0000","last_clean":"2024-03-26T14:00:55.206692+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T14:00:55.206692+0000","last_undegraded":"2024-03-26T14:00:55.206692+0000","last_fullsized":"2024-03-26T14:00:55.206692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434453+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T14:00:55.434453+0000","last_peered":"2024-03-26T14:00:55.434453+0000","last_clean":"2024-03-26T14:00:55.434453+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T14:00:55.434453+0000","last_undegraded":"2024-03-26T14:00:55.434453+0000","last_fullsized":"2024-03-26T14:00:55.434453+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218316+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T14:00:55.218316+0000","last_peered":"2024-03-26T14:00:55.218316+0000","last_clean":"2024-03-26T14:00:55.218316+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T14:00:55.218316+0000","last_undegraded":"2024-03-26T14:00:55.218316+0000","last_fullsized":"2024-03-26T14:00:55.218316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214164+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T14:00:55.214164+0000","last_peered":"2024-03-26T14:00:55.214164+0000","last_clean":"2024-03-26T14:00:55.214164+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T14:00:55.214164+0000","last_undegraded":"2024-03-26T14:00:55.214164+0000","last_fullsized":"2024-03-26T14:00:55.214164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209734+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T14:00:55.209734+0000","last_peered":"2024-03-26T14:00:55.209734+0000","last_clean":"2024-03-26T14:00:55.209734+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T14:00:55.209734+0000","last_undegraded":"2024-03-26T14:00:55.209734+0000","last_fullsized":"2024-03-26T14:00:55.209734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217206+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T14:00:55.217206+0000","last_peered":"2024-03-26T14:00:55.217206+0000","last_clean":"2024-03-26T14:00:55.217206+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T14:00:55.217206+0000","last_undegraded":"2024-03-26T14:00:55.217206+0000","last_fullsized":"2024-03-26T14:00:55.217206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210320+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T14:00:55.210320+0000","last_peered":"2024-03-26T14:00:55.210320+0000","last_clean":"2024-03-26T14:00:55.210320+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T14:00:55.210320+0000","last_undegraded":"2024-03-26T14:00:55.210320+0000","last_fullsized":"2024-03-26T14:00:55.210320+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210676+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T14:00:55.210676+0000","last_peered":"2024-03-26T14:00:55.210676+0000","last_clean":"2024-03-26T14:00:55.210676+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T14:00:55.210676+0000","last_undegraded":"2024-03-26T14:00:55.210676+0000","last_fullsized":"2024-03-26T14:00:55.210676+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433581+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T14:00:55.433581+0000","last_peered":"2024-03-26T14:00:55.433581+0000","last_clean":"2024-03-26T14:00:55.433581+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T14:00:55.433581+0000","last_undegraded":"2024-03-26T14:00:55.433581+0000","last_fullsized":"2024-03-26T14:00:55.433581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222285+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T14:00:55.222285+0000","last_peered":"2024-03-26T14:00:55.222285+0000","last_clean":"2024-03-26T14:00:55.222285+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T14:00:55.222285+0000","last_undegraded":"2024-03-26T14:00:55.222285+0000","last_fullsized":"2024-03-26T14:00:55.222285+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212135+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T14:00:55.212135+0000","last_peered":"2024-03-26T14:00:55.212135+0000","last_clean":"2024-03-26T14:00:55.212135+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T14:00:55.212135+0000","last_undegraded":"2024-03-26T14:00:55.212135+0000","last_fullsized":"2024-03-26T14:00:55.212135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215769+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T14:00:55.215769+0000","last_peered":"2024-03-26T14:00:55.215769+0000","last_clean":"2024-03-26T14:00:55.215769+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T14:00:55.215769+0000","last_undegraded":"2024-03-26T14:00:55.215769+0000","last_fullsized":"2024-03-26T14:00:55.215769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212573+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T14:00:55.212573+0000","last_peered":"2024-03-26T14:00:55.212573+0000","last_clean":"2024-03-26T14:00:55.212573+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T14:00:55.212573+0000","last_undegraded":"2024-03-26T14:00:55.212573+0000","last_fullsized":"2024-03-26T14:00:55.212573+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212930+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T14:00:55.212930+0000","last_peered":"2024-03-26T14:00:55.212930+0000","last_clean":"2024-03-26T14:00:55.212930+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T14:00:55.212930+0000","last_undegraded":"2024-03-26T14:00:55.212930+0000","last_fullsized":"2024-03-26T14:00:55.212930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207749+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T14:00:55.207749+0000","last_peered":"2024-03-26T14:00:55.207749+0000","last_clean":"2024-03-26T14:00:55.207749+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T14:00:55.207749+0000","last_undegraded":"2024-03-26T14:00:55.207749+0000","last_fullsized":"2024-03-26T14:00:55.207749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209121+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T14:00:55.209121+0000","last_peered":"2024-03-26T14:00:55.209121+0000","last_clean":"2024-03-26T14:00:55.209121+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T14:00:55.209121+0000","last_undegraded":"2024-03-26T14:00:55.209121+0000","last_fullsized":"2024-03-26T14:00:55.209121+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207907+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T14:00:55.207907+0000","last_peered":"2024-03-26T14:00:55.207907+0000","last_clean":"2024-03-26T14:00:55.207907+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T14:00:55.207907+0000","last_undegraded":"2024-03-26T14:00:55.207907+0000","last_fullsized":"2024-03-26T14:00:55.207907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211033+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T14:00:55.211033+0000","last_peered":"2024-03-26T14:00:55.211033+0000","last_clean":"2024-03-26T14:00:55.211033+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T14:00:55.211033+0000","last_undegraded":"2024-03-26T14:00:55.211033+0000","last_fullsized":"2024-03-26T14:00:55.211033+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209396+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T14:00:55.209396+0000","last_peered":"2024-03-26T14:00:55.209396+0000","last_clean":"2024-03-26T14:00:55.209396+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T14:00:55.209396+0000","last_undegraded":"2024-03-26T14:00:55.209396+0000","last_fullsized":"2024-03-26T14:00:55.209396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213402+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T14:00:55.213402+0000","last_peered":"2024-03-26T14:00:55.213402+0000","last_clean":"2024-03-26T14:00:55.213402+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T14:00:55.213402+0000","last_undegraded":"2024-03-26T14:00:55.213402+0000","last_fullsized":"2024-03-26T14:00:55.213402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433778+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T14:00:55.433778+0000","last_peered":"2024-03-26T14:00:55.433778+0000","last_clean":"2024-03-26T14:00:55.433778+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T14:00:55.433778+0000","last_undegraded":"2024-03-26T14:00:55.433778+0000","last_fullsized":"2024-03-26T14:00:55.433778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209004+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T14:00:55.209004+0000","last_peered":"2024-03-26T14:00:55.209004+0000","last_clean":"2024-03-26T14:00:55.209004+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T14:00:55.209004+0000","last_undegraded":"2024-03-26T14:00:55.209004+0000","last_fullsized":"2024-03-26T14:00:55.209004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214537+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T14:00:55.214537+0000","last_peered":"2024-03-26T14:00:55.214537+0000","last_clean":"2024-03-26T14:00:55.214537+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T14:00:55.214537+0000","last_undegraded":"2024-03-26T14:00:55.214537+0000","last_fullsized":"2024-03-26T14:00:55.214537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228324+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T14:00:55.228324+0000","last_peered":"2024-03-26T14:00:55.228324+0000","last_clean":"2024-03-26T14:00:55.228324+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T14:00:55.228324+0000","last_undegraded":"2024-03-26T14:00:55.228324+0000","last_fullsized":"2024-03-26T14:00:55.228324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214576+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T14:00:55.214576+0000","last_peered":"2024-03-26T14:00:55.214576+0000","last_clean":"2024-03-26T14:00:55.214576+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T14:00:55.214576+0000","last_undegraded":"2024-03-26T14:00:55.214576+0000","last_fullsized":"2024-03-26T14:00:55.214576+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":130,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:20.494321+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T14:01:20.494321+0000","last_peered":"2024-03-26T14:01:20.494321+0000","last_clean":"2024-03-26T14:01:20.494321+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T14:01:20.494321+0000","last_undegraded":"2024-03-26T14:01:20.494321+0000","last_fullsized":"2024-03-26T14:01:20.494321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214675+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T14:00:55.214675+0000","last_peered":"2024-03-26T14:00:55.214675+0000","last_clean":"2024-03-26T14:00:55.214675+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T14:00:55.214675+0000","last_undegraded":"2024-03-26T14:00:55.214675+0000","last_fullsized":"2024-03-26T14:00:55.214675+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216200+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T14:00:55.216200+0000","last_peered":"2024-03-26T14:00:55.216200+0000","last_clean":"2024-03-26T14:00:55.216200+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T14:00:55.216200+0000","last_undegraded":"2024-03-26T14:00:55.216200+0000","last_fullsized":"2024-03-26T14:00:55.216200+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217548+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T14:00:55.217548+0000","last_peered":"2024-03-26T14:00:55.217548+0000","last_clean":"2024-03-26T14:00:55.217548+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T14:00:55.217548+0000","last_undegraded":"2024-03-26T14:00:55.217548+0000","last_fullsized":"2024-03-26T14:00:55.217548+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227859+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T14:00:55.227859+0000","last_peered":"2024-03-26T14:00:55.227859+0000","last_clean":"2024-03-26T14:00:55.227859+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T14:00:55.227859+0000","last_undegraded":"2024-03-26T14:00:55.227859+0000","last_fullsized":"2024-03-26T14:00:55.227859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222020+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T14:00:55.222020+0000","last_peered":"2024-03-26T14:00:55.222020+0000","last_clean":"2024-03-26T14:00:55.222020+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T14:00:55.222020+0000","last_undegraded":"2024-03-26T14:00:55.222020+0000","last_fullsized":"2024-03-26T14:00:55.222020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217496+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T14:00:55.217496+0000","last_peered":"2024-03-26T14:00:55.217496+0000","last_clean":"2024-03-26T14:00:55.217496+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T14:00:55.217496+0000","last_undegraded":"2024-03-26T14:00:55.217496+0000","last_fullsized":"2024-03-26T14:00:55.217496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210312+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T14:00:55.210312+0000","last_peered":"2024-03-26T14:00:55.210312+0000","last_clean":"2024-03-26T14:00:55.210312+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T14:00:55.210312+0000","last_undegraded":"2024-03-26T14:00:55.210312+0000","last_fullsized":"2024-03-26T14:00:55.210312+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212373+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T14:00:55.212373+0000","last_peered":"2024-03-26T14:00:55.212373+0000","last_clean":"2024-03-26T14:00:55.212373+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T14:00:55.212373+0000","last_undegraded":"2024-03-26T14:00:55.212373+0000","last_fullsized":"2024-03-26T14:00:55.212373+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226392+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T14:00:55.226392+0000","last_peered":"2024-03-26T14:00:55.226392+0000","last_clean":"2024-03-26T14:00:55.226392+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T14:00:55.226392+0000","last_undegraded":"2024-03-26T14:00:55.226392+0000","last_fullsized":"2024-03-26T14:00:55.226392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211021+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T14:00:55.211021+0000","last_peered":"2024-03-26T14:00:55.211021+0000","last_clean":"2024-03-26T14:00:55.211021+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T14:00:55.211021+0000","last_undegraded":"2024-03-26T14:00:55.211021+0000","last_fullsized":"2024-03-26T14:00:55.211021+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206684+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T14:00:55.206684+0000","last_peered":"2024-03-26T14:00:55.206684+0000","last_clean":"2024-03-26T14:00:55.206684+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T14:00:55.206684+0000","last_undegraded":"2024-03-26T14:00:55.206684+0000","last_fullsized":"2024-03-26T14:00:55.206684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":130,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:24.503412+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T14:01:24.503412+0000","last_peered":"2024-03-26T14:01:24.503412+0000","last_clean":"2024-03-26T14:01:24.503412+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T14:01:24.503412+0000","last_undegraded":"2024-03-26T14:01:24.503412+0000","last_fullsized":"2024-03-26T14:01:24.503412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434076+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T14:00:55.434076+0000","last_peered":"2024-03-26T14:00:55.434076+0000","last_clean":"2024-03-26T14:00:55.434076+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T14:00:55.434076+0000","last_undegraded":"2024-03-26T14:00:55.434076+0000","last_fullsized":"2024-03-26T14:00:55.434076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211798+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T14:00:55.211798+0000","last_peered":"2024-03-26T14:00:55.211798+0000","last_clean":"2024-03-26T14:00:55.211798+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T14:00:55.211798+0000","last_undegraded":"2024-03-26T14:00:55.211798+0000","last_fullsized":"2024-03-26T14:00:55.211798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213925+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T14:00:55.213925+0000","last_peered":"2024-03-26T14:00:55.213925+0000","last_clean":"2024-03-26T14:00:55.213925+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T14:00:55.213925+0000","last_undegraded":"2024-03-26T14:00:55.213925+0000","last_fullsized":"2024-03-26T14:00:55.213925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224154+0000","last_change":"2024-03-26T14:00:55.224154+0000","last_active":"2024-03-26T14:00:55.224154+0000","last_peered":"2024-03-26T14:00:55.224154+0000","last_clean":"2024-03-26T14:00:55.224154+0000","last_became_active":"2024-03-26T14:00:55.223674+0000","last_became_peered":"2024-03-26T14:00:55.223674+0000","last_unstale":"2024-03-26T14:00:55.224154+0000","last_undegraded":"2024-03-26T14:00:55.224154+0000","last_fullsized":"2024-03-26T14:00:55.224154+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:48:31.146317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209641+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T14:00:55.209641+0000","last_peered":"2024-03-26T14:00:55.209641+0000","last_clean":"2024-03-26T14:00:55.209641+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T14:00:55.209641+0000","last_undegraded":"2024-03-26T14:00:55.209641+0000","last_fullsized":"2024-03-26T14:00:55.209641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228031+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T14:00:55.228031+0000","last_peered":"2024-03-26T14:00:55.228031+0000","last_clean":"2024-03-26T14:00:55.228031+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T14:00:55.228031+0000","last_undegraded":"2024-03-26T14:00:55.228031+0000","last_fullsized":"2024-03-26T14:00:55.228031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433934+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T14:00:55.433934+0000","last_peered":"2024-03-26T14:00:55.433934+0000","last_clean":"2024-03-26T14:00:55.433934+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T14:00:55.433934+0000","last_undegraded":"2024-03-26T14:00:55.433934+0000","last_fullsized":"2024-03-26T14:00:55.433934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210744+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T14:00:55.210744+0000","last_peered":"2024-03-26T14:00:55.210744+0000","last_clean":"2024-03-26T14:00:55.210744+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T14:00:55.210744+0000","last_undegraded":"2024-03-26T14:00:55.210744+0000","last_fullsized":"2024-03-26T14:00:55.210744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":270,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:25.488367+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T14:01:25.488367+0000","last_peered":"2024-03-26T14:01:25.488367+0000","last_clean":"2024-03-26T14:01:25.488367+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T14:01:25.488367+0000","last_undegraded":"2024-03-26T14:01:25.488367+0000","last_fullsized":"2024-03-26T14:01:25.488367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67843,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":201,"num_read_kb":167,"num_write":39,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217237+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T14:00:55.217237+0000","last_peered":"2024-03-26T14:00:55.217237+0000","last_clean":"2024-03-26T14:00:55.217237+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T14:00:55.217237+0000","last_undegraded":"2024-03-26T14:00:55.217237+0000","last_fullsized":"2024-03-26T14:00:55.217237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.229178+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T14:00:55.229178+0000","last_peered":"2024-03-26T14:00:55.229178+0000","last_clean":"2024-03-26T14:00:55.229178+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T14:00:55.229178+0000","last_undegraded":"2024-03-26T14:00:55.229178+0000","last_fullsized":"2024-03-26T14:00:55.229178+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.240886+0000","last_change":"2024-03-26T14:00:55.239852+0000","last_active":"2024-03-26T14:00:55.240886+0000","last_peered":"2024-03-26T14:00:55.240886+0000","last_clean":"2024-03-26T14:00:55.240886+0000","last_became_active":"2024-03-26T14:00:55.221294+0000","last_became_peered":"2024-03-26T14:00:55.221294+0000","last_unstale":"2024-03-26T14:00:55.240886+0000","last_undegraded":"2024-03-26T14:00:55.240886+0000","last_fullsized":"2024-03-26T14:00:55.240886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:14:53.910832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217230+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T14:00:55.217230+0000","last_peered":"2024-03-26T14:00:55.217230+0000","last_clean":"2024-03-26T14:00:55.217230+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T14:00:55.217230+0000","last_undegraded":"2024-03-26T14:00:55.217230+0000","last_fullsized":"2024-03-26T14:00:55.217230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220506+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T14:00:55.220506+0000","last_peered":"2024-03-26T14:00:55.220506+0000","last_clean":"2024-03-26T14:00:55.220506+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T14:00:55.220506+0000","last_undegraded":"2024-03-26T14:00:55.220506+0000","last_fullsized":"2024-03-26T14:00:55.220506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227792+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T14:00:55.227792+0000","last_peered":"2024-03-26T14:00:55.227792+0000","last_clean":"2024-03-26T14:00:55.227792+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T14:00:55.227792+0000","last_undegraded":"2024-03-26T14:00:55.227792+0000","last_fullsized":"2024-03-26T14:00:55.227792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214282+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T14:00:55.214282+0000","last_peered":"2024-03-26T14:00:55.214282+0000","last_clean":"2024-03-26T14:00:55.214282+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T14:00:55.214282+0000","last_undegraded":"2024-03-26T14:00:55.214282+0000","last_fullsized":"2024-03-26T14:00:55.214282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212753+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T14:00:55.212753+0000","last_peered":"2024-03-26T14:00:55.212753+0000","last_clean":"2024-03-26T14:00:55.212753+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T14:00:55.212753+0000","last_undegraded":"2024-03-26T14:00:55.212753+0000","last_fullsized":"2024-03-26T14:00:55.212753+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":134,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:22.527707+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T14:01:22.527707+0000","last_peered":"2024-03-26T14:01:22.527707+0000","last_clean":"2024-03-26T14:01:22.527707+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T14:01:22.527707+0000","last_undegraded":"2024-03-26T14:01:22.527707+0000","last_fullsized":"2024-03-26T14:01:22.527707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213461+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T14:00:55.213461+0000","last_peered":"2024-03-26T14:00:55.213461+0000","last_clean":"2024-03-26T14:00:55.213461+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T14:00:55.213461+0000","last_undegraded":"2024-03-26T14:00:55.213461+0000","last_fullsized":"2024-03-26T14:00:55.213461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217886+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T14:00:55.217886+0000","last_peered":"2024-03-26T14:00:55.217886+0000","last_clean":"2024-03-26T14:00:55.217886+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T14:00:55.217886+0000","last_undegraded":"2024-03-26T14:00:55.217886+0000","last_fullsized":"2024-03-26T14:00:55.217886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212358+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T14:00:55.212358+0000","last_peered":"2024-03-26T14:00:55.212358+0000","last_clean":"2024-03-26T14:00:55.212358+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T14:00:55.212358+0000","last_undegraded":"2024-03-26T14:00:55.212358+0000","last_fullsized":"2024-03-26T14:00:55.212358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226910+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T14:00:55.226910+0000","last_peered":"2024-03-26T14:00:55.226910+0000","last_clean":"2024-03-26T14:00:55.226910+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T14:00:55.226910+0000","last_undegraded":"2024-03-26T14:00:55.226910+0000","last_fullsized":"2024-03-26T14:00:55.226910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226746+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T14:00:55.226746+0000","last_peered":"2024-03-26T14:00:55.226746+0000","last_clean":"2024-03-26T14:00:55.226746+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T14:00:55.226746+0000","last_undegraded":"2024-03-26T14:00:55.226746+0000","last_fullsized":"2024-03-26T14:00:55.226746+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217023+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T14:00:55.217023+0000","last_peered":"2024-03-26T14:00:55.217023+0000","last_clean":"2024-03-26T14:00:55.217023+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T14:00:55.217023+0000","last_undegraded":"2024-03-26T14:00:55.217023+0000","last_fullsized":"2024-03-26T14:00:55.217023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221673+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T14:00:55.221673+0000","last_peered":"2024-03-26T14:00:55.221673+0000","last_clean":"2024-03-26T14:00:55.221673+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T14:00:55.221673+0000","last_undegraded":"2024-03-26T14:00:55.221673+0000","last_fullsized":"2024-03-26T14:00:55.221673+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214379+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T14:00:55.214379+0000","last_peered":"2024-03-26T14:00:55.214379+0000","last_clean":"2024-03-26T14:00:55.214379+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T14:00:55.214379+0000","last_undegraded":"2024-03-26T14:00:55.214379+0000","last_fullsized":"2024-03-26T14:00:55.214379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224501+0000","last_change":"2024-03-26T14:00:55.224501+0000","last_active":"2024-03-26T14:00:55.224501+0000","last_peered":"2024-03-26T14:00:55.224501+0000","last_clean":"2024-03-26T14:00:55.224501+0000","last_became_active":"2024-03-26T14:00:55.223843+0000","last_became_peered":"2024-03-26T14:00:55.223843+0000","last_unstale":"2024-03-26T14:00:55.224501+0000","last_undegraded":"2024-03-26T14:00:55.224501+0000","last_fullsized":"2024-03-26T14:00:55.224501+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:29:11.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217725+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T14:00:55.217725+0000","last_peered":"2024-03-26T14:00:55.217725+0000","last_clean":"2024-03-26T14:00:55.217725+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T14:00:55.217725+0000","last_undegraded":"2024-03-26T14:00:55.217725+0000","last_fullsized":"2024-03-26T14:00:55.217725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217082+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T14:00:55.217082+0000","last_peered":"2024-03-26T14:00:55.217082+0000","last_clean":"2024-03-26T14:00:55.217082+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T14:00:55.217082+0000","last_undegraded":"2024-03-26T14:00:55.217082+0000","last_fullsized":"2024-03-26T14:00:55.217082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214292+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T14:00:55.214292+0000","last_peered":"2024-03-26T14:00:55.214292+0000","last_clean":"2024-03-26T14:00:55.214292+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T14:00:55.214292+0000","last_undegraded":"2024-03-26T14:00:55.214292+0000","last_fullsized":"2024-03-26T14:00:55.214292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217573+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T14:00:55.217573+0000","last_peered":"2024-03-26T14:00:55.217573+0000","last_clean":"2024-03-26T14:00:55.217573+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T14:00:55.217573+0000","last_undegraded":"2024-03-26T14:00:55.217573+0000","last_fullsized":"2024-03-26T14:00:55.217573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217833+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T14:00:55.217833+0000","last_peered":"2024-03-26T14:00:55.217833+0000","last_clean":"2024-03-26T14:00:55.217833+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T14:00:55.217833+0000","last_undegraded":"2024-03-26T14:00:55.217833+0000","last_fullsized":"2024-03-26T14:00:55.217833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227098+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T14:00:55.227098+0000","last_peered":"2024-03-26T14:00:55.227098+0000","last_clean":"2024-03-26T14:00:55.227098+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T14:00:55.227098+0000","last_undegraded":"2024-03-26T14:00:55.227098+0000","last_fullsized":"2024-03-26T14:00:55.227098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":134,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:26.538576+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T14:01:26.538576+0000","last_peered":"2024-03-26T14:01:26.538576+0000","last_clean":"2024-03-26T14:01:26.538576+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T14:01:26.538576+0000","last_undegraded":"2024-03-26T14:01:26.538576+0000","last_fullsized":"2024-03-26T14:01:26.538576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214001+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T14:00:55.214001+0000","last_peered":"2024-03-26T14:00:55.214001+0000","last_clean":"2024-03-26T14:00:55.214001+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T14:00:55.214001+0000","last_undegraded":"2024-03-26T14:00:55.214001+0000","last_fullsized":"2024-03-26T14:00:55.214001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206949+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T14:00:55.206949+0000","last_peered":"2024-03-26T14:00:55.206949+0000","last_clean":"2024-03-26T14:00:55.206949+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T14:00:55.206949+0000","last_undegraded":"2024-03-26T14:00:55.206949+0000","last_fullsized":"2024-03-26T14:00:55.206949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209531+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T14:00:55.209531+0000","last_peered":"2024-03-26T14:00:55.209531+0000","last_clean":"2024-03-26T14:00:55.209531+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T14:00:55.209531+0000","last_undegraded":"2024-03-26T14:00:55.209531+0000","last_fullsized":"2024-03-26T14:00:55.209531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213309+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T14:00:55.213309+0000","last_peered":"2024-03-26T14:00:55.213309+0000","last_clean":"2024-03-26T14:00:55.213309+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T14:00:55.213309+0000","last_undegraded":"2024-03-26T14:00:55.213309+0000","last_fullsized":"2024-03-26T14:00:55.213309+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228357+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T14:00:55.228357+0000","last_peered":"2024-03-26T14:00:55.228357+0000","last_clean":"2024-03-26T14:00:55.228357+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T14:00:55.228357+0000","last_undegraded":"2024-03-26T14:00:55.228357+0000","last_fullsized":"2024-03-26T14:00:55.228357+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216629+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T14:00:55.216629+0000","last_peered":"2024-03-26T14:00:55.216629+0000","last_clean":"2024-03-26T14:00:55.216629+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T14:00:55.216629+0000","last_undegraded":"2024-03-26T14:00:55.216629+0000","last_fullsized":"2024-03-26T14:00:55.216629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.439884+0000","last_change":"2024-03-26T14:00:55.439230+0000","last_active":"2024-03-26T14:00:55.439884+0000","last_peered":"2024-03-26T14:00:55.439884+0000","last_clean":"2024-03-26T14:00:55.439884+0000","last_became_active":"2024-03-26T14:00:55.434988+0000","last_became_peered":"2024-03-26T14:00:55.434988+0000","last_unstale":"2024-03-26T14:00:55.439884+0000","last_undegraded":"2024-03-26T14:00:55.439884+0000","last_fullsized":"2024-03-26T14:00:55.439884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:58.291575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222955+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T14:00:55.222955+0000","last_peered":"2024-03-26T14:00:55.222955+0000","last_clean":"2024-03-26T14:00:55.222955+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T14:00:55.222955+0000","last_undegraded":"2024-03-26T14:00:55.222955+0000","last_fullsized":"2024-03-26T14:00:55.222955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217076+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T14:00:55.217076+0000","last_peered":"2024-03-26T14:00:55.217076+0000","last_clean":"2024-03-26T14:00:55.217076+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T14:00:55.217076+0000","last_undegraded":"2024-03-26T14:00:55.217076+0000","last_fullsized":"2024-03-26T14:00:55.217076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217284+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T14:00:55.217284+0000","last_peered":"2024-03-26T14:00:55.217284+0000","last_clean":"2024-03-26T14:00:55.217284+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T14:00:55.217284+0000","last_undegraded":"2024-03-26T14:00:55.217284+0000","last_fullsized":"2024-03-26T14:00:55.217284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206683+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T14:00:55.206683+0000","last_peered":"2024-03-26T14:00:55.206683+0000","last_clean":"2024-03-26T14:00:55.206683+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T14:00:55.206683+0000","last_undegraded":"2024-03-26T14:00:55.206683+0000","last_fullsized":"2024-03-26T14:00:55.206683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212027+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T14:00:55.212027+0000","last_peered":"2024-03-26T14:00:55.212027+0000","last_clean":"2024-03-26T14:00:55.212027+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T14:00:55.212027+0000","last_undegraded":"2024-03-26T14:00:55.212027+0000","last_fullsized":"2024-03-26T14:00:55.212027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206914+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T14:00:55.206914+0000","last_peered":"2024-03-26T14:00:55.206914+0000","last_clean":"2024-03-26T14:00:55.206914+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T14:00:55.206914+0000","last_undegraded":"2024-03-26T14:00:55.206914+0000","last_fullsized":"2024-03-26T14:00:55.206914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211560+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T14:00:55.211560+0000","last_peered":"2024-03-26T14:00:55.211560+0000","last_clean":"2024-03-26T14:00:55.211560+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T14:00:55.211560+0000","last_undegraded":"2024-03-26T14:00:55.211560+0000","last_fullsized":"2024-03-26T14:00:55.211560+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210372+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T14:00:55.210372+0000","last_peered":"2024-03-26T14:00:55.210372+0000","last_clean":"2024-03-26T14:00:55.210372+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T14:00:55.210372+0000","last_undegraded":"2024-03-26T14:00:55.210372+0000","last_fullsized":"2024-03-26T14:00:55.210372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212585+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T14:00:55.212585+0000","last_peered":"2024-03-26T14:00:55.212585+0000","last_clean":"2024-03-26T14:00:55.212585+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T14:00:55.212585+0000","last_undegraded":"2024-03-26T14:00:55.212585+0000","last_fullsized":"2024-03-26T14:00:55.212585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210097+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T14:00:55.210097+0000","last_peered":"2024-03-26T14:00:55.210097+0000","last_clean":"2024-03-26T14:00:55.210097+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T14:00:55.210097+0000","last_undegraded":"2024-03-26T14:00:55.210097+0000","last_fullsized":"2024-03-26T14:00:55.210097+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433759+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T14:00:55.433759+0000","last_peered":"2024-03-26T14:00:55.433759+0000","last_clean":"2024-03-26T14:00:55.433759+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T14:00:55.433759+0000","last_undegraded":"2024-03-26T14:00:55.433759+0000","last_fullsized":"2024-03-26T14:00:55.433759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213140+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T14:00:55.213140+0000","last_peered":"2024-03-26T14:00:55.213140+0000","last_clean":"2024-03-26T14:00:55.213140+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T14:00:55.213140+0000","last_undegraded":"2024-03-26T14:00:55.213140+0000","last_fullsized":"2024-03-26T14:00:55.213140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216585+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T14:00:55.216585+0000","last_peered":"2024-03-26T14:00:55.216585+0000","last_clean":"2024-03-26T14:00:55.216585+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T14:00:55.216585+0000","last_undegraded":"2024-03-26T14:00:55.216585+0000","last_fullsized":"2024-03-26T14:00:55.216585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206741+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T14:00:55.206741+0000","last_peered":"2024-03-26T14:00:55.206741+0000","last_clean":"2024-03-26T14:00:55.206741+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T14:00:55.206741+0000","last_undegraded":"2024-03-26T14:00:55.206741+0000","last_fullsized":"2024-03-26T14:00:55.206741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215304+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T14:00:55.215304+0000","last_peered":"2024-03-26T14:00:55.215304+0000","last_clean":"2024-03-26T14:00:55.215304+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T14:00:55.215304+0000","last_undegraded":"2024-03-26T14:00:55.215304+0000","last_fullsized":"2024-03-26T14:00:55.215304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209195+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T14:00:55.209195+0000","last_peered":"2024-03-26T14:00:55.209195+0000","last_clean":"2024-03-26T14:00:55.209195+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T14:00:55.209195+0000","last_undegraded":"2024-03-26T14:00:55.209195+0000","last_fullsized":"2024-03-26T14:00:55.209195+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213884+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T14:00:55.213884+0000","last_peered":"2024-03-26T14:00:55.213884+0000","last_clean":"2024-03-26T14:00:55.213884+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T14:00:55.213884+0000","last_undegraded":"2024-03-26T14:00:55.213884+0000","last_fullsized":"2024-03-26T14:00:55.213884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207328+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T14:00:55.207328+0000","last_peered":"2024-03-26T14:00:55.207328+0000","last_clean":"2024-03-26T14:00:55.207328+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T14:00:55.207328+0000","last_undegraded":"2024-03-26T14:00:55.207328+0000","last_fullsized":"2024-03-26T14:00:55.207328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209775+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T14:00:55.209775+0000","last_peered":"2024-03-26T14:00:55.209775+0000","last_clean":"2024-03-26T14:00:55.209775+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T14:00:55.209775+0000","last_undegraded":"2024-03-26T14:00:55.209775+0000","last_fullsized":"2024-03-26T14:00:55.209775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213124+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T14:00:55.213124+0000","last_peered":"2024-03-26T14:00:55.213124+0000","last_clean":"2024-03-26T14:00:55.213124+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T14:00:55.213124+0000","last_undegraded":"2024-03-26T14:00:55.213124+0000","last_fullsized":"2024-03-26T14:00:55.213124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214058+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T14:00:55.214058+0000","last_peered":"2024-03-26T14:00:55.214058+0000","last_clean":"2024-03-26T14:00:55.214058+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T14:00:55.214058+0000","last_undegraded":"2024-03-26T14:00:55.214058+0000","last_fullsized":"2024-03-26T14:00:55.214058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.432532+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T14:00:55.432532+0000","last_peered":"2024-03-26T14:00:55.432532+0000","last_clean":"2024-03-26T14:00:55.432532+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T14:00:55.432532+0000","last_undegraded":"2024-03-26T14:00:55.432532+0000","last_fullsized":"2024-03-26T14:00:55.432532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212693+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T14:00:55.212693+0000","last_peered":"2024-03-26T14:00:55.212693+0000","last_clean":"2024-03-26T14:00:55.212693+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T14:00:55.212693+0000","last_undegraded":"2024-03-26T14:00:55.212693+0000","last_fullsized":"2024-03-26T14:00:55.212693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214241+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T14:00:55.214241+0000","last_peered":"2024-03-26T14:00:55.214241+0000","last_clean":"2024-03-26T14:00:55.214241+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T14:00:55.214241+0000","last_undegraded":"2024-03-26T14:00:55.214241+0000","last_fullsized":"2024-03-26T14:00:55.214241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210488+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T14:00:55.210488+0000","last_peered":"2024-03-26T14:00:55.210488+0000","last_clean":"2024-03-26T14:00:55.210488+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T14:00:55.210488+0000","last_undegraded":"2024-03-26T14:00:55.210488+0000","last_fullsized":"2024-03-26T14:00:55.210488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228782+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T14:00:55.228782+0000","last_peered":"2024-03-26T14:00:55.228782+0000","last_clean":"2024-03-26T14:00:55.228782+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T14:00:55.228782+0000","last_undegraded":"2024-03-26T14:00:55.228782+0000","last_fullsized":"2024-03-26T14:00:55.228782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214577+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T14:00:55.214577+0000","last_peered":"2024-03-26T14:00:55.214577+0000","last_clean":"2024-03-26T14:00:55.214577+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T14:00:55.214577+0000","last_undegraded":"2024-03-26T14:00:55.214577+0000","last_fullsized":"2024-03-26T14:00:55.214577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225684+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T14:00:55.225684+0000","last_peered":"2024-03-26T14:00:55.225684+0000","last_clean":"2024-03-26T14:00:55.225684+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T14:00:55.225684+0000","last_undegraded":"2024-03-26T14:00:55.225684+0000","last_fullsized":"2024-03-26T14:00:55.225684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217803+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T14:00:55.217803+0000","last_peered":"2024-03-26T14:00:55.217803+0000","last_clean":"2024-03-26T14:00:55.217803+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T14:00:55.217803+0000","last_undegraded":"2024-03-26T14:00:55.217803+0000","last_fullsized":"2024-03-26T14:00:55.217803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214677+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T14:00:55.214677+0000","last_peered":"2024-03-26T14:00:55.214677+0000","last_clean":"2024-03-26T14:00:55.214677+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T14:00:55.214677+0000","last_undegraded":"2024-03-26T14:00:55.214677+0000","last_fullsized":"2024-03-26T14:00:55.214677+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215690+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T14:00:55.215690+0000","last_peered":"2024-03-26T14:00:55.215690+0000","last_clean":"2024-03-26T14:00:55.215690+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T14:00:55.215690+0000","last_undegraded":"2024-03-26T14:00:55.215690+0000","last_fullsized":"2024-03-26T14:00:55.215690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212266+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T14:00:55.212266+0000","last_peered":"2024-03-26T14:00:55.212266+0000","last_clean":"2024-03-26T14:00:55.212266+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T14:00:55.212266+0000","last_undegraded":"2024-03-26T14:00:55.212266+0000","last_fullsized":"2024-03-26T14:00:55.212266+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213444+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T14:00:55.213444+0000","last_peered":"2024-03-26T14:00:55.213444+0000","last_clean":"2024-03-26T14:00:55.213444+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T14:00:55.213444+0000","last_undegraded":"2024-03-26T14:00:55.213444+0000","last_fullsized":"2024-03-26T14:00:55.213444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213850+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T14:00:55.213850+0000","last_peered":"2024-03-26T14:00:55.213850+0000","last_clean":"2024-03-26T14:00:55.213850+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T14:00:55.213850+0000","last_undegraded":"2024-03-26T14:00:55.213850+0000","last_fullsized":"2024-03-26T14:00:55.213850+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217393+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T14:00:55.217393+0000","last_peered":"2024-03-26T14:00:55.217393+0000","last_clean":"2024-03-26T14:00:55.217393+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T14:00:55.217393+0000","last_undegraded":"2024-03-26T14:00:55.217393+0000","last_fullsized":"2024-03-26T14:00:55.217393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221405+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T14:00:55.221405+0000","last_peered":"2024-03-26T14:00:55.221405+0000","last_clean":"2024-03-26T14:00:55.221405+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T14:00:55.221405+0000","last_undegraded":"2024-03-26T14:00:55.221405+0000","last_fullsized":"2024-03-26T14:00:55.221405+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218347+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T14:00:55.218347+0000","last_peered":"2024-03-26T14:00:55.218347+0000","last_clean":"2024-03-26T14:00:55.218347+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T14:00:55.218347+0000","last_undegraded":"2024-03-26T14:00:55.218347+0000","last_fullsized":"2024-03-26T14:00:55.218347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207012+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T14:00:55.207012+0000","last_peered":"2024-03-26T14:00:55.207012+0000","last_clean":"2024-03-26T14:00:55.207012+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T14:00:55.207012+0000","last_undegraded":"2024-03-26T14:00:55.207012+0000","last_fullsized":"2024-03-26T14:00:55.207012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226803+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T14:00:55.226803+0000","last_peered":"2024-03-26T14:00:55.226803+0000","last_clean":"2024-03-26T14:00:55.226803+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T14:00:55.226803+0000","last_undegraded":"2024-03-26T14:00:55.226803+0000","last_fullsized":"2024-03-26T14:00:55.226803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206825+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T14:00:55.206825+0000","last_peered":"2024-03-26T14:00:55.206825+0000","last_clean":"2024-03-26T14:00:55.206825+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T14:00:55.206825+0000","last_undegraded":"2024-03-26T14:00:55.206825+0000","last_fullsized":"2024-03-26T14:00:55.206825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433719+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T14:00:55.433719+0000","last_peered":"2024-03-26T14:00:55.433719+0000","last_clean":"2024-03-26T14:00:55.433719+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T14:00:55.433719+0000","last_undegraded":"2024-03-26T14:00:55.433719+0000","last_fullsized":"2024-03-26T14:00:55.433719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217090+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T14:00:55.217090+0000","last_peered":"2024-03-26T14:00:55.217090+0000","last_clean":"2024-03-26T14:00:55.217090+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T14:00:55.217090+0000","last_undegraded":"2024-03-26T14:00:55.217090+0000","last_fullsized":"2024-03-26T14:00:55.217090+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220969+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T14:00:55.220969+0000","last_peered":"2024-03-26T14:00:55.220969+0000","last_clean":"2024-03-26T14:00:55.220969+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T14:00:55.220969+0000","last_undegraded":"2024-03-26T14:00:55.220969+0000","last_fullsized":"2024-03-26T14:00:55.220969+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71985,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":600,"num_read_kb":380,"num_write":170,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":215955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":522925,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716972,"statfs":{"total":95995035648,"available":95966179328,"internally_reserved":0,"allocated":1183744,"data_stored":1182630,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776183,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":523127,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037822,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":523047,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299461,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1119097,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462987,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518303,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1097728,"data_stored":586056,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738485,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519691,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":587372,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149792,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":518497,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888154,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523485,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":1114153,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:01:28.662 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-26T14:01:28.662 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-26T14:01:28.662 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-26T14:01:28.662 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph health --format=json 2024-03-26T14:01:28.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:28 smithi179 ceph-mon[35472]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:28.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:28 smithi159 ceph-mon[35501]: pgmap v431: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:29.151 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:29.511 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:29 smithi137 ceph-mon[31539]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:01:29.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:29 smithi179 ceph-mon[35472]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:01:29.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:29 smithi159 ceph-mon[35501]: from='client.15192 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:01:30.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:30 smithi137 ceph-mon[31539]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:30 smithi179 ceph-mon[35472]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:30.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:30 smithi159 ceph-mon[35501]: pgmap v432: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:31.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 -- 172.21.15.137:0/876015629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c109160 msgr2=0x7f008c115670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 --2- 172.21.15.137:0/876015629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c109160 0x7f008c115670 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f008400b0a0 tx=0x7f008400b3b0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 -- 172.21.15.137:0/876015629 shutdown_connections 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 --2- 172.21.15.137:0/876015629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c109160 0x7f008c115670 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 --2- 172.21.15.137:0/876015629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f008c10e7a0 0x7f008c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 --2- 172.21.15.137:0/876015629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f008c1087a0 0x7f008c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 -- 172.21.15.137:0/876015629 >> 172.21.15.137:0/876015629 conn(0x7f008c075960 msgr2=0x7f008c077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 -- 172.21.15.137:0/876015629 shutdown_connections 2024-03-26T14:01:31.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.422+0000 7f0094200700 1 -- 172.21.15.137:0/876015629 wait complete. 2024-03-26T14:01:31.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 Processor -- start 2024-03-26T14:01:31.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 -- start start 2024-03-26T14:01:31.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f008c1087a0 0x7f008c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:31.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f008c109160 0x7f008c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:31.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 0x7f008c1a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c118a30 con 0x7f008c10e7a0 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c1188b0 con 0x7f008c1087a0 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.423+0000 7f0094200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c118bb0 con 0x7f008c109160 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 0x7f008c1a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 0x7f008c1a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42818/0 (socket says 172.21.15.137:42818) 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 -- 172.21.15.137:0/2190337085 learned_addr learned my addr 172.21.15.137:0/2190337085 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0091800700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f008c1087a0 0x7f008c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0090e00700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f008c109160 0x7f008c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 -- 172.21.15.137:0/2190337085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f008c109160 msgr2=0x7f008c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f008c109160 0x7f008c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 -- 172.21.15.137:0/2190337085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f008c1087a0 msgr2=0x7f008c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f008c1087a0 0x7f008c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.424+0000 7f0092200700 1 -- 172.21.15.137:0/2190337085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c1a3e50 con 0x7f008c10e7a0 2024-03-26T14:01:31.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.425+0000 7f0092200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 0x7f008c1a3710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f008400b070 tx=0x7f0084004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:31.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.425+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084021070 con 0x7f008c10e7a0 2024-03-26T14:01:31.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.425+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0084007e30 con 0x7f008c10e7a0 2024-03-26T14:01:31.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.425+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c1a40e0 con 0x7f008c10e7a0 2024-03-26T14:01:31.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.425+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084004780 con 0x7f008c10e7a0 2024-03-26T14:01:31.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.425+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c1ab9c0 con 0x7f008c10e7a0 2024-03-26T14:01:31.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.426+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c068e50 con 0x7f008c10e7a0 2024-03-26T14:01:31.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.428+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0084007980 con 0x7f008c10e7a0 2024-03-26T14:01:31.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.429+0000 7f008a000700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00740a2910 0x7f00740a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:31.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.430+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f008400f020 con 0x7f008c10e7a0 2024-03-26T14:01:31.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.430+0000 7f0091800700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00740a2910 0x7f00740a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:31.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.430+0000 7f0091800700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00740a2910 0x7f00740a4dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f00800045b0 tx=0x7f008000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:31.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.432+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f008402a020 con 0x7f008c10e7a0 2024-03-26T14:01:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.704+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f008c1a01a0 con 0x7f008c10e7a0 2024-03-26T14:01:31.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.704+0000 7f008a000700 1 -- 172.21.15.137:0/2190337085 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f008409d390 con 0x7f008c10e7a0 2024-03-26T14:01:31.701 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:31.702 INFO:teuthology.orchestra.run.smithi137.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-26T14:01:31.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00740a2910 msgr2=0x7f00740a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00740a2910 0x7f00740a4dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f00800045b0 tx=0x7f008000b3c0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 msgr2=0x7f008c1a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 0x7f008c1a3710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f008400b070 tx=0x7f0084004cf0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 shutdown_connections 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00740a2910 0x7f00740a4dd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f008c10e7a0 0x7f008c1a3710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f008c1087a0 0x7f008c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 --2- 172.21.15.137:0/2190337085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f008c109160 0x7f008c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 >> 172.21.15.137:0/2190337085 conn(0x7f008c075960 msgr2=0x7f008c075ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.707+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 shutdown_connections 2024-03-26T14:01:31.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:31.708+0000 7f0094200700 1 -- 172.21.15.137:0/2190337085 wait complete. 2024-03-26T14:01:32.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:32 smithi137 ceph-mon[31539]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:32.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2190337085' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T14:01:32.651 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-26T14:01:32.651 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:01:32.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:32 smithi179 ceph-mon[35472]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:32.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2190337085' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T14:01:32.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:32 smithi159 ceph-mon[35501]: pgmap v433: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:32.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2190337085' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-26T14:01:32.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.871+0000 7fdfd2a00640 1 -- 172.21.15.137:0/2275877190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1036f0 msgr2=0x7fdfcc103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:32.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.871+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/2275877190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1036f0 0x7fdfcc103b10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009900 tx=0x7fdfc002bc40 comp rx=0 tx=0).stop 2024-03-26T14:01:32.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.871+0000 7fdfd2a00640 1 -- 172.21.15.137:0/2275877190 shutdown_connections 2024-03-26T14:01:32.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.871+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/2275877190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc10bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:32.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.872+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/2275877190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1048e0 0x7fdfcc104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:32.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.872+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/2275877190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1036f0 0x7fdfcc103b10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:32.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.872+0000 7fdfd2a00640 1 -- 172.21.15.137:0/2275877190 >> 172.21.15.137:0/2275877190 conn(0x7fdfcc0fec80 msgr2=0x7fdfcc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:32.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.872+0000 7fdfd2a00640 1 -- 172.21.15.137:0/2275877190 shutdown_connections 2024-03-26T14:01:32.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.872+0000 7fdfd2a00640 1 -- 172.21.15.137:0/2275877190 wait complete. 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 Processor -- start 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 -- start start 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfcc1036f0 0x7fdfcc199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1048e0 0x7fdfcc199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc10f2c0 con 0x7fdfcc1052a0 2024-03-26T14:01:32.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc10f140 con 0x7fdfcc1036f0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc10f440 con 0x7fdfcc1048e0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc1a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42842/0 (socket says 172.21.15.137:42842) 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd2000640 1 -- 172.21.15.137:0/4112573382 learned_addr learned my addr 172.21.15.137:0/4112573382 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.873+0000 7fdfd1600640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfcc1036f0 0x7fdfcc199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd0c00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1048e0 0x7fdfcc199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2000640 1 -- 172.21.15.137:0/4112573382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1048e0 msgr2=0x7fdfcc199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2000640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1048e0 0x7fdfcc199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2000640 1 -- 172.21.15.137:0/4112573382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfcc1036f0 msgr2=0x7fdfcc199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2000640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfcc1036f0 0x7fdfcc199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2000640 1 -- 172.21.15.137:0/4112573382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc1a1580 con 0x7fdfcc1052a0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2000640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc1a0f10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400cce0 tx=0x7fdfc400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400f450 con 0x7fdfcc1052a0 2024-03-26T14:01:32.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdfc400faa0 con 0x7fdfcc1052a0 2024-03-26T14:01:32.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4018b70 con 0x7fdfcc1052a0 2024-03-26T14:01:32.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.874+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc1a1870 con 0x7fdfcc1052a0 2024-03-26T14:01:32.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.875+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc1a1df0 con 0x7fdfcc1052a0 2024-03-26T14:01:32.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.876+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc10aa60 con 0x7fdfcc1052a0 2024-03-26T14:01:32.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.878+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdfc4018d10 con 0x7fdfcc1052a0 2024-03-26T14:01:32.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.879+0000 7fdfca000640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:32.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.879+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdfc40cf260 con 0x7fdfcc1052a0 2024-03-26T14:01:32.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.879+0000 7fdfd1600640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:32.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.880+0000 7fdfd1600640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdfc002c250 tx=0x7fdfc0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:32.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:32.883+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdfc4014030 con 0x7fdfcc1052a0 2024-03-26T14:01:33.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.085+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfcc0622a0 con 0x7fdfcc1052a0 2024-03-26T14:01:33.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.087+0000 7fdfca000640 1 -- 172.21.15.137:0/4112573382 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7fdfc409bf30 con 0x7fdfcc1052a0 2024-03-26T14:01:33.085 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:33.085 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdfb40a2800 msgr2=0x7fdfb40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdfc002c250 tx=0x7fdfc0000bc0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 msgr2=0x7fdfcc1a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc1a0f10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400cce0 tx=0x7fdfc400a620 comp rx=0 tx=0).stop 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 shutdown_connections 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdfb40a2800 0x7fdfb40a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdfcc1052a0 0x7fdfcc1a0f10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdfcc1048e0 0x7fdfcc199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 --2- 172.21.15.137:0/4112573382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfcc1036f0 0x7fdfcc199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.090+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 >> 172.21.15.137:0/4112573382 conn(0x7fdfcc0fec80 msgr2=0x7fdfcc107c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.091+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 shutdown_connections 2024-03-26T14:01:33.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.091+0000 7fdfd2a00640 1 -- 172.21.15.137:0/4112573382 wait complete. 2024-03-26T14:01:33.107 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:01:33.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:01:33.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 -- 172.21.15.137:0/4062959752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 msgr2=0x7fabc8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 --2- 172.21.15.137:0/4062959752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 0x7fabc8103ce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009900 tx=0x7fabb802bc40 comp rx=0 tx=0).stop 2024-03-26T14:01:33.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 -- 172.21.15.137:0/4062959752 shutdown_connections 2024-03-26T14:01:33.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 --2- 172.21.15.137:0/4062959752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 --2- 172.21.15.137:0/4062959752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabc8104ab0 0x7fabc8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 --2- 172.21.15.137:0/4062959752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 0x7fabc8103ce0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 -- 172.21.15.137:0/4062959752 >> 172.21.15.137:0/4062959752 conn(0x7fabc80fee50 msgr2=0x7fabc8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:33.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 -- 172.21.15.137:0/4062959752 shutdown_connections 2024-03-26T14:01:33.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.327+0000 7fabccc00640 1 -- 172.21.15.137:0/4062959752 wait complete. 2024-03-26T14:01:33.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.328+0000 7fabccc00640 1 Processor -- start 2024-03-26T14:01:33.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.328+0000 7fabccc00640 1 -- start start 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabccc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 0x7fabc8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabccc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabc8104ab0 0x7fabc819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabccc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc81a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabccc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc810f3c0 con 0x7fabc8105470 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabccc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc810f240 con 0x7fabc81038c0 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabccc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc810f540 con 0x7fabc8104ab0 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc81a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc81a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42866/0 (socket says 172.21.15.137:42866) 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 -- 172.21.15.137:0/351658510 learned_addr learned my addr 172.21.15.137:0/351658510 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:33.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc5e00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 0x7fabc8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabbd400640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabc8104ab0 0x7fabc819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 -- 172.21.15.137:0/351658510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabc8104ab0 msgr2=0x7fabc819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabc8104ab0 0x7fabc819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 -- 172.21.15.137:0/351658510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 msgr2=0x7fabc8199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 0x7fabc8199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.329+0000 7fabc6800640 1 -- 172.21.15.137:0/351658510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc81a1770 con 0x7fabc8105470 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.330+0000 7fabc6800640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc81a1100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabc000efd0 tx=0x7fabc000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.330+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0003b70 con 0x7fabc8105470 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.330+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fabc0003d10 con 0x7fabc8105470 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.330+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc81a1a60 con 0x7fabc8105470 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.330+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0010980 con 0x7fabc8105470 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.330+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc81a1f60 con 0x7fabc8105470 2024-03-26T14:01:33.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.332+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc810ad00 con 0x7fabc8105470 2024-03-26T14:01:33.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.333+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fabc0004020 con 0x7fabc8105470 2024-03-26T14:01:33.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.335+0000 7fabbf400640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabac0a2800 0x7fabac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.335+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fabc00cf330 con 0x7fabc8105470 2024-03-26T14:01:33.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.335+0000 7fabc5e00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabac0a2800 0x7fabac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.336+0000 7fabc5e00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabac0a2800 0x7fabac0a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005ab0 tx=0x7fabb8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:33.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.339+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fabc009bfc0 con 0x7fabc8105470 2024-03-26T14:01:33.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:33 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4112573382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:33.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.537+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabc80622a0 con 0x7fabc8105470 2024-03-26T14:01:33.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.538+0000 7fabbf400640 1 -- 172.21.15.137:0/351658510 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabc009b8c0 con 0x7fabc8105470 2024-03-26T14:01:33.535 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:01:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabac0a2800 msgr2=0x7fabac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabac0a2800 0x7fabac0a4cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fabb8005ab0 tx=0x7fabb8005fd0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 msgr2=0x7fabc81a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc81a1100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabc000efd0 tx=0x7fabc000c6a0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 shutdown_connections 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabac0a2800 0x7fabac0a4cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabc8105470 0x7fabc81a1100 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabc8104ab0 0x7fabc819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 --2- 172.21.15.137:0/351658510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabc81038c0 0x7fabc8199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 >> 172.21.15.137:0/351658510 conn(0x7fabc80fee50 msgr2=0x7fabc8107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 shutdown_connections 2024-03-26T14:01:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.541+0000 7fabccc00640 1 -- 172.21.15.137:0/351658510 wait complete. 2024-03-26T14:01:33.558 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:01:33.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:33 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4112573382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:33 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4112573382' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 -- 172.21.15.137:0/2902507410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f8077c40 msgr2=0x7f51f80784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 --2- 172.21.15.137:0/2902507410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f8077c40 0x7f51f80784e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009ca0 tx=0x7f51f002c100 comp rx=0 tx=0).stop 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 -- 172.21.15.137:0/2902507410 shutdown_connections 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 --2- 172.21.15.137:0/2902507410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f8077c40 0x7f51f80784e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 --2- 172.21.15.137:0/2902507410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f80772a0 0x7f51f8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 --2- 172.21.15.137:0/2902507410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f8076060 0x7f51f8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 -- 172.21.15.137:0/2902507410 >> 172.21.15.137:0/2902507410 conn(0x7f51f80fee50 msgr2=0x7f51f8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 -- 172.21.15.137:0/2902507410 shutdown_connections 2024-03-26T14:01:33.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.777+0000 7f51fea00640 1 -- 172.21.15.137:0/2902507410 wait complete. 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 Processor -- start 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 -- start start 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f8076060 0x7f51f819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 0x7f51f819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f8077c40 0x7f51f81a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8113820 con 0x7f51f80772a0 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81136a0 con 0x7f51f8077c40 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51fea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81139a0 con 0x7f51f8076060 2024-03-26T14:01:33.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 0x7f51f819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 0x7f51f819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42884/0 (socket says 172.21.15.137:42884) 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.778+0000 7f51ef400640 1 -- 172.21.15.137:0/2566910327 learned_addr learned my addr 172.21.15.137:0/2566910327 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51fca00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f8077c40 0x7f51f81a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51f7e00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f8076060 0x7f51f819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51ef400640 1 -- 172.21.15.137:0/2566910327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f8076060 msgr2=0x7f51f819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51ef400640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f8076060 0x7f51f819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51ef400640 1 -- 172.21.15.137:0/2566910327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f8077c40 msgr2=0x7f51f81a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51ef400640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f8077c40 0x7f51f81a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51ef400640 1 -- 172.21.15.137:0/2566910327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f81a5ad0 con 0x7f51f80772a0 2024-03-26T14:01:33.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51ef400640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 0x7f51f819e3f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f51e000d950 tx=0x7f51e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:33.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0017070 con 0x7f51f80772a0 2024-03-26T14:01:33.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.779+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f51e0009c30 con 0x7f51f80772a0 2024-03-26T14:01:33.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.780+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f81a5dc0 con 0x7f51f80772a0 2024-03-26T14:01:33.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.780+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e000f620 con 0x7f51f80772a0 2024-03-26T14:01:33.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.780+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81a6280 con 0x7f51f80772a0 2024-03-26T14:01:33.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.781+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f8110250 con 0x7f51f80772a0 2024-03-26T14:01:33.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.783+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f51e00109a0 con 0x7f51f80772a0 2024-03-26T14:01:33.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.785+0000 7f51f5600640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f51dc0a2800 0x7f51dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:33.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.785+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f51e00ceb30 con 0x7f51f80772a0 2024-03-26T14:01:33.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.785+0000 7f51f7e00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f51dc0a2800 0x7f51dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:33.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.786+0000 7f51f7e00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f51dc0a2800 0x7f51dc0a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005fd0 tx=0x7f51e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:33.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.790+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f51e0009de0 con 0x7f51f80772a0 2024-03-26T14:01:33.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.989+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51f80623e0 con 0x7f51f80772a0 2024-03-26T14:01:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.990+0000 7f51f5600640 1 -- 172.21.15.137:0/2566910327 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51e009b800 con 0x7f51f80772a0 2024-03-26T14:01:33.987 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f51dc0a2800 msgr2=0x7f51dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f51dc0a2800 0x7f51dc0a4cc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005fd0 tx=0x7f51e8005850 comp rx=0 tx=0).stop 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 msgr2=0x7f51f819e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 0x7f51f819e3f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f51e000d950 tx=0x7f51e000de10 comp rx=0 tx=0).stop 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 shutdown_connections 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f51dc0a2800 0x7f51dc0a4cc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51f8077c40 0x7f51f81a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f51f80772a0 0x7f51f819e3f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 --2- 172.21.15.137:0/2566910327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f51f8076060 0x7f51f819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:33.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 >> 172.21.15.137:0/2566910327 conn(0x7f51f80fee50 msgr2=0x7f51f8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:33.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 shutdown_connections 2024-03-26T14:01:33.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:33.993+0000 7f51fea00640 1 -- 172.21.15.137:0/2566910327 wait complete. 2024-03-26T14:01:34.009 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 -- 172.21.15.137:0/1529677238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5778077c40 msgr2=0x7f57780784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 --2- 172.21.15.137:0/1529677238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5778077c40 0x7f57780784e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5770009ca0 tx=0x7f577002c100 comp rx=0 tx=0).stop 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 -- 172.21.15.137:0/1529677238 shutdown_connections 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 --2- 172.21.15.137:0/1529677238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57780772a0 0x7f5778077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 --2- 172.21.15.137:0/1529677238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5778077c40 0x7f57780784e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 --2- 172.21.15.137:0/1529677238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778076060 0x7f5778076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 -- 172.21.15.137:0/1529677238 >> 172.21.15.137:0/1529677238 conn(0x7f57780fee50 msgr2=0x7f5778101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 -- 172.21.15.137:0/1529677238 shutdown_connections 2024-03-26T14:01:34.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.227+0000 7f577f400640 1 -- 172.21.15.137:0/1529677238 wait complete. 2024-03-26T14:01:34.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 Processor -- start 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 -- start start 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778076060 0x7f577819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 0x7f577819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778077c40 0x7f57781a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778113820 con 0x7f57780772a0 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57781136a0 con 0x7f5778076060 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.228+0000 7f577f400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57781139a0 con 0x7f5778077c40 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 0x7f577819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 0x7f577819e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42908/0 (socket says 172.21.15.137:42908) 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 -- 172.21.15.137:0/493982920 learned_addr learned my addr 172.21.15.137:0/493982920 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f577ca00640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778076060 0x7f577819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f577d400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778077c40 0x7f57781a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 -- 172.21.15.137:0/493982920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778077c40 msgr2=0x7f57781a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778077c40 0x7f57781a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 -- 172.21.15.137:0/493982920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778076060 msgr2=0x7f577819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778076060 0x7f577819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.229+0000 7f5777e00640 1 -- 172.21.15.137:0/493982920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57781a5aa0 con 0x7f57780772a0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.230+0000 7f5777e00640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 0x7f577819e3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f576800d950 tx=0x7f576800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.230+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768017070 con 0x7f57780772a0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.230+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5768009c30 con 0x7f57780772a0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.230+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781a5d90 con 0x7f57780772a0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.230+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800f620 con 0x7f57780772a0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.230+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781a6260 con 0x7f57780772a0 2024-03-26T14:01:34.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.231+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778110250 con 0x7f57780772a0 2024-03-26T14:01:34.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.233+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f57680109a0 con 0x7f57780772a0 2024-03-26T14:01:34.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.235+0000 7f5775600640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57600a2880 0x7f57600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.236+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f57680cea20 con 0x7f57780772a0 2024-03-26T14:01:34.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.236+0000 7f577ca00640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57600a2880 0x7f57600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.236+0000 7f577ca00640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57600a2880 0x7f57600a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f576c005fd0 tx=0x7f576c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:34.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.240+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5768009de0 con 0x7f57780772a0 2024-03-26T14:01:34.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:34 smithi137 ceph-mon[31539]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:34.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/351658510' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:01:34.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:34 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2566910327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:01:34.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.437+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5778062340 con 0x7f57780772a0 2024-03-26T14:01:34.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.438+0000 7f5775600640 1 -- 172.21.15.137:0/493982920 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f576809b6b0 con 0x7f57780772a0 2024-03-26T14:01:34.435 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57600a2880 msgr2=0x7f57600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57600a2880 0x7f57600a4d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f576c005fd0 tx=0x7f576c00b390 comp rx=0 tx=0).stop 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 msgr2=0x7f577819e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 0x7f577819e3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f576800d950 tx=0x7f576800de10 comp rx=0 tx=0).stop 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 shutdown_connections 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57600a2880 0x7f57600a4d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5778077c40 0x7f57781a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f57780772a0 0x7f577819e3c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 --2- 172.21.15.137:0/493982920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5778076060 0x7f577819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 >> 172.21.15.137:0/493982920 conn(0x7f57780fee50 msgr2=0x7f5778100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:34.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 shutdown_connections 2024-03-26T14:01:34.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.441+0000 7f577f400640 1 -- 172.21.15.137:0/493982920 wait complete. 2024-03-26T14:01:34.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:01:34.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:34 smithi179 ceph-mon[35472]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:34.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/351658510' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:01:34.573 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:34 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2566910327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:01:34.601 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:34 smithi159 ceph-mon[35501]: pgmap v434: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:34.602 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/351658510' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:01:34.602 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:34 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2566910327' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 -- 172.21.15.137:0/473289132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 msgr2=0x7f9cc4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 --2- 172.21.15.137:0/473289132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4101270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009900 tx=0x7f9cac02bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 -- 172.21.15.137:0/473289132 shutdown_connections 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 --2- 172.21.15.137:0/473289132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc41017b0 0x7f9cc4108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 --2- 172.21.15.137:0/473289132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4101270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 --2- 172.21.15.137:0/473289132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc40ffc00 0x7f9cc4100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 -- 172.21.15.137:0/473289132 >> 172.21.15.137:0/473289132 conn(0x7f9cc40fb190 msgr2=0x7f9cc40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:34.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 -- 172.21.15.137:0/473289132 shutdown_connections 2024-03-26T14:01:34.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.670+0000 7f9cc9000640 1 -- 172.21.15.137:0/473289132 wait complete. 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 Processor -- start 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 -- start start 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc40ffc00 0x7f9cc41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc41017b0 0x7f9cc41a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc410af10 con 0x7f9cc4100df0 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc410ad90 con 0x7f9cc40ffc00 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.671+0000 7f9cc9000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc410b090 con 0x7f9cc41017b0 2024-03-26T14:01:34.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42930/0 (socket says 172.21.15.137:42930) 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 -- 172.21.15.137:0/640467683 learned_addr learned my addr 172.21.15.137:0/640467683 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cc3400640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc40ffc00 0x7f9cc41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 -- 172.21.15.137:0/640467683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc41017b0 msgr2=0x7f9cc41a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cc3e00640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc41017b0 0x7f9cc41a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc41017b0 0x7f9cc41a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 -- 172.21.15.137:0/640467683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc40ffc00 msgr2=0x7f9cc41999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc40ffc00 0x7f9cc41999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 -- 172.21.15.137:0/640467683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc41a1610 con 0x7f9cc4100df0 2024-03-26T14:01:34.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.672+0000 7f9cbbe00640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4199f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cac000c00 tx=0x7f9cac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:34.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.673+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac043020 con 0x7f9cc4100df0 2024-03-26T14:01:34.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.673+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9cac004aa0 con 0x7f9cc4100df0 2024-03-26T14:01:34.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.673+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc41a18a0 con 0x7f9cc4100df0 2024-03-26T14:01:34.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.673+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac003d00 con 0x7f9cc4100df0 2024-03-26T14:01:34.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.673+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc41a1df0 con 0x7f9cc4100df0 2024-03-26T14:01:34.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.674+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc4106f20 con 0x7f9cc4100df0 2024-03-26T14:01:34.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.676+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9cac046020 con 0x7f9cc4100df0 2024-03-26T14:01:34.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.677+0000 7f9cc0c00640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ca80a2800 0x7f9ca80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:34.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.677+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9cac0f1250 con 0x7f9cc4100df0 2024-03-26T14:01:34.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.677+0000 7f9cc3400640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ca80a2800 0x7f9ca80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:34.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.678+0000 7f9cc3400640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ca80a2800 0x7f9ca80a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4005d00 tx=0x7f9cb4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:34.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.681+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9cac04d020 con 0x7f9cc4100df0 2024-03-26T14:01:34.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.881+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9cc40622a0 con 0x7f9cc4100df0 2024-03-26T14:01:34.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.882+0000 7f9cc0c00640 1 -- 172.21.15.137:0/640467683 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9cac0bdf20 con 0x7f9cc4100df0 2024-03-26T14:01:34.879 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ca80a2800 msgr2=0x7f9ca80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ca80a2800 0x7f9ca80a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4005d00 tx=0x7f9cb4005c50 comp rx=0 tx=0).stop 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 msgr2=0x7f9cc4199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4199f30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cac000c00 tx=0x7f9cac0042f0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 shutdown_connections 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ca80a2800 0x7f9ca80a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9cc41017b0 0x7f9cc41a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9cc4100df0 0x7f9cc4199f30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 --2- 172.21.15.137:0/640467683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9cc40ffc00 0x7f9cc41999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.885+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 >> 172.21.15.137:0/640467683 conn(0x7f9cc40fb190 msgr2=0x7f9cc41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.886+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 shutdown_connections 2024-03-26T14:01:34.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:34.886+0000 7f9cc9000640 1 -- 172.21.15.137:0/640467683 wait complete. 2024-03-26T14:01:34.902 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 -- 172.21.15.137:0/149524451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0740c0 msgr2=0x7fc55c074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 --2- 172.21.15.137:0/149524451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0740c0 0x7fc55c074540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc544009900 tx=0x7fc54402bc50 comp rx=0 tx=0).stop 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 -- 172.21.15.137:0/149524451 shutdown_connections 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 --2- 172.21.15.137:0/149524451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc55c109560 0x7fc55c10b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 --2- 172.21.15.137:0/149524451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c0740c0 0x7fc55c074540 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 --2- 172.21.15.137:0/149524451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 0x7fc55c073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.104+0000 7fc561600640 1 -- 172.21.15.137:0/149524451 >> 172.21.15.137:0/149524451 conn(0x7fc55c0fcc80 msgr2=0x7fc55c0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.105+0000 7fc561600640 1 -- 172.21.15.137:0/149524451 shutdown_connections 2024-03-26T14:01:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.105+0000 7fc561600640 1 -- 172.21.15.137:0/149524451 wait complete. 2024-03-26T14:01:35.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.105+0000 7fc561600640 1 Processor -- start 2024-03-26T14:01:35.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 -- start start 2024-03-26T14:01:35.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc55c0740c0 0x7fc55c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 0x7fc55c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 0x7fc55c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c10f3c0 con 0x7fc55c109560 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c10f240 con 0x7fc55c0740c0 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc561600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c10f540 con 0x7fc55c0756a0 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc55b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 0x7fc55c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc55b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 0x7fc55c1a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42942/0 (socket says 172.21.15.137:42942) 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc55b400640 1 -- 172.21.15.137:0/945447996 learned_addr learned my addr 172.21.15.137:0/945447996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc55aa00640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc55c0740c0 0x7fc55c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.106+0000 7fc553e00640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 0x7fc55c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc55b400640 1 -- 172.21.15.137:0/945447996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 msgr2=0x7fc55c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc55b400640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 0x7fc55c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc55b400640 1 -- 172.21.15.137:0/945447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc55c0740c0 msgr2=0x7fc55c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc55b400640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc55c0740c0 0x7fc55c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc55b400640 1 -- 172.21.15.137:0/945447996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c1a1700 con 0x7fc55c109560 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc553e00640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 0x7fc55c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc55b400640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 0x7fc55c1a1090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc55400eb10 tx=0x7fc55400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:35.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554004b20 con 0x7fc55c109560 2024-03-26T14:01:35.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc554004cc0 con 0x7fc55c109560 2024-03-26T14:01:35.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554010690 con 0x7fc55c109560 2024-03-26T14:01:35.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c1a19f0 con 0x7fc55c109560 2024-03-26T14:01:35.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.107+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c1a1ef0 con 0x7fc55c109560 2024-03-26T14:01:35.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.109+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c1082e0 con 0x7fc55c109560 2024-03-26T14:01:35.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.110+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc554005020 con 0x7fc55c109560 2024-03-26T14:01:35.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.112+0000 7fc553400640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5400a2880 0x7fc5400a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.112+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc5540cf410 con 0x7fc55c109560 2024-03-26T14:01:35.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.112+0000 7fc55aa00640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5400a2880 0x7fc5400a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.113+0000 7fc55aa00640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5400a2880 0x7fc5400a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005fd0 tx=0x7fc54c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:35.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.116+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc55409c0a0 con 0x7fc55c109560 2024-03-26T14:01:35.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.360+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc55c02d070 con 0x7fc55c109560 2024-03-26T14:01:35.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.362+0000 7fc553400640 1 -- 172.21.15.137:0/945447996 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fc55409b9a0 con 0x7fc55c109560 2024-03-26T14:01:35.360 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:35.360 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:50.245808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34424,34451,34496]},"id":1}]} 2024-03-26T14:01:35.360 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T14:01:35.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5400a2880 msgr2=0x7fc5400a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5400a2880 0x7fc5400a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005fd0 tx=0x7fc54c00b390 comp rx=0 tx=0).stop 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 msgr2=0x7fc55c1a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 0x7fc55c1a1090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc55400eb10 tx=0x7fc55400c2d0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 shutdown_connections 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc5400a2880 0x7fc5400a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc55c109560 0x7fc55c1a1090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc55c0756a0 0x7fc55c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 --2- 172.21.15.137:0/945447996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc55c0740c0 0x7fc55c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 >> 172.21.15.137:0/945447996 conn(0x7fc55c0fcc80 msgr2=0x7fc55c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 shutdown_connections 2024-03-26T14:01:35.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.365+0000 7fc561600640 1 -- 172.21.15.137:0/945447996 wait complete. 2024-03-26T14:01:35.381 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:01:35.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:35 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/493982920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:01:35.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:35 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/640467683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.571+0000 7f17db000640 1 -- 172.21.15.137:0/2942297148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41048a0 msgr2=0x7f17d4106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.571+0000 7f17db000640 1 --2- 172.21.15.137:0/2942297148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41048a0 0x7f17d4106c90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009900 tx=0x7f17c002bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.571+0000 7f17db000640 1 -- 172.21.15.137:0/2942297148 shutdown_connections 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.572+0000 7f17db000640 1 --2- 172.21.15.137:0/2942297148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41071d0 0x7f17d4109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.572+0000 7f17db000640 1 --2- 172.21.15.137:0/2942297148 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41048a0 0x7f17d4106c90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.572+0000 7f17db000640 1 --2- 172.21.15.137:0/2942297148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d4101f70 0x7f17d4104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.572+0000 7f17db000640 1 -- 172.21.15.137:0/2942297148 >> 172.21.15.137:0/2942297148 conn(0x7f17d40fb980 msgr2=0x7f17d40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.572+0000 7f17db000640 1 -- 172.21.15.137:0/2942297148 shutdown_connections 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.572+0000 7f17db000640 1 -- 172.21.15.137:0/2942297148 wait complete. 2024-03-26T14:01:35.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 Processor -- start 2024-03-26T14:01:35.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 -- start start 2024-03-26T14:01:35.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d4101f70 0x7f17d4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 0x7f17d4197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41071d0 0x7f17d419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d410d080 con 0x7f17d41071d0 2024-03-26T14:01:35.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d410cf00 con 0x7f17d41048a0 2024-03-26T14:01:35.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.573+0000 7f17db000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d410d200 con 0x7f17d4101f70 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 0x7f17d4197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17da600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41071d0 0x7f17d419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 0x7f17d4197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55720/0 (socket says 172.21.15.137:55720) 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 -- 172.21.15.137:0/2564430147 learned_addr learned my addr 172.21.15.137:0/2564430147 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9c00640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d4101f70 0x7f17d4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 -- 172.21.15.137:0/2564430147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d4101f70 msgr2=0x7f17d4197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d4101f70 0x7f17d4197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 -- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41071d0 msgr2=0x7f17d419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41071d0 0x7f17d419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.574+0000 7f17d9200640 1 -- 172.21.15.137:0/2564430147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d419f3b0 con 0x7f17d41048a0 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.575+0000 7f17d9200640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 0x7f17d4197cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005ab0 tx=0x7f17c00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.575+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0043020 con 0x7f17d41048a0 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.575+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d419f640 con 0x7f17d41048a0 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.575+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f17c00054c0 con 0x7f17d41048a0 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.575+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0003d10 con 0x7f17d41048a0 2024-03-26T14:01:35.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.575+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d419fbe0 con 0x7f17d41048a0 2024-03-26T14:01:35.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.577+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d40fd470 con 0x7f17d41048a0 2024-03-26T14:01:35.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.579+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f17c0046020 con 0x7f17d41048a0 2024-03-26T14:01:35.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.580+0000 7f17d2000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17bc0a2800 0x7f17bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:35.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.580+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f17c00f1300 con 0x7f17d41048a0 2024-03-26T14:01:35.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.580+0000 7f17d9c00640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17bc0a2800 0x7f17bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:35.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.581+0000 7f17d9c00640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17bc0a2800 0x7f17bc0a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005fd0 tx=0x7f17c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:35.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.598+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17c004d020 con 0x7f17d41048a0 2024-03-26T14:01:35.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:35 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/493982920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:01:35.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:35 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/640467683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:01:35.699 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:35 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/493982920' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:01:35.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:35 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/640467683' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:01:35.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.781+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17d40622a0 con 0x7f17d41048a0 2024-03-26T14:01:35.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.786+0000 7f17d2000640 1 -- 172.21.15.137:0/2564430147 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f17c00be0b0 con 0x7f17d41048a0 2024-03-26T14:01:35.784 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:35.784 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.788+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17bc0a2800 msgr2=0x7f17bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.788+0000 7f17db000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17bc0a2800 0x7f17bc0a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f17c8005fd0 tx=0x7f17c800b390 comp rx=0 tx=0).stop 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.788+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 msgr2=0x7f17d4197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.788+0000 7f17db000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 0x7f17d4197cd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005ab0 tx=0x7f17c00042f0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 shutdown_connections 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17bc0a2800 0x7f17bc0a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d41048a0 0x7f17d4197cd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17d41071d0 0x7f17d419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 --2- 172.21.15.137:0/2564430147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17d4101f70 0x7f17d4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 >> 172.21.15.137:0/2564430147 conn(0x7f17d40fb980 msgr2=0x7f17d41028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 shutdown_connections 2024-03-26T14:01:35.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:35.789+0000 7f17db000640 1 -- 172.21.15.137:0/2564430147 wait complete. 2024-03-26T14:01:35.803 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:01:35.804 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 -- 172.21.15.137:0/3459743301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 msgr2=0x7f703c104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 --2- 172.21.15.137:0/3459743301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c104520 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7028009900 tx=0x7f702802bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 -- 172.21.15.137:0/3459743301 shutdown_connections 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 --2- 172.21.15.137:0/3459743301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f703c107390 0x7f703c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 --2- 172.21.15.137:0/3459743301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f703c104a60 0x7f703c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 --2- 172.21.15.137:0/3459743301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c104520 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.009+0000 7f7042800640 1 -- 172.21.15.137:0/3459743301 >> 172.21.15.137:0/3459743301 conn(0x7f703c0fbb40 msgr2=0x7f703c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.010+0000 7f7042800640 1 -- 172.21.15.137:0/3459743301 shutdown_connections 2024-03-26T14:01:36.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.010+0000 7f7042800640 1 -- 172.21.15.137:0/3459743301 wait complete. 2024-03-26T14:01:36.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.010+0000 7f7042800640 1 Processor -- start 2024-03-26T14:01:36.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 -- start start 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f703c104a60 0x7f703c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f703c107390 0x7f703c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c10d230 con 0x7f703c102130 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c10d0b0 con 0x7f703c104a60 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f7042800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c10d3b0 con 0x7f703c107390 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42972/0 (socket says 172.21.15.137:42972) 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 -- 172.21.15.137:0/1378236835 learned_addr learned my addr 172.21.15.137:0/1378236835 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703aa00640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f703c104a60 0x7f703c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703be00640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f703c107390 0x7f703c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 -- 172.21.15.137:0/1378236835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f703c107390 msgr2=0x7f703c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f703c107390 0x7f703c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 -- 172.21.15.137:0/1378236835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f703c104a60 msgr2=0x7f703c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f703c104a60 0x7f703c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.011+0000 7f703b400640 1 -- 172.21.15.137:0/1378236835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c1a1750 con 0x7f703c102130 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.012+0000 7f703b400640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c199b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f70280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.012+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028043020 con 0x7f703c102130 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.012+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70280054c0 con 0x7f703c102130 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.012+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7028003d70 con 0x7f703c102130 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.012+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c1a19e0 con 0x7f703c102130 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.012+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c1a1ee0 con 0x7f703c102130 2024-03-26T14:01:36.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.014+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c108ee0 con 0x7f703c102130 2024-03-26T14:01:36.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.016+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7028046020 con 0x7f703c102130 2024-03-26T14:01:36.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.017+0000 7f7033e00640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70200a2800 0x7f70200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.017+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f70280f17a0 con 0x7f703c102130 2024-03-26T14:01:36.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.018+0000 7f703aa00640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70200a2800 0x7f70200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.018+0000 7f703aa00640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70200a2800 0x7f70200a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f702c005fd0 tx=0x7f702c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:36.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.019+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f702804d020 con 0x7f703c102130 2024-03-26T14:01:36.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.216+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f703c02d2c0 con 0x7f703c102130 2024-03-26T14:01:36.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.216+0000 7f7033e00640 1 -- 172.21.15.137:0/1378236835 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70280be550 con 0x7f703c102130 2024-03-26T14:01:36.213 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70200a2800 msgr2=0x7f70200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70200a2800 0x7f70200a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f702c005fd0 tx=0x7f702c005850 comp rx=0 tx=0).stop 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 msgr2=0x7f703c199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c199b30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c00 tx=0x7f70280042f0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 shutdown_connections 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70200a2800 0x7f70200a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f703c104a60 0x7f703c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f703c107390 0x7f703c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 --2- 172.21.15.137:0/1378236835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f703c102130 0x7f703c199b30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.219+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 >> 172.21.15.137:0/1378236835 conn(0x7f703c0fbb40 msgr2=0x7f703c1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.220+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 shutdown_connections 2024-03-26T14:01:36.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.220+0000 7f7042800640 1 -- 172.21.15.137:0/1378236835 wait complete. 2024-03-26T14:01:36.236 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:01:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 -- 172.21.15.137:0/2628596645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41028a0 msgr2=0x7f00b4102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 --2- 172.21.15.137:0/2628596645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41028a0 0x7f00b4102d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f009c009900 tx=0x7f009c02bc40 comp rx=0 tx=0).stop 2024-03-26T14:01:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 -- 172.21.15.137:0/2628596645 shutdown_connections 2024-03-26T14:01:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 --2- 172.21.15.137:0/2628596645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b4103260 0x7f00b4109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 --2- 172.21.15.137:0/2628596645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41028a0 0x7f00b4102d20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 --2- 172.21.15.137:0/2628596645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00b41016b0 0x7f00b4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 -- 172.21.15.137:0/2628596645 >> 172.21.15.137:0/2628596645 conn(0x7f00b40fcc40 msgr2=0x7f00b40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:36.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 -- 172.21.15.137:0/2628596645 shutdown_connections 2024-03-26T14:01:36.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.455+0000 7f00bae00640 1 -- 172.21.15.137:0/2628596645 wait complete. 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.456+0000 7f00bae00640 1 Processor -- start 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.456+0000 7f00bae00640 1 -- start start 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00bae00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 0x7f00b4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00bae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00b41028a0 0x7f00b419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00bae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b4103260 0x7f00b41a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00bae00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b410d230 con 0x7f00b41016b0 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00bae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b410d0b0 con 0x7f00b4103260 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00bae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b410d3b0 con 0x7f00b41028a0 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 0x7f00b4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 0x7f00b4199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42994/0 (socket says 172.21.15.137:42994) 2024-03-26T14:01:36.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 -- 172.21.15.137:0/761922335 learned_addr learned my addr 172.21.15.137:0/761922335 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b8e00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b4103260 0x7f00b41a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3400640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00b41028a0 0x7f00b419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 -- 172.21.15.137:0/761922335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00b41028a0 msgr2=0x7f00b419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00b41028a0 0x7f00b419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 -- 172.21.15.137:0/761922335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b4103260 msgr2=0x7f00b41a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b4103260 0x7f00b41a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.457+0000 7f00b3e00640 1 -- 172.21.15.137:0/761922335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b41a1760 con 0x7f00b41016b0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.458+0000 7f00b3e00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 0x7f00b4199b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00a400eaa0 tx=0x7f00a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.458+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4016070 con 0x7f00b41016b0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.458+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00a4003e20 con 0x7f00b41016b0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.458+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b41a1a50 con 0x7f00b41016b0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.458+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a40116d0 con 0x7f00b41016b0 2024-03-26T14:01:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.458+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b41a1f70 con 0x7f00b41016b0 2024-03-26T14:01:36.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.459+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b40688e0 con 0x7f00b41016b0 2024-03-26T14:01:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.461+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f00a4019070 con 0x7f00b41016b0 2024-03-26T14:01:36.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.463+0000 7f00b0c00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00980a2880 0x7f00980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.464+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f00a40cef80 con 0x7f00b41016b0 2024-03-26T14:01:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.464+0000 7f00b3400640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00980a2880 0x7f00980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.464+0000 7f00b3400640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00980a2880 0x7f00980a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f009c006010 tx=0x7f009c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:36.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.468+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00a409bc10 con 0x7f00b41016b0 2024-03-26T14:01:36.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:36 smithi137 ceph-mon[31539]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:36.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:36 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/945447996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:36.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:36 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2564430147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:36.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:36 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1378236835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:01:36.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.663+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00b41a29b0 con 0x7f00b41016b0 2024-03-26T14:01:36.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.664+0000 7f00b0c00640 1 -- 172.21.15.137:0/761922335 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00a409b510 con 0x7f00b41016b0 2024-03-26T14:01:36.662 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:01:36.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00980a2880 msgr2=0x7f00980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00980a2880 0x7f00980a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f009c006010 tx=0x7f009c000bc0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 msgr2=0x7f00b4199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 0x7f00b4199b40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00a400eaa0 tx=0x7f00a400ef60 comp rx=0 tx=0).stop 2024-03-26T14:01:36.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 shutdown_connections 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00980a2880 0x7f00980a4d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b4103260 0x7f00b41a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00b41028a0 0x7f00b419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 --2- 172.21.15.137:0/761922335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00b41016b0 0x7f00b4199b40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 >> 172.21.15.137:0/761922335 conn(0x7f00b40fcc40 msgr2=0x7f00b40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 shutdown_connections 2024-03-26T14:01:36.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.667+0000 7f00bae00640 1 -- 172.21.15.137:0/761922335 wait complete. 2024-03-26T14:01:36.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:36 smithi179 ceph-mon[35472]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:36.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:36 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/945447996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:36.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:36 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2564430147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:36.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:36 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1378236835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:01:36.683 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:01:36.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:36 smithi159 ceph-mon[35501]: pgmap v435: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:36.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:36 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/945447996' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:36.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:36 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2564430147' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:36.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:36 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1378236835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.899+0000 7f2a29600640 1 -- 172.21.15.137:0/2444580124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 msgr2=0x7f2a24107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.899+0000 7f2a29600640 1 --2- 172.21.15.137:0/2444580124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a24107fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009900 tx=0x7f2a1402bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.899+0000 7f2a29600640 1 -- 172.21.15.137:0/2444580124 shutdown_connections 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.899+0000 7f2a29600640 1 --2- 172.21.15.137:0/2444580124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a24109770 0x7f2a24110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.899+0000 7f2a29600640 1 --2- 172.21.15.137:0/2444580124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a24108db0 0x7f2a24109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.900+0000 7f2a29600640 1 --2- 172.21.15.137:0/2444580124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a24107fe0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.900+0000 7f2a29600640 1 -- 172.21.15.137:0/2444580124 >> 172.21.15.137:0/2444580124 conn(0x7f2a24076b70 msgr2=0x7f2a24078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.900+0000 7f2a29600640 1 -- 172.21.15.137:0/2444580124 shutdown_connections 2024-03-26T14:01:36.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.900+0000 7f2a29600640 1 -- 172.21.15.137:0/2444580124 wait complete. 2024-03-26T14:01:36.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 Processor -- start 2024-03-26T14:01:36.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 -- start start 2024-03-26T14:01:36.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a2419de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a24108db0 0x7f2a2419e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a24109770 0x7f2a241a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a241136e0 con 0x7f2a24107bc0 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24113560 con 0x7f2a24109770 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a29600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a24113860 con 0x7f2a24108db0 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a23e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a2419de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a23e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a2419de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43014/0 (socket says 172.21.15.137:43014) 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.901+0000 7f2a23e00640 1 -- 172.21.15.137:0/3083224872 learned_addr learned my addr 172.21.15.137:0/3083224872 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a28c00640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a24109770 0x7f2a241a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a1b400640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a24108db0 0x7f2a2419e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a23e00640 1 -- 172.21.15.137:0/3083224872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a24108db0 msgr2=0x7f2a2419e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a23e00640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a24108db0 0x7f2a2419e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a23e00640 1 -- 172.21.15.137:0/3083224872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a24109770 msgr2=0x7f2a241a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:36.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a23e00640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a24109770 0x7f2a241a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a23e00640 1 -- 172.21.15.137:0/3083224872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a241a5a20 con 0x7f2a24107bc0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a23e00640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a2419de00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000c00 tx=0x7f2a14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14043020 con 0x7f2a24107bc0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a241a5cb0 con 0x7f2a24107bc0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a14004aa0 con 0x7f2a24107bc0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.902+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14003d00 con 0x7f2a24107bc0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.903+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a241a61b0 con 0x7f2a24107bc0 2024-03-26T14:01:36.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.904+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2410efb0 con 0x7f2a24107bc0 2024-03-26T14:01:36.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.906+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2a14046020 con 0x7f2a24107bc0 2024-03-26T14:01:36.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.907+0000 7f2a21600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a080a2800 0x7f2a080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:36.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.907+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2a140f1540 con 0x7f2a24107bc0 2024-03-26T14:01:36.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.907+0000 7f2a1b400640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a080a2800 0x7f2a080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:36.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.908+0000 7f2a1b400640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a080a2800 0x7f2a080a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005fd0 tx=0x7f2a0c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:36.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:36.911+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2a1404d020 con 0x7f2a24107bc0 2024-03-26T14:01:37.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.112+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a240622a0 con 0x7f2a24107bc0 2024-03-26T14:01:37.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.113+0000 7f2a21600640 1 -- 172.21.15.137:0/3083224872 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a140be1d0 con 0x7f2a24107bc0 2024-03-26T14:01:37.110 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:01:37.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a080a2800 msgr2=0x7f2a080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a080a2800 0x7f2a080a4cc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005fd0 tx=0x7f2a0c005850 comp rx=0 tx=0).stop 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 msgr2=0x7f2a2419de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a2419de00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000c00 tx=0x7f2a14004270 comp rx=0 tx=0).stop 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 shutdown_connections 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a080a2800 0x7f2a080a4cc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a24109770 0x7f2a241a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a24108db0 0x7f2a2419e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 --2- 172.21.15.137:0/3083224872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a24107bc0 0x7f2a2419de00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 >> 172.21.15.137:0/3083224872 conn(0x7f2a24076b70 msgr2=0x7f2a2410c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.116+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 shutdown_connections 2024-03-26T14:01:37.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.117+0000 7f2a29600640 1 -- 172.21.15.137:0/3083224872 wait complete. 2024-03-26T14:01:37.131 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.352+0000 7f55db400640 1 -- 172.21.15.137:0/3301355730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 msgr2=0x7f55d40784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.352+0000 7f55db400640 1 --2- 172.21.15.137:0/3301355730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d40784e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55cc009ca0 tx=0x7f55cc02c100 comp rx=0 tx=0).stop 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 -- 172.21.15.137:0/3301355730 shutdown_connections 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 --2- 172.21.15.137:0/3301355730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 0x7f55d4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 --2- 172.21.15.137:0/3301355730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d40784e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 --2- 172.21.15.137:0/3301355730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d4076060 0x7f55d4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 -- 172.21.15.137:0/3301355730 >> 172.21.15.137:0/3301355730 conn(0x7f55d40fee50 msgr2=0x7f55d4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 -- 172.21.15.137:0/3301355730 shutdown_connections 2024-03-26T14:01:37.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.353+0000 7f55db400640 1 -- 172.21.15.137:0/3301355730 wait complete. 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 Processor -- start 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 -- start start 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d4076060 0x7f55d419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 0x7f55d419e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d41a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d4113820 con 0x7f55d4077c40 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41136a0 con 0x7f55d40772a0 2024-03-26T14:01:37.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55db400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41139a0 con 0x7f55d4076060 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.354+0000 7f55d9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d41a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d41a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43018/0 (socket says 172.21.15.137:43018) 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 -- 172.21.15.137:0/3201932273 learned_addr learned my addr 172.21.15.137:0/3201932273 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d3e00640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 0x7f55d419e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d8a00640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d4076060 0x7f55d419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 -- 172.21.15.137:0/3201932273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d4076060 msgr2=0x7f55d419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d4076060 0x7f55d419de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 -- 172.21.15.137:0/3201932273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 msgr2=0x7f55d419e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 0x7f55d419e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 -- 172.21.15.137:0/3201932273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d41a5aa0 con 0x7f55d4077c40 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d3e00640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 0x7f55d419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:37.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d9400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d41a5430 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f55cc032520 tx=0x7f55cc034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:37.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc039040 con 0x7f55d4077c40 2024-03-26T14:01:37.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.355+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55cc044070 con 0x7f55d4077c40 2024-03-26T14:01:37.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.356+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0043c0 con 0x7f55d4077c40 2024-03-26T14:01:37.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.356+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d41a5d30 con 0x7f55d4077c40 2024-03-26T14:01:37.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.356+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d41a61d0 con 0x7f55d4077c40 2024-03-26T14:01:37.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.357+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d4110250 con 0x7f55d4077c40 2024-03-26T14:01:37.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.359+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f55cc004560 con 0x7f55d4077c40 2024-03-26T14:01:37.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.360+0000 7f55d1600640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55bc0a2800 0x7f55bc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.360+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f55cc0f15f0 con 0x7f55d4077c40 2024-03-26T14:01:37.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.360+0000 7f55d8a00640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55bc0a2800 0x7f55bc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.361+0000 7f55d8a00640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55bc0a2800 0x7f55bc0a4cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005fd0 tx=0x7f55c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:37.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.364+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f55cc0333a0 con 0x7f55d4077c40 2024-03-26T14:01:37.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:37 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/761922335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:01:37.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:37 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3083224872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:01:37.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:37 smithi137 ceph-mon[31539]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:37.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.561+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f55d40622a0 con 0x7f55d4077c40 2024-03-26T14:01:37.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.562+0000 7f55d1600640 1 -- 172.21.15.137:0/3201932273 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f55cc0be280 con 0x7f55d4077c40 2024-03-26T14:01:37.559 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:01:37.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.564+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55bc0a2800 msgr2=0x7f55bc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.564+0000 7f55db400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55bc0a2800 0x7f55bc0a4cc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f55c8005fd0 tx=0x7f55c8005850 comp rx=0 tx=0).stop 2024-03-26T14:01:37.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.564+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 msgr2=0x7f55d41a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.564+0000 7f55db400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d41a5430 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f55cc032520 tx=0x7f55cc034880 comp rx=0 tx=0).stop 2024-03-26T14:01:37.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 shutdown_connections 2024-03-26T14:01:37.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55bc0a2800 0x7f55bc0a4cc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55d40772a0 0x7f55d419e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55d4077c40 0x7f55d41a5430 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 --2- 172.21.15.137:0/3201932273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55d4076060 0x7f55d419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 >> 172.21.15.137:0/3201932273 conn(0x7f55d40fee50 msgr2=0x7f55d4100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:37.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 shutdown_connections 2024-03-26T14:01:37.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.565+0000 7f55db400640 1 -- 172.21.15.137:0/3201932273 wait complete. 2024-03-26T14:01:37.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:01:37.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:37 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/761922335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:01:37.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:37 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3083224872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:01:37.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:37 smithi179 ceph-mon[35472]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:37.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:37 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/761922335' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:01:37.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:37 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3083224872' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:01:37.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:37 smithi159 ceph-mon[35501]: pgmap v436: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 -- 172.21.15.137:0/1011187725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e4076060 msgr2=0x7fa4e4076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 --2- 172.21.15.137:0/1011187725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e4076060 0x7fa4e4076480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009900 tx=0x7fa4d402bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 -- 172.21.15.137:0/1011187725 shutdown_connections 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 --2- 172.21.15.137:0/1011187725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4e40772a0 0x7fa4e4077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 --2- 172.21.15.137:0/1011187725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4e4077c40 0x7fa4e40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 --2- 172.21.15.137:0/1011187725 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e4076060 0x7fa4e4076480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 -- 172.21.15.137:0/1011187725 >> 172.21.15.137:0/1011187725 conn(0x7fa4e40fee50 msgr2=0x7fa4e4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:37.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 -- 172.21.15.137:0/1011187725 shutdown_connections 2024-03-26T14:01:37.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.792+0000 7fa4eb000640 1 -- 172.21.15.137:0/1011187725 wait complete. 2024-03-26T14:01:37.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 Processor -- start 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 -- start start 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4e4076060 0x7fa4e419dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 0x7fa4e419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4e4077c40 0x7fa4e41a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41138a0 con 0x7fa4e40772a0 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4113720 con 0x7fa4e4076060 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4eb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4113a20 con 0x7fa4e4077c40 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.793+0000 7fa4e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 0x7fa4e419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 0x7fa4e419e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43030/0 (socket says 172.21.15.137:43030) 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 -- 172.21.15.137:0/2608453311 learned_addr learned my addr 172.21.15.137:0/2608453311 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3e00640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4e4076060 0x7fa4e419dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 -- 172.21.15.137:0/2608453311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4e4077c40 msgr2=0x7fa4e41a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4e4077c40 0x7fa4e41a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 -- 172.21.15.137:0/2608453311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4e4076060 msgr2=0x7fa4e419dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4e4076060 0x7fa4e419dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 -- 172.21.15.137:0/2608453311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e41a5bc0 con 0x7fa4e40772a0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e3400640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 0x7fa4e419e4e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e9f0 tx=0x7fa4d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000cd80 con 0x7fa4e40772a0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.794+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4d00108e0 con 0x7fa4e40772a0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.795+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e41a5eb0 con 0x7fa4e40772a0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.795+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0018920 con 0x7fa4e40772a0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.795+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e41a63f0 con 0x7fa4e40772a0 2024-03-26T14:01:37.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.796+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e4110420 con 0x7fa4e40772a0 2024-03-26T14:01:37.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.799+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa4d0010a90 con 0x7fa4e40772a0 2024-03-26T14:01:37.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.801+0000 7fa4e0c00640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4c80a2800 0x7fa4c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.801+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa4d00cf280 con 0x7fa4e40772a0 2024-03-26T14:01:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.801+0000 7fa4e3e00640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4c80a2800 0x7fa4c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:37.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.802+0000 7fa4e3e00640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4c80a2800 0x7fa4c80a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4006010 tx=0x7fa4d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:37.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:37.805+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa4d009bf50 con 0x7fa4e40772a0 2024-03-26T14:01:38.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.042+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4e4062340 con 0x7fa4e40772a0 2024-03-26T14:01:38.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.044+0000 7fa4e0c00640 1 -- 172.21.15.137:0/2608453311 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fa4d009b850 con 0x7fa4e40772a0 2024-03-26T14:01:38.041 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:38.041 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:50.245808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34424,34451,34496]},"id":1}]} 2024-03-26T14:01:38.042 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4c80a2800 msgr2=0x7fa4c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4c80a2800 0x7fa4c80a4cc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4006010 tx=0x7fa4d4000bc0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 msgr2=0x7fa4e419e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 0x7fa4e419e4e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000e9f0 tx=0x7fa4d000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 shutdown_connections 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4c80a2800 0x7fa4c80a4cc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4e4077c40 0x7fa4e41a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4e40772a0 0x7fa4e419e4e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 --2- 172.21.15.137:0/2608453311 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4e4076060 0x7fa4e419dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 >> 172.21.15.137:0/2608453311 conn(0x7fa4e40fee50 msgr2=0x7fa4e41005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:38.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 shutdown_connections 2024-03-26T14:01:38.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.047+0000 7fa4eb000640 1 -- 172.21.15.137:0/2608453311 wait complete. 2024-03-26T14:01:38.065 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:01:38.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 -- 172.21.15.137:0/3674533484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0074080 msgr2=0x7faef00744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 --2- 172.21.15.137:0/3674533484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0074080 0x7faef00744e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faedc009900 tx=0x7faedc02bcd0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 -- 172.21.15.137:0/3674533484 shutdown_connections 2024-03-26T14:01:38.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 --2- 172.21.15.137:0/3674533484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 0x7faef010b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 --2- 172.21.15.137:0/3674533484 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0074080 0x7faef00744e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 --2- 172.21.15.137:0/3674533484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef0075660 0x7faef0073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 -- 172.21.15.137:0/3674533484 >> 172.21.15.137:0/3674533484 conn(0x7faef00fca50 msgr2=0x7faef00fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:38.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.280+0000 7faef7000640 1 -- 172.21.15.137:0/3674533484 shutdown_connections 2024-03-26T14:01:38.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.281+0000 7faef7000640 1 -- 172.21.15.137:0/3674533484 wait complete. 2024-03-26T14:01:38.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.282+0000 7faef7000640 1 Processor -- start 2024-03-26T14:01:38.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.282+0000 7faef7000640 1 -- start start 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.282+0000 7faef7000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef0074080 0x7faef0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef7000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 0x7faef0199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 0x7faef01a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef7000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010f260 con 0x7faef0075660 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef7000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010f0e0 con 0x7faef0109390 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef7000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef010f3e0 con 0x7faef0074080 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 0x7faef0199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 0x7faef0199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60556/0 (socket says 172.21.15.137:60556) 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef5200640 1 -- 172.21.15.137:0/154268111 learned_addr learned my addr 172.21.15.137:0/154268111 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef6600640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 0x7faef01a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef5c00640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef0074080 0x7faef0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.283+0000 7faef5200640 1 -- 172.21.15.137:0/154268111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef0074080 msgr2=0x7faef0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.284+0000 7faef5200640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef0074080 0x7faef0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.284+0000 7faef5200640 1 -- 172.21.15.137:0/154268111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 msgr2=0x7faef01a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.284+0000 7faef5200640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 0x7faef01a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.284+0000 7faef5200640 1 -- 172.21.15.137:0/154268111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef01a1570 con 0x7faef0075660 2024-03-26T14:01:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.284+0000 7faef6600640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 0x7faef01a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:38.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.284+0000 7faef5200640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 0x7faef0199e90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faedc000c00 tx=0x7faedc02d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:38.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.285+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc043020 con 0x7faef0075660 2024-03-26T14:01:38.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.285+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faedc02dd90 con 0x7faef0075660 2024-03-26T14:01:38.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.285+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc003c20 con 0x7faef0075660 2024-03-26T14:01:38.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.285+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef01a1800 con 0x7faef0075660 2024-03-26T14:01:38.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.285+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef01a1da0 con 0x7faef0075660 2024-03-26T14:01:38.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.286+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef0108150 con 0x7faef0075660 2024-03-26T14:01:38.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.288+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7faedc046020 con 0x7faef0075660 2024-03-26T14:01:38.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.289+0000 7faeee000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faed80a2880 0x7faed80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.290+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faedc0f10e0 con 0x7faef0075660 2024-03-26T14:01:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.290+0000 7faef5c00640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faed80a2880 0x7faed80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.291+0000 7faef5c00640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faed80a2880 0x7faed80a4d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faee4005fd0 tx=0x7faee4009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:38.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.294+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faedc04d020 con 0x7faef0075660 2024-03-26T14:01:38.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.495+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faef00622a0 con 0x7faef0075660 2024-03-26T14:01:38.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.500+0000 7faeee000640 1 -- 172.21.15.137:0/154268111 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7faedc0dd030 con 0x7faef0075660 2024-03-26T14:01:38.497 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:38.498 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:01:38.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faed80a2880 msgr2=0x7faed80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faed80a2880 0x7faed80a4d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faee4005fd0 tx=0x7faee4009870 comp rx=0 tx=0).stop 2024-03-26T14:01:38.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 msgr2=0x7faef0199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 0x7faef0199e90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7faedc000c00 tx=0x7faedc02d990 comp rx=0 tx=0).stop 2024-03-26T14:01:38.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 shutdown_connections 2024-03-26T14:01:38.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faed80a2880 0x7faed80a4d40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faef0109390 0x7faef01a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faef0075660 0x7faef0199e90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 --2- 172.21.15.137:0/154268111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faef0074080 0x7faef0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 >> 172.21.15.137:0/154268111 conn(0x7faef00fca50 msgr2=0x7faef0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:38.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 shutdown_connections 2024-03-26T14:01:38.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.503+0000 7faef7000640 1 -- 172.21.15.137:0/154268111 wait complete. 2024-03-26T14:01:38.516 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:38 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3201932273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:01:38.516 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:38 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2608453311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:38.519 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:01:38.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:38 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3201932273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:01:38.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:38 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2608453311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:38.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:38 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3201932273' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:01:38.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:38 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2608453311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:38.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 -- 172.21.15.137:0/133638512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac0772a0 msgr2=0x7f3cac077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 --2- 172.21.15.137:0/133638512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac0772a0 0x7f3cac077700 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009ca0 tx=0x7f3c9402c100 comp rx=0 tx=0).stop 2024-03-26T14:01:38.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 -- 172.21.15.137:0/133638512 shutdown_connections 2024-03-26T14:01:38.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 --2- 172.21.15.137:0/133638512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac077c40 0x7f3cac0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 --2- 172.21.15.137:0/133638512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac0772a0 0x7f3cac077700 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 --2- 172.21.15.137:0/133638512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cac076060 0x7f3cac076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 -- 172.21.15.137:0/133638512 >> 172.21.15.137:0/133638512 conn(0x7f3cac0fee50 msgr2=0x7f3cac101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:38.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 -- 172.21.15.137:0/133638512 shutdown_connections 2024-03-26T14:01:38.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.730+0000 7f3cb1600640 1 -- 172.21.15.137:0/133638512 wait complete. 2024-03-26T14:01:38.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.731+0000 7f3cb1600640 1 Processor -- start 2024-03-26T14:01:38.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.731+0000 7f3cb1600640 1 -- start start 2024-03-26T14:01:38.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cb1600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cac076060 0x7f3cac19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cb1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac0772a0 0x7f3cac19e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cb1600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 0x7f3cac1a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cb1600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac113820 con 0x7f3cac077c40 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cb1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1136a0 con 0x7f3cac0772a0 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cb1600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac1139a0 con 0x7f3cac076060 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 0x7f3cac1a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 0x7f3cac1a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60564/0 (socket says 172.21.15.137:60564) 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 -- 172.21.15.137:0/1403401183 learned_addr learned my addr 172.21.15.137:0/1403401183 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3ca2000640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac0772a0 0x7f3cac19e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3caaa00640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cac076060 0x7f3cac19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 -- 172.21.15.137:0/1403401183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cac076060 msgr2=0x7f3cac19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cac076060 0x7f3cac19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 -- 172.21.15.137:0/1403401183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac0772a0 msgr2=0x7f3cac19e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac0772a0 0x7f3cac19e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3cab400640 1 -- 172.21.15.137:0/1403401183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac1a5b30 con 0x7f3cac077c40 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.732+0000 7f3ca2000640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac0772a0 0x7f3cac19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:38.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.733+0000 7f3cab400640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 0x7f3cac1a54c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400e970 tx=0x7f3ca400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.733+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400ccc0 con 0x7f3cac077c40 2024-03-26T14:01:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.733+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ca400ce60 con 0x7f3cac077c40 2024-03-26T14:01:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.733+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac1a5e20 con 0x7f3cac077c40 2024-03-26T14:01:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.733+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4010430 con 0x7f3cac077c40 2024-03-26T14:01:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.733+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1a62e0 con 0x7f3cac077c40 2024-03-26T14:01:38.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.734+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac110490 con 0x7f3cac077c40 2024-03-26T14:01:38.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.736+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3ca4004ff0 con 0x7f3cac077c40 2024-03-26T14:01:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.737+0000 7f3ca3e00640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c900a2880 0x7f3c900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.737+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3ca40cf310 con 0x7f3cac077c40 2024-03-26T14:01:38.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.738+0000 7f3caaa00640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c900a2880 0x7f3c900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:38.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.738+0000 7f3caaa00640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c900a2880 0x7f3c900a4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005fd0 tx=0x7f3c9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:38.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.742+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3ca409bfe0 con 0x7f3cac077c40 2024-03-26T14:01:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.982+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cac02d070 con 0x7f3cac077c40 2024-03-26T14:01:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.983+0000 7f3ca3e00640 1 -- 172.21.15.137:0/1403401183 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f3ca409b8e0 con 0x7f3cac077c40 2024-03-26T14:01:38.981 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:01:38.981 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:50.245808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34424,34451,34496]},"id":1}]} 2024-03-26T14:01:38.981 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T14:01:38.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c900a2880 msgr2=0x7f3c900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c900a2880 0x7f3c900a4d40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005fd0 tx=0x7f3c9c005850 comp rx=0 tx=0).stop 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 msgr2=0x7f3cac1a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 0x7f3cac1a54c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400e970 tx=0x7f3ca400ed70 comp rx=0 tx=0).stop 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 shutdown_connections 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c900a2880 0x7f3c900a4d40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cac0772a0 0x7f3cac19e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cac077c40 0x7f3cac1a54c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 --2- 172.21.15.137:0/1403401183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cac076060 0x7f3cac19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 >> 172.21.15.137:0/1403401183 conn(0x7f3cac0fee50 msgr2=0x7f3cac1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 shutdown_connections 2024-03-26T14:01:38.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:38.986+0000 7f3cb1600640 1 -- 172.21.15.137:0/1403401183 wait complete. 2024-03-26T14:01:39.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T13:59:22.728231+0000', 'modified': '2024-03-26T13:59:50.245808+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34496, 'mds_1': 34451, 'mds_2': 34424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6837/7146430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 7146430}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 7146430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6839/2441458416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2441458416}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2441458416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34394': {'gid': 34394, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/1469472888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 1469472888}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 1469472888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34424': {'gid': 34424, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.159:6835/3125472496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 3125472496}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3125472496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34451': {'gid': 34451, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.159:6837/16915360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 16915360}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 16915360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34496': {'gid': 34496, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/2964052982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2964052982}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2964052982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34496, 'qdb_cluster': [34424, 34451, 34496]} 2024-03-26T14:01:39.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-26T14:01:39.004 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 -- 172.21.15.137:0/1620468239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2640772a0 msgr2=0x7fe264077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 --2- 172.21.15.137:0/1620468239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2640772a0 0x7fe264077700 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe250009900 tx=0x7fe25002bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 -- 172.21.15.137:0/1620468239 shutdown_connections 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 --2- 172.21.15.137:0/1620468239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe264077c40 0x7fe2640784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 --2- 172.21.15.137:0/1620468239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2640772a0 0x7fe264077700 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 --2- 172.21.15.137:0/1620468239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe264076060 0x7fe264076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 -- 172.21.15.137:0/1620468239 >> 172.21.15.137:0/1620468239 conn(0x7fe2640fee50 msgr2=0x7fe264101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 -- 172.21.15.137:0/1620468239 shutdown_connections 2024-03-26T14:01:39.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.215+0000 7fe269200640 1 -- 172.21.15.137:0/1620468239 wait complete. 2024-03-26T14:01:39.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 Processor -- start 2024-03-26T14:01:39.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 -- start start 2024-03-26T14:01:39.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 0x7fe26419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2640772a0 0x7fe26419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe264077c40 0x7fe2641a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe264113820 con 0x7fe264076060 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2641136a0 con 0x7fe2640772a0 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.216+0000 7fe269200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2641139a0 con 0x7fe264077c40 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 0x7fe26419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 0x7fe26419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60582/0 (socket says 172.21.15.137:60582) 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 -- 172.21.15.137:0/2845850631 learned_addr learned my addr 172.21.15.137:0/2845850631 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261400640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2640772a0 0x7fe26419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe262800640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe264077c40 0x7fe2641a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 -- 172.21.15.137:0/2845850631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe264077c40 msgr2=0x7fe2641a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe264077c40 0x7fe2641a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 -- 172.21.15.137:0/2845850631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2640772a0 msgr2=0x7fe26419e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2640772a0 0x7fe26419e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.217+0000 7fe261e00640 1 -- 172.21.15.137:0/2845850631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2641a5b60 con 0x7fe264076060 2024-03-26T14:01:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.218+0000 7fe261e00640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 0x7fe26419df40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe25400d770 tx=0x7fe25400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.218+0000 7fe25aa00640 1 -- 172.21.15.137:0/2845850631 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254017070 con 0x7fe264076060 2024-03-26T14:01:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.218+0000 7fe25aa00640 1 -- 172.21.15.137:0/2845850631 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2540054d0 con 0x7fe264076060 2024-03-26T14:01:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.218+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2641a5e50 con 0x7fe264076060 2024-03-26T14:01:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.218+0000 7fe25aa00640 1 -- 172.21.15.137:0/2845850631 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254004cf0 con 0x7fe264076060 2024-03-26T14:01:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.218+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2641a6260 con 0x7fe264076060 2024-03-26T14:01:39.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.221+0000 7fe25aa00640 1 -- 172.21.15.137:0/2845850631 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe254004e90 con 0x7fe264076060 2024-03-26T14:01:39.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.221+0000 7fe26be74740 1 Processor -- start 2024-03-26T14:01:39.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.221+0000 7fe26be74740 1 -- start start 2024-03-26T14:01:39.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe26be74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe26be74740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366bc590 0x561e366c1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe26be74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366c1700 0x561e366c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe26be74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e36620c90 con 0x561e366bbba0 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe26be74740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e365c9980 con 0x561e366bc590 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe26be74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3661cd30 con 0x561e366c1700 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe262800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe262800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366bbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60586/0 (socket says 172.21.15.137:60586) 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe262800640 1 -- 172.21.15.137:0/2402112821 learned_addr learned my addr 172.21.15.137:0/2402112821 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe261e00640 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366bc590 0x561e366c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe261400640 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366c1700 0x561e366c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.222+0000 7fe262800640 1 -- 172.21.15.137:0/2402112821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366c1700 msgr2=0x561e366c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe262800640 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366c1700 0x561e366c3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe262800640 1 -- 172.21.15.137:0/2402112821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366bc590 msgr2=0x561e366c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe262800640 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366bc590 0x561e366c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe262800640 1 -- 172.21.15.137:0/2402112821 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e365e1510 con 0x561e366bbba0 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe261400640 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366c1700 0x561e366c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:39.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe25aa00640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2480a2800 0x7fe2480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe262800640 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366bbfc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009fa0 tx=0x7fe25c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04dcfb887ccdafd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe25aa00640 1 -- 172.21.15.137:0/2845850631 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe2540ce960 con 0x7fe264076060 2024-03-26T14:01:39.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe261e00640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2480a2800 0x7fe2480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe24f400640 1 -- 172.21.15.137:0/2402112821 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c015070 con 0x561e366bbba0 2024-03-26T14:01:39.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe24f400640 1 -- 172.21.15.137:0/2402112821 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe25c01ab10 con 0x561e366bbba0 2024-03-26T14:01:39.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.223+0000 7fe24f400640 1 -- 172.21.15.137:0/2402112821 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c011c80 con 0x561e366bbba0 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 -- 172.21.15.137:0/2402112821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 msgr2=0x561e366bbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe261e00640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2480a2800 0x7fe2480a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe25409abf0 tx=0x7fe2540d4260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366bbfc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009fa0 tx=0x7fe25c00f7f0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 -- 172.21.15.137:0/2402112821 shutdown_connections 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366bc590 0x561e366c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366c1700 0x561e366c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 --2- 172.21.15.137:0/2402112821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366bbfc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 -- 172.21.15.137:0/2402112821 >> 172.21.15.137:0/2402112821 conn(0x561e366b7110 msgr2=0x561e366b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 -- 172.21.15.137:0/2402112821 shutdown_connections 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.224+0000 7fe26be74740 1 -- 172.21.15.137:0/2402112821 wait complete. 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.225+0000 7fe26be74740 1 -- start start 2024-03-26T14:01:39.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.225+0000 7fe26be74740 1 Processor -- start 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.225+0000 7fe26be74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.225+0000 7fe26be74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bc590 0x561e366c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.225+0000 7fe26be74740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366cd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe26be74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366c46b0 con 0x561e366bbba0 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe26be74740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e365dcdd0 con 0x561e366c1700 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe26be74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366c4830 con 0x561e366bc590 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe26be74740 10 client.0 _ll_register_callbacks cb 0x561e36334ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe261400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bc590 0x561e366c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe262800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe262800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60598/0 (socket says 172.21.15.137:60598) 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe262800640 1 -- 172.21.15.137:0/3278978279 learned_addr learned my addr 172.21.15.137:0/3278978279 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.226+0000 7fe261400640 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bc590 0x561e366c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:33548/0 (socket says 172.21.15.137:33548) 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe262800640 1 -- 172.21.15.137:0/3278978279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bc590 msgr2=0x561e366c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe262800640 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bc590 0x561e366c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe262800640 1 -- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 msgr2=0x561e366cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe262800640 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe262800640 1 -- 172.21.15.137:0/3278978279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e363352d0 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe262800640 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c6110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0116e0 tx=0x7fe25c01aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bed82751c9cb99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe243e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe243e00640 1 -- 172.21.15.137:0/3278978279 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c015050 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe243e00640 1 -- 172.21.15.137:0/3278978279 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe25c004a40 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.227+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe25c01ab10 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe243e00640 1 -- 172.21.15.137:0/3278978279 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe25c030050 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 10 client.15294 fetch_fsmap learned FSMap version 23 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561e363ff6b0 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe243e00640 1 -- 172.21.15.137:0/3278978279 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe25c01ab10 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe243e00640 1 -- 172.21.15.137:0/3278978279 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe25c005280 con 0x561e366bbba0 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 10 client.15294 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 10 client.15294 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T14:01:39.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 20 client.15294 populate_metadata read hostname 'smithi137' 2024-03-26T14:01:39.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e366d2380 0x561e366d6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 4 client.15294 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:01:39.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.228+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e365c2520 con 0x561e366d2380 2024-03-26T14:01:39.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.229+0000 7fe261400640 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e366d2380 0x561e366d6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.229+0000 7fe261400640 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e366d2380 0x561e366d6800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de6e66214a301ed5 server_cookie=2a9f9405fae08ac1 in_seq=0 out_seq=0 2024-03-26T14:01:39.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.229+0000 7fe243e00640 10 client.15294 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:01:39.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe243e00640 1 -- 172.21.15.137:0/3278978279 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561e365c2520 con 0x561e366d2380 2024-03-26T14:01:39.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe243e00640 10 client.15294 handle_command_reply: tid=0 2024-03-26T14:01:39.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 client.15294 shutdown 2024-03-26T14:01:39.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 msgr2=0x561e366c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c6110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0116e0 tx=0x7fe25c01aef0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 shutdown_connections 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e366d2380 0x561e366d6800 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bc590 0x561e366c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 --2- 172.21.15.137:0/3278978279 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c6110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe243e00640 0 client.15294 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.231+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 >> 172.21.15.137:0/3278978279 conn(0x561e366b7110 msgr2=0x561e366b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.232+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 shutdown_connections 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.232+0000 7fe26be74740 1 -- 172.21.15.137:0/3278978279 wait complete. 2024-03-26T14:01:39.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.232+0000 7fe26be74740 20 client.15294 trim_cache size 0 max 16384 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 Processor -- start 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 -- start start 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bbba0 0x561e366b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bc590 0x561e366cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366d1ba0 con 0x561e366bc590 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366d2200 con 0x561e366c1700 2024-03-26T14:01:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe26be74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366d2860 con 0x561e366bbba0 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.250+0000 7fe261400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bc590 0x561e366cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe262800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe262800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bbba0 0x561e366b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe262800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:57318/0 (socket says 172.21.15.137:57318) 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe262800640 1 -- 172.21.15.137:0/3093833793 learned_addr learned my addr 172.21.15.137:0/3093833793 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe261400640 1 -- 172.21.15.137:0/3093833793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bbba0 msgr2=0x561e366b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe261400640 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bbba0 0x561e366b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe261400640 1 -- 172.21.15.137:0/3093833793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 msgr2=0x561e366c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe261400640 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe261400640 1 -- 172.21.15.137:0/3093833793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e366c7100 con 0x561e366bc590 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe262800640 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bbba0 0x561e366b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.251+0000 7fe261400640 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bc590 0x561e366cd650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe250009230 tx=0x7fe250009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d7a6c460af2685 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe24ea00640 1 -- 172.21.15.137:0/3093833793 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250004040 con 0x561e366bc590 2024-03-26T14:01:39.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe24ea00640 1 -- 172.21.15.137:0/3093833793 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2500054f0 con 0x561e366bc590 2024-03-26T14:01:39.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe24ea00640 1 -- 172.21.15.137:0/3093833793 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250004ae0 con 0x561e366bc590 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 -- 172.21.15.137:0/3093833793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bc590 msgr2=0x561e366cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bc590 0x561e366cd650 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe250009230 tx=0x7fe250009260 comp rx=0 tx=0).stop 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 -- 172.21.15.137:0/3093833793 shutdown_connections 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bc590 0x561e366cd650 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 --2- 172.21.15.137:0/3093833793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366bbba0 0x561e366b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 -- 172.21.15.137:0/3093833793 >> 172.21.15.137:0/3093833793 conn(0x561e366b7110 msgr2=0x561e366b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 -- 172.21.15.137:0/3093833793 shutdown_connections 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.252+0000 7fe26be74740 1 -- 172.21.15.137:0/3093833793 wait complete. 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 -- start start 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 Processor -- start 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366cc730 0x561e366ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366d3520 con 0x561e366bbba0 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366d2ec0 con 0x561e366c1700 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e366d3b80 con 0x561e366cc730 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.253+0000 7fe26be74740 10 client.0 _ll_register_callbacks cb 0x7fe25c009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60626/0 (socket says 172.21.15.137:60626) 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 -- 172.21.15.137:0/3529533057 learned_addr learned my addr 172.21.15.137:0/3529533057 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe261400640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 -- 172.21.15.137:0/3529533057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366cc730 msgr2=0x561e366ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366cc730 0x561e366ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 -- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 msgr2=0x561e366c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 -- 172.21.15.137:0/3529533057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e366cd0f0 con 0x561e366bbba0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe261400640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe262800640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c91d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0056a0 tx=0x7fe25c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73df2462ae20ab12 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.254+0000 7fe243e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe243e00640 1 -- 172.21.15.137:0/3529533057 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c015040 con 0x561e366bbba0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe243e00640 1 -- 172.21.15.137:0/3529533057 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe25c005110 con 0x561e366bbba0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe2500054f0 con 0x561e366bbba0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe243e00640 1 -- 172.21.15.137:0/3529533057 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe25c005310 con 0x561e366bbba0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 10 client.15300 fetch_fsmap learned FSMap version 23 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561e3672e100 con 0x561e366bbba0 2024-03-26T14:01:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe243e00640 1 -- 172.21.15.137:0/3529533057 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2500054f0 con 0x561e366bbba0 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe243e00640 1 -- 172.21.15.137:0/3529533057 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe25c0116a0 con 0x561e366bbba0 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 10 client.15300 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 10 client.15300 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 20 client.15300 populate_metadata read hostname 'smithi137' 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e367326d0 0x561e3672e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 4 client.15300 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.255+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e3662e1d0 con 0x561e367326d0 2024-03-26T14:01:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.256+0000 7fe261400640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e367326d0 0x561e3672e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.257+0000 7fe261400640 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e367326d0 0x561e3672e330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19500a1b464f5a11 server_cookie=3fe681626cd792d1 in_seq=0 out_seq=0 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.257+0000 7fe243e00640 10 client.15300 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe243e00640 1 -- 172.21.15.137:0/3529533057 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x561e3662e1d0 con 0x561e367326d0 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe243e00640 10 client.15300 handle_command_reply: tid=0 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 client.15300 shutdown 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 msgr2=0x561e366c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c91d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0056a0 tx=0x7fe25c01ac50 comp rx=0 tx=0).stop 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 shutdown_connections 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x561e367326d0 0x561e3672e330 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e366c1700 0x561e366c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561e366cc730 0x561e366ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 --2- 172.21.15.137:0/3529533057 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561e366bbba0 0x561e366c91d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe243e00640 0 client.15300 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 >> 172.21.15.137:0/3529533057 conn(0x561e366b7110 msgr2=0x561e366b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 shutdown_connections 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 1 -- 172.21.15.137:0/3529533057 wait complete. 2024-03-26T14:01:39.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.258+0000 7fe26be74740 20 client.15300 trim_cache size 0 max 16384 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 2, 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34424, 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 110.020185802, 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 94, 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 170.557756267 2024-03-26T14:01:39.259 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2480a2800 msgr2=0x7fe2480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2480a2800 0x7fe2480a4cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe25409abf0 tx=0x7fe2540d4260 comp rx=0 tx=0).stop 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 msgr2=0x7fe26419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 0x7fe26419df40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe25400d770 tx=0x7fe25400dc30 comp rx=0 tx=0).stop 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 shutdown_connections 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe2480a2800 0x7fe2480a4cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe2640772a0 0x7fe26419e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe264077c40 0x7fe2641a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 --2- 172.21.15.137:0/2845850631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe264076060 0x7fe26419df40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 >> 172.21.15.137:0/2845850631 conn(0x7fe2640fee50 msgr2=0x7fe2641021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 shutdown_connections 2024-03-26T14:01:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.259+0000 7fe269200640 1 -- 172.21.15.137:0/2845850631 wait complete. 2024-03-26T14:01:39.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 -- 172.21.15.137:0/3501504442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf98103260 msgr2=0x7fdf98109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 --2- 172.21.15.137:0/3501504442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf98103260 0x7fdf98109af0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009900 tx=0x7fdf9002bc30 comp rx=0 tx=0).stop 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 -- 172.21.15.137:0/3501504442 shutdown_connections 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 --2- 172.21.15.137:0/3501504442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf981028a0 0x7fdf98102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 --2- 172.21.15.137:0/3501504442 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf98103260 0x7fdf98109af0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 --2- 172.21.15.137:0/3501504442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 0x7fdf98101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.482+0000 7fdfa0400640 1 -- 172.21.15.137:0/3501504442 >> 172.21.15.137:0/3501504442 conn(0x7fdf980fcc40 msgr2=0x7fdf980ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.483+0000 7fdfa0400640 1 -- 172.21.15.137:0/3501504442 shutdown_connections 2024-03-26T14:01:39.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.483+0000 7fdfa0400640 1 -- 172.21.15.137:0/3501504442 wait complete. 2024-03-26T14:01:39.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 Processor -- start 2024-03-26T14:01:39.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 -- start start 2024-03-26T14:01:39.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 0x7fdf981978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 0x7fdf98197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf98103260 0x7fdf9819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9810d230 con 0x7fdf981028a0 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9810d0b0 con 0x7fdf98103260 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdfa0400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9810d3b0 con 0x7fdf981016b0 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.484+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 0x7fdf98197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 0x7fdf98197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60654/0 (socket says 172.21.15.137:60654) 2024-03-26T14:01:39.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 -- 172.21.15.137:0/1092418097 learned_addr learned my addr 172.21.15.137:0/1092418097 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9e400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf98103260 0x7fdf9819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9da00640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 0x7fdf981978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 -- 172.21.15.137:0/1092418097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 msgr2=0x7fdf981978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 0x7fdf981978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 -- 172.21.15.137:0/1092418097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf98103260 msgr2=0x7fdf9819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf98103260 0x7fdf9819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 -- 172.21.15.137:0/1092418097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9819f4c0 con 0x7fdf981028a0 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9da00640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 0x7fdf981978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.485+0000 7fdf9d000640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 0x7fdf98197de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400d950 tx=0x7fdf8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.486+0000 7fdf96000640 1 -- 172.21.15.137:0/1092418097 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84017070 con 0x7fdf981028a0 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.486+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9819f7b0 con 0x7fdf981028a0 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.486+0000 7fdf96000640 1 -- 172.21.15.137:0/1092418097 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf84009c30 con 0x7fdf981028a0 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.486+0000 7fdf96000640 1 -- 172.21.15.137:0/1092418097 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84020060 con 0x7fdf981028a0 2024-03-26T14:01:39.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.486+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9819fd00 con 0x7fdf981028a0 2024-03-26T14:01:39.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.489+0000 7fdf96000640 1 -- 172.21.15.137:0/1092418097 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdf840109a0 con 0x7fdf981028a0 2024-03-26T14:01:39.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.489+0000 7fdfa31b7740 1 Processor -- start 2024-03-26T14:01:39.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 -- start start 2024-03-26T14:01:39.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069f9fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 0x56069fa05b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069f962c90 con 0x56069fa03700 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069f90b980 con 0x56069f9fdba0 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069f95ed30 con 0x56069f9fe590 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 0x56069fa05b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 0x56069fa05b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60660/0 (socket says 172.21.15.137:60660) 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdf9d000640 1 -- 172.21.15.137:0/2653031601 learned_addr learned my addr 172.21.15.137:0/2653031601 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.490+0000 7fdf9e400640 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069f9fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9da00640 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9d000640 1 -- 172.21.15.137:0/2653031601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 msgr2=0x56069fa03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9d000640 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9d000640 1 -- 172.21.15.137:0/2653031601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 msgr2=0x56069f9fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9d000640 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069f9fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9d000640 1 -- 172.21.15.137:0/2653031601 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069f923510 con 0x56069fa03700 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf96000640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf7c0a2800 0x7fdf7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf96000640 1 -- 172.21.15.137:0/1092418097 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdf840cec00 con 0x7fdf981028a0 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9d000640 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 0x56069fa05b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdf840d2d70 tx=0x7fdf840d7690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d112d60912fce73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.491+0000 7fdf9e400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf7c0a2800 0x7fdf7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.492+0000 7fdf8ea00640 1 -- 172.21.15.137:0/2653031601 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84097470 con 0x56069fa03700 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.492+0000 7fdf8ea00640 1 -- 172.21.15.137:0/2653031601 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf8409b8d0 con 0x56069fa03700 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.492+0000 7fdf8ea00640 1 -- 172.21.15.137:0/2653031601 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8409ab30 con 0x56069fa03700 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.492+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2653031601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 msgr2=0x56069fa05b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.492+0000 7fdf9e400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf7c0a2800 0x7fdf7c0a4cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf90006010 tx=0x7fdf9002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.492+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 0x56069fa05b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdf840d2d70 tx=0x7fdf840d7690 comp rx=0 tx=0).stop 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2653031601 shutdown_connections 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069fa03700 0x56069fa05b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2653031601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069f9fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2653031601 >> 172.21.15.137:0/2653031601 conn(0x56069f9f9110 msgr2=0x56069f9f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2653031601 shutdown_connections 2024-03-26T14:01:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2653031601 wait complete. 2024-03-26T14:01:39.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 -- start start 2024-03-26T14:01:39.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.493+0000 7fdfa31b7740 1 Processor -- start 2024-03-26T14:01:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069fa08080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069fa03700 0x56069fa0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa06650 con 0x56069f9fe590 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069f91edd0 con 0x56069f9fdba0 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa067d0 con 0x56069fa03700 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdfa31b7740 10 client.0 _ll_register_callbacks cb 0x56069f676ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60672/0 (socket says 172.21.15.137:60672) 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdf9d000640 1 -- 172.21.15.137:0/238707015 learned_addr learned my addr 172.21.15.137:0/238707015 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdf9da00640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069fa08080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.494+0000 7fdf9e400640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069fa03700 0x56069fa0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf9d000640 1 -- 172.21.15.137:0/238707015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069fa03700 msgr2=0x56069fa0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf9d000640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069fa03700 0x56069fa0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf9d000640 1 -- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 msgr2=0x56069fa08080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf9d000640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069fa08080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf9d000640 1 -- 172.21.15.137:0/238707015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56069f6772d0 con 0x56069f9fe590 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf9d000640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa085c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdf840d2d40 tx=0x7fdf840cee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853282d4d98ee318 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf83400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf83400640 1 -- 172.21.15.137:0/238707015 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8409ad10 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf83400640 1 -- 172.21.15.137:0/238707015 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdf840d7da0 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf8409b8d0 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdf83400640 1 -- 172.21.15.137:0/238707015 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf8409afa0 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdfa31b7740 10 client.15312 fetch_fsmap learned FSMap version 23 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.495+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56069f7416b0 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdf83400640 1 -- 172.21.15.137:0/238707015 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf8409b8d0 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdf83400640 1 -- 172.21.15.137:0/238707015 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdf84097840 con 0x56069f9fe590 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdfa31b7740 10 client.15312 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdfa31b7740 10 client.15312 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdfa31b7740 20 client.15312 populate_metadata read hostname 'smithi137' 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa14270 0x56069fa186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdfa31b7740 4 client.15312 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.496+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56069f904520 con 0x56069fa14270 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.497+0000 7fdf9da00640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa14270 0x56069fa186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.497+0000 7fdf9da00640 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa14270 0x56069fa186f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0842b1d4847a358 server_cookie=4cb5851f8f98eee3 in_seq=0 out_seq=0 2024-03-26T14:01:39.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.497+0000 7fdf83400640 10 client.15312 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdf83400640 1 -- 172.21.15.137:0/238707015 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56069f904520 con 0x56069fa14270 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdf83400640 10 client.15312 handle_command_reply: tid=0 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 client.15312 shutdown 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 msgr2=0x56069fa085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa085c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdf840d2d40 tx=0x7fdf840cee80 comp rx=0 tx=0).stop 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 shutdown_connections 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa14270 0x56069fa186f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069fa03700 0x56069fa0f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa085c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/238707015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069f9fdba0 0x56069fa08080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdf83400640 0 client.15312 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 >> 172.21.15.137:0/238707015 conn(0x56069f9f9110 msgr2=0x56069f9f8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 shutdown_connections 2024-03-26T14:01:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 1 -- 172.21.15.137:0/238707015 wait complete. 2024-03-26T14:01:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.500+0000 7fdfa31b7740 20 client.15312 trim_cache size 0 max 16384 2024-03-26T14:01:39.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.518+0000 7fdfa31b7740 1 Processor -- start 2024-03-26T14:01:39.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.518+0000 7fdfa31b7740 1 -- start start 2024-03-26T14:01:39.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 0x56069fa08040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa0f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa08580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa13a90 con 0x56069f9fdba0 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa140f0 con 0x56069fa03700 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa14750 con 0x56069f9fe590 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 0x56069fa08040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 0x56069fa08040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60684/0 (socket says 172.21.15.137:60684) 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 -- 172.21.15.137:0/3152675629 learned_addr learned my addr 172.21.15.137:0/3152675629 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9e400640 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9d000640 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa0f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 -- 172.21.15.137:0/3152675629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 msgr2=0x56069fa0f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa0f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 -- 172.21.15.137:0/3152675629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 msgr2=0x56069fa08580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa08580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.519+0000 7fdf9da00640 1 -- 172.21.15.137:0/3152675629 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56069f9f4470 con 0x56069f9fdba0 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.520+0000 7fdf9da00640 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 0x56069fa08040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x56069f9ff000 tx=0x7fdf88003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc2462cda151500 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.520+0000 7fdf8e000640 1 -- 172.21.15.137:0/3152675629 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88013050 con 0x56069f9fdba0 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.520+0000 7fdf8e000640 1 -- 172.21.15.137:0/3152675629 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf88023070 con 0x56069f9fdba0 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.520+0000 7fdf8e000640 1 -- 172.21.15.137:0/3152675629 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88013b90 con 0x56069f9fdba0 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 -- 172.21.15.137:0/3152675629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 msgr2=0x56069fa08040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 0x56069fa08040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x56069f9ff000 tx=0x7fdf88003d40 comp rx=0 tx=0).stop 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 -- 172.21.15.137:0/3152675629 shutdown_connections 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa08580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fe590 0x56069fa0f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/3152675629 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fdba0 0x56069fa08040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 -- 172.21.15.137:0/3152675629 >> 172.21.15.137:0/3152675629 conn(0x56069f9f9110 msgr2=0x56069f9f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 -- 172.21.15.137:0/3152675629 shutdown_connections 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 -- 172.21.15.137:0/3152675629 wait complete. 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 -- start start 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.521+0000 7fdfa31b7740 1 Processor -- start 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fdba0 0x56069fa0b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa0bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa0ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa15410 con 0x56069f9fe590 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa14db0 con 0x56069fa03700 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069fa15a70 con 0x56069f9fdba0 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdfa31b7740 10 client.0 _ll_register_callbacks cb 0x7fdf90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa0bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdf9d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa0bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60696/0 (socket says 172.21.15.137:60696) 2024-03-26T14:01:39.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdf9d000640 1 -- 172.21.15.137:0/2871806018 learned_addr learned my addr 172.21.15.137:0/2871806018 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdf9e400640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa0ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdf9da00640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fdba0 0x56069fa0b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.522+0000 7fdf9d000640 1 -- 172.21.15.137:0/2871806018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fdba0 msgr2=0x56069fa0b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf9d000640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fdba0 0x56069fa0b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf9d000640 1 -- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 msgr2=0x56069fa0ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf9d000640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa0ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf9d000640 1 -- 172.21.15.137:0/2871806018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56069fa0f280 con 0x56069f9fe590 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf9e400640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa0ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf9d000640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa0bd30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdf840cef30 tx=0x7fdf8409a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11c576092eb5c70 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf83400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf83400640 1 -- 172.21.15.137:0/2871806018 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8409ad10 con 0x56069f9fe590 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf83400640 1 -- 172.21.15.137:0/2871806018 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdf840d2680 con 0x56069f9fe590 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf88023070 con 0x56069f9fe590 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdf83400640 1 -- 172.21.15.137:0/2871806018 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf8409ab30 con 0x56069f9fe590 2024-03-26T14:01:39.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdfa31b7740 10 client.15318 fetch_fsmap learned FSMap version 23 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.523+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56069fa70230 con 0x56069f9fe590 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdf83400640 1 -- 172.21.15.137:0/2871806018 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdf88023070 con 0x56069f9fe590 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdf83400640 1 -- 172.21.15.137:0/2871806018 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdf840a34e0 con 0x56069f9fe590 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdfa31b7740 10 client.15318 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdfa31b7740 10 client.15318 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdfa31b7740 20 client.15318 populate_metadata read hostname 'smithi137' 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa74540 0x56069fa70230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdfa31b7740 4 client.15318 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T14:01:39.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56069f9701d0 con 0x56069fa74540 2024-03-26T14:01:39.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.524+0000 7fdf9da00640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa74540 0x56069fa70230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.525+0000 7fdf9da00640 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa74540 0x56069fa70230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88d660e8419112f7 server_cookie=3039125824e30a33 in_seq=0 out_seq=0 2024-03-26T14:01:39.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.525+0000 7fdf83400640 10 client.15318 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:01:39.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.525+0000 7fdf83400640 1 -- 172.21.15.137:0/2871806018 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56069f9701d0 con 0x56069fa74540 2024-03-26T14:01:39.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.525+0000 7fdf83400640 10 client.15318 handle_command_reply: tid=0 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.525+0000 7fdfa31b7740 1 client.15318 shutdown 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 msgr2=0x56069fa0bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa0bd30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdf840cef30 tx=0x7fdf8409a520 comp rx=0 tx=0).stop 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 shutdown_connections 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56069fa74540 0x56069fa70230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56069fa03700 0x56069fa0ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56069f9fe590 0x56069fa0bd30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 --2- 172.21.15.137:0/2871806018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56069f9fdba0 0x56069fa0b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 >> 172.21.15.137:0/2871806018 conn(0x56069f9f9110 msgr2=0x56069f9ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdf83400640 0 client.15318 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 shutdown_connections 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 1 -- 172.21.15.137:0/2871806018 wait complete. 2024-03-26T14:01:39.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.526+0000 7fdfa31b7740 20 client.15318 trim_cache size 0 max 16384 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 1, 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34451, 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 111.302457077, 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 94, 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 158.62368512099999 2024-03-26T14:01:39.527 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf7c0a2800 msgr2=0x7fdf7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf7c0a2800 0x7fdf7c0a4cc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf90006010 tx=0x7fdf9002c7e0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 msgr2=0x7fdf98197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 0x7fdf98197de0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdf8400d950 tx=0x7fdf8400de10 comp rx=0 tx=0).stop 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 shutdown_connections 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdf7c0a2800 0x7fdf7c0a4cc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf98103260 0x7fdf9819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdf981028a0 0x7fdf98197de0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 --2- 172.21.15.137:0/1092418097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf981016b0 0x7fdf981978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 >> 172.21.15.137:0/1092418097 conn(0x7fdf980fcc40 msgr2=0x7fdf98105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 shutdown_connections 2024-03-26T14:01:39.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.527+0000 7fdfa0400640 1 -- 172.21.15.137:0/1092418097 wait complete. 2024-03-26T14:01:39.531 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:39 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/154268111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:39.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:39 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1403401183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:39.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:39 smithi137 ceph-mon[31539]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:39.540 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T14:01:39.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:39 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/154268111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:39.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:39 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1403401183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:39.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:39 smithi179 ceph-mon[35472]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:39.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:39 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/154268111' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:01:39.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:39 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1403401183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:01:39.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:39 smithi159 ceph-mon[35501]: pgmap v437: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 -- 172.21.15.137:0/70317401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 msgr2=0x7fd1740784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 --2- 172.21.15.137:0/70317401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1740784e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009ca0 tx=0x7fd16c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 -- 172.21.15.137:0/70317401 shutdown_connections 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 --2- 172.21.15.137:0/70317401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1740772a0 0x7fd174077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 --2- 172.21.15.137:0/70317401 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1740784e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 --2- 172.21.15.137:0/70317401 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 0x7fd174076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 -- 172.21.15.137:0/70317401 >> 172.21.15.137:0/70317401 conn(0x7fd1740fee50 msgr2=0x7fd174101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 -- 172.21.15.137:0/70317401 shutdown_connections 2024-03-26T14:01:39.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.744+0000 7fd17ba00640 1 -- 172.21.15.137:0/70317401 wait complete. 2024-03-26T14:01:39.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.745+0000 7fd17ba00640 1 Processor -- start 2024-03-26T14:01:39.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.745+0000 7fd17ba00640 1 -- start start 2024-03-26T14:01:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd17ba00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 0x7fd17419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd17ba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1740772a0 0x7fd17419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd17ba00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1741a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd17ba00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd174113820 con 0x7fd174077c40 2024-03-26T14:01:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd17ba00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1741136a0 con 0x7fd1740772a0 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd17ba00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1741139a0 con 0x7fd174076060 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1741a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1741a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60716/0 (socket says 172.21.15.137:60716) 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 -- 172.21.15.137:0/4042571688 learned_addr learned my addr 172.21.15.137:0/4042571688 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd173e00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1740772a0 0x7fd17419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179000640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 0x7fd17419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 -- 172.21.15.137:0/4042571688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 msgr2=0x7fd17419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 0x7fd17419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 -- 172.21.15.137:0/4042571688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1740772a0 msgr2=0x7fd17419e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1740772a0 0x7fd17419e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179a00640 1 -- 172.21.15.137:0/4042571688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1741a5a90 con 0x7fd174077c40 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.746+0000 7fd179000640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 0x7fd17419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.747+0000 7fd179a00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1741a5420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd16c032380 tx=0x7fd16c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.747+0000 7fd171600640 1 -- 172.21.15.137:0/4042571688 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c041020 con 0x7fd174077c40 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.747+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1741a5d20 con 0x7fd174077c40 2024-03-26T14:01:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.747+0000 7fd171600640 1 -- 172.21.15.137:0/4042571688 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd16c003e20 con 0x7fd174077c40 2024-03-26T14:01:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.747+0000 7fd171600640 1 -- 172.21.15.137:0/4042571688 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c039040 con 0x7fd174077c40 2024-03-26T14:01:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.747+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1741a61b0 con 0x7fd174077c40 2024-03-26T14:01:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.750+0000 7fd171600640 1 -- 172.21.15.137:0/4042571688 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd16c044020 con 0x7fd174077c40 2024-03-26T14:01:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.750+0000 7fd17e783740 1 Processor -- start 2024-03-26T14:01:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.750+0000 7fd17e783740 1 -- start start 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd17e783740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad2ba0 0x56523cad2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd17e783740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 0x56523cad8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd17e783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cadab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd17e783740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523ca37c90 con 0x56523cad3590 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd17e783740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523c9e0980 con 0x56523cad2ba0 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd17e783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523ca33d30 con 0x56523cad8700 2024-03-26T14:01:39.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd179000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 0x56523cad8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd179000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 0x56523cad8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60730/0 (socket says 172.21.15.137:60730) 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd179000640 1 -- 172.21.15.137:0/4005911869 learned_addr learned my addr 172.21.15.137:0/4005911869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.751+0000 7fd173e00640 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad2ba0 0x56523cad2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd179000640 1 -- 172.21.15.137:0/4005911869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 msgr2=0x56523cadab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd179000640 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cadab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd179000640 1 -- 172.21.15.137:0/4005911869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad2ba0 msgr2=0x56523cad2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd179000640 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad2ba0 0x56523cad2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd179000640 1 -- 172.21.15.137:0/4005911869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523c9f8510 con 0x56523cad3590 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd179000640 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 0x56523cad8090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd164009c50 tx=0x7fd16400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bc019aa92b3e86 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd171600640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd15c0a2800 0x7fd15c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd162000640 1 -- 172.21.15.137:0/4005911869 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164016070 con 0x56523cad3590 2024-03-26T14:01:39.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd162000640 1 -- 172.21.15.137:0/4005911869 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd164011920 con 0x56523cad3590 2024-03-26T14:01:39.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd171600640 1 -- 172.21.15.137:0/4042571688 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd16c0f1120 con 0x7fd174077c40 2024-03-26T14:01:39.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd162000640 1 -- 172.21.15.137:0/4005911869 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640107f0 con 0x56523cad3590 2024-03-26T14:01:39.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.752+0000 7fd173e00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd15c0a2800 0x7fd15c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 -- 172.21.15.137:0/4005911869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 msgr2=0x56523cad8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 0x56523cad8090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd164009c50 tx=0x7fd16400e6c0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd173e00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd15c0a2800 0x7fd15c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x56523cad3df0 tx=0x7fd16800c3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 -- 172.21.15.137:0/4005911869 shutdown_connections 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cadab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad3590 0x56523cad8090 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 --2- 172.21.15.137:0/4005911869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad2ba0 0x56523cad2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 -- 172.21.15.137:0/4005911869 >> 172.21.15.137:0/4005911869 conn(0x56523cace110 msgr2=0x56523cace510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 -- 172.21.15.137:0/4005911869 shutdown_connections 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.753+0000 7fd17e783740 1 -- 172.21.15.137:0/4005911869 wait complete. 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 -- start start 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 Processor -- start 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cadd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cadd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cae45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cadb6b0 con 0x56523cad2ba0 2024-03-26T14:01:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523c9f3dd0 con 0x56523cad3590 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cadb830 con 0x56523cad8700 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.754+0000 7fd17e783740 10 client.0 _ll_register_callbacks cb 0x56523c74bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cadd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cadd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60740/0 (socket says 172.21.15.137:60740) 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 -- 172.21.15.137:0/3088578520 learned_addr learned my addr 172.21.15.137:0/3088578520 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd173e00640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cadd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179a00640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cae45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 -- 172.21.15.137:0/3088578520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 msgr2=0x56523cae45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cae45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 -- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 msgr2=0x56523cadd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cadd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 -- 172.21.15.137:0/3088578520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523c74c2d0 con 0x56523cad2ba0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd173e00640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cadd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd179000640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cadd040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd16400ef30 tx=0x7fd164009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b38920e511a0e74 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.755+0000 7fd156a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:01:39.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd156a00640 1 -- 172.21.15.137:0/3088578520 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164028060 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd156a00640 1 -- 172.21.15.137:0/3088578520 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd164005040 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd164011920 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd156a00640 1 -- 172.21.15.137:0/3088578520 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd16402f050 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd17e783740 10 client.15330 fetch_fsmap learned FSMap version 23 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56523c8166b0 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.756+0000 7fd156a00640 1 -- 172.21.15.137:0/3088578520 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd164011920 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd156a00640 1 -- 172.21.15.137:0/3088578520 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd16400f040 con 0x56523cad2ba0 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd17e783740 10 client.15330 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd17e783740 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd17e783740 20 client.15330 populate_metadata read hostname 'smithi137' 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd17e783740 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cae8890 0x56523caed990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd17e783740 4 client.15330 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56523c9d9520 con 0x56523cae8890 2024-03-26T14:01:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.757+0000 7fd173e00640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cae8890 0x56523caed990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.758+0000 7fd173e00640 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cae8890 0x56523caed990 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c695e2a11a4f41ba server_cookie=94e0cea892be5a76 in_seq=0 out_seq=0 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.758+0000 7fd156a00640 10 client.15330 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.760+0000 7fd156a00640 1 -- 172.21.15.137:0/3088578520 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56523c9d9520 con 0x56523cae8890 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.760+0000 7fd156a00640 10 client.15330 handle_command_reply: tid=0 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.760+0000 7fd17e783740 1 client.15330 shutdown 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.760+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 msgr2=0x56523cadd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.760+0000 7fd17e783740 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cadd040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd16400ef30 tx=0x7fd164009d40 comp rx=0 tx=0).stop 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 shutdown_connections 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cae8890 0x56523caed990 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cadd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad8700 0x56523cae45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 --2- 172.21.15.137:0/3088578520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cadd040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 >> 172.21.15.137:0/3088578520 conn(0x56523cace110 msgr2=0x56523cacd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd156a00640 0 client.15330 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:01:39.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 shutdown_connections 2024-03-26T14:01:39.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 1 -- 172.21.15.137:0/3088578520 wait complete. 2024-03-26T14:01:39.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.761+0000 7fd17e783740 20 client.15330 trim_cache size 0 max 16384 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 Processor -- start 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 -- start start 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cad3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad3590 0x56523cae4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad8700 0x56523cac8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cae8d30 con 0x56523cad2ba0 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cae9390 con 0x56523cad8700 2024-03-26T14:01:39.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd17e783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cae99f0 con 0x56523cad3590 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd179000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cad3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.779+0000 7fd179000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cad3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60750/0 (socket says 172.21.15.137:60750) 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 -- 172.21.15.137:0/993088592 learned_addr learned my addr 172.21.15.137:0/993088592 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179a00640 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad8700 0x56523cac8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd173e00640 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad3590 0x56523cae4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 -- 172.21.15.137:0/993088592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad3590 msgr2=0x56523cae4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad3590 0x56523cae4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 -- 172.21.15.137:0/993088592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad8700 msgr2=0x56523cac8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad8700 0x56523cac8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 -- 172.21.15.137:0/993088592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523cad4440 con 0x56523cad2ba0 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd173e00640 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad3590 0x56523cae4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd179000640 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cad3f00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd1640099e0 tx=0x7fd164009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fe21f670d8b70e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd161600640 1 -- 172.21.15.137:0/993088592 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164028070 con 0x56523cad2ba0 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.780+0000 7fd161600640 1 -- 172.21.15.137:0/993088592 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd164004af0 con 0x56523cad2ba0 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd161600640 1 -- 172.21.15.137:0/993088592 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164016070 con 0x56523cad2ba0 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 -- 172.21.15.137:0/993088592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 msgr2=0x56523cad3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cad3f00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd1640099e0 tx=0x7fd164009db0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 -- 172.21.15.137:0/993088592 shutdown_connections 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad8700 0x56523cac8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad3590 0x56523cae4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 --2- 172.21.15.137:0/993088592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad2ba0 0x56523cad3f00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 -- 172.21.15.137:0/993088592 >> 172.21.15.137:0/993088592 conn(0x56523cace110 msgr2=0x56523cad1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 -- 172.21.15.137:0/993088592 shutdown_connections 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.781+0000 7fd17e783740 1 -- 172.21.15.137:0/993088592 wait complete. 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 -- start start 2024-03-26T14:01:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 Processor -- start 2024-03-26T14:01:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad2ba0 0x56523cae07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cae0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 0x56523cae35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523caea6b0 con 0x56523cad8700 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523caea050 con 0x56523cad3590 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523caead10 con 0x56523cad2ba0 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd17e783740 10 client.0 _ll_register_callbacks cb 0x7fd164009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.782+0000 7fd179a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 0x56523cae35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 0x56523cae35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60754/0 (socket says 172.21.15.137:60754) 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 -- 172.21.15.137:0/4279376500 learned_addr learned my addr 172.21.15.137:0/4279376500 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd173e00640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cae0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 -- 172.21.15.137:0/4279376500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad2ba0 msgr2=0x56523cae07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179000640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad2ba0 0x56523cae07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad2ba0 0x56523cae07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 -- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 msgr2=0x56523cae0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cae0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 -- 172.21.15.137:0/4279376500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523cae3c00 con 0x56523cad8700 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd173e00640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cae0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd179a00640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 0x56523cae35c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd16c006010 tx=0x7fd16c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c49a712de03996 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd156a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd156a00640 1 -- 172.21.15.137:0/4279376500 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c004040 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd156a00640 1 -- 172.21.15.137:0/4279376500 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd16c04dd60 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.783+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd164004af0 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd156a00640 1 -- 172.21.15.137:0/4279376500 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd16c0bddb0 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 10 client.15336 fetch_fsmap learned FSMap version 23 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56523cb44da0 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd156a00640 1 -- 172.21.15.137:0/4279376500 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd164004af0 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd156a00640 1 -- 172.21.15.137:0/4279376500 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd16c0efce0 con 0x56523cad8700 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 10 client.15336 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 10 client.15336 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 20 client.15336 populate_metadata read hostname 'smithi137' 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cb49510 0x56523cb45010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 4 client.15336 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T14:01:39.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.784+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56523ca451d0 con 0x56523cb49510 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.785+0000 7fd179000640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cb49510 0x56523cb45010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.785+0000 7fd179000640 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cb49510 0x56523cb45010 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f26718b2d39a39e server_cookie=eadcfd387978dee1 in_seq=0 out_seq=0 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.785+0000 7fd156a00640 10 client.15336 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd156a00640 1 -- 172.21.15.137:0/4279376500 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56523ca451d0 con 0x56523cb49510 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd156a00640 10 client.15336 handle_command_reply: tid=0 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 client.15336 shutdown 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 msgr2=0x56523cae35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 0x56523cae35c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd16c006010 tx=0x7fd16c009540 comp rx=0 tx=0).stop 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 shutdown_connections 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56523cb49510 0x56523cb45010 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56523cad3590 0x56523cae0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56523cad8700 0x56523cae35c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 --2- 172.21.15.137:0/4279376500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56523cad2ba0 0x56523cae07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd156a00640 0 client.15336 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 >> 172.21.15.137:0/4279376500 conn(0x56523cace110 msgr2=0x56523cad1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 shutdown_connections 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.786+0000 7fd17e783740 1 -- 172.21.15.137:0/4279376500 wait complete. 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17e783740 20 client.15336 trim_cache size 0 max 16384 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 0, 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34496, 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 136.49127331899999, 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 94, 2024-03-26T14:01:39.788 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T14:01:39.789 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 142.804336697 2024-03-26T14:01:39.789 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:01:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd15c0a2800 msgr2=0x7fd15c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd15c0a2800 0x7fd15c0a4cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x56523cad3df0 tx=0x7fd16800c3b0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 msgr2=0x7fd1741a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1741a5420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd16c032380 tx=0x7fd16c032820 comp rx=0 tx=0).stop 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 shutdown_connections 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd15c0a2800 0x7fd15c0a4cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1740772a0 0x7fd17419e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd174077c40 0x7fd1741a5420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 --2- 172.21.15.137:0/4042571688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd174076060 0x7fd17419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 >> 172.21.15.137:0/4042571688 conn(0x7fd1740fee50 msgr2=0x7fd1741005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 shutdown_connections 2024-03-26T14:01:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:39.787+0000 7fd17ba00640 1 -- 172.21.15.137:0/4042571688 wait complete. 2024-03-26T14:01:39.803 INFO:tasks.cephadm:Setup complete, yielding 2024-03-26T14:01:39.803 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T14:01:39.812 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T14:01:39.812 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph orch status' 2024-03-26T14:01:40.265 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:42.378 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:42 smithi137 ceph-mon[31539]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:42.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:42 smithi159 ceph-mon[35501]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:42.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.544+0000 7f6177c00700 1 -- 172.21.15.137:0/3068825976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 msgr2=0x7f617010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:42.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 --2- 172.21.15.137:0/3068825976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f617010a610 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f616800b0d0 tx=0x7f616800b3e0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 -- 172.21.15.137:0/3068825976 shutdown_connections 2024-03-26T14:01:42.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 --2- 172.21.15.137:0/3068825976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f617010a610 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 --2- 172.21.15.137:0/3068825976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701021c0 0x7f61701025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 --2- 172.21.15.137:0/3068825976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701033c0 0x7f6170103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 -- 172.21.15.137:0/3068825976 >> 172.21.15.137:0/3068825976 conn(0x7f61700fd7a0 msgr2=0x7f61700ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:42.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 -- 172.21.15.137:0/3068825976 shutdown_connections 2024-03-26T14:01:42.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.545+0000 7f6177c00700 1 -- 172.21.15.137:0/3068825976 wait complete. 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 Processor -- start 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 -- start start 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701021c0 0x7f617019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701033c0 0x7f617019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f61701a1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617010dca0 con 0x7f6170103d60 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617010db20 con 0x7f61701021c0 2024-03-26T14:01:42.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.546+0000 7f6177c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617010de20 con 0x7f61701033c0 2024-03-26T14:01:42.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f61701a1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:42.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f61701a1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60770/0 (socket says 172.21.15.137:60770) 2024-03-26T14:01:42.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 -- 172.21.15.137:0/1814101130 learned_addr learned my addr 172.21.15.137:0/1814101130 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:42.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175200700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701021c0 0x7f617019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:42.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f616fe00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701033c0 0x7f617019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:42.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 -- 172.21.15.137:0/1814101130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701033c0 msgr2=0x7f617019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701033c0 0x7f617019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 -- 172.21.15.137:0/1814101130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701021c0 msgr2=0x7f617019a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701021c0 0x7f617019a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175c00700 1 -- 172.21.15.137:0/1814101130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61701a2190 con 0x7f6170103d60 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f6175200700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701021c0 0x7f617019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.547+0000 7f616fe00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701033c0 0x7f617019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.548+0000 7f6175c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f61701a1af0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6168004cb0 tx=0x7f6168004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.548+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616801f030 con 0x7f6170103d60 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.548+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6168007ac0 con 0x7f6170103d60 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.548+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616801b690 con 0x7f6170103d60 2024-03-26T14:01:42.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.548+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61701a2420 con 0x7f6170103d60 2024-03-26T14:01:42.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.548+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701a2870 con 0x7f6170103d60 2024-03-26T14:01:42.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.550+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6170068e50 con 0x7f6170103d60 2024-03-26T14:01:42.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.553+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6168022030 con 0x7f6170103d60 2024-03-26T14:01:42.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.554+0000 7f616d600700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61580a2910 0x7f61580a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:42.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.554+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61680cf9f0 con 0x7f6170103d60 2024-03-26T14:01:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.555+0000 7f6175200700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61580a2910 0x7f61580a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:42.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.555+0000 7f6175200700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61580a2910 0x7f61580a4dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6164005950 tx=0x7f61640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:42.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.562+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f616809c710 con 0x7f6170103d60 2024-03-26T14:01:42.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:42 smithi179 ceph-mon[35472]: pgmap v438: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:42.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.812+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6170075ae0 con 0x7f61580a2910 2024-03-26T14:01:42.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.814+0000 7f616d600700 1 -- 172.21.15.137:0/1814101130 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6170075ae0 con 0x7f61580a2910 2024-03-26T14:01:42.811 INFO:teuthology.orchestra.run.smithi137.stdout:Backend: cephadm 2024-03-26T14:01:42.811 INFO:teuthology.orchestra.run.smithi137.stdout:Available: Yes 2024-03-26T14:01:42.811 INFO:teuthology.orchestra.run.smithi137.stdout:Paused: No 2024-03-26T14:01:42.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61580a2910 msgr2=0x7f61580a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:42.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61580a2910 0x7f61580a4dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6164005950 tx=0x7f61640058e0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 msgr2=0x7f61701a1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:42.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f61701a1af0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6168004cb0 tx=0x7f6168004d90 comp rx=0 tx=0).stop 2024-03-26T14:01:42.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 shutdown_connections 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61580a2910 0x7f61580a4dd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6170103d60 0x7f61701a1af0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61701021c0 0x7f617019a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 --2- 172.21.15.137:0/1814101130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61701033c0 0x7f617019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 >> 172.21.15.137:0/1814101130 conn(0x7f61700fd7a0 msgr2=0x7f61700743c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 shutdown_connections 2024-03-26T14:01:42.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:42.817+0000 7f6177c00700 1 -- 172.21.15.137:0/1814101130 wait complete. 2024-03-26T14:01:43.596 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-26T14:01:44.030 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:44.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:44 smithi159 ceph-mon[35501]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:44.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:44 smithi159 ceph-mon[35501]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:44.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:44 smithi137 ceph-mon[31539]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:44.502 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:44 smithi137 ceph-mon[31539]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:44.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:44 smithi179 ceph-mon[35472]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:44.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:44 smithi179 ceph-mon[35472]: pgmap v439: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 -- 172.21.15.137:0/995663891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724104410 msgr2=0x7f5724104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 --2- 172.21.15.137:0/995663891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724104410 0x7f5724104830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f57140099c0 tx=0x7f5714009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 -- 172.21.15.137:0/995663891 shutdown_connections 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 --2- 172.21.15.137:0/995663891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724105fb0 0x7f572410c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 --2- 172.21.15.137:0/995663891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724104410 0x7f5724104830 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 --2- 172.21.15.137:0/995663891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5724105610 0x7f5724105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 -- 172.21.15.137:0/995663891 >> 172.21.15.137:0/995663891 conn(0x7f57240ff9b0 msgr2=0x7f5724101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 -- 172.21.15.137:0/995663891 shutdown_connections 2024-03-26T14:01:46.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.181+0000 7f572b000700 1 -- 172.21.15.137:0/995663891 wait complete. 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.182+0000 7f572b000700 1 Processor -- start 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.182+0000 7f572b000700 1 -- start start 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f572b000700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724104410 0x7f572419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f572b000700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5724105610 0x7f572419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f572b000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 0x7f57241a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f572b000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572410ff00 con 0x7f5724105fb0 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f572b000700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572410fd80 con 0x7f5724105610 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f572b000700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5724110080 con 0x7f5724104410 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f5729000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 0x7f57241a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:46.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f5729000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 0x7f57241a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60784/0 (socket says 172.21.15.137:60784) 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f5729000700 1 -- 172.21.15.137:0/3009505321 learned_addr learned my addr 172.21.15.137:0/3009505321 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f5723e00700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724104410 0x7f572419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.183+0000 7f5723400700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5724105610 0x7f572419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5729000700 1 -- 172.21.15.137:0/3009505321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724104410 msgr2=0x7f572419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5729000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724104410 0x7f572419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5729000700 1 -- 172.21.15.137:0/3009505321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5724105610 msgr2=0x7f572419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5729000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5724105610 0x7f572419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5729000700 1 -- 172.21.15.137:0/3009505321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57241a2180 con 0x7f5724105fb0 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5723e00700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724104410 0x7f572419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:46.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5729000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 0x7f57241a1ae0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f571c007f10 tx=0x7f571c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:46.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c004bb0 con 0x7f5724105fb0 2024-03-26T14:01:46.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f571c004d50 con 0x7f5724105fb0 2024-03-26T14:01:46.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57241a2470 con 0x7f5724105fb0 2024-03-26T14:01:46.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c005990 con 0x7f5724105fb0 2024-03-26T14:01:46.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.184+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57241a2a00 con 0x7f5724105fb0 2024-03-26T14:01:46.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.186+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5724068e50 con 0x7f5724105fb0 2024-03-26T14:01:46.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.188+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f571c01a020 con 0x7f5724105fb0 2024-03-26T14:01:46.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.189+0000 7f5720c00700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57080a2910 0x7f57080a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:46.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.189+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f571c0ce670 con 0x7f5724105fb0 2024-03-26T14:01:46.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.189+0000 7f5723e00700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57080a2910 0x7f57080a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:46.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.190+0000 7f5723e00700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57080a2910 0x7f57080a4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f571400b580 tx=0x7f5714005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:46.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.194+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f571c010070 con 0x7f5724105fb0 2024-03-26T14:01:46.282 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:46 smithi137 ceph-mon[31539]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:46.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.432+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57240625d0 con 0x7f57080a2910 2024-03-26T14:01:46.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.446+0000 7f5720c00700 1 -- 172.21.15.137:0/3009505321 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f57240625d0 con 0x7f57080a2910 2024-03-26T14:01:46.444 INFO:teuthology.orchestra.run.smithi137.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-26T14:01:46.444 INFO:teuthology.orchestra.run.smithi137.stdout:mds.a smithi137 running (2m) 37s ago 2m 12.3M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 cfbcad657b2a 2024-03-26T14:01:46.444 INFO:teuthology.orchestra.run.smithi137.stdout:mds.b smithi159 running (2m) 76s ago 2m 24.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 12dac58e7caf 2024-03-26T14:01:46.444 INFO:teuthology.orchestra.run.smithi137.stdout:mds.c smithi179 running (2m) 77s ago 2m 14.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 2a298516b5dc 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.d smithi137 running (3m) 37s ago 3m 12.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 4b9571b37119 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.e smithi159 running (2m) 76s ago 2m 16.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 d84026a869aa 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.f smithi179 running (2m) 77s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 a78f758c342b 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.g smithi137 running (2m) 37s ago 2m 12.4M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 17d9460e4d86 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.h smithi159 running (2m) 76s ago 2m 16.2M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 66af434dd95e 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.i smithi179 running (2m) 77s ago 2m 14.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 9e79f1a555a6 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.j smithi137 running (3m) 37s ago 3m 12.6M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 667c102acc8c 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.k smithi159 running (3m) 76s ago 3m 15.9M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 69ca7f16293f 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mds.l smithi179 running (3m) 77s ago 3m 12.5M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 c586187f6c74 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.x smithi137 *:9283,8765 running (12m) 37s ago 12m 514M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 fc3bed4ebd10 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.y smithi159 *:8443,8765 running (10m) 76s ago 10m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 14dc75baac4f 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mgr.z smithi179 *:8443,8765 running (10m) 77s ago 10m 451M - 19.0.0-2145-g3be9a5c9 c664afe56bb0 c08f9860921c 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mon.a smithi137 running (12m) 37s ago 12m 51.8M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 15e9559f90d0 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mon.b smithi159 running (10m) 76s ago 10m 38.9M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 a88340e27b39 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:mon.c smithi179 running (10m) 77s ago 10m 39.5M 2048M 19.0.0-2145-g3be9a5c9 c664afe56bb0 df3ce41e900c 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 smithi137 running (9m) 37s ago 9m 45.2M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 1ff2c4b90707 2024-03-26T14:01:46.445 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 smithi159 running (9m) 76s ago 9m 46.3M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 8b7036724f6f 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 smithi179 running (8m) 77s ago 8m 57.6M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 98a0f8a9c54d 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 smithi137 running (8m) 37s ago 8m 43.6M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c6c07313ead3 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 smithi159 running (7m) 76s ago 7m 41.8M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 964ea10d4c88 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 smithi179 running (7m) 77s ago 7m 53.9M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 6764f31fd2b3 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.6 smithi137 running (6m) 37s ago 6m 51.6M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 c049927a6c53 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.7 smithi159 running (5m) 76s ago 5m 53.3M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 714b40031f41 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.8 smithi179 running (5m) 77s ago 5m 53.0M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 fd27a7ee2636 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.9 smithi137 running (4m) 37s ago 4m 42.6M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 649c0c7e6115 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.10 smithi159 running (4m) 76s ago 4m 42.2M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 e2b3fe1a1f69 2024-03-26T14:01:46.446 INFO:teuthology.orchestra.run.smithi137.stdout:osd.11 smithi179 running (3m) 77s ago 3m 51.8M 4272M 19.0.0-2145-g3be9a5c9 c664afe56bb0 259b8bc17bca 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57080a2910 msgr2=0x7f57080a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57080a2910 0x7f57080a4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f571400b580 tx=0x7f5714005fb0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 msgr2=0x7f57241a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 0x7f57241a1ae0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f571c007f10 tx=0x7f571c00d3b0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 shutdown_connections 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f57080a2910 0x7f57080a4dd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5724105fb0 0x7f57241a1ae0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5724104410 0x7f572419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 --2- 172.21.15.137:0/3009505321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5724105610 0x7f572419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:46.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 >> 172.21.15.137:0/3009505321 conn(0x7f57240ff9b0 msgr2=0x7f5724108910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:46.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.449+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 shutdown_connections 2024-03-26T14:01:46.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:46.450+0000 7f572b000700 1 -- 172.21.15.137:0/3009505321 wait complete. 2024-03-26T14:01:46.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:46 smithi159 ceph-mon[35501]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:46.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:46 smithi179 ceph-mon[35472]: pgmap v440: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:47.377 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-26T14:01:47.814 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:48.269 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:48 smithi137 ceph-mon[31539]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:48.270 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:48 smithi137 ceph-mon[31539]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:48.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:48 smithi179 ceph-mon[35472]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:48.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:48 smithi179 ceph-mon[35472]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:48.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:48 smithi159 ceph-mon[35501]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:48.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:48 smithi159 ceph-mon[35501]: pgmap v441: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 -- 172.21.15.137:0/2458817304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841010c0 msgr2=0x7f01841014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 --2- 172.21.15.137:0/2458817304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841010c0 0x7f01841014e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f01780099c0 tx=0x7f0178009cd0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 -- 172.21.15.137:0/2458817304 shutdown_connections 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 --2- 172.21.15.137:0/2458817304 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0184102c60 0x7f0184109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 --2- 172.21.15.137:0/2458817304 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841010c0 0x7f01841014e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 --2- 172.21.15.137:0/2458817304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01841022c0 0x7f0184102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 -- 172.21.15.137:0/2458817304 >> 172.21.15.137:0/2458817304 conn(0x7f01840fc660 msgr2=0x7f01840feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 -- 172.21.15.137:0/2458817304 shutdown_connections 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.113+0000 7f018be00700 1 -- 172.21.15.137:0/2458817304 wait complete. 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.114+0000 7f018be00700 1 Processor -- start 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 -- start start 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01841010c0 0x7f0184198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 0x7f01841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102c60 0x7f018419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018410cb90 con 0x7f01841022c0 2024-03-26T14:01:50.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018410ca10 con 0x7f0184102c60 2024-03-26T14:01:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f018be00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018410cd10 con 0x7f01841010c0 2024-03-26T14:01:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f0188a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 0x7f01841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f0188a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 0x7f01841989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50750/0 (socket says 172.21.15.137:50750) 2024-03-26T14:01:50.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.115+0000 7f0188a00700 1 -- 172.21.15.137:0/397760528 learned_addr learned my addr 172.21.15.137:0/397760528 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:50.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0189e00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102c60 0x7f018419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:50.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0189400700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01841010c0 0x7f0184198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:50.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0188a00700 1 -- 172.21.15.137:0/397760528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01841010c0 msgr2=0x7f0184198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0188a00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01841010c0 0x7f0184198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0188a00700 1 -- 172.21.15.137:0/397760528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102c60 msgr2=0x7f018419f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0188a00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102c60 0x7f018419f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0188a00700 1 -- 172.21.15.137:0/397760528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01841a0080 con 0x7f01841022c0 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0189400700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01841010c0 0x7f0184198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0188a00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 0x7f01841989b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f017000d980 tx=0x7f017000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170016070 con 0x7f01841022c0 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0170010460 con 0x7f01841022c0 2024-03-26T14:01:50.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01841a0370 con 0x7f01841022c0 2024-03-26T14:01:50.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.116+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000f620 con 0x7f01841022c0 2024-03-26T14:01:50.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.117+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01841a0900 con 0x7f01841022c0 2024-03-26T14:01:50.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.118+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0184068e50 con 0x7f01841022c0 2024-03-26T14:01:50.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.120+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0170019070 con 0x7f01841022c0 2024-03-26T14:01:50.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.121+0000 7f0182000700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f016c0a2910 0x7f016c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:50.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.121+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f01700ce980 con 0x7f01841022c0 2024-03-26T14:01:50.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.121+0000 7f0189400700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f016c0a2910 0x7f016c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:50.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.122+0000 7f0189400700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f016c0a2910 0x7f016c0a4dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0178006010 tx=0x7f0178009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:50.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.124+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f017009b650 con 0x7f01841022c0 2024-03-26T14:01:50.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.354+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f01841a0c20 con 0x7f016c0a2910 2024-03-26T14:01:50.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.370+0000 7f0182000700 1 -- 172.21.15.137:0/397760528 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f01841a0c20 con 0x7f016c0a2910 2024-03-26T14:01:50.368 INFO:teuthology.orchestra.run.smithi137.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-26T14:01:50.368 INFO:teuthology.orchestra.run.smithi137.stdout:mds.all 12/12 81s ago 3m smithi137=a;smithi137=d;smithi137=g;smithi137=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-03-26T14:01:50.368 INFO:teuthology.orchestra.run.smithi137.stdout:mgr 3/3 81s ago 10m smithi137=x;smithi159=y;smithi179=z;count:3 2024-03-26T14:01:50.369 INFO:teuthology.orchestra.run.smithi137.stdout:mon 3/3 81s ago 11m smithi137:172.21.15.137=a;smithi159:172.21.15.159=b;smithi179:172.21.15.179=c;count:3 2024-03-26T14:01:50.369 INFO:teuthology.orchestra.run.smithi137.stdout:osd 12 81s ago - 2024-03-26T14:01:50.369 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:50 smithi137 ceph-mon[31539]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:50.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f016c0a2910 msgr2=0x7f016c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:50.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f016c0a2910 0x7f016c0a4dd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0178006010 tx=0x7f0178009f30 comp rx=0 tx=0).stop 2024-03-26T14:01:50.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 msgr2=0x7f01841989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:50.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 0x7f01841989b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f017000d980 tx=0x7f017000dd40 comp rx=0 tx=0).stop 2024-03-26T14:01:50.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 shutdown_connections 2024-03-26T14:01:50.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f016c0a2910 0x7f016c0a4dd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0184102c60 0x7f018419f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01841010c0 0x7f0184198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 --2- 172.21.15.137:0/397760528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01841022c0 0x7f01841989b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:50.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 >> 172.21.15.137:0/397760528 conn(0x7f01840fc660 msgr2=0x7f0184069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:50.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.374+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 shutdown_connections 2024-03-26T14:01:50.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:50.375+0000 7f018be00700 1 -- 172.21.15.137:0/397760528 wait complete. 2024-03-26T14:01:50.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:50 smithi179 ceph-mon[35472]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:50.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:50 smithi159 ceph-mon[35501]: pgmap v442: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:51.174 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-26T14:01:51.628 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:52.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:52 smithi137 ceph-mon[31539]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:52.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:52 smithi137 ceph-mon[31539]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:52.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:52 smithi179 ceph-mon[35472]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:52.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:52 smithi179 ceph-mon[35472]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:52.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:52 smithi159 ceph-mon[35501]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:52.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:52 smithi159 ceph-mon[35501]: pgmap v443: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 -- 172.21.15.137:0/1096741921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 msgr2=0x7fbac010c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 --2- 172.21.15.137:0/1096741921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac010c760 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbab800b0d0 tx=0x7fbab800b3e0 comp rx=0 tx=0).stop 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 -- 172.21.15.137:0/1096741921 shutdown_connections 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 --2- 172.21.15.137:0/1096741921 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac010c760 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 --2- 172.21.15.137:0/1096741921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0104310 0x7fbac0104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 --2- 172.21.15.137:0/1096741921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0105510 0x7fbac0105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 -- 172.21.15.137:0/1096741921 >> 172.21.15.137:0/1096741921 conn(0x7fbac00ff8b0 msgr2=0x7fbac0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 -- 172.21.15.137:0/1096741921 shutdown_connections 2024-03-26T14:01:53.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.867+0000 7fbabf200700 1 -- 172.21.15.137:0/1096741921 wait complete. 2024-03-26T14:01:53.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 Processor -- start 2024-03-26T14:01:53.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 -- start start 2024-03-26T14:01:53.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0104310 0x7fbac019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:53.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0105510 0x7fbac019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:53.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:53.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac010fe00 con 0x7fbac0105eb0 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac010fc80 con 0x7fbac0104310 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.869+0000 7fbabf200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac010ff80 con 0x7fbac0105510 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac01a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50770/0 (socket says 172.21.15.137:50770) 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 -- 172.21.15.137:0/152029044 learned_addr learned my addr 172.21.15.137:0/152029044 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabde00700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0104310 0x7fbac019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbab7e00700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0105510 0x7fbac019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 -- 172.21.15.137:0/152029044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0105510 msgr2=0x7fbac019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0105510 0x7fbac019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 -- 172.21.15.137:0/152029044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0104310 msgr2=0x7fbac019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0104310 0x7fbac019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabe800700 1 -- 172.21.15.137:0/152029044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac01a2050 con 0x7fbac0105eb0 2024-03-26T14:01:53.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.870+0000 7fbabde00700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0104310 0x7fbac019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:01:53.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.871+0000 7fbabe800700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac01a19b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbab8004cb0 tx=0x7fbab8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:53.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.871+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab801f030 con 0x7fbac0105eb0 2024-03-26T14:01:53.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.871+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01a22e0 con 0x7fbac0105eb0 2024-03-26T14:01:53.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.871+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbab8007ac0 con 0x7fbac0105eb0 2024-03-26T14:01:53.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.871+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab801b690 con 0x7fbac0105eb0 2024-03-26T14:01:53.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.871+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac01a2730 con 0x7fbac0105eb0 2024-03-26T14:01:53.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.872+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac0068e50 con 0x7fbac0105eb0 2024-03-26T14:01:53.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.874+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbab8022030 con 0x7fbac0105eb0 2024-03-26T14:01:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.875+0000 7fbab6a00700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa40a2890 0x7fbaa40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.876+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbab800f020 con 0x7fbac0105eb0 2024-03-26T14:01:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.876+0000 7fbabde00700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa40a2890 0x7fbaa40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:53.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.876+0000 7fbabde00700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa40a2890 0x7fbaa40a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005950 tx=0x7fbab0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:53.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:53.880+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbab8029020 con 0x7fbac0105eb0 2024-03-26T14:01:54.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.111+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbac00625d0 con 0x7fbaa40a2890 2024-03-26T14:01:54.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.114+0000 7fbab6a00700 1 -- 172.21.15.137:0/152029044 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fbac00625d0 con 0x7fbaa40a2890 2024-03-26T14:01:54.115 INFO:teuthology.orchestra.run.smithi137.stdout:HOST ADDR LABELS STATUS 2024-03-26T14:01:54.115 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137 172.21.15.137 2024-03-26T14:01:54.115 INFO:teuthology.orchestra.run.smithi137.stdout:smithi159 172.21.15.159 2024-03-26T14:01:54.115 INFO:teuthology.orchestra.run.smithi137.stdout:smithi179 172.21.15.179 2024-03-26T14:01:54.116 INFO:teuthology.orchestra.run.smithi137.stdout:3 hosts in cluster 2024-03-26T14:01:54.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.117+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa40a2890 msgr2=0x7fbaa40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:54.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.117+0000 7fbabf200700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa40a2890 0x7fbaa40a4d50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005950 tx=0x7fbab0005f50 comp rx=0 tx=0).stop 2024-03-26T14:01:54.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.117+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 msgr2=0x7fbac01a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:54.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.117+0000 7fbabf200700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac01a19b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbab8004cb0 tx=0x7fbab8004d90 comp rx=0 tx=0).stop 2024-03-26T14:01:54.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 shutdown_connections 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa40a2890 0x7fbaa40a4d50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0105eb0 0x7fbac01a19b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0104310 0x7fbac019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 --2- 172.21.15.137:0/152029044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0105510 0x7fbac019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 >> 172.21.15.137:0/152029044 conn(0x7fbac00ff8b0 msgr2=0x7fbac0108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 shutdown_connections 2024-03-26T14:01:54.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:54.118+0000 7fbabf200700 1 -- 172.21.15.137:0/152029044 wait complete. 2024-03-26T14:01:54.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:54 smithi137 ceph-mon[31539]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:54.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:54 smithi179 ceph-mon[35472]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:54 smithi159 ceph-mon[35501]: pgmap v444: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:01:55.014 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-26T14:01:55.464 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:55.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:55 smithi137 ceph-mon[31539]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:55.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:55 smithi179 ceph-mon[35472]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:55.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:55 smithi159 ceph-mon[35501]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:56.488 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:56 smithi137 ceph-mon[31539]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:56.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:56 smithi179 ceph-mon[35472]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:56.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:56 smithi159 ceph-mon[35501]: pgmap v445: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:01:57.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.801+0000 7ff0d7200700 1 -- 172.21.15.137:0/2335344009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0105570 msgr2=0x7ff0d0107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:57.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.801+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2335344009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0105570 0x7ff0d0107960 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009990 tx=0x7ff0b8009ca0 comp rx=0 tx=0).stop 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 -- 172.21.15.137:0/2335344009 shutdown_connections 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2335344009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d0107ea0 0x7ff0d010a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2335344009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d0102c40 0x7ff0d0105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2335344009 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0105570 0x7ff0d0107960 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 -- 172.21.15.137:0/2335344009 >> 172.21.15.137:0/2335344009 conn(0x7ff0d00fc650 msgr2=0x7ff0d00fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 -- 172.21.15.137:0/2335344009 shutdown_connections 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.802+0000 7ff0d7200700 1 -- 172.21.15.137:0/2335344009 wait complete. 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.803+0000 7ff0d7200700 1 Processor -- start 2024-03-26T14:01:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.803+0000 7ff0d7200700 1 -- start start 2024-03-26T14:01:57.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d7200700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d0102c40 0x7ff0d0102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:57.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d7200700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d0105570 0x7ff0d0100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:57.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d7200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 0x7ff0d0101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:57.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d7200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d010dc40 con 0x7ff0d0107ea0 2024-03-26T14:01:57.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d7200700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d010dac0 con 0x7ff0d0102c40 2024-03-26T14:01:57.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d7200700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d010ddc0 con 0x7ff0d0105570 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d5200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 0x7ff0d0101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d5200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 0x7ff0d0101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50800/0 (socket says 172.21.15.137:50800) 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0d5200700 1 -- 172.21.15.137:0/2196029118 learned_addr learned my addr 172.21.15.137:0/2196029118 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0cfe00700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d0102c40 0x7ff0d0102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.804+0000 7ff0c7400700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d0105570 0x7ff0d0100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d5200700 1 -- 172.21.15.137:0/2196029118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d0105570 msgr2=0x7ff0d0100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:57.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d5200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d0105570 0x7ff0d0100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d5200700 1 -- 172.21.15.137:0/2196029118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d0102c40 msgr2=0x7ff0d0102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d5200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d0102c40 0x7ff0d0102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d5200700 1 -- 172.21.15.137:0/2196029118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0101a00 con 0x7ff0d0107ea0 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d5200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 0x7ff0d0101130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8007d40 tx=0x7ff0c800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8016020 con 0x7ff0d0107ea0 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff0c800ad40 con 0x7ff0d0107ea0 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d01a23e0 con 0x7ff0d0107ea0 2024-03-26T14:01:57.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.805+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8011680 con 0x7ff0d0107ea0 2024-03-26T14:01:57.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.806+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d01a28f0 con 0x7ff0d0107ea0 2024-03-26T14:01:57.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.807+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d0068e50 con 0x7ff0d0107ea0 2024-03-26T14:01:57.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.809+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff0c8019020 con 0x7ff0d0107ea0 2024-03-26T14:01:57.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.811+0000 7ff0cd600700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff0b40a2910 0x7ff0b40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:01:57.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.811+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff0c80cf310 con 0x7ff0d0107ea0 2024-03-26T14:01:57.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.811+0000 7ff0cfe00700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff0b40a2910 0x7ff0b40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:01:57.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.812+0000 7ff0cfe00700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff0b40a2910 0x7ff0b40a4dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005950 tx=0x7ff0c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:01:57.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:57.814+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff0c8020020 con 0x7ff0d0107ea0 2024-03-26T14:01:58.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.049+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0d00625d0 con 0x7ff0b40a2910 2024-03-26T14:01:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.051+0000 7ff0cd600700 1 -- 172.21.15.137:0/2196029118 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7ff0d00625d0 con 0x7ff0b40a2910 2024-03-26T14:01:58.049 INFO:teuthology.orchestra.run.smithi137.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-26T14:01:58.049 INFO:teuthology.orchestra.run.smithi137.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-26T14:01:58.049 INFO:teuthology.orchestra.run.smithi137.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-26T14:01:58.049 INFO:teuthology.orchestra.run.smithi137.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.054+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff0b40a2910 msgr2=0x7ff0b40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.054+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff0b40a2910 0x7ff0b40a4dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0005950 tx=0x7ff0c00058e0 comp rx=0 tx=0).stop 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.054+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 msgr2=0x7ff0d0101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.054+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 0x7ff0d0101130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8007d40 tx=0x7ff0c800a340 comp rx=0 tx=0).stop 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 shutdown_connections 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff0b40a2910 0x7ff0b40a4dd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff0d0107ea0 0x7ff0d0101130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d0102c40 0x7ff0d0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 --2- 172.21.15.137:0/2196029118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d0105570 0x7ff0d0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:01:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 >> 172.21.15.137:0/2196029118 conn(0x7ff0d00fc650 msgr2=0x7ff0d01035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:01:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 shutdown_connections 2024-03-26T14:01:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:01:58.055+0000 7ff0d7200700 1 -- 172.21.15.137:0/2196029118 wait complete. 2024-03-26T14:01:58.429 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:58 smithi137 ceph-mon[31539]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:58.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:58 smithi179 ceph-mon[35472]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:58.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:58 smithi159 ceph-mon[35501]: pgmap v446: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:01:58.843 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-26T14:01:58.853 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T14:01:58.853 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-26T14:01:59.321 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:01:59.509 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:01:59 smithi137 ceph-mon[31539]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:59.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:01:59 smithi179 ceph-mon[35472]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:01:59.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:01:59 smithi159 ceph-mon[35501]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:02:00.519 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:00 smithi137 ceph-mon[31539]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:00.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:00 smithi179 ceph-mon[35472]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:00.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:00 smithi159 ceph-mon[35501]: pgmap v447: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:01.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 -- 172.21.15.137:0/577981127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105510 msgr2=0x7f76dc105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 --2- 172.21.15.137:0/577981127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105510 0x7f76dc105970 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76c40099c0 tx=0x7f76c4009cd0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 -- 172.21.15.137:0/577981127 shutdown_connections 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 --2- 172.21.15.137:0/577981127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76dc105eb0 0x7f76dc10c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 --2- 172.21.15.137:0/577981127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76dc104310 0x7f76dc104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 --2- 172.21.15.137:0/577981127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105510 0x7f76dc105970 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 -- 172.21.15.137:0/577981127 >> 172.21.15.137:0/577981127 conn(0x7f76dc0ff8b0 msgr2=0x7f76dc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 -- 172.21.15.137:0/577981127 shutdown_connections 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.632+0000 7f76e0e00700 1 -- 172.21.15.137:0/577981127 wait complete. 2024-03-26T14:02:01.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.633+0000 7f76e0e00700 1 Processor -- start 2024-03-26T14:02:01.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.633+0000 7f76e0e00700 1 -- start start 2024-03-26T14:02:01.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.633+0000 7f76e0e00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76dc104310 0x7f76dc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:01.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.633+0000 7f76e0e00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76dc105510 0x7f76dc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:01.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76e0e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 0x7f76dc1a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:01.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76e0e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc10fe00 con 0x7f76dc105eb0 2024-03-26T14:02:01.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76e0e00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc10fc80 con 0x7f76dc105510 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76e0e00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc10ff80 con 0x7f76dc104310 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 0x7f76dc1a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 0x7f76dc1a19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47512/0 (socket says 172.21.15.137:47512) 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 -- 172.21.15.137:0/2434022823 learned_addr learned my addr 172.21.15.137:0/2434022823 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76db400700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76dc104310 0x7f76dc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76d2a00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76dc105510 0x7f76dc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 -- 172.21.15.137:0/2434022823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76dc104310 msgr2=0x7f76dc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76dc104310 0x7f76dc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 -- 172.21.15.137:0/2434022823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76dc105510 msgr2=0x7f76dc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76dc105510 0x7f76dc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.634+0000 7f76dbe00700 1 -- 172.21.15.137:0/2434022823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc1a2060 con 0x7f76dc105eb0 2024-03-26T14:02:01.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.635+0000 7f76dbe00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 0x7f76dc1a19c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76d400d9b0 tx=0x7f76d400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:01.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.635+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4017070 con 0x7f76dc105eb0 2024-03-26T14:02:01.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.635+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76d4009df0 con 0x7f76dc105eb0 2024-03-26T14:02:01.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.635+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc1a2350 con 0x7f76dc105eb0 2024-03-26T14:02:01.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.635+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d40053a0 con 0x7f76dc105eb0 2024-03-26T14:02:01.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.635+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc1a28e0 con 0x7f76dc105eb0 2024-03-26T14:02:01.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.636+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc068e50 con 0x7f76dc105eb0 2024-03-26T14:02:01.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.638+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f76d4004750 con 0x7f76dc105eb0 2024-03-26T14:02:01.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.640+0000 7f76d8c00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76c00a2910 0x7f76c00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:01.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.640+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f76d40cec30 con 0x7f76dc105eb0 2024-03-26T14:02:01.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.640+0000 7f76db400700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76c00a2910 0x7f76c00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:01.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.641+0000 7f76db400700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76c00a2910 0x7f76c00a4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f76dc105370 tx=0x7f76cc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:01.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.645+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f76d4020020 con 0x7f76dc105eb0 2024-03-26T14:02:01.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.912+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76dc062da0 con 0x7f76dc105eb0 2024-03-26T14:02:01.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.913+0000 7f76d8c00700 1 -- 172.21.15.137:0/2434022823 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+3004 (secure 0 0 0) 0x7f76d409b8c0 con 0x7f76dc105eb0 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:e23 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:legacy client fscid: 1 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:Filesystem 'cephfs' (1) 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:fs_name cephfs 2024-03-26T14:02:01.912 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 23 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-03-26T13:59:22.728231+0000 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-03-26T13:59:50.245808+0000 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:tableserver 0 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:root 0 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:session_timeout 300 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:session_autoclose 300 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:max_file_size 1099511627776 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:max_xattr_size 65536 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:required_client_features {} 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure 0 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:last_failure_osd_epoch 0 2024-03-26T14:02:01.913 INFO:teuthology.orchestra.run.smithi137.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:max_mds 3 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:in 0,1,2 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:up {0=34496,1=34451,2=34424} 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:failed 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:damaged 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:stopped 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:data_pools [3,4] 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:metadata_pool 2 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:inline_data disabled 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:balancer 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:bal_rank_mask -1 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:standby_count_wanted 0 2024-03-26T14:02:01.914 INFO:teuthology.orchestra.run.smithi137.stdout:qdb_cluster leader: 34496 members: 34424,34451,34496 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.b{0:34496} state up:active seq 3 addr [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.k{0:34394} state up:standby-replay seq 1 addr [v2:172.21.15.159:6832/1469472888,v1:172.21.15.159:6833/1469472888] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.h{1:34451} state up:active seq 13 addr [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.i{1:24481} state up:standby-replay seq 1 addr [v2:172.21.15.179:6838/2441458416,v1:172.21.15.179:6839/2441458416] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.e{2:34424} state up:active seq 17 addr [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.c{2:24460} state up:standby-replay seq 1 addr [v2:172.21.15.179:6836/7146430,v1:172.21.15.179:6837/7146430] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:Standby daemons: 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.137:6834/4090447897,v1:172.21.15.137:6835/4090447897] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.137:6836/4057650912,v1:172.21.15.137:6837/4057650912] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.915 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.g{-1:14643} state up:standby seq 1 addr [v2:172.21.15.137:6838/2542329898,v1:172.21.15.137:6839/2542329898] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.a{-1:14655} state up:standby seq 1 addr [v2:172.21.15.137:6840/2350775776,v1:172.21.15.137:6841/2350775776] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.l{-1:24391} state up:standby seq 1 addr [v2:172.21.15.179:6832/2724223163,v1:172.21.15.179:6833/2724223163] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.916 INFO:teuthology.orchestra.run.smithi137.stdout:[mds.f{-1:24442} state up:standby seq 1 addr [v2:172.21.15.179:6834/2045604256,v1:172.21.15.179:6835/2045604256] compat {c=[1],r=[1],i=[fff]}] 2024-03-26T14:02:01.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76c00a2910 msgr2=0x7f76c00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76c00a2910 0x7f76c00a4dd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f76dc105370 tx=0x7f76cc009040 comp rx=0 tx=0).stop 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 msgr2=0x7f76dc1a19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 0x7f76dc1a19c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76d400d9b0 tx=0x7f76d400dd70 comp rx=0 tx=0).stop 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 shutdown_connections 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f76c00a2910 0x7f76c00a4dd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f76dc105eb0 0x7f76dc1a19c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76dc104310 0x7f76dc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 --2- 172.21.15.137:0/2434022823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76dc105510 0x7f76dc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 >> 172.21.15.137:0/2434022823 conn(0x7f76dc0ff8b0 msgr2=0x7f76dc108810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 shutdown_connections 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:01.916+0000 7f76e0e00700 1 -- 172.21.15.137:0/2434022823 wait complete. 2024-03-26T14:02:01.917 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T14:02:02.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:02 smithi137 ceph-mon[31539]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:02.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:02 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2434022823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T14:02:02.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:02 smithi179 ceph-mon[35472]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:02.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:02 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2434022823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T14:02:02.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:02 smithi159 ceph-mon[35501]: pgmap v448: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:02.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:02 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2434022823' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-26T14:02:02.885 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-26T14:02:03.339 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:02:04.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:04 smithi137 ceph-mon[31539]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:04.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:04 smithi179 ceph-mon[35472]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:04.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:04 smithi159 ceph-mon[35501]: pgmap v449: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 -- 172.21.15.137:0/2106470191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084108800 msgr2=0x7f7084108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 --2- 172.21.15.137:0/2106470191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084108800 0x7f7084108c20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f70740099c0 tx=0x7f7074009cd0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 -- 172.21.15.137:0/2106470191 shutdown_connections 2024-03-26T14:02:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 --2- 172.21.15.137:0/2106470191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f708410a3a0 0x7f7084110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 --2- 172.21.15.137:0/2106470191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084108800 0x7f7084108c20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 --2- 172.21.15.137:0/2106470191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7084109a00 0x7f7084109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 -- 172.21.15.137:0/2106470191 >> 172.21.15.137:0/2106470191 conn(0x7f7084077810 msgr2=0x7f7084079c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 -- 172.21.15.137:0/2106470191 shutdown_connections 2024-03-26T14:02:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.670+0000 7f7089c00700 1 -- 172.21.15.137:0/2106470191 wait complete. 2024-03-26T14:02:05.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.671+0000 7f7089c00700 1 Processor -- start 2024-03-26T14:02:05.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 -- start start 2024-03-26T14:02:05.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7084108800 0x7f708419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 0x7f708419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f708410a3a0 0x7f70841a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084114290 con 0x7f7084109a00 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084114110 con 0x7f7084108800 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7089c00700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084114410 con 0x7f708410a3a0 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.672+0000 7f7082000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 0x7f708419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 0x7f708419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47526/0 (socket says 172.21.15.137:47526) 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 -- 172.21.15.137:0/3932378106 learned_addr learned my addr 172.21.15.137:0/3932378106 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082a00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7084108800 0x7f708419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7083400700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f708410a3a0 0x7f70841a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 -- 172.21.15.137:0/3932378106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f708410a3a0 msgr2=0x7f70841a5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f708410a3a0 0x7f70841a5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 -- 172.21.15.137:0/3932378106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7084108800 msgr2=0x7f708419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7084108800 0x7f708419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 -- 172.21.15.137:0/3932378106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70841a65c0 con 0x7f7084109a00 2024-03-26T14:02:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.673+0000 7f7082000700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 0x7f708419eea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f707000d9b0 tx=0x7f707000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.674+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070017070 con 0x7f7084109a00 2024-03-26T14:02:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.674+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7070004830 con 0x7f7084109a00 2024-03-26T14:02:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.674+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70841a68b0 con 0x7f7084109a00 2024-03-26T14:02:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.674+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70700053a0 con 0x7f7084109a00 2024-03-26T14:02:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.674+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70841a6df0 con 0x7f7084109a00 2024-03-26T14:02:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.675+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7084068e50 con 0x7f7084109a00 2024-03-26T14:02:05.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.677+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f70700049e0 con 0x7f7084109a00 2024-03-26T14:02:05.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.678+0000 7f707b400700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70680a2910 0x7f70680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:05.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.679+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f70700ced50 con 0x7f7084109a00 2024-03-26T14:02:05.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.679+0000 7f7082a00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70680a2910 0x7f70680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:05.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.679+0000 7f7082a00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70680a2910 0x7f70680a4dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7074000c00 tx=0x7f7074005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.684+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7070020020 con 0x7f7084109a00 2024-03-26T14:02:05.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.910+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f7084062da0 con 0x7f7084109a00 2024-03-26T14:02:05.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.911+0000 7f707b400700 1 -- 172.21.15.137:0/3932378106 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) v1 ==== 56+0+5739 (secure 0 0 0) 0x7f707009b9e0 con 0x7f7084109a00 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:epoch 94 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:created 2024-03-26T13:48:46.854612+0000 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:modified 2024-03-26T14:00:55.191525+0000 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:crush_version 27 2024-03-26T14:02:05.911 INFO:teuthology.orchestra.run.smithi137.stdout:full_ratio 0.95 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:backfillfull_ratio 0.9 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:nearfull_ratio 0.85 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:require_min_compat_client luminous 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:min_compat_client luminous 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:require_osd_release squid 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:stretch_mode_enabled false 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:max_osd 12 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6802/4170736921,v1:172.21.15.137:6803/4170736921] [v2:172.21.15.137:6804/4170736921,v1:172.21.15.137:6805/4170736921] exists,up 0f6b26e4-ccfd-4457-91b0-240679242c9e 2024-03-26T14:02:05.912 INFO:teuthology.orchestra.run.smithi137.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/3508415474,v1:172.21.15.159:6801/3508415474] [v2:172.21.15.159:6802/3508415474,v1:172.21.15.159:6803/3508415474] exists,up 40fd05e9-e41a-4c27-9f67-43693dae21d8 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.2 up in weight 1 up_from 18 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/395733293,v1:172.21.15.179:6801/395733293] [v2:172.21.15.179:6802/395733293,v1:172.21.15.179:6803/395733293] exists,up 76a74368-6c04-4a19-b56d-432b5145e385 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6810/3541368776,v1:172.21.15.137:6811/3541368776] [v2:172.21.15.137:6812/3541368776,v1:172.21.15.137:6813/3541368776] exists,up ba6d4b45-933b-4346-879f-ead75f3faf3b 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.4 up in weight 1 up_from 34 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/2050135218,v1:172.21.15.159:6809/2050135218] [v2:172.21.15.159:6810/2050135218,v1:172.21.15.159:6811/2050135218] exists,up ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/3906205051,v1:172.21.15.179:6809/3906205051] [v2:172.21.15.179:6810/3906205051,v1:172.21.15.179:6811/3906205051] exists,up bdbb0679-03d9-4423-873d-2c9c968896ac 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6818/102905149,v1:172.21.15.137:6819/102905149] [v2:172.21.15.137:6820/102905149,v1:172.21.15.137:6821/102905149] exists,up d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.7 up in weight 1 up_from 52 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/1788160471,v1:172.21.15.159:6817/1788160471] [v2:172.21.15.159:6818/1788160471,v1:172.21.15.159:6819/1788160471] exists,up 54ae5a2e-89d5-4a31-801f-0d04c514f63c 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/4118554699,v1:172.21.15.179:6817/4118554699] [v2:172.21.15.179:6818/4118554699,v1:172.21.15.179:6819/4118554699] exists,up 54d0b6d1-2751-465f-89fb-d866d56a94e1 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.9 up in weight 1 up_from 68 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6826/3720717247,v1:172.21.15.137:6827/3720717247] [v2:172.21.15.137:6828/3720717247,v1:172.21.15.137:6829/3720717247] exists,up 86070f8d-a3f6-4a0f-9936-8c52710b2e85 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/2917781375,v1:172.21.15.159:6825/2917781375] [v2:172.21.15.159:6826/2917781375,v1:172.21.15.159:6827/2917781375] exists,up 95b5ff77-4547-400f-96ef-142faa248b14 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:osd.11 up in weight 1 up_from 81 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/1258425145,v1:172.21.15.179:6825/1258425145] [v2:172.21.15.179:6826/1258425145,v1:172.21.15.179:6827/1258425145] exists,up cefee215-43f2-4ed3-adbc-b2784f7cbddc 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 2.0 [8,1] 2024-03-26T14:02:05.913 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 2.4 [2,11] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 2.13 [2,11] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.e [4,6] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.17 [4,8,5,6] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.18 [3,9] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.1a [5,7] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.2c [4,9] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.31 [4,8] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 3.3b [3,9] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 4.c [9,10] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 4.f [9,8] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 4.1c [9,8] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:pg_upmap_items 4.1d [5,8] 2024-03-26T14:02:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6801/1621047019 expires 2024-03-27T13:49:52.910355+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6800/1621047019 expires 2024-03-27T13:49:52.910355+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/460394311 expires 2024-03-27T13:49:52.910355+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/1426358112 expires 2024-03-27T13:49:19.001493+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/2012111099 expires 2024-03-27T13:49:52.910355+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/1602182932 expires 2024-03-27T13:49:19.001493+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6800/1955125876 expires 2024-03-27T13:49:19.001493+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/3592180121 expires 2024-03-27T13:49:19.001493+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:6801/1955125876 expires 2024-03-27T13:49:19.001493+0000 2024-03-26T14:02:05.915 INFO:teuthology.orchestra.run.smithi137.stdout:blocklist 172.21.15.137:0/487629871 expires 2024-03-27T13:49:52.910355+0000 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70680a2910 msgr2=0x7f70680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70680a2910 0x7f70680a4dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7074000c00 tx=0x7f7074005fd0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 msgr2=0x7f708419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 0x7f708419eea0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f707000d9b0 tx=0x7f707000dd70 comp rx=0 tx=0).stop 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 shutdown_connections 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70680a2910 0x7f70680a4dd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f708410a3a0 0x7f70841a5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7084108800 0x7f708419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.914+0000 7f7089c00700 1 --2- 172.21.15.137:0/3932378106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7084109a00 0x7f708419eea0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.915+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 >> 172.21.15.137:0/3932378106 conn(0x7f7084077810 msgr2=0x7f708410cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.915+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 shutdown_connections 2024-03-26T14:02:05.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:05.915+0000 7f7089c00700 1 -- 172.21.15.137:0/3932378106 wait complete. 2024-03-26T14:02:06.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:06 smithi137 ceph-mon[31539]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:06.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:06 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3932378106' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T14:02:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:06 smithi179 ceph-mon[35472]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:06 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3932378106' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T14:02:06.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:06 smithi159 ceph-mon[35501]: pgmap v450: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:06.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3932378106' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-26T14:02:06.702 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-26T14:02:06.714 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:06.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 -- 172.21.15.137:0/3470417993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8104ad0 msgr2=0x7f84d8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:06.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 --2- 172.21.15.137:0/3470417993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8104ad0 0x7f84d8104f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009860 tx=0x7f84c402bb70 comp rx=0 tx=0).stop 2024-03-26T14:02:06.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 -- 172.21.15.137:0/3470417993 shutdown_connections 2024-03-26T14:02:06.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 --2- 172.21.15.137:0/3470417993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8104ad0 0x7f84d8104f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:06.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 --2- 172.21.15.137:0/3470417993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8105490 0x7f84d810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:06.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 --2- 172.21.15.137:0/3470417993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:06.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 -- 172.21.15.137:0/3470417993 >> 172.21.15.137:0/3470417993 conn(0x7f84d80fee50 msgr2=0x7f84d8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:06.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 -- 172.21.15.137:0/3470417993 shutdown_connections 2024-03-26T14:02:06.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.936+0000 7f84e0400640 1 -- 172.21.15.137:0/3470417993 wait complete. 2024-03-26T14:02:06.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.937+0000 7f84e0400640 1 Processor -- start 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.937+0000 7f84e0400640 1 -- start start 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84e0400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84e0400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8104ad0 0x7f84d819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84e0400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8105490 0x7f84d81a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84e0400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d810f3f0 con 0x7f84d81038e0 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84e0400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d810f270 con 0x7f84d8105490 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84e0400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d810f570 con 0x7f84d8104ad0 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47556/0 (socket says 172.21.15.137:47556) 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 -- 172.21.15.137:0/2572241122 learned_addr learned my addr 172.21.15.137:0/2572241122 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:06.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84de400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8105490 0x7f84d81a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dd000640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8104ad0 0x7f84d819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 -- 172.21.15.137:0/2572241122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8104ad0 msgr2=0x7f84d819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8104ad0 0x7f84d819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 -- 172.21.15.137:0/2572241122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8105490 msgr2=0x7f84d81a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8105490 0x7f84d81a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.938+0000 7f84dda00640 1 -- 172.21.15.137:0/2572241122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d81a1740 con 0x7f84d81038e0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.939+0000 7f84dda00640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8199b20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9a0 tx=0x7f84cc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.939+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc00cd80 con 0x7f84d81038e0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.939+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84cc0108e0 con 0x7f84d81038e0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.939+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d81a1a30 con 0x7f84d81038e0 2024-03-26T14:02:06.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.939+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc018950 con 0x7f84d81038e0 2024-03-26T14:02:06.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.939+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d81a1fa0 con 0x7f84d81038e0 2024-03-26T14:02:06.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.940+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d810acd0 con 0x7f84d81038e0 2024-03-26T14:02:06.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.942+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f84cc010a90 con 0x7f84d81038e0 2024-03-26T14:02:06.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.943+0000 7f84d6000640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84c00a2880 0x7f84c00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:06.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.944+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84cc0cf1d0 con 0x7f84d81038e0 2024-03-26T14:02:06.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.944+0000 7f84dd000640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84c00a2880 0x7f84c00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:06.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.945+0000 7f84dd000640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84c00a2880 0x7f84c00a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f84c4000c00 tx=0x7f84c4034980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:06.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:06.948+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f84cc053740 con 0x7f84d81038e0 2024-03-26T14:02:07.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.153+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84d802d070 con 0x7f84d81038e0 2024-03-26T14:02:07.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.157+0000 7f84d6000640 1 -- 172.21.15.137:0/2572241122 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f84cc09bea0 con 0x7f84d81038e0 2024-03-26T14:02:07.156 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:07.156 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:07.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84c00a2880 msgr2=0x7f84c00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84c00a2880 0x7f84c00a4d40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f84c4000c00 tx=0x7f84c4034980 comp rx=0 tx=0).stop 2024-03-26T14:02:07.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 msgr2=0x7f84d8199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8199b20 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e9a0 tx=0x7f84cc00ee60 comp rx=0 tx=0).stop 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 shutdown_connections 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84c00a2880 0x7f84c00a4d40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d8105490 0x7f84d81a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8104ad0 0x7f84d819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 --2- 172.21.15.137:0/2572241122 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d81038e0 0x7f84d8199b20 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.161+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 >> 172.21.15.137:0/2572241122 conn(0x7f84d80fee50 msgr2=0x7f84d8107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.162+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 shutdown_connections 2024-03-26T14:02:07.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.162+0000 7f84e0400640 1 -- 172.21.15.137:0/2572241122 wait complete. 2024-03-26T14:02:07.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:02:07.180 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 -- 172.21.15.137:0/2675872513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420102150 msgr2=0x7fd4201089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 --2- 172.21.15.137:0/2675872513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420102150 0x7fd4201089e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd418009900 tx=0x7fd41802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 -- 172.21.15.137:0/2675872513 shutdown_connections 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 --2- 172.21.15.137:0/2675872513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101790 0x7fd420101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 --2- 172.21.15.137:0/2675872513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd420102150 0x7fd4201089e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 --2- 172.21.15.137:0/2675872513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4201005a0 0x7fd4201009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 -- 172.21.15.137:0/2675872513 >> 172.21.15.137:0/2675872513 conn(0x7fd4200fbb30 msgr2=0x7fd4200fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:07.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 -- 172.21.15.137:0/2675872513 shutdown_connections 2024-03-26T14:02:07.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.403+0000 7fd425400640 1 -- 172.21.15.137:0/2675872513 wait complete. 2024-03-26T14:02:07.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.404+0000 7fd425400640 1 Processor -- start 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.404+0000 7fd425400640 1 -- start start 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.404+0000 7fd425400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 0x7fd420199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.404+0000 7fd425400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101790 0x7fd420199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd425400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd420102150 0x7fd4201a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd425400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010c0e0 con 0x7fd4201005a0 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd425400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010bf60 con 0x7fd420101790 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd425400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010c260 con 0x7fd420102150 2024-03-26T14:02:07.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 0x7fd420199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 0x7fd420199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47566/0 (socket says 172.21.15.137:47566) 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 -- 172.21.15.137:0/1760651318 learned_addr learned my addr 172.21.15.137:0/1760651318 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41de00640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101790 0x7fd420199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41f200640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd420102150 0x7fd4201a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 -- 172.21.15.137:0/1760651318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd420102150 msgr2=0x7fd4201a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd420102150 0x7fd4201a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 -- 172.21.15.137:0/1760651318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101790 msgr2=0x7fd420199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101790 0x7fd420199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.405+0000 7fd41e800640 1 -- 172.21.15.137:0/1760651318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4201a16d0 con 0x7fd4201005a0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.406+0000 7fd41e800640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 0x7fd420199ab0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd41000d950 tx=0x7fd41000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.406+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410017070 con 0x7fd4201005a0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.406+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd410009c30 con 0x7fd4201005a0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.406+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000f620 con 0x7fd4201005a0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.406+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4201a19c0 con 0x7fd4201005a0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.406+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4201a1ec0 con 0x7fd4201005a0 2024-03-26T14:02:07.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.407+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4201a20f0 con 0x7fd4201005a0 2024-03-26T14:02:07.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.409+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd41001a070 con 0x7fd4201005a0 2024-03-26T14:02:07.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.411+0000 7fd417400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4000a2800 0x7fd4000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.411+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd4100ce8a0 con 0x7fd4201005a0 2024-03-26T14:02:07.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.412+0000 7fd41de00640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4000a2800 0x7fd4000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.412+0000 7fd41de00640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4000a2800 0x7fd4000a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd42019afc0 tx=0x7fd408009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:07.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.416+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd410009de0 con 0x7fd4201005a0 2024-03-26T14:02:07.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:07 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2572241122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:07.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.614+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4201a23e0 con 0x7fd4201005a0 2024-03-26T14:02:07.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.615+0000 7fd417400640 1 -- 172.21.15.137:0/1760651318 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd41009b530 con 0x7fd4201005a0 2024-03-26T14:02:07.613 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4000a2800 msgr2=0x7fd4000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4000a2800 0x7fd4000a4cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd42019afc0 tx=0x7fd408009500 comp rx=0 tx=0).stop 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 msgr2=0x7fd420199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 0x7fd420199ab0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd41000d950 tx=0x7fd41000de10 comp rx=0 tx=0).stop 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 shutdown_connections 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4000a2800 0x7fd4000a4cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd420101790 0x7fd420199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd420102150 0x7fd4201a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 --2- 172.21.15.137:0/1760651318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4201005a0 0x7fd420199ab0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.618+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 >> 172.21.15.137:0/1760651318 conn(0x7fd4200fbb30 msgr2=0x7fd4200fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:07.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.619+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 shutdown_connections 2024-03-26T14:02:07.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.619+0000 7fd425400640 1 -- 172.21.15.137:0/1760651318 wait complete. 2024-03-26T14:02:07.636 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:02:07.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:07 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2572241122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:07.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:07 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2572241122' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 -- 172.21.15.137:0/1279733657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c109560 msgr2=0x7fa73c10b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 --2- 172.21.15.137:0/1279733657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c109560 0x7fa73c10b9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa734009ca0 tx=0x7fa73402c100 comp rx=0 tx=0).stop 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 -- 172.21.15.137:0/1279733657 shutdown_connections 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 --2- 172.21.15.137:0/1279733657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c109560 0x7fa73c10b9d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 --2- 172.21.15.137:0/1279733657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0740c0 0x7fa73c074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 --2- 172.21.15.137:0/1279733657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa73c0756a0 0x7fa73c073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 -- 172.21.15.137:0/1279733657 >> 172.21.15.137:0/1279733657 conn(0x7fa73c0fcc40 msgr2=0x7fa73c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 -- 172.21.15.137:0/1279733657 shutdown_connections 2024-03-26T14:02:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.855+0000 7fa741200640 1 -- 172.21.15.137:0/1279733657 wait complete. 2024-03-26T14:02:07.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.856+0000 7fa741200640 1 Processor -- start 2024-03-26T14:02:07.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.856+0000 7fa741200640 1 -- start start 2024-03-26T14:02:07.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa741200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 0x7fa73c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa741200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0756a0 0x7fa73c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa741200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa73c109560 0x7fa73c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa741200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c10f3c0 con 0x7fa73c0740c0 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa741200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c10f240 con 0x7fa73c109560 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa741200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c10f540 con 0x7fa73c0756a0 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 0x7fa73c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 0x7fa73c199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47584/0 (socket says 172.21.15.137:47584) 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 -- 172.21.15.137:0/1861136269 learned_addr learned my addr 172.21.15.137:0/1861136269 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739400640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0756a0 0x7fa73c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa73a800640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa73c109560 0x7fa73c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 -- 172.21.15.137:0/1861136269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0756a0 msgr2=0x7fa73c199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0756a0 0x7fa73c199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 -- 172.21.15.137:0/1861136269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa73c109560 msgr2=0x7fa73c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa73c109560 0x7fa73c1a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739e00640 1 -- 172.21.15.137:0/1861136269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c1a16b0 con 0x7fa73c0740c0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.857+0000 7fa739400640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0756a0 0x7fa73c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.858+0000 7fa739e00640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 0x7fa73c199a90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00d950 tx=0x7fa72c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.858+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c017030 con 0x7fa73c0740c0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.858+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa72c004020 con 0x7fa73c0740c0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.858+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0047a0 con 0x7fa73c0740c0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.858+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c1a19a0 con 0x7fa73c0740c0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.858+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c1a1ef0 con 0x7fa73c0740c0 2024-03-26T14:02:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.859+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c1082e0 con 0x7fa73c0740c0 2024-03-26T14:02:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.861+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa72c0041d0 con 0x7fa73c0740c0 2024-03-26T14:02:07.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.864+0000 7fa732a00640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7200a2800 0x7fa7200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.864+0000 7fa739400640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7200a2800 0x7fa7200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.864+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa72c0ce940 con 0x7fa73c0740c0 2024-03-26T14:02:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.864+0000 7fa739400640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7200a2800 0x7fa7200a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa73c19afa0 tx=0x7fa728005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:07.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:07.868+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa72c09b610 con 0x7fa73c0740c0 2024-03-26T14:02:08.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.067+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa73c0622a0 con 0x7fa73c0740c0 2024-03-26T14:02:08.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.068+0000 7fa732a00640 1 -- 172.21.15.137:0/1861136269 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa72c09af10 con 0x7fa73c0740c0 2024-03-26T14:02:08.065 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7200a2800 msgr2=0x7fa7200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7200a2800 0x7fa7200a4cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa73c19afa0 tx=0x7fa728005e50 comp rx=0 tx=0).stop 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 msgr2=0x7fa73c199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 0x7fa73c199a90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00d950 tx=0x7fa72c00de10 comp rx=0 tx=0).stop 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 shutdown_connections 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa7200a2800 0x7fa7200a4cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa73c109560 0x7fa73c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa73c0756a0 0x7fa73c199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 --2- 172.21.15.137:0/1861136269 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa73c0740c0 0x7fa73c199a90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 >> 172.21.15.137:0/1861136269 conn(0x7fa73c0fcc40 msgr2=0x7fa73c1054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:08.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 shutdown_connections 2024-03-26T14:02:08.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.071+0000 7fa741200640 1 -- 172.21.15.137:0/1861136269 wait complete. 2024-03-26T14:02:08.086 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:02:08.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 -- 172.21.15.137:0/2031457684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4481038c0 msgr2=0x7fb448103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 --2- 172.21.15.137:0/2031457684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4481038c0 0x7fb448103ce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fb438009900 tx=0x7fb43802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 -- 172.21.15.137:0/2031457684 shutdown_connections 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 --2- 172.21.15.137:0/2031457684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb448104ab0 0x7fb448104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 --2- 172.21.15.137:0/2031457684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb448105470 0x7fb44810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 --2- 172.21.15.137:0/2031457684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb4481038c0 0x7fb448103ce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 -- 172.21.15.137:0/2031457684 >> 172.21.15.137:0/2031457684 conn(0x7fb4480fee50 msgr2=0x7fb448101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 -- 172.21.15.137:0/2031457684 shutdown_connections 2024-03-26T14:02:08.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.293+0000 7fb44ce00640 1 -- 172.21.15.137:0/2031457684 wait complete. 2024-03-26T14:02:08.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.294+0000 7fb44ce00640 1 Processor -- start 2024-03-26T14:02:08.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.294+0000 7fb44ce00640 1 -- start start 2024-03-26T14:02:08.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.294+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4481038c0 0x7fb448199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 0x7fb44819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb44ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb448105470 0x7fb4481a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb44ce00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44810f410 con 0x7fb448104ab0 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb44ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44810f290 con 0x7fb448105470 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb44ce00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44810f590 con 0x7fb4481038c0 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 0x7fb44819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 0x7fb44819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40534/0 (socket says 172.21.15.137:40534) 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 -- 172.21.15.137:0/619211406 learned_addr learned my addr 172.21.15.137:0/619211406 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb445e00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4481038c0 0x7fb448199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb446800640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb448105470 0x7fb4481a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 -- 172.21.15.137:0/619211406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4481038c0 msgr2=0x7fb448199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4481038c0 0x7fb448199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 -- 172.21.15.137:0/619211406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb448105470 msgr2=0x7fb4481a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb448105470 0x7fb4481a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.295+0000 7fb43d400640 1 -- 172.21.15.137:0/619211406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4481a1730 con 0x7fb448104ab0 2024-03-26T14:02:08.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.296+0000 7fb43d400640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 0x7fb44819a050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb43000d950 tx=0x7fb43000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:08.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.296+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430017030 con 0x7fb448104ab0 2024-03-26T14:02:08.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.296+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb430004020 con 0x7fb448104ab0 2024-03-26T14:02:08.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.296+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300047a0 con 0x7fb448104ab0 2024-03-26T14:02:08.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.296+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4481a1a20 con 0x7fb448104ab0 2024-03-26T14:02:08.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.296+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4481a1f20 con 0x7fb448104ab0 2024-03-26T14:02:08.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.297+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44810ad00 con 0x7fb448104ab0 2024-03-26T14:02:08.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.299+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb4300041d0 con 0x7fb448104ab0 2024-03-26T14:02:08.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.301+0000 7fb43f400640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb4280a2880 0x7fb4280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.302+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb4300ce9c0 con 0x7fb448104ab0 2024-03-26T14:02:08.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.302+0000 7fb445e00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb4280a2880 0x7fb4280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.302+0000 7fb445e00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb4280a2880 0x7fb4280a4d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb43802c240 tx=0x7fb438005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:08.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.306+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb43009b650 con 0x7fb448104ab0 2024-03-26T14:02:08.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.508+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb4480622d0 con 0x7fb448104ab0 2024-03-26T14:02:08.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.509+0000 7fb43f400640 1 -- 172.21.15.137:0/619211406 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb43009af50 con 0x7fb448104ab0 2024-03-26T14:02:08.506 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:02:08.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb4280a2880 msgr2=0x7fb4280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb4280a2880 0x7fb4280a4d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb43802c240 tx=0x7fb438005fd0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 msgr2=0x7fb44819a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 0x7fb44819a050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb43000d950 tx=0x7fb43000de10 comp rx=0 tx=0).stop 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 shutdown_connections 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb4280a2880 0x7fb4280a4d40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb448105470 0x7fb4481a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb448104ab0 0x7fb44819a050 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 --2- 172.21.15.137:0/619211406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb4481038c0 0x7fb448199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 >> 172.21.15.137:0/619211406 conn(0x7fb4480fee50 msgr2=0x7fb448107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 shutdown_connections 2024-03-26T14:02:08.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.512+0000 7fb44ce00640 1 -- 172.21.15.137:0/619211406 wait complete. 2024-03-26T14:02:08.524 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:08 smithi137 ceph-mon[31539]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:08.525 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1760651318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:08.525 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1861136269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:08.527 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:02:08.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:08 smithi179 ceph-mon[35472]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:08.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1760651318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:08.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1861136269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:08.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:08 smithi159 ceph-mon[35501]: pgmap v451: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:08.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1760651318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:08.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1861136269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 -- 172.21.15.137:0/1011572783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c8103260 msgr2=0x7f75c8109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 --2- 172.21.15.137:0/1011572783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c8103260 0x7f75c8109af0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009900 tx=0x7f75c002bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 -- 172.21.15.137:0/1011572783 shutdown_connections 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 --2- 172.21.15.137:0/1011572783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c81028a0 0x7f75c8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 --2- 172.21.15.137:0/1011572783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c8103260 0x7f75c8109af0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 --2- 172.21.15.137:0/1011572783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c81016b0 0x7f75c8101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 -- 172.21.15.137:0/1011572783 >> 172.21.15.137:0/1011572783 conn(0x7f75c80fcc20 msgr2=0x7f75c80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 -- 172.21.15.137:0/1011572783 shutdown_connections 2024-03-26T14:02:08.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.744+0000 7f75cf400640 1 -- 172.21.15.137:0/1011572783 wait complete. 2024-03-26T14:02:08.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.745+0000 7f75cf400640 1 Processor -- start 2024-03-26T14:02:08.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.745+0000 7f75cf400640 1 -- start start 2024-03-26T14:02:08.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 0x7f75c8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cf400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c81028a0 0x7f75c819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8103260 0x7f75c81a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cf400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c810d210 con 0x7f75c81016b0 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cf400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c810d090 con 0x7f75c8103260 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cf400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c810d390 con 0x7f75c81028a0 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 0x7f75c8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 0x7f75c8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40548/0 (socket says 172.21.15.137:40548) 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cca00640 1 -- 172.21.15.137:0/2386349944 learned_addr learned my addr 172.21.15.137:0/2386349944 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75cd400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8103260 0x7f75c81a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.746+0000 7f75c7e00640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c81028a0 0x7f75c819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cca00640 1 -- 172.21.15.137:0/2386349944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c81028a0 msgr2=0x7f75c819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cca00640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c81028a0 0x7f75c819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cca00640 1 -- 172.21.15.137:0/2386349944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8103260 msgr2=0x7f75c81a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cca00640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8103260 0x7f75c81a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cca00640 1 -- 172.21.15.137:0/2386349944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c81a16a0 con 0x7f75c81016b0 2024-03-26T14:02:08.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cca00640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 0x7f75c8199ad0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ea80 tx=0x7f75b800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:08.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b800ce40 con 0x7f75c81016b0 2024-03-26T14:02:08.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c81a1990 con 0x7f75c81016b0 2024-03-26T14:02:08.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75b8011920 con 0x7f75c81016b0 2024-03-26T14:02:08.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b801aaa0 con 0x7f75c81016b0 2024-03-26T14:02:08.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.747+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c81a1ee0 con 0x7f75c81016b0 2024-03-26T14:02:08.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.748+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c81a2170 con 0x7f75c81016b0 2024-03-26T14:02:08.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.750+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f75b8011ad0 con 0x7f75c81016b0 2024-03-26T14:02:08.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.751+0000 7f75c5600640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75ac0a2800 0x7f75ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:08.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.752+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f75b80cf420 con 0x7f75c81016b0 2024-03-26T14:02:08.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.752+0000 7f75c7e00640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75ac0a2800 0x7f75ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:08.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.753+0000 7f75c7e00640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75ac0a2800 0x7f75ac0a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f75c819af90 tx=0x7f75b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:08.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.756+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f75b8016030 con 0x7f75c81016b0 2024-03-26T14:02:08.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.956+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f75c81a2340 con 0x7f75c81016b0 2024-03-26T14:02:08.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.957+0000 7f75c5600640 1 -- 172.21.15.137:0/2386349944 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f75b809c0b0 con 0x7f75c81016b0 2024-03-26T14:02:08.954 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:08.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75ac0a2800 msgr2=0x7f75ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75ac0a2800 0x7f75ac0a4cc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f75c819af90 tx=0x7f75b0005f90 comp rx=0 tx=0).stop 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 msgr2=0x7f75c8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 0x7f75c8199ad0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f75b800ea80 tx=0x7f75b800ef40 comp rx=0 tx=0).stop 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 shutdown_connections 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75ac0a2800 0x7f75ac0a4cc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75c8103260 0x7f75c81a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75c81028a0 0x7f75c819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 --2- 172.21.15.137:0/2386349944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75c81016b0 0x7f75c8199ad0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 >> 172.21.15.137:0/2386349944 conn(0x7f75c80fcc20 msgr2=0x7f75c80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 shutdown_connections 2024-03-26T14:02:08.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:08.960+0000 7f75cf400640 1 -- 172.21.15.137:0/2386349944 wait complete. 2024-03-26T14:02:08.976 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 -- 172.21.15.137:0/2532426717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 msgr2=0x7fa2a410bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2532426717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a410bd20 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009ca0 tx=0x7fa29c02c100 comp rx=0 tx=0).stop 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 -- 172.21.15.137:0/2532426717 shutdown_connections 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2532426717 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a410bd20 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2532426717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a4104ad0 0x7fa2a4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2532426717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2a41038e0 0x7fa2a4103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 -- 172.21.15.137:0/2532426717 >> 172.21.15.137:0/2532426717 conn(0x7fa2a40fee50 msgr2=0x7fa2a4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 -- 172.21.15.137:0/2532426717 shutdown_connections 2024-03-26T14:02:09.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.189+0000 7fa2aa400640 1 -- 172.21.15.137:0/2532426717 wait complete. 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.190+0000 7fa2aa400640 1 Processor -- start 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.190+0000 7fa2aa400640 1 -- start start 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2aa400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a41038e0 0x7fa2a4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2aa400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2a4104ad0 0x7fa2a4199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2aa400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a41a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2aa400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a410f410 con 0x7fa2a4105490 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2aa400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a410f290 con 0x7fa2a4104ad0 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2aa400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a410f590 con 0x7fa2a41038e0 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a41a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a41a1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40562/0 (socket says 172.21.15.137:40562) 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 -- 172.21.15.137:0/2371723178 learned_addr learned my addr 172.21.15.137:0/2371723178 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa29aa00640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2a4104ad0 0x7fa2a4199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a41038e0 0x7fa2a4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 -- 172.21.15.137:0/2371723178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a41038e0 msgr2=0x7fa2a4199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a41038e0 0x7fa2a4199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 -- 172.21.15.137:0/2371723178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2a4104ad0 msgr2=0x7fa2a4199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2a4104ad0 0x7fa2a4199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.191+0000 7fa2a3e00640 1 -- 172.21.15.137:0/2371723178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a41a1670 con 0x7fa2a4105490 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.192+0000 7fa2a3e00640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a41a1000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa29c032380 tx=0x7fa29c035e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.192+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c039040 con 0x7fa2a4105490 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.192+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa29c044070 con 0x7fa2a4105490 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.192+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a41a1900 con 0x7fa2a4105490 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.192+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004670 con 0x7fa2a4105490 2024-03-26T14:02:09.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.192+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a41a1e40 con 0x7fa2a4105490 2024-03-26T14:02:09.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.193+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a410ac00 con 0x7fa2a4105490 2024-03-26T14:02:09.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.195+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa29c003970 con 0x7fa2a4105490 2024-03-26T14:02:09.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.196+0000 7fa2a0c00640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2880a2880 0x7fa2880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.196+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa29c0f1800 con 0x7fa2a4105490 2024-03-26T14:02:09.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.197+0000 7fa2a3400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2880a2880 0x7fa2880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.197+0000 7fa2a3400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2880a2880 0x7fa2880a4d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2940058c0 tx=0x7fa294005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:09.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.201+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa29c0be490 con 0x7fa2a4105490 2024-03-26T14:02:09.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.443+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2a40622a0 con 0x7fa2a4105490 2024-03-26T14:02:09.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.445+0000 7fa2a0c00640 1 -- 172.21.15.137:0/2371723178 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fa29c0098b0 con 0x7fa2a4105490 2024-03-26T14:02:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:09.443 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:50.245808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34424,34451,34496]},"id":1}]} 2024-03-26T14:02:09.443 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2880a2880 msgr2=0x7fa2880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2880a2880 0x7fa2880a4d40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2940058c0 tx=0x7fa294005850 comp rx=0 tx=0).stop 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 msgr2=0x7fa2a41a1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a41a1000 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa29c032380 tx=0x7fa29c035e20 comp rx=0 tx=0).stop 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 shutdown_connections 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa2880a2880 0x7fa2880a4d40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2a4104ad0 0x7fa2a4199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2a4105490 0x7fa2a41a1000 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 --2- 172.21.15.137:0/2371723178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa2a41038e0 0x7fa2a4199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 >> 172.21.15.137:0/2371723178 conn(0x7fa2a40fee50 msgr2=0x7fa2a4107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 shutdown_connections 2024-03-26T14:02:09.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.449+0000 7fa2aa400640 1 -- 172.21.15.137:0/2371723178 wait complete. 2024-03-26T14:02:09.467 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:09.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/619211406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:09.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2386349944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:09.543 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:09 smithi137 ceph-mon[31539]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:09.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/619211406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:09.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2386349944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:09.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:09 smithi179 ceph-mon[35472]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:09.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 -- 172.21.15.137:0/1508632924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 msgr2=0x7fb19c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 --2- 172.21.15.137:0/1508632924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c0784e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ca0 tx=0x7fb19402c100 comp rx=0 tx=0).stop 2024-03-26T14:02:09.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 -- 172.21.15.137:0/1508632924 shutdown_connections 2024-03-26T14:02:09.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 --2- 172.21.15.137:0/1508632924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb19c0772a0 0x7fb19c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 --2- 172.21.15.137:0/1508632924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c0784e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 --2- 172.21.15.137:0/1508632924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb19c076060 0x7fb19c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.679+0000 7fb1a2000640 1 -- 172.21.15.137:0/1508632924 >> 172.21.15.137:0/1508632924 conn(0x7fb19c0fee50 msgr2=0x7fb19c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:09.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.680+0000 7fb1a2000640 1 -- 172.21.15.137:0/1508632924 shutdown_connections 2024-03-26T14:02:09.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.680+0000 7fb1a2000640 1 -- 172.21.15.137:0/1508632924 wait complete. 2024-03-26T14:02:09.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.680+0000 7fb1a2000640 1 Processor -- start 2024-03-26T14:02:09.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 -- start start 2024-03-26T14:02:09.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb19c076060 0x7fb19c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb19c0772a0 0x7fb19c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c113820 con 0x7fb19c077c40 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1136a0 con 0x7fb19c076060 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb1a2000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19c1139a0 con 0x7fb19c0772a0 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb19be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb19be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c1a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40594/0 (socket says 172.21.15.137:40594) 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb19be00640 1 -- 172.21.15.137:0/2318043550 learned_addr learned my addr 172.21.15.137:0/2318043550 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.681+0000 7fb19b400640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb19c076060 0x7fb19c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb193e00640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb19c0772a0 0x7fb19c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb19be00640 1 -- 172.21.15.137:0/2318043550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb19c0772a0 msgr2=0x7fb19c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb19be00640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb19c0772a0 0x7fb19c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb19be00640 1 -- 172.21.15.137:0/2318043550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb19c076060 msgr2=0x7fb19c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb19be00640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb19c076060 0x7fb19c19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb19be00640 1 -- 172.21.15.137:0/2318043550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c1a5a40 con 0x7fb19c077c40 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb19be00640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c1a53d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb194032520 tx=0x7fb194034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb194039040 con 0x7fb19c077c40 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb194044070 con 0x7fb19c077c40 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19c1a5cd0 con 0x7fb19c077c40 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940043c0 con 0x7fb19c077c40 2024-03-26T14:02:09.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.682+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19c1a61c0 con 0x7fb19c077c40 2024-03-26T14:02:09.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.684+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19c1103f0 con 0x7fb19c077c40 2024-03-26T14:02:09.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.686+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb194004560 con 0x7fb19c077c40 2024-03-26T14:02:09.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.687+0000 7fb198c00640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb1800a2800 0x7fb1800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:09.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.687+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb1940f15b0 con 0x7fb19c077c40 2024-03-26T14:02:09.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.687+0000 7fb19b400640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb1800a2800 0x7fb1800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:09.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.688+0000 7fb19b400640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb1800a2800 0x7fb1800a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005e10 tx=0x7fb18c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:09.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.691+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb1940333a0 con 0x7fb19c077c40 2024-03-26T14:02:09.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/619211406' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:09.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2386349944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:09.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:09 smithi159 ceph-mon[35501]: pgmap v452: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:09.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.889+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb19c068490 con 0x7fb19c077c40 2024-03-26T14:02:09.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.893+0000 7fb198c00640 1 -- 172.21.15.137:0/2318043550 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7fb1940be240 con 0x7fb19c077c40 2024-03-26T14:02:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:09.892 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:09.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb1800a2800 msgr2=0x7fb1800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb1800a2800 0x7fb1800a4cc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb18c005e10 tx=0x7fb18c005da0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 msgr2=0x7fb19c1a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c1a53d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb194032520 tx=0x7fb194034880 comp rx=0 tx=0).stop 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 shutdown_connections 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb1800a2800 0x7fb1800a4cc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb19c077c40 0x7fb19c1a53d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb19c0772a0 0x7fb19c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 --2- 172.21.15.137:0/2318043550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb19c076060 0x7fb19c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 >> 172.21.15.137:0/2318043550 conn(0x7fb19c0fee50 msgr2=0x7fb19c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 shutdown_connections 2024-03-26T14:02:09.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:09.896+0000 7fb1a2000640 1 -- 172.21.15.137:0/2318043550 wait complete. 2024-03-26T14:02:09.913 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:02:09.914 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 -- 172.21.15.137:0/1592940787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36681038e0 msgr2=0x7f3668103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 --2- 172.21.15.137:0/1592940787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36681038e0 0x7f3668103d00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3658009ca0 tx=0x7f365802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 -- 172.21.15.137:0/1592940787 shutdown_connections 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 --2- 172.21.15.137:0/1592940787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3668105490 0x7f366810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 --2- 172.21.15.137:0/1592940787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3668104ad0 0x7f3668104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 --2- 172.21.15.137:0/1592940787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36681038e0 0x7f3668103d00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 -- 172.21.15.137:0/1592940787 >> 172.21.15.137:0/1592940787 conn(0x7f36680fee50 msgr2=0x7f3668101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 -- 172.21.15.137:0/1592940787 shutdown_connections 2024-03-26T14:02:10.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.119+0000 7f366dc00640 1 -- 172.21.15.137:0/1592940787 wait complete. 2024-03-26T14:02:10.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.120+0000 7f366dc00640 1 Processor -- start 2024-03-26T14:02:10.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.120+0000 7f366dc00640 1 -- start start 2024-03-26T14:02:10.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f366dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36681038e0 0x7f3668199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f366dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3668104ad0 0x7f366819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f366dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 0x7f36681a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f366dc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366810f400 con 0x7f3668105490 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f366dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366810f280 con 0x7f3668104ad0 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f366dc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366810f580 con 0x7f36681038e0 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f3667400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 0x7f36681a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f3667400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 0x7f36681a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40600/0 (socket says 172.21.15.137:40600) 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f3667400640 1 -- 172.21.15.137:0/2812146044 learned_addr learned my addr 172.21.15.137:0/2812146044 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f3666000640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3668104ad0 0x7f366819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.121+0000 7f3666a00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36681038e0 0x7f3668199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3667400640 1 -- 172.21.15.137:0/2812146044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36681038e0 msgr2=0x7f3668199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3667400640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36681038e0 0x7f3668199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3667400640 1 -- 172.21.15.137:0/2812146044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3668104ad0 msgr2=0x7f366819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3667400640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3668104ad0 0x7f366819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3667400640 1 -- 172.21.15.137:0/2812146044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36681a1770 con 0x7f3668105490 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3666a00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36681038e0 0x7f3668199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f3667400640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 0x7f36681a1100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f366000d750 tx=0x7f366000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660017070 con 0x7f3668105490 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36600054d0 con 0x7f3668105490 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36681a1a60 con 0x7f3668105490 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660004c60 con 0x7f3668105490 2024-03-26T14:02:10.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.122+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36681a1fb0 con 0x7f3668105490 2024-03-26T14:02:10.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.124+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366810acd0 con 0x7f3668105490 2024-03-26T14:02:10.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.126+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3660004e00 con 0x7f3668105490 2024-03-26T14:02:10.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.127+0000 7f365f400640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f364c0a2880 0x7f364c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.127+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f36600cec00 con 0x7f3668105490 2024-03-26T14:02:10.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.127+0000 7f3666a00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f364c0a2880 0x7f364c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.128+0000 7f3666a00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f364c0a2880 0x7f364c0a4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f365802c630 tx=0x7f3658032510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:10.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.130+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f366009b910 con 0x7f3668105490 2024-03-26T14:02:10.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.332+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f366802d220 con 0x7f3668105490 2024-03-26T14:02:10.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.332+0000 7f365f400640 1 -- 172.21.15.137:0/2812146044 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f366009b210 con 0x7f3668105490 2024-03-26T14:02:10.330 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f364c0a2880 msgr2=0x7f364c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f364c0a2880 0x7f364c0a4d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f365802c630 tx=0x7f3658032510 comp rx=0 tx=0).stop 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 msgr2=0x7f36681a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 0x7f36681a1100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f366000d750 tx=0x7f366000dc10 comp rx=0 tx=0).stop 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 shutdown_connections 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f364c0a2880 0x7f364c0a4d40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3668104ad0 0x7f366819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3668105490 0x7f36681a1100 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 --2- 172.21.15.137:0/2812146044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36681038e0 0x7f3668199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 >> 172.21.15.137:0/2812146044 conn(0x7f36680fee50 msgr2=0x7f3668107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 shutdown_connections 2024-03-26T14:02:10.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.335+0000 7f366dc00640 1 -- 172.21.15.137:0/2812146044 wait complete. 2024-03-26T14:02:10.353 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:02:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 -- 172.21.15.137:0/3428548341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c104ab0 msgr2=0x7f1e9c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3428548341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c104ab0 0x7f1e9c106ea0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009900 tx=0x7f1e8802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 -- 172.21.15.137:0/3428548341 shutdown_connections 2024-03-26T14:02:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3428548341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e9c1073e0 0x7f1e9c1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3428548341 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c104ab0 0x7f1e9c106ea0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3428548341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e9c102180 0x7f1e9c104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 -- 172.21.15.137:0/3428548341 >> 172.21.15.137:0/3428548341 conn(0x7f1e9c0fbb30 msgr2=0x7f1e9c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:10.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 -- 172.21.15.137:0/3428548341 shutdown_connections 2024-03-26T14:02:10.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.608+0000 7f1ea1400640 1 -- 172.21.15.137:0/3428548341 wait complete. 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 Processor -- start 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 -- start start 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 0x7f1e9c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e9c104ab0 0x7f1e9c100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e9c1073e0 0x7f1e9c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c10d1f0 con 0x7f1e9c102180 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c10d070 con 0x7f1e9c104ab0 2024-03-26T14:02:10.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.609+0000 7f1ea1400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c10d370 con 0x7f1e9c1073e0 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 0x7f1e9c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 0x7f1e9c101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40622/0 (socket says 172.21.15.137:40622) 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 -- 172.21.15.137:0/3330517594 learned_addr learned my addr 172.21.15.137:0/3330517594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e99e00640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e9c104ab0 0x7f1e9c100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9b200640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e9c1073e0 0x7f1e9c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 -- 172.21.15.137:0/3330517594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e9c1073e0 msgr2=0x7f1e9c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e9c1073e0 0x7f1e9c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 -- 172.21.15.137:0/3330517594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e9c104ab0 msgr2=0x7f1e9c100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e9c104ab0 0x7f1e9c100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 -- 172.21.15.137:0/3330517594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c100f00 con 0x7f1e9c102180 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.610+0000 7f1e9a800640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 0x7f1e9c101af0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00d950 tx=0x7f1e8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.611+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c017030 con 0x7f1e9c102180 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.611+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e8c004020 con 0x7f1e9c102180 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.611+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0047a0 con 0x7f1e9c102180 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.611+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c1a1a90 con 0x7f1e9c102180 2024-03-26T14:02:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.611+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c1a2000 con 0x7f1e9c102180 2024-03-26T14:02:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.612+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c0fd620 con 0x7f1e9c102180 2024-03-26T14:02:10.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.614+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1e8c0041d0 con 0x7f1e9c102180 2024-03-26T14:02:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.616+0000 7f1e93400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1e800a2880 0x7f1e800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.616+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1e8c0ce990 con 0x7f1e9c102180 2024-03-26T14:02:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.617+0000 7f1e99e00640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1e800a2880 0x7f1e800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:10.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.618+0000 7f1e99e00640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1e800a2880 0x7f1e800a4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1e8802c240 tx=0x7f1e8802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.621+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1e8c09b620 con 0x7f1e9c102180 2024-03-26T14:02:10.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:10 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2371723178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:10.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:10 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2318043550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:10.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:10 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2371723178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:10.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:10 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2318043550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:10.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2371723178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:10.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2318043550' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:10.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.825+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e9c062340 con 0x7f1e9c102180 2024-03-26T14:02:10.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.825+0000 7f1e93400640 1 -- 172.21.15.137:0/3330517594 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e8c09af20 con 0x7f1e9c102180 2024-03-26T14:02:10.823 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1e800a2880 msgr2=0x7f1e800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1e800a2880 0x7f1e800a4d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1e8802c240 tx=0x7f1e8802c7e0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 msgr2=0x7f1e9c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 0x7f1e9c101af0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00d950 tx=0x7f1e8c00de10 comp rx=0 tx=0).stop 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 shutdown_connections 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1e800a2880 0x7f1e800a4d40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e9c104ab0 0x7f1e9c100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e9c1073e0 0x7f1e9c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 --2- 172.21.15.137:0/3330517594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1e9c102180 0x7f1e9c101af0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 >> 172.21.15.137:0/3330517594 conn(0x7f1e9c0fbb30 msgr2=0x7f1e9c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 shutdown_connections 2024-03-26T14:02:10.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:10.828+0000 7f1ea1400640 1 -- 172.21.15.137:0/3330517594 wait complete. 2024-03-26T14:02:10.844 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:02:11.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 -- 172.21.15.137:0/893985373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 msgr2=0x7fb0581099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 --2- 172.21.15.137:0/893985373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb0581099e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb050009900 tx=0x7fb05002bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 -- 172.21.15.137:0/893985373 shutdown_connections 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 --2- 172.21.15.137:0/893985373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 0x7fb058102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 --2- 172.21.15.137:0/893985373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb0581099e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 --2- 172.21.15.137:0/893985373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0581015a0 0x7fb0581019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 -- 172.21.15.137:0/893985373 >> 172.21.15.137:0/893985373 conn(0x7fb0580fcb10 msgr2=0x7fb0580fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 -- 172.21.15.137:0/893985373 shutdown_connections 2024-03-26T14:02:11.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.034+0000 7fb05f000640 1 -- 172.21.15.137:0/893985373 wait complete. 2024-03-26T14:02:11.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 Processor -- start 2024-03-26T14:02:11.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 -- start start 2024-03-26T14:02:11.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0581015a0 0x7fb0581976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 0x7fb058197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb05819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05810d0e0 con 0x7fb058103150 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05810cf60 con 0x7fb058102790 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.035+0000 7fb05f000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05810d260 con 0x7fb0581015a0 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 0x7fb058197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05e600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb05819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 0x7fb058197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37840/0 (socket says 172.21.15.137:37840) 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05d200640 1 -- 172.21.15.137:0/2713333210 learned_addr learned my addr 172.21.15.137:0/2713333210 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05dc00640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0581015a0 0x7fb0581976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05e600640 1 -- 172.21.15.137:0/2713333210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0581015a0 msgr2=0x7fb0581976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05e600640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0581015a0 0x7fb0581976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05e600640 1 -- 172.21.15.137:0/2713333210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 msgr2=0x7fb058197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05e600640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 0x7fb058197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.036+0000 7fb05e600640 1 -- 172.21.15.137:0/2713333210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05819f310 con 0x7fb058103150 2024-03-26T14:02:11.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.037+0000 7fb05e600640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb05819eca0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb050009230 tx=0x7fb0500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.037+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050043020 con 0x7fb058103150 2024-03-26T14:02:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.037+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05819f5a0 con 0x7fb058103150 2024-03-26T14:02:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.037+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb050004aa0 con 0x7fb058103150 2024-03-26T14:02:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.037+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050003c40 con 0x7fb058103150 2024-03-26T14:02:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.037+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05819faf0 con 0x7fb058103150 2024-03-26T14:02:11.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.038+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0581088c0 con 0x7fb058103150 2024-03-26T14:02:11.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.041+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb050046020 con 0x7fb058103150 2024-03-26T14:02:11.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.043+0000 7fb056000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb0400a2800 0x7fb0400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.043+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb0500f11b0 con 0x7fb058103150 2024-03-26T14:02:11.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.043+0000 7fb05dc00640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb0400a2800 0x7fb0400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.044+0000 7fb05dc00640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb0400a2800 0x7fb0400a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0581025f0 tx=0x7fb04c009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:11.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.047+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb05004d020 con 0x7fb058103150 2024-03-26T14:02:11.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.247+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0580622a0 con 0x7fb058103150 2024-03-26T14:02:11.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.247+0000 7fb056000640 1 -- 172.21.15.137:0/2713333210 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0500bde40 con 0x7fb058103150 2024-03-26T14:02:11.245 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb0400a2800 msgr2=0x7fb0400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb0400a2800 0x7fb0400a4cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb0581025f0 tx=0x7fb04c009670 comp rx=0 tx=0).stop 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 msgr2=0x7fb05819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb05819eca0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb050009230 tx=0x7fb0500042f0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 shutdown_connections 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb0400a2800 0x7fb0400a4cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb058102790 0x7fb058197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb058103150 0x7fb05819eca0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 --2- 172.21.15.137:0/2713333210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0581015a0 0x7fb0581976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 >> 172.21.15.137:0/2713333210 conn(0x7fb0580fcb10 msgr2=0x7fb058105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:11.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 shutdown_connections 2024-03-26T14:02:11.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.250+0000 7fb05f000640 1 -- 172.21.15.137:0/2713333210 wait complete. 2024-03-26T14:02:11.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:02:11.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 -- 172.21.15.137:0/3908457656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44102790 msgr2=0x7fcf44102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 --2- 172.21.15.137:0/3908457656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44102790 0x7fcf44102c10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009900 tx=0x7fcf2c02bc50 comp rx=0 tx=0).stop 2024-03-26T14:02:11.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 -- 172.21.15.137:0/3908457656 shutdown_connections 2024-03-26T14:02:11.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 --2- 172.21.15.137:0/3908457656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf44103150 0x7fcf441099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 --2- 172.21.15.137:0/3908457656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf44102790 0x7fcf44102c10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 --2- 172.21.15.137:0/3908457656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf441015a0 0x7fcf441019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.466+0000 7fcf49000640 1 -- 172.21.15.137:0/3908457656 >> 172.21.15.137:0/3908457656 conn(0x7fcf440fcb10 msgr2=0x7fcf440fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.467+0000 7fcf49000640 1 -- 172.21.15.137:0/3908457656 shutdown_connections 2024-03-26T14:02:11.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.467+0000 7fcf49000640 1 -- 172.21.15.137:0/3908457656 wait complete. 2024-03-26T14:02:11.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.467+0000 7fcf49000640 1 Processor -- start 2024-03-26T14:02:11.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 -- start start 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 0x7fcf441999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf44102790 0x7fcf44199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf44103150 0x7fcf441a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4410d120 con 0x7fcf441015a0 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4410cfa0 con 0x7fcf44103150 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf49000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4410d2a0 con 0x7fcf44102790 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 0x7fcf441999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 0x7fcf441999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40654/0 (socket says 172.21.15.137:40654) 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf43400640 1 -- 172.21.15.137:0/1011080614 learned_addr learned my addr 172.21.15.137:0/1011080614 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:11.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf3be00640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf44102790 0x7fcf44199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf43e00640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf44103150 0x7fcf441a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.468+0000 7fcf43400640 1 -- 172.21.15.137:0/1011080614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf44102790 msgr2=0x7fcf44199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf43400640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf44102790 0x7fcf44199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf43400640 1 -- 172.21.15.137:0/1011080614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf44103150 msgr2=0x7fcf441a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf43400640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf44103150 0x7fcf441a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf43400640 1 -- 172.21.15.137:0/1011080614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf441a15f0 con 0x7fcf441015a0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf43400640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 0x7fcf441999d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400e9c0 tx=0x7fcf3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400cd80 con 0x7fcf441015a0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcf340035f0 con 0x7fcf441015a0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf441a18e0 con 0x7fcf441015a0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34010690 con 0x7fcf441015a0 2024-03-26T14:02:11.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.469+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf441a1db0 con 0x7fcf441015a0 2024-03-26T14:02:11.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.470+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf441a2010 con 0x7fcf441015a0 2024-03-26T14:02:11.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.474+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcf340037a0 con 0x7fcf441015a0 2024-03-26T14:02:11.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.475+0000 7fcf40c00640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcf280a2800 0x7fcf280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.475+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcf340cf4a0 con 0x7fcf441015a0 2024-03-26T14:02:11.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.476+0000 7fcf3be00640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcf280a2800 0x7fcf280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.476+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcf3409c170 con 0x7fcf441015a0 2024-03-26T14:02:11.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.476+0000 7fcf3be00640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcf280a2800 0x7fcf280a4cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c000c00 tx=0x7fcf2c02c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:11.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:11 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2812146044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:02:11.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:11 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3330517594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:11 smithi137 ceph-mon[31539]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:11.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:11 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2713333210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:11 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2812146044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:02:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:11 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3330517594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:11 smithi179 ceph-mon[35472]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:11 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2713333210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.679+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcf441a21e0 con 0x7fcf441015a0 2024-03-26T14:02:11.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.680+0000 7fcf40c00640 1 -- 172.21.15.137:0/1011080614 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcf3409ba70 con 0x7fcf441015a0 2024-03-26T14:02:11.678 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:11.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.682+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcf280a2800 msgr2=0x7fcf280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.682+0000 7fcf49000640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcf280a2800 0x7fcf280a4cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c000c00 tx=0x7fcf2c02c800 comp rx=0 tx=0).stop 2024-03-26T14:02:11.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.682+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 msgr2=0x7fcf441999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.682+0000 7fcf49000640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 0x7fcf441999d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400e9c0 tx=0x7fcf3400ee80 comp rx=0 tx=0).stop 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 shutdown_connections 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcf280a2800 0x7fcf280a4cc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf44103150 0x7fcf441a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf44102790 0x7fcf44199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 --2- 172.21.15.137:0/1011080614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcf441015a0 0x7fcf441999d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 >> 172.21.15.137:0/1011080614 conn(0x7fcf440fcb10 msgr2=0x7fcf440738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 shutdown_connections 2024-03-26T14:02:11.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.683+0000 7fcf49000640 1 -- 172.21.15.137:0/1011080614 wait complete. 2024-03-26T14:02:11.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:02:11.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:11 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2812146044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:02:11.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:11 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3330517594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:11 smithi159 ceph-mon[35501]: pgmap v453: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:11.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:11 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2713333210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:11.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.877+0000 7f346c400640 1 -- 172.21.15.137:0/3384916521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 msgr2=0x7f3464104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.877+0000 7f346c400640 1 --2- 172.21.15.137:0/3384916521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f3464104f50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3450009900 tx=0x7f345002bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:11.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 -- 172.21.15.137:0/3384916521 shutdown_connections 2024-03-26T14:02:11.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 --2- 172.21.15.137:0/3384916521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3464105490 0x7f346410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 --2- 172.21.15.137:0/3384916521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f3464104f50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 --2- 172.21.15.137:0/3384916521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34641038e0 0x7f3464103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 -- 172.21.15.137:0/3384916521 >> 172.21.15.137:0/3384916521 conn(0x7f34640fee50 msgr2=0x7f3464101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:11.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 -- 172.21.15.137:0/3384916521 shutdown_connections 2024-03-26T14:02:11.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 -- 172.21.15.137:0/3384916521 wait complete. 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.878+0000 7f346c400640 1 Processor -- start 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 -- start start 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34641038e0 0x7f3464199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f346419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3464105490 0x7f34641a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346410f410 con 0x7f3464104ad0 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346410f290 con 0x7f34641038e0 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346c400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346410f590 con 0x7f3464105490 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f3469000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f346419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f3469a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34641038e0 0x7f3464199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f3469000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f346419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40686/0 (socket says 172.21.15.137:40686) 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f3469000640 1 -- 172.21.15.137:0/3225183247 learned_addr learned my addr 172.21.15.137:0/3225183247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f3469a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34641038e0 0x7f3464199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37870/0 (socket says 172.21.15.137:37870) 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.879+0000 7f346a400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3464105490 0x7f34641a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3469000640 1 -- 172.21.15.137:0/3225183247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3464105490 msgr2=0x7f34641a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3469000640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3464105490 0x7f34641a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3469000640 1 -- 172.21.15.137:0/3225183247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34641038e0 msgr2=0x7f3464199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3469000640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34641038e0 0x7f3464199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3469000640 1 -- 172.21.15.137:0/3225183247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34641a1760 con 0x7f3464104ad0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3469000640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f346419a080 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f3450004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450043020 con 0x7f3464104ad0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34641a19f0 con 0x7f3464104ad0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3450004440 con 0x7f3464104ad0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450003bf0 con 0x7f3464104ad0 2024-03-26T14:02:11.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.880+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34641a1f40 con 0x7f3464104ad0 2024-03-26T14:02:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.881+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464068490 con 0x7f3464104ad0 2024-03-26T14:02:11.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.883+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3450046020 con 0x7f3464104ad0 2024-03-26T14:02:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.884+0000 7f3462000640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f344c0a2800 0x7f344c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.884+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f34500f1860 con 0x7f3464104ad0 2024-03-26T14:02:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.885+0000 7f3469a00640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f344c0a2800 0x7f344c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:11.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.885+0000 7f3469a00640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f344c0a2800 0x7f344c0a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3464104930 tx=0x7f3458005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:11.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:11.887+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f345004d020 con 0x7f3464104ad0 2024-03-26T14:02:12.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.122+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f346402d140 con 0x7f3464104ad0 2024-03-26T14:02:12.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.124+0000 7f3462000640 1 -- 172.21.15.137:0/3225183247 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f346402d140 con 0x7f3464104ad0 2024-03-26T14:02:12.122 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:12.122 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T13:59:50.245808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34424,34451,34496]},"id":1}]} 2024-03-26T14:02:12.122 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 23 2024-03-26T14:02:12.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f344c0a2800 msgr2=0x7f344c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f344c0a2800 0x7f344c0a4cc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3464104930 tx=0x7f3458005f90 comp rx=0 tx=0).stop 2024-03-26T14:02:12.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 msgr2=0x7f346419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f346419a080 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f3450004040 comp rx=0 tx=0).stop 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 shutdown_connections 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f344c0a2800 0x7f344c0a4cc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3464105490 0x7f34641a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3464104ad0 0x7f346419a080 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 --2- 172.21.15.137:0/3225183247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34641038e0 0x7f3464199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 >> 172.21.15.137:0/3225183247 conn(0x7f34640fee50 msgr2=0x7f3464107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 shutdown_connections 2024-03-26T14:02:12.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.128+0000 7f346c400640 1 -- 172.21.15.137:0/3225183247 wait complete. 2024-03-26T14:02:12.143 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.332+0000 7f2d76000640 1 -- 172.21.15.137:0/2225908251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 msgr2=0x7f2d70106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.332+0000 7f2d76000640 1 --2- 172.21.15.137:0/2225908251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d70106e80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009900 tx=0x7f2d5c02bc40 comp rx=0 tx=0).stop 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 -- 172.21.15.137:0/2225908251 shutdown_connections 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 --2- 172.21.15.137:0/2225908251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d701073c0 0x7f2d70109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 --2- 172.21.15.137:0/2225908251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d70106e80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 --2- 172.21.15.137:0/2225908251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70102160 0x7f2d70104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 -- 172.21.15.137:0/2225908251 >> 172.21.15.137:0/2225908251 conn(0x7f2d700fbb50 msgr2=0x7f2d700fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 -- 172.21.15.137:0/2225908251 shutdown_connections 2024-03-26T14:02:12.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.333+0000 7f2d76000640 1 -- 172.21.15.137:0/2225908251 wait complete. 2024-03-26T14:02:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 Processor -- start 2024-03-26T14:02:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 -- start start 2024-03-26T14:02:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d70102160 0x7f2d70199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d7019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701073c0 0x7f2d701a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7010d270 con 0x7f2d70104a90 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7010d0f0 con 0x7f2d70102160 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d76000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7010d3f0 con 0x7f2d701073c0 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d7019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d6ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d7019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40708/0 (socket says 172.21.15.137:40708) 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d6ea00640 1 -- 172.21.15.137:0/1898806408 learned_addr learned my addr 172.21.15.137:0/1898806408 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d6f400640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d70102160 0x7f2d70199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.334+0000 7f2d6fe00640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701073c0 0x7f2d701a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d6ea00640 1 -- 172.21.15.137:0/1898806408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701073c0 msgr2=0x7f2d701a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d6ea00640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701073c0 0x7f2d701a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d6ea00640 1 -- 172.21.15.137:0/1898806408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d70102160 msgr2=0x7f2d70199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d6ea00640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d70102160 0x7f2d70199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d6ea00640 1 -- 172.21.15.137:0/1898806408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d701a1760 con 0x7f2d70104a90 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d6ea00640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d7019a080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c043020 con 0x7f2d70104a90 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2d5c0044c0 con 0x7f2d70104a90 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d701a19f0 con 0x7f2d70104a90 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c003c60 con 0x7f2d70104a90 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.335+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d701a1f40 con 0x7f2d70104a90 2024-03-26T14:02:12.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.337+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d70108f10 con 0x7f2d70104a90 2024-03-26T14:02:12.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.338+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2d5c046020 con 0x7f2d70104a90 2024-03-26T14:02:12.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.340+0000 7f2d67e00640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2d540a2800 0x7f2d540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.340+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2d5c0f1190 con 0x7f2d70104a90 2024-03-26T14:02:12.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.340+0000 7f2d6f400640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2d540a2800 0x7f2d540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.341+0000 7f2d6f400640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2d540a2800 0x7f2d540a4cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009b60 tx=0x7f2d60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.344+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2d5c04d020 con 0x7f2d70104a90 2024-03-26T14:02:12.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.572+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d700622a0 con 0x7f2d70104a90 2024-03-26T14:02:12.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.574+0000 7f2d67e00640 1 -- 172.21.15.137:0/1898806408 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f2d5c0bde20 con 0x7f2d70104a90 2024-03-26T14:02:12.572 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:12.572 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:12.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2d540a2800 msgr2=0x7f2d540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2d540a2800 0x7f2d540a4cc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009b60 tx=0x7f2d60005f90 comp rx=0 tx=0).stop 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 msgr2=0x7f2d7019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d7019a080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c0040c0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 shutdown_connections 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2d540a2800 0x7f2d540a4cc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701073c0 0x7f2d701a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d70104a90 0x7f2d7019a080 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 --2- 172.21.15.137:0/1898806408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d70102160 0x7f2d70199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 >> 172.21.15.137:0/1898806408 conn(0x7f2d700fbb50 msgr2=0x7f2d701017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 shutdown_connections 2024-03-26T14:02:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.577+0000 7f2d76000640 1 -- 172.21.15.137:0/1898806408 wait complete. 2024-03-26T14:02:12.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-26T13:59:22.728231+0000', 'modified': '2024-03-26T13:59:50.245808+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 34496, 'mds_1': 34451, 'mds_2': 34424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6837/7146430', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 7146430}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 7146430}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24481': {'gid': 24481, 'name': 'i', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6839/2441458416', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 2441458416}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 2441458416}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34394': {'gid': 34394, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/1469472888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 1469472888}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 1469472888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34424': {'gid': 34424, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.159:6835/3125472496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 3125472496}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 3125472496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34451': {'gid': 34451, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.159:6837/16915360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 16915360}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 16915360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34496': {'gid': 34496, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.159:6839/2964052982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2964052982}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2964052982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34496, 'qdb_cluster': [34424, 34451, 34496]} 2024-03-26T14:02:12.595 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-26T14:02:12.595 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-26T14:02:12.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 -- 172.21.15.137:0/1580670734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 msgr2=0x7fb5a8073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 --2- 172.21.15.137:0/1580670734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8073ad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb598009900 tx=0x7fb59802bcd0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 -- 172.21.15.137:0/1580670734 shutdown_connections 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 --2- 172.21.15.137:0/1580670734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5a8109430 0x7fb5a810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 --2- 172.21.15.137:0/1580670734 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a80740a0 0x7fb5a8074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 --2- 172.21.15.137:0/1580670734 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8073ad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 -- 172.21.15.137:0/1580670734 >> 172.21.15.137:0/1580670734 conn(0x7fb5a80fcb10 msgr2=0x7fb5a80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 -- 172.21.15.137:0/1580670734 shutdown_connections 2024-03-26T14:02:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.797+0000 7fb5ad400640 1 -- 172.21.15.137:0/1580670734 wait complete. 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.798+0000 7fb5ad400640 1 Processor -- start 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.798+0000 7fb5ad400640 1 -- start start 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5ad400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5a80740a0 0x7fb5a8199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5ad400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5ad400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8109430 0x7fb5a81a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5ad400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a810f2f0 con 0x7fb5a8075680 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5ad400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a810f170 con 0x7fb5a80740a0 2024-03-26T14:02:12.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5ad400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a810f470 con 0x7fb5a8109430 2024-03-26T14:02:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40722/0 (socket says 172.21.15.137:40722) 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 -- 172.21.15.137:0/2059635275 learned_addr learned my addr 172.21.15.137:0/2059635275 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5a80740a0 0x7fb5a8199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5aca00640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8109430 0x7fb5a81a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 -- 172.21.15.137:0/2059635275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8109430 msgr2=0x7fb5a81a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8109430 0x7fb5a81a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 -- 172.21.15.137:0/2059635275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5a80740a0 msgr2=0x7fb5a8199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.799+0000 7fb5a7400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5a80740a0 0x7fb5a8199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5a7400640 1 -- 172.21.15.137:0/2059635275 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a81a15f0 con 0x7fb5a8075680 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5a7400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8199f60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c890 tx=0x7fb59400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5a4c00640 1 -- 172.21.15.137:0/2059635275 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594004220 con 0x7fb5a8075680 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5a4c00640 1 -- 172.21.15.137:0/2059635275 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb5940043c0 con 0x7fb5a8075680 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5a4c00640 1 -- 172.21.15.137:0/2059635275 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594004a00 con 0x7fb5a8075680 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a81a18e0 con 0x7fb5a8075680 2024-03-26T14:02:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.800+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a81a1de0 con 0x7fb5a8075680 2024-03-26T14:02:12.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 Processor -- start 2024-03-26T14:02:12.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 -- start start 2024-03-26T14:02:12.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18e9d80 0x560ee18ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee184ec90 con 0x560ee18ea770 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee17f7980 con 0x560ee18ef8e0 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.804+0000 7fb5b1790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee184ad30 con 0x560ee18e9d80 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a4c00640 1 -- 172.21.15.137:0/2059635275 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb594004ba0 con 0x7fb5a8075680 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18ef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40726/0 (socket says 172.21.15.137:40726) 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 -- 172.21.15.137:0/970925770 learned_addr learned my addr 172.21.15.137:0/970925770 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5aca00640 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18e9d80 0x560ee18ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7400640 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 -- 172.21.15.137:0/970925770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18e9d80 msgr2=0x560ee18ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18e9d80 0x560ee18ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 -- 172.21.15.137:0/970925770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 msgr2=0x560ee18f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.805+0000 7fb5a7e00640 1 -- 172.21.15.137:0/970925770 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee180f510 con 0x560ee18ea770 2024-03-26T14:02:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.806+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18ef270 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb598009230 tx=0x7fb5980095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20907404d5207b80 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.806+0000 7fb59d200640 1 -- 172.21.15.137:0/970925770 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598004200 con 0x560ee18ea770 2024-03-26T14:02:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.806+0000 7fb59d200640 1 -- 172.21.15.137:0/970925770 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59802ce30 con 0x560ee18ea770 2024-03-26T14:02:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.806+0000 7fb59d200640 1 -- 172.21.15.137:0/970925770 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980046c0 con 0x560ee18ea770 2024-03-26T14:02:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.806+0000 7fb5a4c00640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb58c0a2800 0x7fb58c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5a4c00640 1 -- 172.21.15.137:0/2059635275 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb5940ceb40 con 0x7fb5a8075680 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5aca00640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb58c0a2800 0x7fb58c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 -- 172.21.15.137:0/970925770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 msgr2=0x560ee18ef270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18ef270 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb598009230 tx=0x7fb5980095e0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 -- 172.21.15.137:0/970925770 shutdown_connections 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18ef270 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 --2- 172.21.15.137:0/970925770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18e9d80 0x560ee18ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 -- 172.21.15.137:0/970925770 >> 172.21.15.137:0/970925770 conn(0x560ee18e52f0 msgr2=0x560ee18e56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.807+0000 7fb5b1790740 1 -- 172.21.15.137:0/970925770 shutdown_connections 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.808+0000 7fb5aca00640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb58c0a2800 0x7fb58c0a4cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x560ee18eafd0 tx=0x7fb5a0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.808+0000 7fb5b1790740 1 -- 172.21.15.137:0/970925770 wait complete. 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.808+0000 7fb5b1790740 1 -- start start 2024-03-26T14:02:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.808+0000 7fb5b1790740 1 Processor -- start 2024-03-26T14:02:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18f4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ef8e0 0x560ee18fb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee18f2830 con 0x560ee18ea770 2024-03-26T14:02:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee180add0 con 0x560ee18e9d80 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee18f29b0 con 0x560ee18ef8e0 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5b1790740 10 client.0 _ll_register_callbacks cb 0x560ee1562ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37926/0 (socket says 172.21.15.137:37926) 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 -- 172.21.15.137:0/3628895647 learned_addr learned my addr 172.21.15.137:0/3628895647 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5aca00640 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ef8e0 0x560ee18fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 -- 172.21.15.137:0/3628895647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ef8e0 msgr2=0x560ee18fb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ef8e0 0x560ee18fb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 -- 172.21.15.137:0/3628895647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 msgr2=0x560ee18f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.809+0000 7fb5a7e00640 1 -- 172.21.15.137:0/3628895647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ee15632d0 con 0x560ee18e9d80 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f4200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c2e0 tx=0x7fb59802c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b6e21e2492d10c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb591600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb591600640 1 -- 172.21.15.137:0/3628895647 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5980035f0 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb591600640 1 -- 172.21.15.137:0/3628895647 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb598003790 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb59802ce30 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb591600640 1 -- 172.21.15.137:0/3628895647 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb598003990 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb5b1790740 10 client.35198 fetch_fsmap learned FSMap version 23 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.810+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560ee162d6b0 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb591600640 1 -- 172.21.15.137:0/3628895647 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59802ce30 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb591600640 1 -- 172.21.15.137:0/3628895647 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb59803f430 con 0x560ee18e9d80 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5b1790740 10 client.35198 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5b1790740 10 client.35198 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5b1790740 20 client.35198 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5b1790740 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee18ff8e0 0x560ee1904970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5b1790740 4 client.35198 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ee17f0520 con 0x560ee18ff8e0 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.811+0000 7fb5a7400640 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee18ff8e0 0x560ee1904970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.812+0000 7fb591600640 10 client.35198 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:02:12.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.812+0000 7fb5a7400640 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee18ff8e0 0x560ee1904970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55f022673906472b server_cookie=7bb9a393c26090f in_seq=0 out_seq=0 2024-03-26T14:02:12.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.813+0000 7fb591600640 1 -- 172.21.15.137:0/3628895647 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560ee17f0520 con 0x560ee18ff8e0 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.813+0000 7fb591600640 10 client.35198 handle_command_reply: tid=0 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.813+0000 7fb5b1790740 1 client.35198 shutdown 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 msgr2=0x560ee18f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f4200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c2e0 tx=0x7fb59802c6d0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 shutdown_connections 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee18ff8e0 0x560ee1904970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ef8e0 0x560ee18fb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ea770 0x560ee18f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb591600640 0 client.35198 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 --2- 172.21.15.137:0/3628895647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f4200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 >> 172.21.15.137:0/3628895647 conn(0x560ee18e52f0 msgr2=0x560ee18e44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 shutdown_connections 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 1 -- 172.21.15.137:0/3628895647 wait complete. 2024-03-26T14:02:12.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.814+0000 7fb5b1790740 20 client.35198 trim_cache size 0 max 16384 2024-03-26T14:02:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 Processor -- start 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 -- start start 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 0x560ee18e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18fb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee18ffd10 con 0x560ee18e9d80 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee1900370 con 0x560ee18ef8e0 2024-03-26T14:02:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.834+0000 7fb5b1790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee19009d0 con 0x560ee18ea770 2024-03-26T14:02:12.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 0x560ee18e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 0x560ee18e8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40748/0 (socket says 172.21.15.137:40748) 2024-03-26T14:02:12.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 -- 172.21.15.137:0/1113998522 learned_addr learned my addr 172.21.15.137:0/1113998522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:12.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5aca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7400640 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 -- 172.21.15.137:0/1113998522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 msgr2=0x560ee18fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 -- 172.21.15.137:0/1113998522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 msgr2=0x560ee18f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5a7e00640 1 -- 172.21.15.137:0/1113998522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ee18e6060 con 0x560ee18e9d80 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.835+0000 7fb5aca00640 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.836+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 0x560ee18e8c60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c2b0 tx=0x7fb598009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb746caf636b390 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.836+0000 7fb593e00640 1 -- 172.21.15.137:0/1113998522 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59803fa40 con 0x560ee18e9d80 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.836+0000 7fb593e00640 1 -- 172.21.15.137:0/1113998522 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59803f430 con 0x560ee18e9d80 2024-03-26T14:02:12.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.836+0000 7fb593e00640 1 -- 172.21.15.137:0/1113998522 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59802cdf0 con 0x560ee18e9d80 2024-03-26T14:02:12.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 -- 172.21.15.137:0/1113998522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 msgr2=0x560ee18e8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 0x560ee18e8c60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb59802c2b0 tx=0x7fb598009780 comp rx=0 tx=0).stop 2024-03-26T14:02:12.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 -- 172.21.15.137:0/1113998522 shutdown_connections 2024-03-26T14:02:12.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18ef8e0 0x560ee18f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 --2- 172.21.15.137:0/1113998522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18e9d80 0x560ee18e8c60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 -- 172.21.15.137:0/1113998522 >> 172.21.15.137:0/1113998522 conn(0x560ee18e52f0 msgr2=0x560ee18e8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 -- 172.21.15.137:0/1113998522 shutdown_connections 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 -- 172.21.15.137:0/1113998522 wait complete. 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.837+0000 7fb5b1790740 1 -- start start 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 Processor -- start 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 0x560ee18faa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee1901690 con 0x560ee18ef8e0 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee1901030 con 0x560ee18e9d80 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ee1901cf0 con 0x560ee18ea770 2024-03-26T14:02:12.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5b1790740 10 client.0 _ll_register_callbacks cb 0x7fb5a00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:12.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5aca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 0x560ee18faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5aca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 0x560ee18faa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40758/0 (socket says 172.21.15.137:40758) 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5aca00640 1 -- 172.21.15.137:0/2974255178 learned_addr learned my addr 172.21.15.137:0/2974255178 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.838+0000 7fb5a7400640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5aca00640 1 -- 172.21.15.137:0/2974255178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 msgr2=0x560ee18f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5aca00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5aca00640 1 -- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 msgr2=0x560ee18f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5aca00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5aca00640 1 -- 172.21.15.137:0/2974255178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ee18fb020 con 0x560ee18ef8e0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5aca00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 0x560ee18faa30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb5a001ea00 tx=0x7fb5a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d9230e6bb84eac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb591600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb591600640 1 -- 172.21.15.137:0/2974255178 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a001db20 con 0x560ee18ef8e0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb591600640 1 -- 172.21.15.137:0/2974255178 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb5a001dcc0 con 0x560ee18ef8e0 2024-03-26T14:02:12.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.839+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb59803f430 con 0x560ee18ef8e0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb591600640 1 -- 172.21.15.137:0/2974255178 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5a0038050 con 0x560ee18ef8e0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 10 client.15474 fetch_fsmap learned FSMap version 23 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560ee195c150 con 0x560ee18ef8e0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb591600640 1 -- 172.21.15.137:0/2974255178 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb59803f430 con 0x560ee18ef8e0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb591600640 1 -- 172.21.15.137:0/2974255178 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb5a0011440 con 0x560ee18ef8e0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 10 client.15474 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 10 client.15474 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 20 client.15474 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee1960920 0x560ee195c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 4 client.15474 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.840+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560ee185c1d0 con 0x560ee1960920 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.841+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee1960920 0x560ee195c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.841+0000 7fb5a7e00640 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee1960920 0x560ee195c490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2dea42b02613047 server_cookie=6e96a90d10103e70 in_seq=0 out_seq=0 2024-03-26T14:02:12.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.841+0000 7fb591600640 10 client.15474 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb591600640 1 -- 172.21.15.137:0/2974255178 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560ee185c1d0 con 0x560ee1960920 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb591600640 10 client.15474 handle_command_reply: tid=0 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 client.15474 shutdown 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 msgr2=0x560ee18faa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 0x560ee18faa30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb5a001ea00 tx=0x7fb5a0004ab0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 shutdown_connections 2024-03-26T14:02:12.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560ee1960920 0x560ee195c490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ee18ef8e0 0x560ee18faa30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ee18ea770 0x560ee18f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 --2- 172.21.15.137:0/2974255178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ee18e9d80 0x560ee18f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb591600640 0 client.15474 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 >> 172.21.15.137:0/2974255178 conn(0x560ee18e52f0 msgr2=0x560ee18e8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 shutdown_connections 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 1 -- 172.21.15.137:0/2974255178 wait complete. 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.842+0000 7fb5b1790740 20 client.15474 trim_cache size 0 max 16384 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 2, 2024-03-26T14:02:12.842 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34424, 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 143.60455539500001, 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 94, 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 204.14211826799999 2024-03-26T14:02:12.843 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb58c0a2800 msgr2=0x7fb58c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb58c0a2800 0x7fb58c0a4cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x560ee18eafd0 tx=0x7fb5a0009500 comp rx=0 tx=0).stop 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 msgr2=0x7fb5a8199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8199f60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c890 tx=0x7fb59400cd50 comp rx=0 tx=0).stop 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 shutdown_connections 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb58c0a2800 0x7fb58c0a4cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a8109430 0x7fb5a81a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb5a8075680 0x7fb5a8199f60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 --2- 172.21.15.137:0/2059635275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5a80740a0 0x7fb5a8199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 >> 172.21.15.137:0/2059635275 conn(0x7fb5a80fcb10 msgr2=0x7fb5a81053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 shutdown_connections 2024-03-26T14:02:12.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:12.843+0000 7fb5ad400640 1 -- 172.21.15.137:0/2059635275 wait complete. 2024-03-26T14:02:12.856 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:12.856 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:12 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:12.856 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1011080614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:12.856 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3225183247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:12.856 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1898806408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:12.858 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-26T14:02:12.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:12.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:12 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:12.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1011080614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:12.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3225183247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:12.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1898806408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:12.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:12.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:12 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:12.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1011080614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:12.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3225183247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:12.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1898806408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 -- 172.21.15.137:0/4291042724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec102130 msgr2=0x7fe6ec104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/4291042724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec102130 0x7fe6ec104520 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009900 tx=0x7fe6dc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 -- 172.21.15.137:0/4291042724 shutdown_connections 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/4291042724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec107390 0x7fe6ec1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/4291042724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec104a60 0x7fe6ec106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/4291042724 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec102130 0x7fe6ec104520 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 -- 172.21.15.137:0/4291042724 >> 172.21.15.137:0/4291042724 conn(0x7fe6ec0fbb40 msgr2=0x7fe6ec0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 -- 172.21.15.137:0/4291042724 shutdown_connections 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.052+0000 7fe6f1c00640 1 -- 172.21.15.137:0/4291042724 wait complete. 2024-03-26T14:02:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.053+0000 7fe6f1c00640 1 Processor -- start 2024-03-26T14:02:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 -- start start 2024-03-26T14:02:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec102130 0x7fe6ec101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec104a60 0x7fe6ec1000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 0x7fe6ec100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec10d200 con 0x7fe6ec107390 2024-03-26T14:02:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec10d080 con 0x7fe6ec104a60 2024-03-26T14:02:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6f1c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec10d380 con 0x7fe6ec102130 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.054+0000 7fe6eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 0x7fe6ec100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6ea000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec104a60 0x7fe6ec1000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 0x7fe6ec100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40764/0 (socket says 172.21.15.137:40764) 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 -- 172.21.15.137:0/3621931343 learned_addr learned my addr 172.21.15.137:0/3621931343 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec102130 0x7fe6ec101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 -- 172.21.15.137:0/3621931343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec102130 msgr2=0x7fe6ec101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec102130 0x7fe6ec101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 -- 172.21.15.137:0/3621931343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec104a60 msgr2=0x7fe6ec1000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec104a60 0x7fe6ec1000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 -- 172.21.15.137:0/3621931343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec100eb0 con 0x7fe6ec107390 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec102130 0x7fe6ec101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6ea000640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec104a60 0x7fe6ec1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.055+0000 7fe6eb400640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 0x7fe6ec100620 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400eb10 tx=0x7fe6e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.056+0000 7fe6e3400640 1 -- 172.21.15.137:0/3621931343 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4004b20 con 0x7fe6ec107390 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.056+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec1a1a60 con 0x7fe6ec107390 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.056+0000 7fe6e3400640 1 -- 172.21.15.137:0/3621931343 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6e4004cc0 con 0x7fe6ec107390 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.056+0000 7fe6e3400640 1 -- 172.21.15.137:0/3621931343 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4018630 con 0x7fe6ec107390 2024-03-26T14:02:13.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.056+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec1a1fb0 con 0x7fe6ec107390 2024-03-26T14:02:13.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.059+0000 7fe6f480e740 1 Processor -- start 2024-03-26T14:02:13.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.059+0000 7fe6e3400640 1 -- 172.21.15.137:0/3621931343 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe6e4005000 con 0x7fe6ec107390 2024-03-26T14:02:13.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 -- start start 2024-03-26T14:02:13.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafedfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3baff3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3baff3700 0x55d3baff5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3baf52c90 con 0x55d3bafee590 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3baefb980 con 0x55d3bafedba0 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6f480e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3baf4ed30 con 0x55d3baff3700 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6ea000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafedfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6ea000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafedfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37972/0 (socket says 172.21.15.137:37972) 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6eaa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3baff3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6ea000640 1 -- 172.21.15.137:0/2256696868 learned_addr learned my addr 172.21.15.137:0/2256696868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.060+0000 7fe6ea000640 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3baff3700 0x55d3baff5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6ea000640 1 -- 172.21.15.137:0/2256696868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3baff3700 msgr2=0x55d3baff5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6ea000640 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3baff3700 0x55d3baff5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6ea000640 1 -- 172.21.15.137:0/2256696868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 msgr2=0x55d3baff3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6ea000640 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3baff3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6ea000640 1 -- 172.21.15.137:0/2256696868 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3baf13510 con 0x55d3bafedba0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6e3400640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6d00a2800 0x7fe6d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6e3400640 1 -- 172.21.15.137:0/3621931343 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe6e40cf660 con 0x7fe6ec107390 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6ea000640 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafedfc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8012950 tx=0x7fe6d800bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc2dd0767721d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6d00a2800 0x7fe6d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.061+0000 7fe6d7e00640 1 -- 172.21.15.137:0/2256696868 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8004230 con 0x55d3bafedba0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6d7e00640 1 -- 172.21.15.137:0/2256696868 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6d80043d0 con 0x55d3bafedba0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6d7e00640 1 -- 172.21.15.137:0/2256696868 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8003c60 con 0x55d3bafedba0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 -- 172.21.15.137:0/2256696868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 msgr2=0x55d3bafedfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6d00a2800 0x7fe6d00a4cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc000c00 tx=0x7fe6dc0058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafedfc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8012950 tx=0x7fe6d800bb70 comp rx=0 tx=0).stop 2024-03-26T14:02:13.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 -- 172.21.15.137:0/2256696868 shutdown_connections 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3baff3700 0x55d3baff5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3baff3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2256696868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafedfc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.062+0000 7fe6f480e740 1 -- 172.21.15.137:0/2256696868 >> 172.21.15.137:0/2256696868 conn(0x55d3bafe9110 msgr2=0x55d3bafe9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.063+0000 7fe6f480e740 1 -- 172.21.15.137:0/2256696868 shutdown_connections 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.063+0000 7fe6f480e740 1 -- 172.21.15.137:0/2256696868 wait complete. 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.063+0000 7fe6f480e740 1 -- start start 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.063+0000 7fe6f480e740 1 Processor -- start 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3baff8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafee590 0x55d3baff8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 0x55d3bafff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3baff66b0 con 0x55d3baff3700 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3baf0edd0 con 0x55d3bafedba0 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3baff6830 con 0x55d3bafee590 2024-03-26T14:02:13.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6f480e740 10 client.0 _ll_register_callbacks cb 0x55d3bac66ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 0x55d3bafff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 0x55d3bafff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40784/0 (socket says 172.21.15.137:40784) 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3baff8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 -- 172.21.15.137:0/1139136406 learned_addr learned my addr 172.21.15.137:0/1139136406 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6ea000640 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafee590 0x55d3baff8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 -- 172.21.15.137:0/1139136406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafee590 msgr2=0x55d3baff8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafee590 0x55d3baff8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 -- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 msgr2=0x55d3baff8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3baff8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.064+0000 7fe6eb400640 1 -- 172.21.15.137:0/1139136406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3bac672d0 con 0x55d3baff3700 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6eb400640 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 0x55d3bafff690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6e409bb80 tx=0x7fe6e409c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebaf2b72c4194e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6d4c00640 1 -- 172.21.15.137:0/1139136406 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e409fd40 con 0x55d3baff3700 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6d4c00640 1 -- 172.21.15.137:0/1139136406 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe6e40052f0 con 0x55d3baff3700 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6d80043d0 con 0x55d3baff3700 2024-03-26T14:02:13.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6d4c00640 1 -- 172.21.15.137:0/1139136406 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6e40a4390 con 0x55d3baff3700 2024-03-26T14:02:13.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6f480e740 10 client.15486 fetch_fsmap learned FSMap version 23 2024-03-26T14:02:13.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d3bad316b0 con 0x55d3baff3700 2024-03-26T14:02:13.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.065+0000 7fe6d4c00640 1 -- 172.21.15.137:0/1139136406 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6d80043d0 con 0x55d3baff3700 2024-03-26T14:02:13.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6d4c00640 1 -- 172.21.15.137:0/1139136406 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe6e409aab0 con 0x55d3baff3700 2024-03-26T14:02:13.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6f480e740 10 client.15486 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:02:13.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6f480e740 10 client.15486 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T14:02:13.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6f480e740 20 client.15486 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:13.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6f480e740 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3bb0048b0 0x55d3bb003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6f480e740 4 client.15486 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3baef4520 con 0x55d3bb0048b0 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.066+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3bb0048b0 0x55d3bb003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.067+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3bb0048b0 0x55d3bb003980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b05d0c48983a624 server_cookie=1ac359e9b37b18bf in_seq=0 out_seq=0 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.067+0000 7fe6d4c00640 10 client.15486 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6d4c00640 1 -- 172.21.15.137:0/1139136406 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3baef4520 con 0x55d3bb0048b0 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6d4c00640 10 client.15486 handle_command_reply: tid=0 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 client.15486 shutdown 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 msgr2=0x55d3bafff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 0x55d3bafff690 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe6e409bb80 tx=0x7fe6e409c2f0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 shutdown_connections 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3bb0048b0 0x55d3bb003980 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3baff3700 0x55d3bafff690 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafee590 0x55d3baff8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 --2- 172.21.15.137:0/1139136406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3baff8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 >> 172.21.15.137:0/1139136406 conn(0x55d3bafe9110 msgr2=0x55d3bafe8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.068+0000 7fe6d4c00640 0 client.15486 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:02:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.069+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 shutdown_connections 2024-03-26T14:02:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.069+0000 7fe6f480e740 1 -- 172.21.15.137:0/1139136406 wait complete. 2024-03-26T14:02:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.069+0000 7fe6f480e740 20 client.15486 trim_cache size 0 max 16384 2024-03-26T14:02:13.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 Processor -- start 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 -- start start 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafeffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3bafff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb0040d0 con 0x55d3bafee590 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb004730 con 0x55d3bafedba0 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6f480e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb004d90 con 0x55d3bafe9110 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3bafff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6eb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3bafff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40786/0 (socket says 172.21.15.137:40786) 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6ea000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafeffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.087+0000 7fe6eb400640 1 -- 172.21.15.137:0/2136129284 learned_addr learned my addr 172.21.15.137:0/2136129284 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eb400640 1 -- 172.21.15.137:0/2136129284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 msgr2=0x55d3bafefa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eb400640 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafefa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eb400640 1 -- 172.21.15.137:0/2136129284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 msgr2=0x55d3bafeffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eb400640 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafeffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eb400640 1 -- 172.21.15.137:0/2136129284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3bafed990 con 0x55d3bafee590 2024-03-26T14:02:13.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6eb400640 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3bafff650 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4098000 tx=0x7fe6e409a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b772d626cacaa5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6d7400640 1 -- 172.21.15.137:0/2136129284 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e409fd40 con 0x55d3bafee590 2024-03-26T14:02:13.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6d7400640 1 -- 172.21.15.137:0/2136129284 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6e40b4070 con 0x55d3bafee590 2024-03-26T14:02:13.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.088+0000 7fe6d7400640 1 -- 172.21.15.137:0/2136129284 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40052f0 con 0x55d3bafee590 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.089+0000 7fe6f480e740 1 -- 172.21.15.137:0/2136129284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 msgr2=0x55d3bafff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.089+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3bafff650 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4098000 tx=0x7fe6e409a100 comp rx=0 tx=0).stop 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 -- 172.21.15.137:0/2136129284 shutdown_connections 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafedba0 0x55d3bafeffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafee590 0x55d3bafff650 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 --2- 172.21.15.137:0/2136129284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafefa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 -- 172.21.15.137:0/2136129284 >> 172.21.15.137:0/2136129284 conn(0x55d3bb003cf0 msgr2=0x55d3bafe3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 -- 172.21.15.137:0/2136129284 shutdown_connections 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 -- 172.21.15.137:0/2136129284 wait complete. 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.090+0000 7fe6f480e740 1 -- start start 2024-03-26T14:02:13.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 Processor -- start 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 0x55d3bafff540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafee590 0x55d3baffbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb005a50 con 0x55d3bafedba0 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb0053f0 con 0x55d3bafee590 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3bb0060b0 con 0x55d3bafe9110 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6f480e740 10 client.0 _ll_register_callbacks cb 0x7fe6e40057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6ea000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 0x55d3bafff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6ea000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 0x55d3bafff540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40796/0 (socket says 172.21.15.137:40796) 2024-03-26T14:02:13.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6ea000640 1 -- 172.21.15.137:0/807142449 learned_addr learned my addr 172.21.15.137:0/807142449 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.091+0000 7fe6eb400640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafee590 0x55d3baffbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6ea000640 1 -- 172.21.15.137:0/807142449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 msgr2=0x55d3bafff000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6ea000640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafff000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6ea000640 1 -- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafee590 msgr2=0x55d3baffbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6ea000640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafee590 0x55d3baffbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6ea000640 1 -- 172.21.15.137:0/807142449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3baffc1e0 con 0x55d3bafedba0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6ea000640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 0x55d3bafff540 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80128f0 tx=0x7fe6d8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717b1197479558f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6d4c00640 1 -- 172.21.15.137:0/807142449 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8005e50 con 0x55d3bafedba0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6d4c00640 1 -- 172.21.15.137:0/807142449 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe6d80107f0 con 0x55d3bafedba0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.092+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6e40b4070 con 0x55d3bafedba0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6d4c00640 1 -- 172.21.15.137:0/807142449 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6d8036050 con 0x55d3bafedba0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 10 client.15492 fetch_fsmap learned FSMap version 23 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d3bb0605a0 con 0x55d3bafedba0 2024-03-26T14:02:13.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6d4c00640 1 -- 172.21.15.137:0/807142449 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6e40b4070 con 0x55d3bafedba0 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6d4c00640 1 -- 172.21.15.137:0/807142449 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe6d8031030 con 0x55d3bafedba0 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 10 client.15492 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 10 client.15492 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 20 client.15492 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3baff3700 0x55d3bb060830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 4 client.15492 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.093+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d3baf601d0 con 0x55d3baff3700 2024-03-26T14:02:13.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.094+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3baff3700 0x55d3bb060830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.094+0000 7fe6eaa00640 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3baff3700 0x55d3bb060830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29fa38012cfd61a5 server_cookie=3adb30a32d40796f in_seq=0 out_seq=0 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.094+0000 7fe6d4c00640 10 client.15492 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6d4c00640 1 -- 172.21.15.137:0/807142449 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d3baf601d0 con 0x55d3baff3700 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6d4c00640 10 client.15492 handle_command_reply: tid=0 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 client.15492 shutdown 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 msgr2=0x55d3bafff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 0x55d3bafff540 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80128f0 tx=0x7fe6d8005270 comp rx=0 tx=0).stop 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 shutdown_connections 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55d3baff3700 0x55d3bb060830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3bafee590 0x55d3baffbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3bafedba0 0x55d3bafff540 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 --2- 172.21.15.137:0/807142449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3bafe9110 0x55d3bafff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 >> 172.21.15.137:0/807142449 conn(0x55d3bb003cf0 msgr2=0x55d3baff6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6d4c00640 0 client.15492 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 shutdown_connections 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 1 -- 172.21.15.137:0/807142449 wait complete. 2024-03-26T14:02:13.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.095+0000 7fe6f480e740 20 client.15492 trim_cache size 0 max 16384 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 1, 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34451, 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 144.87184832599999, 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 94, 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 192.19307405699999 2024-03-26T14:02:13.095 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6d00a2800 msgr2=0x7fe6d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6d00a2800 0x7fe6d00a4cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc000c00 tx=0x7fe6dc0058b0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 msgr2=0x7fe6ec100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 0x7fe6ec100620 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400eb10 tx=0x7fe6e400c2d0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 shutdown_connections 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6d00a2800 0x7fe6d00a4cc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6ec104a60 0x7fe6ec1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6ec107390 0x7fe6ec100620 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 --2- 172.21.15.137:0/3621931343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6ec102130 0x7fe6ec101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 >> 172.21.15.137:0/3621931343 conn(0x7fe6ec0fbb40 msgr2=0x7fe6ec103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 shutdown_connections 2024-03-26T14:02:13.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.096+0000 7fe6f1c00640 1 -- 172.21.15.137:0/3621931343 wait complete. 2024-03-26T14:02:13.112 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-26T14:02:13.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 -- 172.21.15.137:0/2030601191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc04103150 msgr2=0x7fdc041099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 --2- 172.21.15.137:0/2030601191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc04103150 0x7fdc041099e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc009be0 tx=0x7fdbfc02bef0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 -- 172.21.15.137:0/2030601191 shutdown_connections 2024-03-26T14:02:13.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 --2- 172.21.15.137:0/2030601191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc04102790 0x7fdc04102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 --2- 172.21.15.137:0/2030601191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc04103150 0x7fdc041099e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 --2- 172.21.15.137:0/2030601191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc041015a0 0x7fdc041019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 -- 172.21.15.137:0/2030601191 >> 172.21.15.137:0/2030601191 conn(0x7fdc040fcb10 msgr2=0x7fdc040fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 -- 172.21.15.137:0/2030601191 shutdown_connections 2024-03-26T14:02:13.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.306+0000 7fdc09c00640 1 -- 172.21.15.137:0/2030601191 wait complete. 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 Processor -- start 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 -- start start 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 0x7fdc04197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc04102790 0x7fdc04197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc04103150 0x7fdc0419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0410d100 con 0x7fdc041015a0 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0410cf80 con 0x7fdc04102790 2024-03-26T14:02:13.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.307+0000 7fdc09c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0410d280 con 0x7fdc04103150 2024-03-26T14:02:13.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 0x7fdc04197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 0x7fdc04197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40804/0 (socket says 172.21.15.137:40804) 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03e00640 1 -- 172.21.15.137:0/4197237403 learned_addr learned my addr 172.21.15.137:0/4197237403 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03400640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc04102790 0x7fdc04197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc09200640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc04103150 0x7fdc0419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03e00640 1 -- 172.21.15.137:0/4197237403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc04103150 msgr2=0x7fdc0419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03e00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc04103150 0x7fdc0419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.308+0000 7fdc03e00640 1 -- 172.21.15.137:0/4197237403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc04102790 msgr2=0x7fdc04197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc03e00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc04102790 0x7fdc04197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc03e00640 1 -- 172.21.15.137:0/4197237403 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0419f340 con 0x7fdc041015a0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc03e00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 0x7fdc04197720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c970 tx=0x7fdbf400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc00c00640 1 -- 172.21.15.137:0/4197237403 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4007c10 con 0x7fdc041015a0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc00c00640 1 -- 172.21.15.137:0/4197237403 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdbf4007db0 con 0x7fdc041015a0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc00c00640 1 -- 172.21.15.137:0/4197237403 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400f870 con 0x7fdc041015a0 2024-03-26T14:02:13.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.309+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0419f630 con 0x7fdc041015a0 2024-03-26T14:02:13.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.310+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0419fbd0 con 0x7fdc041015a0 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc00c00640 1 -- 172.21.15.137:0/4197237403 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdbf400fa10 con 0x7fdc041015a0 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 Processor -- start 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 -- start start 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e894190 0x55c10e8945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 0x55c10e899680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e899cf0 0x55c10e89c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e7f4c90 con 0x55c10e894b80 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e79d980 con 0x55c10e899cf0 2024-03-26T14:02:13.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.313+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e7f0d30 con 0x55c10e894190 2024-03-26T14:02:13.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 0x55c10e899680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 0x55c10e899680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40812/0 (socket says 172.21.15.137:40812) 2024-03-26T14:02:13.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 -- 172.21.15.137:0/4262760093 learned_addr learned my addr 172.21.15.137:0/4262760093 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03400640 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e899cf0 0x55c10e89c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc09200640 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e894190 0x55c10e8945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 -- 172.21.15.137:0/4262760093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e894190 msgr2=0x55c10e8945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e894190 0x55c10e8945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 -- 172.21.15.137:0/4262760093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e899cf0 msgr2=0x55c10e89c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e899cf0 0x55c10e89c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03e00640 1 -- 172.21.15.137:0/4262760093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c10e7b5510 con 0x55c10e894b80 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.314+0000 7fdc03400640 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e899cf0 0x55c10e89c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdc00c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdbe80a2880 0x7fdbe80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdc03e00640 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 0x55c10e899680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40d74b0 tx=0x7fdbf40d75a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd2480aada99832 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdc00c00640 1 -- 172.21.15.137:0/4197237403 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdbf40cf1f0 con 0x7fdc041015a0 2024-03-26T14:02:13.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdc09200640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdbe80a2880 0x7fdbe80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdbf9600640 1 -- 172.21.15.137:0/4262760093 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4097ab0 con 0x55c10e894b80 2024-03-26T14:02:13.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdbf9600640 1 -- 172.21.15.137:0/4262760093 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdbf4018b50 con 0x55c10e894b80 2024-03-26T14:02:13.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.315+0000 7fdbf9600640 1 -- 172.21.15.137:0/4262760093 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf409b170 con 0x55c10e894b80 2024-03-26T14:02:13.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc0e00c740 1 -- 172.21.15.137:0/4262760093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 msgr2=0x55c10e899680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 0x55c10e899680 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40d74b0 tx=0x7fdbf40d75a0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc09200640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdbe80a2880 0x7fdbe80a4d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc034fd0 tx=0x7fdbfc009840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc0e00c740 1 -- 172.21.15.137:0/4262760093 shutdown_connections 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e899cf0 0x55c10e89c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894b80 0x55c10e899680 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.316+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/4262760093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e894190 0x55c10e8945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.317+0000 7fdc0e00c740 1 -- 172.21.15.137:0/4262760093 >> 172.21.15.137:0/4262760093 conn(0x55c10e88f700 msgr2=0x55c10e88fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.317+0000 7fdc0e00c740 1 -- 172.21.15.137:0/4262760093 shutdown_connections 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.317+0000 7fdc0e00c740 1 -- 172.21.15.137:0/4262760093 wait complete. 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.317+0000 7fdc0e00c740 1 -- start start 2024-03-26T14:02:13.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 Processor -- start 2024-03-26T14:02:13.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e89ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e8a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e89cca0 con 0x55c10e894190 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e7b0dd0 con 0x55c10e894b80 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e89ce20 con 0x55c10e899cf0 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc0e00c740 10 client.0 _ll_register_callbacks cb 0x55c10e508ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40814/0 (socket says 172.21.15.137:40814) 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc03e00640 1 -- 172.21.15.137:0/380781002 learned_addr learned my addr 172.21.15.137:0/380781002 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.318+0000 7fdc03400640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e89ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03e00640 1 -- 172.21.15.137:0/380781002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 msgr2=0x55c10e8a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc09200640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e8a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03e00640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e8a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03e00640 1 -- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 msgr2=0x55c10e89ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03e00640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e89ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03e00640 1 -- 172.21.15.137:0/380781002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c10e5092d0 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03400640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e89ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc03e00640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89e690 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40d7480 tx=0x7fdbf40d2e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd839fea22e50f77 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdbee000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdbee000640 1 -- 172.21.15.137:0/380781002 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf409b350 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdbee000640 1 -- 172.21.15.137:0/380781002 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdbf40d7cb0 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.319+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c10e796520 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdbee000640 1 -- 172.21.15.137:0/380781002 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf409b5e0 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 10 client.15504 fetch_fsmap learned FSMap version 23 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c10e5d36b0 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdbee000640 1 -- 172.21.15.137:0/380781002 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55c10e796520 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdbee000640 1 -- 172.21.15.137:0/380781002 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdbf4086070 con 0x55c10e894190 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 10 client.15504 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 10 client.15504 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 20 client.15504 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e8aa740 0x55c10e8aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 4 client.15504 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:02:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.320+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c10e8021d0 con 0x55c10e8aa740 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.321+0000 7fdc03400640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e8aa740 0x55c10e8aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.321+0000 7fdc03400640 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e8aa740 0x55c10e8aebc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52ee34a7ddd1a01 server_cookie=88411492b0594127 in_seq=0 out_seq=0 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.321+0000 7fdbee000640 10 client.15504 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdbee000640 1 -- 172.21.15.137:0/380781002 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c10e8021d0 con 0x55c10e8aa740 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdbee000640 10 client.15504 handle_command_reply: tid=0 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 client.15504 shutdown 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 msgr2=0x55c10e89e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89e690 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40d7480 tx=0x7fdbf40d2e20 comp rx=0 tx=0).stop 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 shutdown_connections 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e8aa740 0x55c10e8aebc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e89ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e8a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdbee000640 0 client.15504 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/380781002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89e690 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 >> 172.21.15.137:0/380781002 conn(0x55c10e88f700 msgr2=0x55c10e88a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 shutdown_connections 2024-03-26T14:02:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 1 -- 172.21.15.137:0/380781002 wait complete. 2024-03-26T14:02:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.323+0000 7fdc0e00c740 20 client.15504 trim_cache size 0 max 16384 2024-03-26T14:02:13.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.341+0000 7fdc0e00c740 1 Processor -- start 2024-03-26T14:02:13.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 -- start start 2024-03-26T14:02:13.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e8a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e893070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e8a9f60 con 0x55c10e894190 2024-03-26T14:02:13.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e8aa5c0 con 0x55c10e894b80 2024-03-26T14:02:13.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e8aac20 con 0x55c10e899cf0 2024-03-26T14:02:13.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40830/0 (socket says 172.21.15.137:40830) 2024-03-26T14:02:13.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc03e00640 1 -- 172.21.15.137:0/2557109605 learned_addr learned my addr 172.21.15.137:0/2557109605 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.342+0000 7fdc03400640 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e8a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc09200640 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e893070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc03e00640 1 -- 172.21.15.137:0/2557109605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 msgr2=0x55c10e893070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc03e00640 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e893070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc03e00640 1 -- 172.21.15.137:0/2557109605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 msgr2=0x55c10e8a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc03e00640 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e8a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc03e00640 1 -- 172.21.15.137:0/2557109605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c10e87d170 con 0x55c10e894190 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdc03e00640 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89eb30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c940 tx=0x7fdbf409fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cc130872bce8fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdbf8c00640 1 -- 172.21.15.137:0/2557109605 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf40b4080 con 0x55c10e894190 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdbf8c00640 1 -- 172.21.15.137:0/2557109605 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdbf409bf10 con 0x55c10e894190 2024-03-26T14:02:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.343+0000 7fdbf8c00640 1 -- 172.21.15.137:0/2557109605 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf409b170 con 0x55c10e894190 2024-03-26T14:02:13.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 -- 172.21.15.137:0/2557109605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 msgr2=0x55c10e89eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89eb30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c940 tx=0x7fdbf409fa70 comp rx=0 tx=0).stop 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 -- 172.21.15.137:0/2557109605 shutdown_connections 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e8a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e893070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/2557109605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e89eb30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 -- 172.21.15.137:0/2557109605 >> 172.21.15.137:0/2557109605 conn(0x55c10e88f700 msgr2=0x55c10e892960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 -- 172.21.15.137:0/2557109605 shutdown_connections 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.344+0000 7fdc0e00c740 1 -- 172.21.15.137:0/2557109605 wait complete. 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 -- start start 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 Processor -- start 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e8a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e901cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e904370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e8ab8e0 con 0x55c10e894190 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e8ab280 con 0x55c10e894b80 2024-03-26T14:02:13.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10e8abf40 con 0x55c10e899cf0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.345+0000 7fdc0e00c740 10 client.0 _ll_register_callbacks cb 0x7fdbf0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e8a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e8a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40842/0 (socket says 172.21.15.137:40842) 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 -- 172.21.15.137:0/3766332767 learned_addr learned my addr 172.21.15.137:0/3766332767 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03400640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e901cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc09200640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e904370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 -- 172.21.15.137:0/3766332767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 msgr2=0x55c10e904370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e904370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 -- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 msgr2=0x55c10e901cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e901cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 -- 172.21.15.137:0/3766332767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c10e904ba0 con 0x55c10e894190 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdc03e00640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e8a3bc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4099ac0 tx=0x7fdbf400ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574391c0cd3e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdbee000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.346+0000 7fdbee000640 1 -- 172.21.15.137:0/3766332767 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf40b4070 con 0x55c10e894190 2024-03-26T14:02:13.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdbee000640 1 -- 172.21.15.137:0/3766332767 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdbf409bc70 con 0x55c10e894190 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbf409bf10 con 0x55c10e894190 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdbee000640 1 -- 172.21.15.137:0/3766332767 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf409b170 con 0x55c10e894190 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 10 client.15510 fetch_fsmap learned FSMap version 23 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c10e905ea0 con 0x55c10e894190 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdbee000640 1 -- 172.21.15.137:0/3766332767 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdbf409bf10 con 0x55c10e894190 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdbee000640 1 -- 172.21.15.137:0/3766332767 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 23) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdbf40afb60 con 0x55c10e894190 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 10 client.15510 fetch_fsmap finished waiting for FSMap version 23 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 10 client.15510 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 20 client.15510 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e90a140 0x55c10e906130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 4 client.15510 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-26T14:02:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.347+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c10e7fc520 con 0x55c10e90a140 2024-03-26T14:02:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.348+0000 7fdc03400640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e90a140 0x55c10e906130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.348+0000 7fdc03400640 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e90a140 0x55c10e906130 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb69b415e3d1d31 server_cookie=d72d8f9a83aaff5e in_seq=0 out_seq=0 2024-03-26T14:02:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.348+0000 7fdbee000640 10 client.15510 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdbee000640 1 -- 172.21.15.137:0/3766332767 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c10e7fc520 con 0x55c10e90a140 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdbee000640 10 client.15510 handle_command_reply: tid=0 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 client.15510 shutdown 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 msgr2=0x55c10e8a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e8a3bc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4099ac0 tx=0x7fdbf400ff30 comp rx=0 tx=0).stop 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 shutdown_connections 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c10e90a140 0x55c10e906130 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c10e894b80 0x55c10e901cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c10e899cf0 0x55c10e904370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdbee000640 0 client.15510 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 --2- 172.21.15.137:0/3766332767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c10e894190 0x55c10e8a3bc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 >> 172.21.15.137:0/3766332767 conn(0x55c10e88f700 msgr2=0x55c10e89da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 shutdown_connections 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 1 -- 172.21.15.137:0/3766332767 wait complete. 2024-03-26T14:02:13.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.349+0000 7fdc0e00c740 20 client.15510 trim_cache size 0 max 16384 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "cluster_fsid": "66aee2a0-eb77-11ee-95d0-87774f69a715", 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "whoami": 0, 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 34496, 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "want_state": "up:active", 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "state": "up:active", 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs", 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "rank_uptime": 170.054387147, 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "mdsmap_epoch": 23, 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch": 94, 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_epoch_barrier": 90, 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout: "uptime": 176.36743943499999 2024-03-26T14:02:13.350 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdbe80a2880 msgr2=0x7fdbe80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdbe80a2880 0x7fdbe80a4d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc034fd0 tx=0x7fdbfc009840 comp rx=0 tx=0).stop 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 msgr2=0x7fdc04197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 0x7fdc04197720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400c970 tx=0x7fdbf400ce30 comp rx=0 tx=0).stop 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 shutdown_connections 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdbe80a2880 0x7fdbe80a4d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc04102790 0x7fdc04197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc04103150 0x7fdc0419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 --2- 172.21.15.137:0/4197237403 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdc041015a0 0x7fdc04197720 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 >> 172.21.15.137:0/4197237403 conn(0x7fdc040fcb10 msgr2=0x7fdc04105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:13.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 shutdown_connections 2024-03-26T14:02:13.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:13.350+0000 7fdc09c00640 1 -- 172.21.15.137:0/4197237403 wait complete. 2024-03-26T14:02:13.365 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-26T14:02:13.376 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-26T14:02:13.376 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-26T14:02:13.376 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-26T14:02:13.376 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-26T14:02:13.376 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:13.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:13.376 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-03-26T14:02:13.418 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-26T14:02:13.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:13.419 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-26T14:02:13.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:13.531 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-03-26T14:02:13.656 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-26T14:02:13.656 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph fs ls 2024-03-26T14:02:13.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:13 smithi159 ceph-mon[35501]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:02:13.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:13.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:13.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:13.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:13 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:13 smithi137 ceph-mon[31539]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:02:14.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.032 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:13 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.084 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:02:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:13 smithi179 ceph-mon[35472]: pgmap v454: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:02:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:14.171 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:13 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.782 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:15 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:15 smithi137 ceph-mon[31539]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:15.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:15 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:15 smithi179 ceph-mon[35472]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:15.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:15 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:15.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:15 smithi159 ceph-mon[35501]: pgmap v455: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:16.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.438+0000 7fc9a7600700 1 -- 172.21.15.137:0/945639986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a006a7e0 msgr2=0x7fc9a010f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:16.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.438+0000 7fc9a7600700 1 --2- 172.21.15.137:0/945639986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a006a7e0 0x7fc9a010f020 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc99800b0d0 tx=0x7fc99800b3e0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 -- 172.21.15.137:0/945639986 shutdown_connections 2024-03-26T14:02:16.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 --2- 172.21.15.137:0/945639986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a006a7e0 0x7fc9a010f020 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 --2- 172.21.15.137:0/945639986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9a01081e0 0x7fc9a01085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 --2- 172.21.15.137:0/945639986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a0069e40 0x7fc9a006a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 -- 172.21.15.137:0/945639986 >> 172.21.15.137:0/945639986 conn(0x7fc9a0077fc0 msgr2=0x7fc9a00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:16.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 -- 172.21.15.137:0/945639986 shutdown_connections 2024-03-26T14:02:16.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.439+0000 7fc9a7600700 1 -- 172.21.15.137:0/945639986 wait complete. 2024-03-26T14:02:16.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.440+0000 7fc9a7600700 1 Processor -- start 2024-03-26T14:02:16.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.440+0000 7fc9a7600700 1 -- start start 2024-03-26T14:02:16.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a7600700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9a0069e40 0x7fc9a01a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:16.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a7600700 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a006a7e0 0x7fc9a01a5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:16.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a7600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 0x7fc9a010d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a7600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a0112360 con 0x7fc9a01081e0 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a7600700 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a01121e0 con 0x7fc9a006a7e0 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a7600700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a01124e0 con 0x7fc9a0069e40 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a5600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 0x7fc9a010d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a5600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 0x7fc9a010d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40858/0 (socket says 172.21.15.137:40858) 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a5600700 1 -- 172.21.15.137:0/2126820919 learned_addr learned my addr 172.21.15.137:0/2126820919 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc99fe00700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a006a7e0 0x7fc9a01a5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:16.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.441+0000 7fc9a4c00700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9a0069e40 0x7fc9a01a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc9a5600700 1 -- 172.21.15.137:0/2126820919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9a0069e40 msgr2=0x7fc9a01a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc9a5600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9a0069e40 0x7fc9a01a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc9a5600700 1 -- 172.21.15.137:0/2126820919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a006a7e0 msgr2=0x7fc9a01a5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc9a5600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a006a7e0 0x7fc9a01a5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc9a5600700 1 -- 172.21.15.137:0/2126820919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a010d7e0 con 0x7fc9a01081e0 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc99fe00700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a006a7e0 0x7fc9a01a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc9a5600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 0x7fc9a010d010 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc9980079a0 tx=0x7fc9980079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.442+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99801f070 con 0x7fc9a01081e0 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.443+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc998003c00 con 0x7fc9a01081e0 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.443+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99801b690 con 0x7fc9a01081e0 2024-03-26T14:02:16.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.443+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a010da70 con 0x7fc9a01081e0 2024-03-26T14:02:16.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.443+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a01ab940 con 0x7fc9a01081e0 2024-03-26T14:02:16.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.444+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a0068e50 con 0x7fc9a01081e0 2024-03-26T14:02:16.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.449+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc998022070 con 0x7fc9a01081e0 2024-03-26T14:02:16.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.450+0000 7fc99d600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9880a2890 0x7fc9880a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:16.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.451+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc99800f020 con 0x7fc9a01081e0 2024-03-26T14:02:16.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.451+0000 7fc9a4c00700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9880a2890 0x7fc9880a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:16.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.452+0000 7fc9a4c00700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9880a2890 0x7fc9880a4d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9940045b0 tx=0x7fc99400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:16.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.453+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc998029020 con 0x7fc9a01081e0 2024-03-26T14:02:16.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.724+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9a01abb10 con 0x7fc9a01081e0 2024-03-26T14:02:16.723 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:02:16.723 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:02:16.723 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:16.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.725+0000 7fc99d600700 1 -- 172.21.15.137:0/2126820919 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc99809d0c0 con 0x7fc9a01081e0 2024-03-26T14:02:16.723 INFO:teuthology.orchestra.run.smithi137.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9880a2890 msgr2=0x7fc9880a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9880a2890 0x7fc9880a4d50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc9940045b0 tx=0x7fc99400b3c0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 msgr2=0x7fc9a010d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 0x7fc9a010d010 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc9980079a0 tx=0x7fc9980079d0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 shutdown_connections 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc9880a2890 0x7fc9880a4d50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc9a01081e0 0x7fc9a010d010 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9a0069e40 0x7fc9a01a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 --2- 172.21.15.137:0/2126820919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc9a006a7e0 0x7fc9a01a5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:16.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 >> 172.21.15.137:0/2126820919 conn(0x7fc9a0077fc0 msgr2=0x7fc9a0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:16.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 shutdown_connections 2024-03-26T14:02:16.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:16.729+0000 7fc9a7600700 1 -- 172.21.15.137:0/2126820919 wait complete. 2024-03-26T14:02:16.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:02:16.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:02:16.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:16.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:02:16.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:02:16.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:02:17.580 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-26T14:02:17.580 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-26T14:02:17.580 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi137.front.sepia.ceph.com 2024-03-26T14:02:17.580 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-26T14:02:17.581 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:17.581 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-26T14:02:17.581 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-26T14:02:17.581 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-26T14:02:17.581 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-26T14:02:17.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:17.581 DEBUG:teuthology.orchestra.run.smithi137:> ip addr 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout: inet 127.0.0.1/8 scope host lo 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 ::1/128 scope host 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T14:02:17.636 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7e brd ff:ff:ff:ff:ff:ff 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f0 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:d9:36:7f brd ff:ff:ff:ff:ff:ff 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: altname enp7s0f1 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:36 brd ff:ff:ff:ff:ff:ff 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: link/ether 0c:c4:7a:88:81:37 brd ff:ff:ff:ff:ff:ff 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: inet 172.21.15.137/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft 41729sec preferred_lft 41729sec 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: inet6 fe80::ec4:7aff:fe88:8137/64 scope link 2024-03-26T14:02:17.637 INFO:teuthology.orchestra.run.smithi137.stdout: valid_lft forever preferred_lft forever 2024-03-26T14:02:17.637 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-26T14:02:17.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add name ceph-brx type bridge 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr flush dev ceph-brx 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set ceph-brx up 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-26T14:02:17.638 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T14:02:17.760 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:17 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2126820919' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T14:02:17.761 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:17 smithi137 ceph-mon[31539]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:17.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:17.853 DEBUG:teuthology.orchestra.run.smithi137:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-26T14:02:17.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:17 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2126820919' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T14:02:17.921 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:17 smithi179 ceph-mon[35472]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:17.927 INFO:teuthology.orchestra.run.smithi137.stdout:1 2024-03-26T14:02:17.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:17.931 DEBUG:teuthology.orchestra.run.smithi137:> route 2024-03-26T14:02:17.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:17 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2126820919' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-26T14:02:17.950 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:17 smithi159 ceph-mon[35501]: pgmap v456: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:02:17.982 INFO:teuthology.orchestra.run.smithi137.stdout:Kernel IP routing table 2024-03-26T14:02:17.982 INFO:teuthology.orchestra.run.smithi137.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-26T14:02:17.982 INFO:teuthology.orchestra.run.smithi137.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-26T14:02:17.982 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-26T14:02:17.982 INFO:teuthology.orchestra.run.smithi137.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-26T14:02:17.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:17.982 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T14:02:17.983 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T14:02:17.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-26T14:02:17.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-26T14:02:17.983 DEBUG:teuthology.orchestra.run.smithi137:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-26T14:02:17.983 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T14:02:18.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:18.158 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-03-26T14:02:18.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:18.208 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list-id 2024-03-26T14:02:18.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:18.257 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T14:02:18.257 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T14:02:18.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:18.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-26T14:02:18.257 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T14:02:18.394 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-26T14:02:18.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-26T14:02:18.395 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T14:02:18.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:18.644 DEBUG:teuthology.orchestra.run.smithi137:> (cd / && exec bash -c ' 2024-03-26T14:02:18.644 DEBUG:teuthology.orchestra.run.smithi137:> set -e 2024-03-26T14:02:18.644 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set brx.0 up 2024-03-26T14:02:18.644 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link set dev brx.0 master ceph-brx 2024-03-26T14:02:18.644 DEBUG:teuthology.orchestra.run.smithi137:> ') 2024-03-26T14:02:18.808 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-26T14:02:18.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T14:02:18.809 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:18.857 INFO:teuthology.orchestra.run.smithi137.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-26T14:02:18.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-26T14:02:18.857 DEBUG:teuthology.orchestra.run.smithi137:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:18.948 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-03-26T14:02:19.039 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-03-26T14:02:19.092 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay 2024-03-26T14:02:19.093 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/6d5176c50bb0924b5697fe484de22ac7910bd0182cb6638d95cb41ddc003720a/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/ef7699130a1edbef8eca4ceb5dee06744f5f7dce4795f904cf09d267f7a491af/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/0 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/aa69ad5c31bdc2ed88bdc366aaadc85a8ea7d3a0b1728d83208028cec057bb5f/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/f5ef712a5805662c301f59e342c4dc1c0b4d4d0dbe51bc5c1a22a3290c7704d8/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/5f4b573d55b2740cad2b22819d0378791b0e9bab5aa9b4a7cf8915ce4a2892ae/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/b68dcbdc09abd5f92e66e58475d97fd1488aa21a66a54942044551525518fa32/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/e553abe0ad1f21fb92e8b35c38f139a5239ff1495e80ff77e44f7036b09abc4d/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/525c7d55962104ce64112cfaef76e05acc035d714ebccbdd99e5d57e87cbf5b8/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/dac0fd409288119dad4627664a2cf3f341d8f8920adf422a09110be7380bd00f/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/194c9c7a044b837c9d3bb06ae7d20494d1994a0ae34421cefa18bccdd79adf3b/merged 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:19.094 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:19.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:19.095 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-03-26T14:02:19.143 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-26T14:02:19.143 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/9f3c23c9-2b6e-40ba-9f8e-0d9d1fbd6b47 --client_fs=cephfs) 2024-03-26T14:02:19.148 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-03-26T14:02:19.226 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T14:02:19.270 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-03-26T14:02:19.270 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-03-26T14:02:19.271 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/6d5176c50bb0924b5697fe484de22ac7910bd0182cb6638d95cb41ddc003720a/merged 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/ef7699130a1edbef8eca4ceb5dee06744f5f7dce4795f904cf09d267f7a491af/merged 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/0 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/aa69ad5c31bdc2ed88bdc366aaadc85a8ea7d3a0b1728d83208028cec057bb5f/merged 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/f5ef712a5805662c301f59e342c4dc1c0b4d4d0dbe51bc5c1a22a3290c7704d8/merged 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/5f4b573d55b2740cad2b22819d0378791b0e9bab5aa9b4a7cf8915ce4a2892ae/merged 2024-03-26T14:02:19.272 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/b68dcbdc09abd5f92e66e58475d97fd1488aa21a66a54942044551525518fa32/merged 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/e553abe0ad1f21fb92e8b35c38f139a5239ff1495e80ff77e44f7036b09abc4d/merged 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/525c7d55962104ce64112cfaef76e05acc035d714ebccbdd99e5d57e87cbf5b8/merged 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/dac0fd409288119dad4627664a2cf3f341d8f8920adf422a09110be7380bd00f/merged 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/194c9c7a044b837c9d3bb06ae7d20494d1994a0ae34421cefa18bccdd79adf3b/merged 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:19.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:19.274 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-03-26T14:02:19.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:2024-03-26T14:02:19.354+0000 7fbc10c39480 -1 init, newargv = 0x55d23e432230 newargc=15 2024-03-26T14:02:19.351 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:ceph-fuse[86381]: starting ceph client 2024-03-26T14:02:19.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:ceph-fuse[86381]: starting fuse 2024-03-26T14:02:20.461 DEBUG:teuthology.orchestra.run.smithi137:> sudo modprobe fuse 2024-03-26T14:02:20.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:20 smithi137 ceph-mon[31539]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:20.561 DEBUG:teuthology.orchestra.run.smithi137:> cat /proc/self/mounts | awk '{print $2}' 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/proc 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/sys 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/dev 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/security 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/shm 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/pts 2024-03-26T14:02:20.615 INFO:teuthology.orchestra.run.smithi137.stdout:/run 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/cgroup 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/pstore 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/bpf 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/ 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/selinux 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/mqueue 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/dev/hugepages 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/debug 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/tracing 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/fs/fuse/connections 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/sys/kernel/config 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-sysctl.service 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-26T14:02:20.616 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/nfs/rpc_pipefs 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/1000 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/ceph 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/6d5176c50bb0924b5697fe484de22ac7910bd0182cb6638d95cb41ddc003720a/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/ef7699130a1edbef8eca4ceb5dee06744f5f7dce4795f904cf09d267f7a491af/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/run/user/0 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/proc/sys/fs/binfmt_misc 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/aa69ad5c31bdc2ed88bdc366aaadc85a8ea7d3a0b1728d83208028cec057bb5f/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/f5ef712a5805662c301f59e342c4dc1c0b4d4d0dbe51bc5c1a22a3290c7704d8/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/5f4b573d55b2740cad2b22819d0378791b0e9bab5aa9b4a7cf8915ce4a2892ae/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/b68dcbdc09abd5f92e66e58475d97fd1488aa21a66a54942044551525518fa32/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/e553abe0ad1f21fb92e8b35c38f139a5239ff1495e80ff77e44f7036b09abc4d/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/525c7d55962104ce64112cfaef76e05acc035d714ebccbdd99e5d57e87cbf5b8/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/dac0fd409288119dad4627664a2cf3f341d8f8920adf422a09110be7380bd00f/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/var/lib/containers/storage/overlay/194c9c7a044b837c9d3bb06ae7d20494d1994a0ae34421cefa18bccdd79adf3b/merged 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns 2024-03-26T14:02:20.617 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:20.618 INFO:teuthology.orchestra.run.smithi137.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:02:20.618 INFO:teuthology.orchestra.run.smithi137.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:20.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:20.619 DEBUG:teuthology.orchestra.run.smithi137:> ls /sys/fs/fuse/connections 2024-03-26T14:02:20.670 INFO:teuthology.orchestra.run.smithi137.stdout:118 2024-03-26T14:02:20.670 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo stdin-killer -- python3 -c ' 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> import glob 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> import re 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> import os 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> import subprocess 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T14:02:20.670 DEBUG:teuthology.orchestra.run.smithi137:> def _find_admin_socket(client_name): 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> files = glob.glob(asok_path) 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> # Given a non-glob path, it better be there 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> if "*" not in asok_path: 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> assert(len(files) == 1) 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> return files[0] 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> for f in files: 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> contents = proc_f.read() 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> if mountpoint in contents: 2024-03-26T14:02:20.671 DEBUG:teuthology.orchestra.run.smithi137:> return f 2024-03-26T14:02:20.672 DEBUG:teuthology.orchestra.run.smithi137:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-26T14:02:20.672 DEBUG:teuthology.orchestra.run.smithi137:> 2024-03-26T14:02:20.672 DEBUG:teuthology.orchestra.run.smithi137:> print(_find_admin_socket("client.0")) 2024-03-26T14:02:20.672 DEBUG:teuthology.orchestra.run.smithi137:> ' 2024-03-26T14:02:20.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:20 smithi179 ceph-mon[35472]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:20.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:20 smithi159 ceph-mon[35501]: pgmap v457: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:02:20.844 INFO:teuthology.orchestra.run.smithi137.stdout:/var/run/ceph/ceph-client.0.86381.asok 2024-03-26T14:02:20.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-26T14:02:20.862 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.86381.asok 2024-03-26T14:02:20.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:20.862 DEBUG:teuthology.orchestra.run.smithi137:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.86381.asok status 2024-03-26T14:02:21.095 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:02:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "metadata": { 2024-03-26T14:02:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_sha1": "3be9a5c9bc793e11e2800a8c0c696e8b46742033", 2024-03-26T14:02:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-2145-g3be9a5c9 (3be9a5c9bc793e11e2800a8c0c696e8b46742033) squid (dev)", 2024-03-26T14:02:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "entity_id": "0", 2024-03-26T14:02:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": "smithi137", 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "pid": "86381", 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "root": "/volumes/qa/sv_0/9f3c23c9-2b6e-40ba-9f8e-0d9d1fbd6b47" 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_count": 0, 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "dentry_pinned_count": 0, 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "id": 15522, 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "inst": { 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "name": { 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "client", 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "num": 15522 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-03-26T14:02:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 4020873486 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": { 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "192.168.0.1:0", 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 4020873486 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "inst_str": "client.15522 192.168.0.1:0/4020873486", 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "addr_str": "192.168.0.1:0/4020873486", 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "inode_count": 5, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_epoch": 23, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch": 94, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_epoch_barrier": 0, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "blocklisted": false, 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "fs_name": "cephfs" 2024-03-26T14:02:21.097 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:02:21.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:21.110 DEBUG:teuthology.orchestra.run.smithi137:> stat --file-system '--printf=%T 2024-03-26T14:02:21.110 DEBUG:teuthology.orchestra.run.smithi137:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:21.161 INFO:teuthology.orchestra.run.smithi137.stdout:fuseblk 2024-03-26T14:02:21.161 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:21.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:21.161 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:21.254 INFO:teuthology.run_tasks:Running task exec... 2024-03-26T14:02:21.264 INFO:teuthology.task.exec:Executing custom commands... 2024-03-26T14:02:21.264 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T14:02:21.264 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-26T14:02:21.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 -- 172.21.15.137:0/2894615895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1114103280 msgr2=0x7f1114109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:21.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 --2- 172.21.15.137:0/2894615895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1114103280 0x7f1114109b10 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f110c009ca0 tx=0x7f110c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:21.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 -- 172.21.15.137:0/2894615895 shutdown_connections 2024-03-26T14:02:21.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 --2- 172.21.15.137:0/2894615895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11141028c0 0x7f1114102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:21.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 --2- 172.21.15.137:0/2894615895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1114103280 0x7f1114109b10 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:21.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 --2- 172.21.15.137:0/2894615895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11141016d0 0x7f1114101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:21.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 -- 172.21.15.137:0/2894615895 >> 172.21.15.137:0/2894615895 conn(0x7f11140fcc60 msgr2=0x7f11140ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:21.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 -- 172.21.15.137:0/2894615895 shutdown_connections 2024-03-26T14:02:21.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.473+0000 7f111b400640 1 -- 172.21.15.137:0/2894615895 wait complete. 2024-03-26T14:02:21.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 Processor -- start 2024-03-26T14:02:21.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 -- start start 2024-03-26T14:02:21.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 0x7f11141999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:21.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11141028c0 0x7f1114199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1114103280 0x7f11141a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111410d200 con 0x7f11141016d0 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111410d080 con 0x7f11141028c0 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.474+0000 7f111b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111410d380 con 0x7f1114103280 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 0x7f11141999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 0x7f11141999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36656/0 (socket says 172.21.15.137:36656) 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 -- 172.21.15.137:0/3191946245 learned_addr learned my addr 172.21.15.137:0/3191946245 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1113e00640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11141028c0 0x7f1114199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1119400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1114103280 0x7f11141a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 -- 172.21.15.137:0/3191946245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1114103280 msgr2=0x7f11141a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1114103280 0x7f11141a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 -- 172.21.15.137:0/3191946245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11141028c0 msgr2=0x7f1114199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11141028c0 0x7f1114199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:21.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 -- 172.21.15.137:0/3191946245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11141a15f0 con 0x7f11141016d0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.475+0000 7f1118a00640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 0x7f11141999d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f110800d950 tx=0x7f110800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.476+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1108017070 con 0x7f11141016d0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.476+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1108009c30 con 0x7f11141016d0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.476+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110800f620 con 0x7f11141016d0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.476+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11141a18e0 con 0x7f11141016d0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.476+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11141a1de0 con 0x7f11141016d0 2024-03-26T14:02:21.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.477+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11141a1fe0 con 0x7f11141016d0 2024-03-26T14:02:21.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.480+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f110801a070 con 0x7f11141016d0 2024-03-26T14:02:21.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.481+0000 7f1111600640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f10fc0a28a0 0x7f10fc0a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:21.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.481+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f11080ce8a0 con 0x7f11141016d0 2024-03-26T14:02:21.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.481+0000 7f1113e00640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f10fc0a28a0 0x7f10fc0a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:21.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.482+0000 7f1113e00640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f10fc0a28a0 0x7f10fc0a4d60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f111419aee0 tx=0x7f1100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:21.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.486+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1108009de0 con 0x7f11141016d0 2024-03-26T14:02:21.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:21.726+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f11141a2370 con 0x7f11141016d0 2024-03-26T14:02:22.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.225+0000 7f1111600640 1 -- 172.21.15.137:0/3191946245 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f110809b530 con 0x7f11141016d0 2024-03-26T14:02:22.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.228+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f10fc0a28a0 msgr2=0x7f10fc0a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.228+0000 7f111b400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f10fc0a28a0 0x7f10fc0a4d60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f111419aee0 tx=0x7f1100005f90 comp rx=0 tx=0).stop 2024-03-26T14:02:22.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 msgr2=0x7f11141999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 0x7f11141999d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f110800d950 tx=0x7f110800de10 comp rx=0 tx=0).stop 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 shutdown_connections 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f10fc0a28a0 0x7f10fc0a4d60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11141028c0 0x7f1114199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1114103280 0x7f11141a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 --2- 172.21.15.137:0/3191946245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f11141016d0 0x7f11141999d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 >> 172.21.15.137:0/3191946245 conn(0x7f11140fcc60 msgr2=0x7f11140738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 shutdown_connections 2024-03-26T14:02:22.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.229+0000 7f111b400640 1 -- 172.21.15.137:0/3191946245 wait complete. 2024-03-26T14:02:22.246 DEBUG:teuthology.orchestra.run.smithi137:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 -- 172.21.15.137:0/340885634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 msgr2=0x7fd4e4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/340885634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e4109190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009900 tx=0x7fd4cc02bc40 comp rx=0 tx=0).stop 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 -- 172.21.15.137:0/340885634 shutdown_connections 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/340885634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e4109190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/340885634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e41096d0 0x7fd4e410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/340885634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e4107b20 0x7fd4e4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.454+0000 7fd4e3e00640 1 -- 172.21.15.137:0/340885634 >> 172.21.15.137:0/340885634 conn(0x7fd4e4076ae0 msgr2=0x7fd4e4078f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.455+0000 7fd4e3e00640 1 -- 172.21.15.137:0/340885634 shutdown_connections 2024-03-26T14:02:22.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.455+0000 7fd4e3e00640 1 -- 172.21.15.137:0/340885634 wait complete. 2024-03-26T14:02:22.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 Processor -- start 2024-03-26T14:02:22.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 -- start start 2024-03-26T14:02:22.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 0x7fd4e419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e419e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e41096d0 0x7fd4e41a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e4113630 con 0x7fd4e4107b20 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e41134b0 con 0x7fd4e4108d10 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.456+0000 7fd4e3e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e41137b0 con 0x7fd4e41096d0 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 0x7fd4e419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 0x7fd4e419dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36666/0 (socket says 172.21.15.137:36666) 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 -- 172.21.15.137:0/1619578382 learned_addr learned my addr 172.21.15.137:0/1619578382 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4dbe00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e419e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 -- 172.21.15.137:0/1619578382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e41096d0 msgr2=0x7fd4e41a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e3400640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e41096d0 0x7fd4e41a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e41096d0 0x7fd4e41a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 -- 172.21.15.137:0/1619578382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 msgr2=0x7fd4e419e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e419e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 -- 172.21.15.137:0/1619578382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e41a59b0 con 0x7fd4e4107b20 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4dbe00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:22.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.457+0000 7fd4e2a00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 0x7fd4e419dda0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9c0 tx=0x7fd4d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:22.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.458+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400cd80 con 0x7fd4e4107b20 2024-03-26T14:02:22.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.458+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e41a5ca0 con 0x7fd4e4107b20 2024-03-26T14:02:22.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.458+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd4d40035f0 con 0x7fd4e4107b20 2024-03-26T14:02:22.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.458+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4010690 con 0x7fd4e4107b20 2024-03-26T14:02:22.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.458+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e41a61a0 con 0x7fd4e4107b20 2024-03-26T14:02:22.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.459+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e410ef10 con 0x7fd4e4107b20 2024-03-26T14:02:22.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.461+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd4d40037a0 con 0x7fd4e4107b20 2024-03-26T14:02:22.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.462+0000 7fd4e8a00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4c80a27f0 0x7fd4c80a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.462+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd4d40cf3d0 con 0x7fd4e4107b20 2024-03-26T14:02:22.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.462+0000 7fd4dbe00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4c80a27f0 0x7fd4c80a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.463+0000 7fd4dbe00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4c80a27f0 0x7fd4c80a4cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc006010 tx=0x7fd4cc02c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:22.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.466+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4d409c060 con 0x7fd4e4107b20 2024-03-26T14:02:22.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-mon[31539]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:22.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3191946245' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T14:02:22.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:22 smithi179 ceph-mon[35472]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:22.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:22 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3191946245' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T14:02:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.695+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4e4061b80 con 0x7fd4c80a27f0 2024-03-26T14:02:22.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:22 smithi159 ceph-mon[35501]: pgmap v458: 129 pgs: 129 active+clean; 647 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:02:22.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:22 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3191946245' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-26T14:02:22.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.727+0000 7fd4e8a00640 1 -- 172.21.15.137:0/1619578382 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7fd4e4061b80 con 0x7fd4c80a27f0 2024-03-26T14:02:22.725 INFO:teuthology.orchestra.run.smithi137.stdout:{} 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4c80a27f0 msgr2=0x7fd4c80a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4c80a27f0 0x7fd4c80a4cb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc006010 tx=0x7fd4cc02c7f0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 msgr2=0x7fd4e419dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 0x7fd4e419dda0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd4d400e9c0 tx=0x7fd4d400ee80 comp rx=0 tx=0).stop 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 shutdown_connections 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4c80a27f0 0x7fd4c80a4cb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4e4108d10 0x7fd4e419e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4e41096d0 0x7fd4e41a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 --2- 172.21.15.137:0/1619578382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e4107b20 0x7fd4e419dda0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 >> 172.21.15.137:0/1619578382 conn(0x7fd4e4076ae0 msgr2=0x7fd4e410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 shutdown_connections 2024-03-26T14:02:22.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.730+0000 7fd4e3e00640 1 -- 172.21.15.137:0/1619578382 wait complete. 2024-03-26T14:02:22.745 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-26T14:02:22.755 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-03-26T14:02:22.765 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 -- 172.21.15.137:0/760219884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c105490 msgr2=0x7ffb6c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 --2- 172.21.15.137:0/760219884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c105490 0x7ffb6c10bd20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009ca0 tx=0x7ffb6402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 -- 172.21.15.137:0/760219884 shutdown_connections 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 --2- 172.21.15.137:0/760219884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 --2- 172.21.15.137:0/760219884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c105490 0x7ffb6c10bd20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 --2- 172.21.15.137:0/760219884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6c1038e0 0x7ffb6c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 -- 172.21.15.137:0/760219884 >> 172.21.15.137:0/760219884 conn(0x7ffb6c0fee50 msgr2=0x7ffb6c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 -- 172.21.15.137:0/760219884 shutdown_connections 2024-03-26T14:02:22.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.963+0000 7ffb73400640 1 -- 172.21.15.137:0/760219884 wait complete. 2024-03-26T14:02:22.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.964+0000 7ffb73400640 1 Processor -- start 2024-03-26T14:02:22.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.964+0000 7ffb73400640 1 -- start start 2024-03-26T14:02:22.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb73400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6c1038e0 0x7ffb6c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb73400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb6c105490 0x7ffb6c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb73400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10f410 con 0x7ffb6c104ad0 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb73400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10f290 con 0x7ffb6c105490 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb73400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c10f590 con 0x7ffb6c1038e0 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36700/0 (socket says 172.21.15.137:36700) 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 -- 172.21.15.137:0/1836654409 learned_addr learned my addr 172.21.15.137:0/1836654409 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb70a00640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6c1038e0 0x7ffb6c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb71400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb6c105490 0x7ffb6c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 -- 172.21.15.137:0/1836654409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6c1038e0 msgr2=0x7ffb6c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6c1038e0 0x7ffb6c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 -- 172.21.15.137:0/1836654409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb6c105490 msgr2=0x7ffb6c1a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb6c105490 0x7ffb6c1a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:22.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.965+0000 7ffb63e00640 1 -- 172.21.15.137:0/1836654409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c1a1700 con 0x7ffb6c104ad0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.966+0000 7ffb63e00640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c19a020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9f0 tx=0x7ffb5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.966+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400cd80 con 0x7ffb6c104ad0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.966+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb540108e0 con 0x7ffb6c104ad0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.966+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c1a19f0 con 0x7ffb6c104ad0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.966+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54018920 con 0x7ffb6c104ad0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.966+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c1a1f40 con 0x7ffb6c104ad0 2024-03-26T14:02:22.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.967+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c10ac50 con 0x7ffb6c104ad0 2024-03-26T14:02:22.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.971+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ffb54010a90 con 0x7ffb6c104ad0 2024-03-26T14:02:22.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.972+0000 7ffb6a000640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb500a2800 0x7ffb500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:22.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.972+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ffb540cf720 con 0x7ffb6c104ad0 2024-03-26T14:02:22.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.972+0000 7ffb70a00640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb500a2800 0x7ffb500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:22.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.973+0000 7ffb70a00640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb500a2800 0x7ffb500a4cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c005fd0 tx=0x7ffb5c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:22.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:22.975+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffb5409c3f0 con 0x7ffb6c104ad0 2024-03-26T14:02:23.033 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:02:22.705+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T14:02:23.033 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:02:22.705+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T14:02:23.033 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:02:22.705+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T14:02:23.033 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:02:22.705+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T14:02:23.033 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:02:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:02:22.705+0000 7f8b99400700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-26T14:02:23.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.176+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb6c02d240 con 0x7ffb6c104ad0 2024-03-26T14:02:23.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.180+0000 7ffb6a000640 1 -- 172.21.15.137:0/1836654409 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7ffb5409bcf0 con 0x7ffb6c104ad0 2024-03-26T14:02:23.178 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:23.178 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb500a2800 msgr2=0x7ffb500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb500a2800 0x7ffb500a4cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c005fd0 tx=0x7ffb5c005850 comp rx=0 tx=0).stop 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 msgr2=0x7ffb6c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c19a020 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e9f0 tx=0x7ffb5400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 shutdown_connections 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb500a2800 0x7ffb500a4cc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb6c105490 0x7ffb6c1a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb6c104ad0 0x7ffb6c19a020 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 --2- 172.21.15.137:0/1836654409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb6c1038e0 0x7ffb6c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 >> 172.21.15.137:0/1836654409 conn(0x7ffb6c0fee50 msgr2=0x7ffb6c107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:23.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.183+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 shutdown_connections 2024-03-26T14:02:23.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.184+0000 7ffb73400640 1 -- 172.21.15.137:0/1836654409 wait complete. 2024-03-26T14:02:23.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:02:23.199 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 -- 172.21.15.137:0/321714368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 msgr2=0x7f7bb0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/321714368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0101ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009900 tx=0x7f7ba402bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 -- 172.21.15.137:0/321714368 shutdown_connections 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/321714368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb0103260 0x7f7bb0109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/321714368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb01028a0 0x7f7bb0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/321714368 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0101ad0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 -- 172.21.15.137:0/321714368 >> 172.21.15.137:0/321714368 conn(0x7f7bb00fcc40 msgr2=0x7f7bb00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 -- 172.21.15.137:0/321714368 shutdown_connections 2024-03-26T14:02:23.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.415+0000 7f7bb7a00640 1 -- 172.21.15.137:0/321714368 wait complete. 2024-03-26T14:02:23.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.416+0000 7f7bb7a00640 1 Processor -- start 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.416+0000 7f7bb7a00640 1 -- start start 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb7a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb7a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb01028a0 0x7f7bb019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb0103260 0x7f7bb01a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb7a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb010d1d0 con 0x7f7bb01016b0 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb7a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb010d050 con 0x7f7bb01028a0 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb010d350 con 0x7f7bb0103260 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36718/0 (socket says 172.21.15.137:36718) 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 -- 172.21.15.137:0/1856590455 learned_addr learned my addr 172.21.15.137:0/1856590455 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bafe00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb01028a0 0x7f7bb019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb0103260 0x7f7bb01a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 -- 172.21.15.137:0/1856590455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb0103260 msgr2=0x7f7bb01a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb0103260 0x7f7bb01a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 -- 172.21.15.137:0/1856590455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb01028a0 msgr2=0x7f7bb019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb01028a0 0x7f7bb019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5000640 1 -- 172.21.15.137:0/1856590455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb01a1780 con 0x7f7bb01016b0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bb5a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb0103260 0x7f7bb01a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.417+0000 7f7bafe00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb01028a0 0x7f7bb019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.418+0000 7f7bb5000640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0199b60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000c00 tx=0x7f7ba4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.418+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4043020 con 0x7f7bb01016b0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.418+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7ba4004440 con 0x7f7bb01016b0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.418+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb01a1a10 con 0x7f7bb01016b0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.418+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4003c90 con 0x7f7bb01016b0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.418+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb01a1f60 con 0x7f7bb01016b0 2024-03-26T14:02:23.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.419+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb01a21f0 con 0x7f7bb01016b0 2024-03-26T14:02:23.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.421+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7ba4046020 con 0x7f7bb01016b0 2024-03-26T14:02:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.423+0000 7f7bad600640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b980a2800 0x7f7b980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.423+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7ba40f1160 con 0x7f7bb01016b0 2024-03-26T14:02:23.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.423+0000 7f7bafe00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b980a2800 0x7f7b980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.424+0000 7f7bafe00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b980a2800 0x7f7b980a4cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0058c0 tx=0x7f7b9c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:23.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.427+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ba404d020 con 0x7f7bb01016b0 2024-03-26T14:02:23.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:23 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3191946245' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T14:02:23.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:23 smithi137 ceph-mon[31539]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T14:02:23.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:23 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1836654409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:23.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.626+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bb01a2580 con 0x7f7bb01016b0 2024-03-26T14:02:23.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.627+0000 7f7bad600640 1 -- 172.21.15.137:0/1856590455 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ba40bde30 con 0x7f7bb01016b0 2024-03-26T14:02:23.624 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.629+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b980a2800 msgr2=0x7f7b980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.629+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b980a2800 0x7f7b980a4cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0058c0 tx=0x7f7b9c005850 comp rx=0 tx=0).stop 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.629+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 msgr2=0x7f7bb0199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.629+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0199b60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000c00 tx=0x7f7ba4004040 comp rx=0 tx=0).stop 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.629+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 shutdown_connections 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b980a2800 0x7f7b980a4cc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bb01028a0 0x7f7bb019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7bb0103260 0x7f7bb01a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 --2- 172.21.15.137:0/1856590455 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7bb01016b0 0x7f7bb0199b60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 >> 172.21.15.137:0/1856590455 conn(0x7f7bb00fcc40 msgr2=0x7f7bb0108130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:23.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 shutdown_connections 2024-03-26T14:02:23.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.630+0000 7f7bb7a00640 1 -- 172.21.15.137:0/1856590455 wait complete. 2024-03-26T14:02:23.645 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:02:23.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:23 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3191946245' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T14:02:23.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:23 smithi179 ceph-mon[35472]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T14:02:23.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:23 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1836654409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:23.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:23 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3191946245' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-26T14:02:23.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:23 smithi159 ceph-mon[35501]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-03-26T14:02:23.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:23 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1836654409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 -- 172.21.15.137:0/2840561703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 msgr2=0x7f75e0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 --2- 172.21.15.137:0/2840561703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e0109330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009900 tx=0x7f75c802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 -- 172.21.15.137:0/2840561703 shutdown_connections 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 --2- 172.21.15.137:0/2840561703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75e0109870 0x7f75e0110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 --2- 172.21.15.137:0/2840561703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e0109330 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 --2- 172.21.15.137:0/2840561703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e0107cc0 0x7f75e01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 -- 172.21.15.137:0/2840561703 >> 172.21.15.137:0/2840561703 conn(0x7f75e0076c60 msgr2=0x7f75e00790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 -- 172.21.15.137:0/2840561703 shutdown_connections 2024-03-26T14:02:23.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.856+0000 7f75e7a00640 1 -- 172.21.15.137:0/2840561703 wait complete. 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.857+0000 7f75e7a00640 1 Processor -- start 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.857+0000 7f75e7a00640 1 -- start start 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e7a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e0107cc0 0x7f75e019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e7a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e7a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75e0109870 0x7f75e01a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e7a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0113820 con 0x7f75e0108eb0 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e7a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e01136a0 con 0x7f75e0109870 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e7a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e01139a0 con 0x7f75e0107cc0 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e019e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36742/0 (socket says 172.21.15.137:36742) 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 -- 172.21.15.137:0/1022432967 learned_addr learned my addr 172.21.15.137:0/1022432967 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e5a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75e0109870 0x7f75e01a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75e5000640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e0107cc0 0x7f75e019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 -- 172.21.15.137:0/1022432967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e0107cc0 msgr2=0x7f75e019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e0107cc0 0x7f75e019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 -- 172.21.15.137:0/1022432967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75e0109870 msgr2=0x7f75e01a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:23.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75e0109870 0x7f75e01a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:23.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.858+0000 7f75d7e00640 1 -- 172.21.15.137:0/1022432967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e01a5af0 con 0x7f75e0108eb0 2024-03-26T14:02:23.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.859+0000 7f75d7e00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e019e460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75c8000c00 tx=0x7f75c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:23.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.859+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8043020 con 0x7f75e0108eb0 2024-03-26T14:02:23.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.859+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f75c8004aa0 con 0x7f75e0108eb0 2024-03-26T14:02:23.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.859+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8003d00 con 0x7f75e0108eb0 2024-03-26T14:02:23.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.859+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e01a5d80 con 0x7f75e0108eb0 2024-03-26T14:02:23.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.859+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e01a6220 con 0x7f75e0108eb0 2024-03-26T14:02:23.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.860+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e010f0b0 con 0x7f75e0108eb0 2024-03-26T14:02:23.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.862+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f75c8046020 con 0x7f75e0108eb0 2024-03-26T14:02:23.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.863+0000 7f75de000640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75c40a2880 0x7f75c40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:23.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.864+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f75c80f11f0 con 0x7f75e0108eb0 2024-03-26T14:02:23.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.864+0000 7f75e5000640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75c40a2880 0x7f75c40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:23.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.864+0000 7f75e5000640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75c40a2880 0x7f75c40a4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005fd0 tx=0x7f75d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:23.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:23.868+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f75c804d020 con 0x7f75e0108eb0 2024-03-26T14:02:24.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.066+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75e00623a0 con 0x7f75e0108eb0 2024-03-26T14:02:24.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.067+0000 7f75de000640 1 -- 172.21.15.137:0/1022432967 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75c80bdfa0 con 0x7f75e0108eb0 2024-03-26T14:02:24.064 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.069+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75c40a2880 msgr2=0x7f75c40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.069+0000 7f75e7a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75c40a2880 0x7f75c40a4d40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75d0005fd0 tx=0x7f75d0005850 comp rx=0 tx=0).stop 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.069+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 msgr2=0x7f75e019e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.069+0000 7f75e7a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e019e460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75c8000c00 tx=0x7f75c80042f0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 shutdown_connections 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f75c40a2880 0x7f75c40a4d40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75e0109870 0x7f75e01a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f75e0108eb0 0x7f75e019e460 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 --2- 172.21.15.137:0/1022432967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f75e0107cc0 0x7f75e019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 >> 172.21.15.137:0/1022432967 conn(0x7f75e0076c60 msgr2=0x7f75e010c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:24.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 shutdown_connections 2024-03-26T14:02:24.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.070+0000 7f75e7a00640 1 -- 172.21.15.137:0/1022432967 wait complete. 2024-03-26T14:02:24.086 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:02:24.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 -- 172.21.15.137:0/1566930843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38108db0 msgr2=0x7fbd38109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/1566930843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38108db0 0x7fbd38109230 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009900 tx=0x7fbd2402bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:24.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 -- 172.21.15.137:0/1566930843 shutdown_connections 2024-03-26T14:02:24.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/1566930843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 0x7fbd38110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/1566930843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38108db0 0x7fbd38109230 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/1566930843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd38107bc0 0x7fbd38107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 -- 172.21.15.137:0/1566930843 >> 172.21.15.137:0/1566930843 conn(0x7fbd38076b70 msgr2=0x7fbd38078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:24.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.312+0000 7fbd3ee00640 1 -- 172.21.15.137:0/1566930843 shutdown_connections 2024-03-26T14:02:24.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.313+0000 7fbd3ee00640 1 -- 172.21.15.137:0/1566930843 wait complete. 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.313+0000 7fbd3ee00640 1 Processor -- start 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 -- start start 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 0x7fbd3819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd38108db0 0x7fbd3819e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 0x7fbd381a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd381136d0 con 0x7fbd38107bc0 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd38113550 con 0x7fbd38108db0 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3ee00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd38113850 con 0x7fbd38109770 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 0x7fbd3819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 0x7fbd3819ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36752/0 (socket says 172.21.15.137:36752) 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3da00640 1 -- 172.21.15.137:0/2610339520 learned_addr learned my addr 172.21.15.137:0/2610339520 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd38108db0 0x7fbd3819e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3e400640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 0x7fbd381a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.314+0000 7fbd3da00640 1 -- 172.21.15.137:0/2610339520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 msgr2=0x7fbd381a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3da00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 0x7fbd381a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3da00640 1 -- 172.21.15.137:0/2610339520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd38108db0 msgr2=0x7fbd3819e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3da00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd38108db0 0x7fbd3819e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3da00640 1 -- 172.21.15.137:0/2610339520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd381a5a10 con 0x7fbd38107bc0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3e400640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 0x7fbd381a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3da00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 0x7fbd3819ddf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00e9c0 tx=0x7fbd2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00cd80 con 0x7fbd38107bc0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbd2c0035f0 con 0x7fbd38107bc0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd381a5d00 con 0x7fbd38107bc0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c010690 con 0x7fbd38107bc0 2024-03-26T14:02:24.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.315+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd381a61c0 con 0x7fbd38107bc0 2024-03-26T14:02:24.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.316+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3810efb0 con 0x7fbd38107bc0 2024-03-26T14:02:24.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.318+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fbd2c0037a0 con 0x7fbd38107bc0 2024-03-26T14:02:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.320+0000 7fbd36000640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbd200a2910 0x7fbd200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.320+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbd2c0cf340 con 0x7fbd38107bc0 2024-03-26T14:02:24.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.320+0000 7fbd3d000640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbd200a2910 0x7fbd200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.321+0000 7fbd3d000640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbd200a2910 0x7fbd200a4dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009230 tx=0x7fbd24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:24.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.324+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd2c09beb0 con 0x7fbd38107bc0 2024-03-26T14:02:24.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.520+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd3802d240 con 0x7fbd38107bc0 2024-03-26T14:02:24.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.521+0000 7fbd36000640 1 -- 172.21.15.137:0/2610339520 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd2c09b7b0 con 0x7fbd38107bc0 2024-03-26T14:02:24.518 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:02:24.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbd200a2910 msgr2=0x7fbd200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbd200a2910 0x7fbd200a4dd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009230 tx=0x7fbd24005fd0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 msgr2=0x7fbd3819ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 0x7fbd3819ddf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00e9c0 tx=0x7fbd2c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 shutdown_connections 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbd200a2910 0x7fbd200a4dd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd38108db0 0x7fbd3819e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd38109770 0x7fbd381a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 --2- 172.21.15.137:0/2610339520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbd38107bc0 0x7fbd3819ddf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 >> 172.21.15.137:0/2610339520 conn(0x7fbd38076b70 msgr2=0x7fbd3810c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 shutdown_connections 2024-03-26T14:02:24.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.523+0000 7fbd3ee00640 1 -- 172.21.15.137:0/2610339520 wait complete. 2024-03-26T14:02:24.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:24 smithi137 ceph-mon[31539]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:02:24.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:24 smithi137 ceph-mon[31539]: pgmap v459: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-26T14:02:24.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:24 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1856590455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:24.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:24 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1022432967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:24.540 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:02:24.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:24 smithi179 ceph-mon[35472]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:02:24.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:24 smithi179 ceph-mon[35472]: pgmap v459: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-26T14:02:24.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:24 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1856590455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:24.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:24 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1022432967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:24.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:24 smithi159 ceph-mon[35501]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-26T14:02:24.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:24 smithi159 ceph-mon[35501]: pgmap v459: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 511 B/s wr, 7 op/s 2024-03-26T14:02:24.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:24 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1856590455' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:24.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:24 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1022432967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:24.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 -- 172.21.15.137:0/1888357446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 msgr2=0x7f83c01003b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 --2- 172.21.15.137:0/1888357446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c01003b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009900 tx=0x7f83b002bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 -- 172.21.15.137:0/1888357446 shutdown_connections 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 --2- 172.21.15.137:0/1888357446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83c0101290 0x7f83c0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 --2- 172.21.15.137:0/1888357446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c01008f0 0x7f83c0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 --2- 172.21.15.137:0/1888357446 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c01003b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 -- 172.21.15.137:0/1888357446 >> 172.21.15.137:0/1888357446 conn(0x7f83c00fbb50 msgr2=0x7f83c00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 -- 172.21.15.137:0/1888357446 shutdown_connections 2024-03-26T14:02:24.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.759+0000 7f83c5c00640 1 -- 172.21.15.137:0/1888357446 wait complete. 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.760+0000 7f83c5c00640 1 Processor -- start 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.760+0000 7f83c5c00640 1 -- start start 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83c5c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c0199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83c5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83c01008f0 0x7f83c019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83c5c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101290 0x7f83c01a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83c5c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c010f400 con 0x7f83c00fff90 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83c5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c010f280 con 0x7f83c01008f0 2024-03-26T14:02:24.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83c5c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c010f580 con 0x7f83c0101290 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83bea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c0199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83bea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c0199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36776/0 (socket says 172.21.15.137:36776) 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83bea00640 1 -- 172.21.15.137:0/2346257996 learned_addr learned my addr 172.21.15.137:0/2346257996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83be000640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83c01008f0 0x7f83c019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83bf400640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101290 0x7f83c01a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83bea00640 1 -- 172.21.15.137:0/2346257996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101290 msgr2=0x7f83c01a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.761+0000 7f83bea00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101290 0x7f83c01a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83bea00640 1 -- 172.21.15.137:0/2346257996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83c01008f0 msgr2=0x7f83c019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83bea00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83c01008f0 0x7f83c019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83bea00640 1 -- 172.21.15.137:0/2346257996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c01a16f0 con 0x7f83c00fff90 2024-03-26T14:02:24.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83bea00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c0199ad0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000c00 tx=0x7f83b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:24.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0043020 con 0x7f83c00fff90 2024-03-26T14:02:24.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83b0004aa0 con 0x7f83c00fff90 2024-03-26T14:02:24.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c01a1980 con 0x7f83c00fff90 2024-03-26T14:02:24.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0003d00 con 0x7f83c00fff90 2024-03-26T14:02:24.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.762+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c01a1f20 con 0x7f83c00fff90 2024-03-26T14:02:24.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.764+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c010ace0 con 0x7f83c00fff90 2024-03-26T14:02:24.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.765+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f83b0046020 con 0x7f83c00fff90 2024-03-26T14:02:24.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.767+0000 7f83b7400640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83a40a2910 0x7f83a40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:24.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.767+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f83b00f1230 con 0x7f83c00fff90 2024-03-26T14:02:24.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.767+0000 7f83be000640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83a40a2910 0x7f83a40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:24.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.768+0000 7f83be000640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83a40a2910 0x7f83a40a4dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f83ac005d00 tx=0x7f83ac005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:24.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.771+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83b004d020 con 0x7f83c00fff90 2024-03-26T14:02:24.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.969+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f83c00622a0 con 0x7f83c00fff90 2024-03-26T14:02:24.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.970+0000 7f83b7400640 1 -- 172.21.15.137:0/2346257996 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f83b00bdfd0 con 0x7f83c00fff90 2024-03-26T14:02:24.967 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83a40a2910 msgr2=0x7f83a40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83a40a2910 0x7f83a40a4dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f83ac005d00 tx=0x7f83ac005c50 comp rx=0 tx=0).stop 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 msgr2=0x7f83c0199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c0199ad0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f83b0000c00 tx=0x7f83b00042f0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 shutdown_connections 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83a40a2910 0x7f83a40a4dd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83c01008f0 0x7f83c019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83c0101290 0x7f83c01a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 --2- 172.21.15.137:0/2346257996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f83c00fff90 0x7f83c0199ad0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:24.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 >> 172.21.15.137:0/2346257996 conn(0x7f83c00fbb50 msgr2=0x7f83c00fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:24.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 shutdown_connections 2024-03-26T14:02:24.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:24.972+0000 7f83c5c00640 1 -- 172.21.15.137:0/2346257996 wait complete. 2024-03-26T14:02:24.989 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:25.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.205+0000 7f24e0200640 1 -- 172.21.15.137:0/4143330417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8107ce0 msgr2=0x7f24d8108100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.205+0000 7f24e0200640 1 --2- 172.21.15.137:0/4143330417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8107ce0 0x7f24d8108100 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009900 tx=0x7f24c802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:25.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 -- 172.21.15.137:0/4143330417 shutdown_connections 2024-03-26T14:02:25.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 --2- 172.21.15.137:0/4143330417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 0x7f24d8110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 --2- 172.21.15.137:0/4143330417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d8108ed0 0x7f24d8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 --2- 172.21.15.137:0/4143330417 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8107ce0 0x7f24d8108100 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 -- 172.21.15.137:0/4143330417 >> 172.21.15.137:0/4143330417 conn(0x7f24d8076cb0 msgr2=0x7f24d80790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 -- 172.21.15.137:0/4143330417 shutdown_connections 2024-03-26T14:02:25.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.206+0000 7f24e0200640 1 -- 172.21.15.137:0/4143330417 wait complete. 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 Processor -- start 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 -- start start 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d8107ce0 0x7f24d819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 0x7f24d819e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 0x7f24d81a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8113850 con 0x7f24d8108ed0 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d81136d0 con 0x7f24d8109890 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24e0200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d81139d0 con 0x7f24d8107ce0 2024-03-26T14:02:25.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24dce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 0x7f24d819e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24dce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 0x7f24d819e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36794/0 (socket says 172.21.15.137:36794) 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24dce00640 1 -- 172.21.15.137:0/1402143192 learned_addr learned my addr 172.21.15.137:0/1402143192 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.207+0000 7f24de200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 0x7f24d81a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dd800640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d8107ce0 0x7f24d819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dce00640 1 -- 172.21.15.137:0/1402143192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d8107ce0 msgr2=0x7f24d819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dce00640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d8107ce0 0x7f24d819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dce00640 1 -- 172.21.15.137:0/1402143192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 msgr2=0x7f24d81a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dce00640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 0x7f24d81a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dce00640 1 -- 172.21.15.137:0/1402143192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d81a5b70 con 0x7f24d8108ed0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24de200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 0x7f24d81a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24dce00640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 0x7f24d819e490 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ea80 tx=0x7f24cc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00ce40 con 0x7f24d8108ed0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24cc005020 con 0x7f24d8108ed0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d81a5e60 con 0x7f24d8108ed0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0116d0 con 0x7f24d8108ed0 2024-03-26T14:02:25.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.208+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d81a62d0 con 0x7f24d8108ed0 2024-03-26T14:02:25.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.210+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d810f050 con 0x7f24d8108ed0 2024-03-26T14:02:25.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.212+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f24cc0051d0 con 0x7f24d8108ed0 2024-03-26T14:02:25.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.213+0000 7f24d6000640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24c00a2890 0x7f24c00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.213+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f24cc0cf380 con 0x7f24d8108ed0 2024-03-26T14:02:25.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.213+0000 7f24dd800640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24c00a2890 0x7f24c00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.214+0000 7f24dd800640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24c00a2890 0x7f24c00a4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24c80096c0 tx=0x7f24c8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:25.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.217+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f24cc016030 con 0x7f24d8108ed0 2024-03-26T14:02:25.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.422+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24d802d2c0 con 0x7f24d8108ed0 2024-03-26T14:02:25.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.425+0000 7f24d6000640 1 -- 172.21.15.137:0/1402143192 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f24cc09beb0 con 0x7f24d8108ed0 2024-03-26T14:02:25.423 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:25.423 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:25.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24c00a2890 msgr2=0x7f24c00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24c00a2890 0x7f24c00a4d50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f24c80096c0 tx=0x7f24c8005fd0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 msgr2=0x7f24d819e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 0x7f24d819e490 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ea80 tx=0x7f24cc00ef40 comp rx=0 tx=0).stop 2024-03-26T14:02:25.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 shutdown_connections 2024-03-26T14:02:25.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24c00a2890 0x7f24c00a4d50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d8109890 0x7f24d81a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d8108ed0 0x7f24d819e490 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 --2- 172.21.15.137:0/1402143192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d8107ce0 0x7f24d819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.428+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 >> 172.21.15.137:0/1402143192 conn(0x7f24d8076cb0 msgr2=0x7f24d810c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:25.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.429+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 shutdown_connections 2024-03-26T14:02:25.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.429+0000 7f24e0200640 1 -- 172.21.15.137:0/1402143192 wait complete. 2024-03-26T14:02:25.445 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:02:25.446 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:02:25.520 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:25 smithi137 ceph-mon[31539]: mgrmap e19: x(active, since 12m), standbys: y, z 2024-03-26T14:02:25.520 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:25 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2610339520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:25.520 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:25 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2346257996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1841107494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0105350 msgr2=0x7f3ca010bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1841107494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0105350 0x7f3ca010bbe0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009ca0 tx=0x7f3c9802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1841107494 shutdown_connections 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1841107494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ca0104990 0x7f3ca0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1841107494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0105350 0x7f3ca010bbe0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1841107494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca01037a0 0x7f3ca0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1841107494 >> 172.21.15.137:0/1841107494 conn(0x7f3ca00fed10 msgr2=0x7f3ca0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1841107494 shutdown_connections 2024-03-26T14:02:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.659+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1841107494 wait complete. 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.660+0000 7f3c9fe00640 1 Processor -- start 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.660+0000 7f3c9fe00640 1 -- start start 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca01037a0 0x7f3ca0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 0x7f3ca0199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ca0105350 0x7f3ca01a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca010f2b0 con 0x7f3ca0104990 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca010f130 con 0x7f3ca0105350 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca010f430 con 0x7f3ca01037a0 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 0x7f3ca0199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 0x7f3ca0199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36818/0 (socket says 172.21.15.137:36818) 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 -- 172.21.15.137:0/1381920736 learned_addr learned my addr 172.21.15.137:0/1381920736 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:25.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9f400640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ca0105350 0x7f3ca01a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c9ea00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca01037a0 0x7f3ca0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 -- 172.21.15.137:0/1381920736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca01037a0 msgr2=0x7f3ca0199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca01037a0 0x7f3ca0199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 -- 172.21.15.137:0/1381920736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ca0105350 msgr2=0x7f3ca01a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ca0105350 0x7f3ca01a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.661+0000 7f3c97e00640 1 -- 172.21.15.137:0/1381920736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca01a1520 con 0x7f3ca0104990 2024-03-26T14:02:25.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.662+0000 7f3c97e00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 0x7f3ca0199e40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d950 tx=0x7f3c8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:25.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.662+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88017030 con 0x7f3ca0104990 2024-03-26T14:02:25.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.662+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c88004020 con 0x7f3ca0104990 2024-03-26T14:02:25.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.662+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca01a1810 con 0x7f3ca0104990 2024-03-26T14:02:25.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.662+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c880047a0 con 0x7f3ca0104990 2024-03-26T14:02:25.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.662+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca01a1d60 con 0x7f3ca0104990 2024-03-26T14:02:25.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.663+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca010aac0 con 0x7f3ca0104990 2024-03-26T14:02:25.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.665+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3c880041d0 con 0x7f3ca0104990 2024-03-26T14:02:25.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.667+0000 7f3c97400640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c840a2890 0x7f3c840a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:25.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.668+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3c880ceb50 con 0x7f3ca0104990 2024-03-26T14:02:25.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.668+0000 7f3c9ea00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c840a2890 0x7f3c840a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:25.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.668+0000 7f3c9ea00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c840a2890 0x7f3c840a4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005fd0 tx=0x7f3c90005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:25.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.672+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c8809b6c0 con 0x7f3ca0104990 2024-03-26T14:02:25.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:25 smithi179 ceph-mon[35472]: mgrmap e19: x(active, since 12m), standbys: y, z 2024-03-26T14:02:25.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:25 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2610339520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:25.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:25 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2346257996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:25.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:25 smithi159 ceph-mon[35501]: mgrmap e19: x(active, since 12m), standbys: y, z 2024-03-26T14:02:25.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:25 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2610339520' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:25.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:25 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2346257996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.870+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ca002d2d0 con 0x7f3ca0104990 2024-03-26T14:02:25.869 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.871+0000 7f3c97400640 1 -- 172.21.15.137:0/1381920736 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c8809afc0 con 0x7f3ca0104990 2024-03-26T14:02:25.869 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:02:25.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c840a2890 msgr2=0x7f3c840a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c840a2890 0x7f3c840a4d50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3c90005fd0 tx=0x7f3c90005850 comp rx=0 tx=0).stop 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 msgr2=0x7f3ca0199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 0x7f3ca0199e40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d950 tx=0x7f3c8800de10 comp rx=0 tx=0).stop 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 shutdown_connections 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c840a2890 0x7f3c840a4d50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ca0105350 0x7f3ca01a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ca0104990 0x7f3ca0199e40 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 --2- 172.21.15.137:0/1381920736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ca01037a0 0x7f3ca0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 >> 172.21.15.137:0/1381920736 conn(0x7f3ca00fed10 msgr2=0x7f3ca0107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 shutdown_connections 2024-03-26T14:02:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:25.874+0000 7f3c9fe00640 1 -- 172.21.15.137:0/1381920736 wait complete. 2024-03-26T14:02:25.892 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.108+0000 7f6ff9600640 1 -- 172.21.15.137:0/726982292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff40772a0 msgr2=0x7f6ff4077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.108+0000 7f6ff9600640 1 --2- 172.21.15.137:0/726982292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff40772a0 0x7f6ff4077700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009900 tx=0x7f6fdc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 -- 172.21.15.137:0/726982292 shutdown_connections 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 --2- 172.21.15.137:0/726982292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff4077c40 0x7f6ff40784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 --2- 172.21.15.137:0/726982292 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff40772a0 0x7f6ff4077700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 --2- 172.21.15.137:0/726982292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 0x7f6ff4076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 -- 172.21.15.137:0/726982292 >> 172.21.15.137:0/726982292 conn(0x7f6ff40fee50 msgr2=0x7f6ff4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:26.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 -- 172.21.15.137:0/726982292 shutdown_connections 2024-03-26T14:02:26.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.109+0000 7f6ff9600640 1 -- 172.21.15.137:0/726982292 wait complete. 2024-03-26T14:02:26.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 Processor -- start 2024-03-26T14:02:26.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 -- start start 2024-03-26T14:02:26.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 0x7f6ff419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff40772a0 0x7f6ff419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 0x7f6ff41a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff4113820 con 0x7f6ff4077c40 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff41136a0 con 0x7f6ff4076060 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.110+0000 7f6ff9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff41139a0 con 0x7f6ff40772a0 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 0x7f6ff41a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 0x7f6ff419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 0x7f6ff41a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36832/0 (socket says 172.21.15.137:36832) 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 0x7f6ff419df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:34822/0 (socket says 172.21.15.137:34822) 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 -- 172.21.15.137:0/3824096377 learned_addr learned my addr 172.21.15.137:0/3824096377 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6fea000640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff40772a0 0x7f6ff419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 -- 172.21.15.137:0/3824096377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff40772a0 msgr2=0x7f6ff419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff40772a0 0x7f6ff419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 -- 172.21.15.137:0/3824096377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 msgr2=0x7f6ff419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 0x7f6ff419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 -- 172.21.15.137:0/3824096377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff41a5b30 con 0x7f6ff4077c40 2024-03-26T14:02:26.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.111+0000 7f6ff3400640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 0x7f6ff41a54c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00eaa0 tx=0x7f6fec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:26.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.112+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00cdb0 con 0x7f6ff4077c40 2024-03-26T14:02:26.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.112+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fec005020 con 0x7f6ff4077c40 2024-03-26T14:02:26.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.112+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff41a5e20 con 0x7f6ff4077c40 2024-03-26T14:02:26.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.112+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0116d0 con 0x7f6ff4077c40 2024-03-26T14:02:26.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.112+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff41a6330 con 0x7f6ff4077c40 2024-03-26T14:02:26.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.113+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff4110250 con 0x7f6ff4077c40 2024-03-26T14:02:26.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.115+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6fec0051d0 con 0x7f6ff4077c40 2024-03-26T14:02:26.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.116+0000 7f6febe00640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6fd80a2910 0x7f6fd80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.117+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6fec0cf120 con 0x7f6ff4077c40 2024-03-26T14:02:26.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.117+0000 7f6ff2a00640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6fd80a2910 0x7f6fd80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.117+0000 7f6ff2a00640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6fd80a2910 0x7f6fd80a4dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009bd0 tx=0x7f6fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:26.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.121+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6fec016030 con 0x7f6ff4077c40 2024-03-26T14:02:26.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.321+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ff402d2c0 con 0x7f6ff4077c40 2024-03-26T14:02:26.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.322+0000 7f6febe00640 1 -- 172.21.15.137:0/3824096377 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fec09bc50 con 0x7f6ff4077c40 2024-03-26T14:02:26.319 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:26.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.324+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6fd80a2910 msgr2=0x7f6fd80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.324+0000 7f6ff9600640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6fd80a2910 0x7f6fd80a4dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009bd0 tx=0x7f6fe4005f90 comp rx=0 tx=0).stop 2024-03-26T14:02:26.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.324+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 msgr2=0x7f6ff41a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.324+0000 7f6ff9600640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 0x7f6ff41a54c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00eaa0 tx=0x7f6fec00ef60 comp rx=0 tx=0).stop 2024-03-26T14:02:26.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 shutdown_connections 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6fd80a2910 0x7f6fd80a4dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ff4077c40 0x7f6ff41a54c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff40772a0 0x7f6ff419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 --2- 172.21.15.137:0/3824096377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff4076060 0x7f6ff419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 >> 172.21.15.137:0/3824096377 conn(0x7f6ff40fee50 msgr2=0x7f6ff4100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 shutdown_connections 2024-03-26T14:02:26.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.325+0000 7f6ff9600640 1 -- 172.21.15.137:0/3824096377 wait complete. 2024-03-26T14:02:26.340 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:02:26.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:26 smithi137 ceph-mon[31539]: pgmap v460: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:26.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:26 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1402143192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:26.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:26 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1381920736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 -- 172.21.15.137:0/367944525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5610104ad0 msgr2=0x7f5610104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 --2- 172.21.15.137:0/367944525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5610104ad0 0x7f5610104f50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009900 tx=0x7f55fc02bc40 comp rx=0 tx=0).stop 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 -- 172.21.15.137:0/367944525 shutdown_connections 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 --2- 172.21.15.137:0/367944525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5610105490 0x7f561010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 --2- 172.21.15.137:0/367944525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5610104ad0 0x7f5610104f50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 --2- 172.21.15.137:0/367944525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56101038e0 0x7f5610103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 -- 172.21.15.137:0/367944525 >> 172.21.15.137:0/367944525 conn(0x7f56100fee50 msgr2=0x7f5610101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.554+0000 7f5615a00640 1 -- 172.21.15.137:0/367944525 shutdown_connections 2024-03-26T14:02:26.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.555+0000 7f5615a00640 1 -- 172.21.15.137:0/367944525 wait complete. 2024-03-26T14:02:26.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.555+0000 7f5615a00640 1 Processor -- start 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 -- start start 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 0x7f5610199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5610104ad0 0x7f561019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5610105490 0x7f56101a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561010f4c0 con 0x7f56101038e0 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561010f340 con 0x7f5610105490 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f5615a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561010f640 con 0x7f5610104ad0 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 0x7f5610199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 0x7f5610199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36846/0 (socket says 172.21.15.137:36846) 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560ea00640 1 -- 172.21.15.137:0/1669691011 learned_addr learned my addr 172.21.15.137:0/1669691011 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560ea00640 1 -- 172.21.15.137:0/1669691011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5610104ad0 msgr2=0x7f561019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560ea00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5610104ad0 0x7f561019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560ea00640 1 -- 172.21.15.137:0/1669691011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5610105490 msgr2=0x7f56101a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.556+0000 7f560f400640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5610105490 0x7f56101a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f560ea00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5610105490 0x7f56101a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f560ea00640 1 -- 172.21.15.137:0/1669691011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56101a1760 con 0x7f56101038e0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f560f400640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5610105490 0x7f56101a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f560ea00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 0x7f5610199b40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f560000d950 tx=0x7f560000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5600017020 con 0x7f56101038e0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5600009c30 con 0x7f56101038e0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56101a1a50 con 0x7f56101038e0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560000f6b0 con 0x7f56101038e0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.557+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56101a1fa0 con 0x7f56101038e0 2024-03-26T14:02:26.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.558+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561010ac80 con 0x7f56101038e0 2024-03-26T14:02:26.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.560+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f560001a020 con 0x7f56101038e0 2024-03-26T14:02:26.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.562+0000 7f5607400640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55f40a2890 0x7f55f40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.562+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f56000cefd0 con 0x7f56101038e0 2024-03-26T14:02:26.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.562+0000 7f560e000640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55f40a2890 0x7f55f40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.563+0000 7f560e000640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55f40a2890 0x7f55f40a4d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005c20 tx=0x7f55fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:26.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.566+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5600021020 con 0x7f56101038e0 2024-03-26T14:02:26.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:26 smithi179 ceph-mon[35472]: pgmap v460: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:26.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:26 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1402143192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:26.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:26 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1381920736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:26.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:26 smithi159 ceph-mon[35501]: pgmap v460: 129 pgs: 129 active+clean; 649 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:26.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:26 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1402143192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:26.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:26 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1381920736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:26.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.763+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56100622a0 con 0x7f56101038e0 2024-03-26T14:02:26.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.764+0000 7f5607400640 1 -- 172.21.15.137:0/1669691011 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5600009de0 con 0x7f56101038e0 2024-03-26T14:02:26.761 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:02:26.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55f40a2890 msgr2=0x7f55f40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55f40a2890 0x7f55f40a4d50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005c20 tx=0x7f55fc000bc0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 msgr2=0x7f5610199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 0x7f5610199b40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f560000d950 tx=0x7f560000de10 comp rx=0 tx=0).stop 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 shutdown_connections 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f55f40a2890 0x7f55f40a4d50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5610105490 0x7f56101a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5610104ad0 0x7f561019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 --2- 172.21.15.137:0/1669691011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56101038e0 0x7f5610199b40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 >> 172.21.15.137:0/1669691011 conn(0x7f56100fee50 msgr2=0x7f5610107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 shutdown_connections 2024-03-26T14:02:26.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.767+0000 7f5615a00640 1 -- 172.21.15.137:0/1669691011 wait complete. 2024-03-26T14:02:26.782 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:02:26.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.996+0000 7f8bdd600640 1 -- 172.21.15.137:0/3844822595 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc02cbe0 con 0x7f8be010b2d0 2024-03-26T14:02:26.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.996+0000 7f8be5a00640 1 -- 172.21.15.137:0/3844822595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 msgr2=0x7f8be010d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.996+0000 7f8be5a00640 1 --2- 172.21.15.137:0/3844822595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be010d6c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009370 tx=0x7f8bcc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:26.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 -- 172.21.15.137:0/3844822595 shutdown_connections 2024-03-26T14:02:26.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 --2- 172.21.15.137:0/3844822595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 0x7f8be0110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 --2- 172.21.15.137:0/3844822595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be010d6c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 --2- 172.21.15.137:0/3844822595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be0108910 0x7f8be010ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 -- 172.21.15.137:0/3844822595 >> 172.21.15.137:0/3844822595 conn(0x7f8be006e8a0 msgr2=0x7f8be0070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:26.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 -- 172.21.15.137:0/3844822595 shutdown_connections 2024-03-26T14:02:26.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.997+0000 7f8be5a00640 1 -- 172.21.15.137:0/3844822595 wait complete. 2024-03-26T14:02:26.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 Processor -- start 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 -- start start 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be0108910 0x7f8be01a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be01a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 0x7f8be01a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be01159f0 con 0x7f8be010b2d0 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be0115870 con 0x7f8be0108910 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8be5a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be0115b70 con 0x7f8be010dd50 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.998+0000 7f8bde000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be01a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be01a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36868/0 (socket says 172.21.15.137:36868) 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 -- 172.21.15.137:0/1262792527 learned_addr learned my addr 172.21.15.137:0/1262792527 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bdea00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be0108910 0x7f8be01a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bdf400640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 0x7f8be01a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 -- 172.21.15.137:0/1262792527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 msgr2=0x7f8be01a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 0x7f8be01a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 -- 172.21.15.137:0/1262792527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be0108910 msgr2=0x7f8be01a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be0108910 0x7f8be01a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 -- 172.21.15.137:0/1262792527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be01a9fb0 con 0x7f8be010b2d0 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bdf400640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 0x7f8be01a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:26.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:26.999+0000 7f8bde000640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be01a28d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0042f0 tx=0x7f8bcc004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:26.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.000+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc042020 con 0x7f8be010b2d0 2024-03-26T14:02:26.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.000+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8bcc0054c0 con 0x7f8be010b2d0 2024-03-26T14:02:26.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.000+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be01aa240 con 0x7f8be010b2d0 2024-03-26T14:02:26.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.000+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc003d70 con 0x7f8be010b2d0 2024-03-26T14:02:26.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.000+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be01aa780 con 0x7f8be010b2d0 2024-03-26T14:02:26.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.001+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be010f910 con 0x7f8be010b2d0 2024-03-26T14:02:27.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.004+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f8bcc045020 con 0x7f8be010b2d0 2024-03-26T14:02:27.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.005+0000 7f8bd7400640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8bc40a2910 0x7f8bc40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.005+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8bcc0f10a0 con 0x7f8be010b2d0 2024-03-26T14:02:27.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.005+0000 7f8bdea00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8bc40a2910 0x7f8bc40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.006+0000 7f8bdea00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8bc40a2910 0x7f8bc40a4dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000bce0 tx=0x7f8bd000b490 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:27.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.009+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bcc04c020 con 0x7f8be010b2d0 2024-03-26T14:02:27.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.202+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8be002d2c0 con 0x7f8be010b2d0 2024-03-26T14:02:27.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.203+0000 7f8bd7400640 1 -- 172.21.15.137:0/1262792527 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8bcc0bdce0 con 0x7f8be010b2d0 2024-03-26T14:02:27.200 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8bc40a2910 msgr2=0x7f8bc40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8bc40a2910 0x7f8bc40a4dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000bce0 tx=0x7f8bd000b490 comp rx=0 tx=0).stop 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 msgr2=0x7f8be01a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be01a28d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0042f0 tx=0x7f8bcc004320 comp rx=0 tx=0).stop 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 shutdown_connections 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8bc40a2910 0x7f8bc40a4dd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8be010dd50 0x7f8be01a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8be010b2d0 0x7f8be01a28d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 --2- 172.21.15.137:0/1262792527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8be0108910 0x7f8be01a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.205+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 >> 172.21.15.137:0/1262792527 conn(0x7f8be006e8a0 msgr2=0x7f8be0114330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.206+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 shutdown_connections 2024-03-26T14:02:27.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.206+0000 7f8be5a00640 1 -- 172.21.15.137:0/1262792527 wait complete. 2024-03-26T14:02:27.221 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-03-26T14:02:27.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.438+0000 7fa457200640 1 -- 172.21.15.137:0/2061490510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa458101f60 msgr2=0x7fa458104350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.438+0000 7fa457200640 1 --2- 172.21.15.137:0/2061490510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa458101f60 0x7fa458104350 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa448009900 tx=0x7fa44802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.438+0000 7fa457200640 1 -- 172.21.15.137:0/2061490510 shutdown_connections 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.438+0000 7fa457200640 1 --2- 172.21.15.137:0/2061490510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa458104890 0x7fa458106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.438+0000 7fa457200640 1 --2- 172.21.15.137:0/2061490510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4581071c0 0x7fa458109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.438+0000 7fa457200640 1 --2- 172.21.15.137:0/2061490510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa458101f60 0x7fa458104350 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.439+0000 7fa457200640 1 -- 172.21.15.137:0/2061490510 >> 172.21.15.137:0/2061490510 conn(0x7fa4580fb950 msgr2=0x7fa4580fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.439+0000 7fa457200640 1 -- 172.21.15.137:0/2061490510 shutdown_connections 2024-03-26T14:02:27.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.439+0000 7fa457200640 1 -- 172.21.15.137:0/2061490510 wait complete. 2024-03-26T14:02:27.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 Processor -- start 2024-03-26T14:02:27.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 -- start start 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa458101f60 0x7fa458197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa458104890 0x7fa458197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 0x7fa45819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45810d070 con 0x7fa4581071c0 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45810cef0 con 0x7fa458104890 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa457200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45810d1f0 con 0x7fa458101f60 2024-03-26T14:02:27.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.440+0000 7fa456800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 0x7fa45819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 0x7fa45819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36884/0 (socket says 172.21.15.137:36884) 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 -- 172.21.15.137:0/927882287 learned_addr learned my addr 172.21.15.137:0/927882287 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa44fe00640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa458104890 0x7fa458197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa455e00640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa458101f60 0x7fa458197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 -- 172.21.15.137:0/927882287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa458101f60 msgr2=0x7fa458197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa458101f60 0x7fa458197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 -- 172.21.15.137:0/927882287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa458104890 msgr2=0x7fa458197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa458104890 0x7fa458197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 -- 172.21.15.137:0/927882287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45819f3a0 con 0x7fa4581071c0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.441+0000 7fa456800640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 0x7fa45819ed30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa45000cad0 tx=0x7fa45000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.442+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45000ae20 con 0x7fa4581071c0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.442+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa450005020 con 0x7fa4581071c0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.442+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45819f690 con 0x7fa4581071c0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.442+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45000f6b0 con 0x7fa4581071c0 2024-03-26T14:02:27.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.442+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45819fbe0 con 0x7fa4581071c0 2024-03-26T14:02:27.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.443+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4580fd440 con 0x7fa4581071c0 2024-03-26T14:02:27.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.445+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa4500051d0 con 0x7fa4581071c0 2024-03-26T14:02:27.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.446+0000 7fa44ea00640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa43c0a2910 0x7fa43c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.446+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa4500cefe0 con 0x7fa4581071c0 2024-03-26T14:02:27.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.446+0000 7fa455e00640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa43c0a2910 0x7fa43c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.447+0000 7fa455e00640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa43c0a2910 0x7fa43c0a4dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa448006010 tx=0x7fa448000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:27.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.450+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa450014030 con 0x7fa4581071c0 2024-03-26T14:02:27.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:27 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3824096377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:27 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1669691011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:27 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1262792527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:27 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3824096377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:27 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1669691011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:27 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1262792527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.688+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa45802d270 con 0x7fa4581071c0 2024-03-26T14:02:27.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.690+0000 7fa44ea00640 1 -- 172.21.15.137:0/927882287 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fa45009bb50 con 0x7fa4581071c0 2024-03-26T14:02:27.687 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:27.688 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:02:27.688 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:02:27.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa43c0a2910 msgr2=0x7fa43c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa43c0a2910 0x7fa43c0a4dd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa448006010 tx=0x7fa448000bc0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 msgr2=0x7fa45819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 0x7fa45819ed30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa45000cad0 tx=0x7fa45000cf90 comp rx=0 tx=0).stop 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 shutdown_connections 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa43c0a2910 0x7fa43c0a4dd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa458104890 0x7fa458197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa4581071c0 0x7fa45819ed30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 --2- 172.21.15.137:0/927882287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa458101f60 0x7fa458197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 >> 172.21.15.137:0/927882287 conn(0x7fa4580fb950 msgr2=0x7fa458102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 shutdown_connections 2024-03-26T14:02:27.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.693+0000 7fa457200640 1 -- 172.21.15.137:0/927882287 wait complete. 2024-03-26T14:02:27.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:27 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3824096377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:27 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1669691011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:27 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1262792527' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:27.708 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-03-26T14:02:27.708 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-03-26T14:02:27.708 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 -- 172.21.15.137:0/3774860165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01052a0 msgr2=0x7f0eb010bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/3774860165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01052a0 0x7f0eb010bb30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8009ca0 tx=0x7f0ea802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 -- 172.21.15.137:0/3774860165 shutdown_connections 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/3774860165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb01048e0 0x7f0eb0104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/3774860165 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01052a0 0x7f0eb010bb30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/3774860165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb01036f0 0x7f0eb0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 -- 172.21.15.137:0/3774860165 >> 172.21.15.137:0/3774860165 conn(0x7f0eb00fec80 msgr2=0x7f0eb01010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 -- 172.21.15.137:0/3774860165 shutdown_connections 2024-03-26T14:02:27.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.919+0000 7f0eb6a00640 1 -- 172.21.15.137:0/3774860165 wait complete. 2024-03-26T14:02:27.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.920+0000 7f0eb6a00640 1 Processor -- start 2024-03-26T14:02:27.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.920+0000 7f0eb6a00640 1 -- start start 2024-03-26T14:02:27.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 0x7f0eb0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb6a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb01048e0 0x7f0eb0199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb01052a0 0x7f0eb01a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb6a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb010f220 con 0x7f0eb01036f0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb010f0a0 con 0x7f0eb01052a0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb6a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb010f3a0 con 0x7f0eb01048e0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 0x7f0eb0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 0x7f0eb0199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60766/0 (socket says 172.21.15.137:60766) 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 -- 172.21.15.137:0/4137737221 learned_addr learned my addr 172.21.15.137:0/4137737221 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb4c00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb01048e0 0x7f0eb0199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 -- 172.21.15.137:0/4137737221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb01048e0 msgr2=0x7f0eb0199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb01048e0 0x7f0eb0199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 -- 172.21.15.137:0/4137737221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb01052a0 msgr2=0x7f0eb01a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb01052a0 0x7f0eb01a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.921+0000 7f0eb5600640 1 -- 172.21.15.137:0/4137737221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb01a1450 con 0x7f0eb01036f0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.922+0000 7f0eb5600640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 0x7f0eb0199830 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c970 tx=0x7f0ea400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.922+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4004220 con 0x7f0eb01036f0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.922+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0ea40043c0 con 0x7f0eb01036f0 2024-03-26T14:02:27.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.922+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4004a00 con 0x7f0eb01036f0 2024-03-26T14:02:27.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.922+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb01a1740 con 0x7f0eb01036f0 2024-03-26T14:02:27.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.922+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb01a1c40 con 0x7f0eb01036f0 2024-03-26T14:02:27.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.923+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb010a9b0 con 0x7f0eb01036f0 2024-03-26T14:02:27.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.925+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f0ea40079b0 con 0x7f0eb01036f0 2024-03-26T14:02:27.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.926+0000 7f0eae000640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2910 0x7f0e980a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:27.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.926+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0ea40cede0 con 0x7f0eb01036f0 2024-03-26T14:02:27.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.927+0000 7f0eb4c00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2910 0x7f0e980a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:27.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.927+0000 7f0eb4c00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2910 0x7f0e980a4dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005e10 tx=0x7f0e9c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:27.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:27.930+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ea4014030 con 0x7f0eb01036f0 2024-03-26T14:02:28.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.126+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0eb0061ae0 con 0x7f0e980a2910 2024-03-26T14:02:28.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.147+0000 7f0eae000640 1 -- 172.21.15.137:0/4137737221 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340825 (secure 0 0 0) 0x7f0eb0061ae0 con 0x7f0e980a2910 2024-03-26T14:02:28.148 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:28.150 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.153+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2910 msgr2=0x7f0e980a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.153+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2910 0x7f0e980a4dd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005e10 tx=0x7f0e9c005da0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.153+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 msgr2=0x7f0eb0199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.153+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 0x7f0eb0199830 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400c970 tx=0x7f0ea400ce30 comp rx=0 tx=0).stop 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 shutdown_connections 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0e980a2910 0x7f0e980a4dd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eb01052a0 0x7f0eb01a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0eb01048e0 0x7f0eb0199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 --2- 172.21.15.137:0/4137737221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb01036f0 0x7f0eb0199830 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 >> 172.21.15.137:0/4137737221 conn(0x7f0eb00fec80 msgr2=0x7f0eb0107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:28.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 shutdown_connections 2024-03-26T14:02:28.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.154+0000 7f0eb6a00640 1 -- 172.21.15.137:0/4137737221 wait complete. 2024-03-26T14:02:28.168 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-03-26T14:02:27.155287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668771,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1090,"num_read_kb":678,"num_write":367,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361756,"kb_used_data":12572,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580068,"statfs":{"total":1151940427776,"available":1151569989632,"internally_reserved":0,"allocated":12873728,"data_stored":8257717,"data_compressed":427101,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6184,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":54,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005672"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222603+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T14:00:55.222603+0000","last_peered":"2024-03-26T14:00:55.222603+0000","last_clean":"2024-03-26T14:00:55.222603+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T14:00:55.222603+0000","last_undegraded":"2024-03-26T14:00:55.222603+0000","last_fullsized":"2024-03-26T14:00:55.222603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212530+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T14:00:55.212530+0000","last_peered":"2024-03-26T14:00:55.212530+0000","last_clean":"2024-03-26T14:00:55.212530+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T14:00:55.212530+0000","last_undegraded":"2024-03-26T14:00:55.212530+0000","last_fullsized":"2024-03-26T14:00:55.212530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211035+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T14:00:55.211035+0000","last_peered":"2024-03-26T14:00:55.211035+0000","last_clean":"2024-03-26T14:00:55.211035+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T14:00:55.211035+0000","last_undegraded":"2024-03-26T14:00:55.211035+0000","last_fullsized":"2024-03-26T14:00:55.211035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210071+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T14:00:55.210071+0000","last_peered":"2024-03-26T14:00:55.210071+0000","last_clean":"2024-03-26T14:00:55.210071+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T14:00:55.210071+0000","last_undegraded":"2024-03-26T14:00:55.210071+0000","last_fullsized":"2024-03-26T14:00:55.210071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224530+0000","last_change":"2024-03-26T14:00:55.221738+0000","last_active":"2024-03-26T14:00:55.224530+0000","last_peered":"2024-03-26T14:00:55.224530+0000","last_clean":"2024-03-26T14:00:55.224530+0000","last_became_active":"2024-03-26T14:00:55.220626+0000","last_became_peered":"2024-03-26T14:00:55.220626+0000","last_unstale":"2024-03-26T14:00:55.224530+0000","last_undegraded":"2024-03-26T14:00:55.224530+0000","last_fullsized":"2024-03-26T14:00:55.224530+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:50.208896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217727+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T14:00:55.217727+0000","last_peered":"2024-03-26T14:00:55.217727+0000","last_clean":"2024-03-26T14:00:55.217727+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T14:00:55.217727+0000","last_undegraded":"2024-03-26T14:00:55.217727+0000","last_fullsized":"2024-03-26T14:00:55.217727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213817+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T14:00:55.213817+0000","last_peered":"2024-03-26T14:00:55.213817+0000","last_clean":"2024-03-26T14:00:55.213817+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T14:00:55.213817+0000","last_undegraded":"2024-03-26T14:00:55.213817+0000","last_fullsized":"2024-03-26T14:00:55.213817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206757+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T14:00:55.206757+0000","last_peered":"2024-03-26T14:00:55.206757+0000","last_clean":"2024-03-26T14:00:55.206757+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T14:00:55.206757+0000","last_undegraded":"2024-03-26T14:00:55.206757+0000","last_fullsized":"2024-03-26T14:00:55.206757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434521+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T14:00:55.434521+0000","last_peered":"2024-03-26T14:00:55.434521+0000","last_clean":"2024-03-26T14:00:55.434521+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T14:00:55.434521+0000","last_undegraded":"2024-03-26T14:00:55.434521+0000","last_fullsized":"2024-03-26T14:00:55.434521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218359+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T14:00:55.218359+0000","last_peered":"2024-03-26T14:00:55.218359+0000","last_clean":"2024-03-26T14:00:55.218359+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T14:00:55.218359+0000","last_undegraded":"2024-03-26T14:00:55.218359+0000","last_fullsized":"2024-03-26T14:00:55.218359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214217+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T14:00:55.214217+0000","last_peered":"2024-03-26T14:00:55.214217+0000","last_clean":"2024-03-26T14:00:55.214217+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T14:00:55.214217+0000","last_undegraded":"2024-03-26T14:00:55.214217+0000","last_fullsized":"2024-03-26T14:00:55.214217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209834+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T14:00:55.209834+0000","last_peered":"2024-03-26T14:00:55.209834+0000","last_clean":"2024-03-26T14:00:55.209834+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T14:00:55.209834+0000","last_undegraded":"2024-03-26T14:00:55.209834+0000","last_fullsized":"2024-03-26T14:00:55.209834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217235+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T14:00:55.217235+0000","last_peered":"2024-03-26T14:00:55.217235+0000","last_clean":"2024-03-26T14:00:55.217235+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T14:00:55.217235+0000","last_undegraded":"2024-03-26T14:00:55.217235+0000","last_fullsized":"2024-03-26T14:00:55.217235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210375+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T14:00:55.210375+0000","last_peered":"2024-03-26T14:00:55.210375+0000","last_clean":"2024-03-26T14:00:55.210375+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T14:00:55.210375+0000","last_undegraded":"2024-03-26T14:00:55.210375+0000","last_fullsized":"2024-03-26T14:00:55.210375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214095+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T14:00:55.214095+0000","last_peered":"2024-03-26T14:00:55.214095+0000","last_clean":"2024-03-26T14:00:55.214095+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T14:00:55.214095+0000","last_undegraded":"2024-03-26T14:00:55.214095+0000","last_fullsized":"2024-03-26T14:00:55.214095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433666+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T14:00:55.433666+0000","last_peered":"2024-03-26T14:00:55.433666+0000","last_clean":"2024-03-26T14:00:55.433666+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T14:00:55.433666+0000","last_undegraded":"2024-03-26T14:00:55.433666+0000","last_fullsized":"2024-03-26T14:00:55.433666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222404+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T14:00:55.222404+0000","last_peered":"2024-03-26T14:00:55.222404+0000","last_clean":"2024-03-26T14:00:55.222404+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T14:00:55.222404+0000","last_undegraded":"2024-03-26T14:00:55.222404+0000","last_fullsized":"2024-03-26T14:00:55.222404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212182+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T14:00:55.212182+0000","last_peered":"2024-03-26T14:00:55.212182+0000","last_clean":"2024-03-26T14:00:55.212182+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T14:00:55.212182+0000","last_undegraded":"2024-03-26T14:00:55.212182+0000","last_fullsized":"2024-03-26T14:00:55.212182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215913+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T14:00:55.215913+0000","last_peered":"2024-03-26T14:00:55.215913+0000","last_clean":"2024-03-26T14:00:55.215913+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T14:00:55.215913+0000","last_undegraded":"2024-03-26T14:00:55.215913+0000","last_fullsized":"2024-03-26T14:00:55.215913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212701+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T14:00:55.212701+0000","last_peered":"2024-03-26T14:00:55.212701+0000","last_clean":"2024-03-26T14:00:55.212701+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T14:00:55.212701+0000","last_undegraded":"2024-03-26T14:00:55.212701+0000","last_fullsized":"2024-03-26T14:00:55.212701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212980+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T14:00:55.212980+0000","last_peered":"2024-03-26T14:00:55.212980+0000","last_clean":"2024-03-26T14:00:55.212980+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T14:00:55.212980+0000","last_undegraded":"2024-03-26T14:00:55.212980+0000","last_fullsized":"2024-03-26T14:00:55.212980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207749+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T14:00:55.207749+0000","last_peered":"2024-03-26T14:00:55.207749+0000","last_clean":"2024-03-26T14:00:55.207749+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T14:00:55.207749+0000","last_undegraded":"2024-03-26T14:00:55.207749+0000","last_fullsized":"2024-03-26T14:00:55.207749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209187+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T14:00:55.209187+0000","last_peered":"2024-03-26T14:00:55.209187+0000","last_clean":"2024-03-26T14:00:55.209187+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T14:00:55.209187+0000","last_undegraded":"2024-03-26T14:00:55.209187+0000","last_fullsized":"2024-03-26T14:00:55.209187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207907+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T14:00:55.207907+0000","last_peered":"2024-03-26T14:00:55.207907+0000","last_clean":"2024-03-26T14:00:55.207907+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T14:00:55.207907+0000","last_undegraded":"2024-03-26T14:00:55.207907+0000","last_fullsized":"2024-03-26T14:00:55.207907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211102+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T14:00:55.211102+0000","last_peered":"2024-03-26T14:00:55.211102+0000","last_clean":"2024-03-26T14:00:55.211102+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T14:00:55.211102+0000","last_undegraded":"2024-03-26T14:00:55.211102+0000","last_fullsized":"2024-03-26T14:00:55.211102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209455+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T14:00:55.209455+0000","last_peered":"2024-03-26T14:00:55.209455+0000","last_clean":"2024-03-26T14:00:55.209455+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T14:00:55.209455+0000","last_undegraded":"2024-03-26T14:00:55.209455+0000","last_fullsized":"2024-03-26T14:00:55.209455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213566+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T14:00:55.213566+0000","last_peered":"2024-03-26T14:00:55.213566+0000","last_clean":"2024-03-26T14:00:55.213566+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T14:00:55.213566+0000","last_undegraded":"2024-03-26T14:00:55.213566+0000","last_fullsized":"2024-03-26T14:00:55.213566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433842+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T14:00:55.433842+0000","last_peered":"2024-03-26T14:00:55.433842+0000","last_clean":"2024-03-26T14:00:55.433842+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T14:00:55.433842+0000","last_undegraded":"2024-03-26T14:00:55.433842+0000","last_fullsized":"2024-03-26T14:00:55.433842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209081+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T14:00:55.209081+0000","last_peered":"2024-03-26T14:00:55.209081+0000","last_clean":"2024-03-26T14:00:55.209081+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T14:00:55.209081+0000","last_undegraded":"2024-03-26T14:00:55.209081+0000","last_fullsized":"2024-03-26T14:00:55.209081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214791+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T14:00:55.214791+0000","last_peered":"2024-03-26T14:00:55.214791+0000","last_clean":"2024-03-26T14:00:55.214791+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T14:00:55.214791+0000","last_undegraded":"2024-03-26T14:00:55.214791+0000","last_fullsized":"2024-03-26T14:00:55.214791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228631+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T14:00:55.228631+0000","last_peered":"2024-03-26T14:00:55.228631+0000","last_clean":"2024-03-26T14:00:55.228631+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T14:00:55.228631+0000","last_undegraded":"2024-03-26T14:00:55.228631+0000","last_fullsized":"2024-03-26T14:00:55.228631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214716+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T14:00:55.214716+0000","last_peered":"2024-03-26T14:00:55.214716+0000","last_clean":"2024-03-26T14:00:55.214716+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T14:00:55.214716+0000","last_undegraded":"2024-03-26T14:00:55.214716+0000","last_fullsized":"2024-03-26T14:00:55.214716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":193,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:20.648087+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T14:02:20.648087+0000","last_peered":"2024-03-26T14:02:20.648087+0000","last_clean":"2024-03-26T14:02:20.648087+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T14:02:20.648087+0000","last_undegraded":"2024-03-26T14:02:20.648087+0000","last_fullsized":"2024-03-26T14:02:20.648087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214812+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T14:00:55.214812+0000","last_peered":"2024-03-26T14:00:55.214812+0000","last_clean":"2024-03-26T14:00:55.214812+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T14:00:55.214812+0000","last_undegraded":"2024-03-26T14:00:55.214812+0000","last_fullsized":"2024-03-26T14:00:55.214812+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216341+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T14:00:55.216341+0000","last_peered":"2024-03-26T14:00:55.216341+0000","last_clean":"2024-03-26T14:00:55.216341+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T14:00:55.216341+0000","last_undegraded":"2024-03-26T14:00:55.216341+0000","last_fullsized":"2024-03-26T14:00:55.216341+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217641+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T14:00:55.217641+0000","last_peered":"2024-03-26T14:00:55.217641+0000","last_clean":"2024-03-26T14:00:55.217641+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T14:00:55.217641+0000","last_undegraded":"2024-03-26T14:00:55.217641+0000","last_fullsized":"2024-03-26T14:00:55.217641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227975+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T14:00:55.227975+0000","last_peered":"2024-03-26T14:00:55.227975+0000","last_clean":"2024-03-26T14:00:55.227975+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T14:00:55.227975+0000","last_undegraded":"2024-03-26T14:00:55.227975+0000","last_fullsized":"2024-03-26T14:00:55.227975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222150+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T14:00:55.222150+0000","last_peered":"2024-03-26T14:00:55.222150+0000","last_clean":"2024-03-26T14:00:55.222150+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T14:00:55.222150+0000","last_undegraded":"2024-03-26T14:00:55.222150+0000","last_fullsized":"2024-03-26T14:00:55.222150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217537+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T14:00:55.217537+0000","last_peered":"2024-03-26T14:00:55.217537+0000","last_clean":"2024-03-26T14:00:55.217537+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T14:00:55.217537+0000","last_undegraded":"2024-03-26T14:00:55.217537+0000","last_fullsized":"2024-03-26T14:00:55.217537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210377+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T14:00:55.210377+0000","last_peered":"2024-03-26T14:00:55.210377+0000","last_clean":"2024-03-26T14:00:55.210377+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T14:00:55.210377+0000","last_undegraded":"2024-03-26T14:00:55.210377+0000","last_fullsized":"2024-03-26T14:00:55.210377+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212422+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T14:00:55.212422+0000","last_peered":"2024-03-26T14:00:55.212422+0000","last_clean":"2024-03-26T14:00:55.212422+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T14:00:55.212422+0000","last_undegraded":"2024-03-26T14:00:55.212422+0000","last_fullsized":"2024-03-26T14:00:55.212422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226654+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T14:00:55.226654+0000","last_peered":"2024-03-26T14:00:55.226654+0000","last_clean":"2024-03-26T14:00:55.226654+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T14:00:55.226654+0000","last_undegraded":"2024-03-26T14:00:55.226654+0000","last_fullsized":"2024-03-26T14:00:55.226654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211075+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T14:00:55.211075+0000","last_peered":"2024-03-26T14:00:55.211075+0000","last_clean":"2024-03-26T14:00:55.211075+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T14:00:55.211075+0000","last_undegraded":"2024-03-26T14:00:55.211075+0000","last_fullsized":"2024-03-26T14:00:55.211075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206766+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T14:00:55.206766+0000","last_peered":"2024-03-26T14:00:55.206766+0000","last_clean":"2024-03-26T14:00:55.206766+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T14:00:55.206766+0000","last_undegraded":"2024-03-26T14:00:55.206766+0000","last_fullsized":"2024-03-26T14:00:55.206766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:24.660131+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T14:02:24.660131+0000","last_peered":"2024-03-26T14:02:24.660131+0000","last_clean":"2024-03-26T14:02:24.660131+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T14:02:24.660131+0000","last_undegraded":"2024-03-26T14:02:24.660131+0000","last_fullsized":"2024-03-26T14:02:24.660131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434135+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T14:00:55.434135+0000","last_peered":"2024-03-26T14:00:55.434135+0000","last_clean":"2024-03-26T14:00:55.434135+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T14:00:55.434135+0000","last_undegraded":"2024-03-26T14:00:55.434135+0000","last_fullsized":"2024-03-26T14:00:55.434135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211849+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T14:00:55.211849+0000","last_peered":"2024-03-26T14:00:55.211849+0000","last_clean":"2024-03-26T14:00:55.211849+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T14:00:55.211849+0000","last_undegraded":"2024-03-26T14:00:55.211849+0000","last_fullsized":"2024-03-26T14:00:55.211849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214014+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T14:00:55.214014+0000","last_peered":"2024-03-26T14:00:55.214014+0000","last_clean":"2024-03-26T14:00:55.214014+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T14:00:55.214014+0000","last_undegraded":"2024-03-26T14:00:55.214014+0000","last_fullsized":"2024-03-26T14:00:55.214014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225122+0000","last_change":"2024-03-26T14:00:55.224154+0000","last_active":"2024-03-26T14:00:55.225122+0000","last_peered":"2024-03-26T14:00:55.225122+0000","last_clean":"2024-03-26T14:00:55.225122+0000","last_became_active":"2024-03-26T14:00:55.223674+0000","last_became_peered":"2024-03-26T14:00:55.223674+0000","last_unstale":"2024-03-26T14:00:55.225122+0000","last_undegraded":"2024-03-26T14:00:55.225122+0000","last_fullsized":"2024-03-26T14:00:55.225122+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:48:31.146317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209697+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T14:00:55.209697+0000","last_peered":"2024-03-26T14:00:55.209697+0000","last_clean":"2024-03-26T14:00:55.209697+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T14:00:55.209697+0000","last_undegraded":"2024-03-26T14:00:55.209697+0000","last_fullsized":"2024-03-26T14:00:55.209697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228174+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T14:00:55.228174+0000","last_peered":"2024-03-26T14:00:55.228174+0000","last_clean":"2024-03-26T14:00:55.228174+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T14:00:55.228174+0000","last_undegraded":"2024-03-26T14:00:55.228174+0000","last_fullsized":"2024-03-26T14:00:55.228174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433993+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T14:00:55.433993+0000","last_peered":"2024-03-26T14:00:55.433993+0000","last_clean":"2024-03-26T14:00:55.433993+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T14:00:55.433993+0000","last_undegraded":"2024-03-26T14:00:55.433993+0000","last_fullsized":"2024-03-26T14:00:55.433993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210803+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T14:00:55.210803+0000","last_peered":"2024-03-26T14:00:55.210803+0000","last_clean":"2024-03-26T14:00:55.210803+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T14:00:55.210803+0000","last_undegraded":"2024-03-26T14:00:55.210803+0000","last_fullsized":"2024-03-26T14:00:55.210803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":398,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:25.633621+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T14:02:25.633621+0000","last_peered":"2024-03-26T14:02:25.633621+0000","last_clean":"2024-03-26T14:02:25.633621+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T14:02:25.633621+0000","last_undegraded":"2024-03-26T14:02:25.633621+0000","last_fullsized":"2024-03-26T14:02:25.633621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73041,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":324,"num_read_kb":234,"num_write":43,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217378+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T14:00:55.217378+0000","last_peered":"2024-03-26T14:00:55.217378+0000","last_clean":"2024-03-26T14:00:55.217378+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T14:00:55.217378+0000","last_undegraded":"2024-03-26T14:00:55.217378+0000","last_fullsized":"2024-03-26T14:00:55.217378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.229375+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T14:00:55.229375+0000","last_peered":"2024-03-26T14:00:55.229375+0000","last_clean":"2024-03-26T14:00:55.229375+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T14:00:55.229375+0000","last_undegraded":"2024-03-26T14:00:55.229375+0000","last_fullsized":"2024-03-26T14:00:55.229375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.240886+0000","last_change":"2024-03-26T14:00:55.239852+0000","last_active":"2024-03-26T14:00:55.240886+0000","last_peered":"2024-03-26T14:00:55.240886+0000","last_clean":"2024-03-26T14:00:55.240886+0000","last_became_active":"2024-03-26T14:00:55.221294+0000","last_became_peered":"2024-03-26T14:00:55.221294+0000","last_unstale":"2024-03-26T14:00:55.240886+0000","last_undegraded":"2024-03-26T14:00:55.240886+0000","last_fullsized":"2024-03-26T14:00:55.240886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:14:53.910832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217300+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T14:00:55.217300+0000","last_peered":"2024-03-26T14:00:55.217300+0000","last_clean":"2024-03-26T14:00:55.217300+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T14:00:55.217300+0000","last_undegraded":"2024-03-26T14:00:55.217300+0000","last_fullsized":"2024-03-26T14:00:55.217300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220773+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T14:00:55.220773+0000","last_peered":"2024-03-26T14:00:55.220773+0000","last_clean":"2024-03-26T14:00:55.220773+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T14:00:55.220773+0000","last_undegraded":"2024-03-26T14:00:55.220773+0000","last_fullsized":"2024-03-26T14:00:55.220773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227998+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T14:00:55.227998+0000","last_peered":"2024-03-26T14:00:55.227998+0000","last_clean":"2024-03-26T14:00:55.227998+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T14:00:55.227998+0000","last_undegraded":"2024-03-26T14:00:55.227998+0000","last_fullsized":"2024-03-26T14:00:55.227998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214454+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T14:00:55.214454+0000","last_peered":"2024-03-26T14:00:55.214454+0000","last_clean":"2024-03-26T14:00:55.214454+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T14:00:55.214454+0000","last_undegraded":"2024-03-26T14:00:55.214454+0000","last_fullsized":"2024-03-26T14:00:55.214454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212803+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T14:00:55.212803+0000","last_peered":"2024-03-26T14:00:55.212803+0000","last_clean":"2024-03-26T14:00:55.212803+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T14:00:55.212803+0000","last_undegraded":"2024-03-26T14:00:55.212803+0000","last_fullsized":"2024-03-26T14:00:55.212803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:21.693198+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T14:02:21.693198+0000","last_peered":"2024-03-26T14:02:21.693198+0000","last_clean":"2024-03-26T14:02:21.693198+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T14:02:21.693198+0000","last_undegraded":"2024-03-26T14:02:21.693198+0000","last_fullsized":"2024-03-26T14:02:21.693198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":159,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213552+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T14:00:55.213552+0000","last_peered":"2024-03-26T14:00:55.213552+0000","last_clean":"2024-03-26T14:00:55.213552+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T14:00:55.213552+0000","last_undegraded":"2024-03-26T14:00:55.213552+0000","last_fullsized":"2024-03-26T14:00:55.213552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217932+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T14:00:55.217932+0000","last_peered":"2024-03-26T14:00:55.217932+0000","last_clean":"2024-03-26T14:00:55.217932+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T14:00:55.217932+0000","last_undegraded":"2024-03-26T14:00:55.217932+0000","last_fullsized":"2024-03-26T14:00:55.217932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212507+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T14:00:55.212507+0000","last_peered":"2024-03-26T14:00:55.212507+0000","last_clean":"2024-03-26T14:00:55.212507+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T14:00:55.212507+0000","last_undegraded":"2024-03-26T14:00:55.212507+0000","last_fullsized":"2024-03-26T14:00:55.212507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227066+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T14:00:55.227066+0000","last_peered":"2024-03-26T14:00:55.227066+0000","last_clean":"2024-03-26T14:00:55.227066+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T14:00:55.227066+0000","last_undegraded":"2024-03-26T14:00:55.227066+0000","last_fullsized":"2024-03-26T14:00:55.227066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226862+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T14:00:55.226862+0000","last_peered":"2024-03-26T14:00:55.226862+0000","last_clean":"2024-03-26T14:00:55.226862+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T14:00:55.226862+0000","last_undegraded":"2024-03-26T14:00:55.226862+0000","last_fullsized":"2024-03-26T14:00:55.226862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217173+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T14:00:55.217173+0000","last_peered":"2024-03-26T14:00:55.217173+0000","last_clean":"2024-03-26T14:00:55.217173+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T14:00:55.217173+0000","last_undegraded":"2024-03-26T14:00:55.217173+0000","last_fullsized":"2024-03-26T14:00:55.217173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221832+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T14:00:55.221832+0000","last_peered":"2024-03-26T14:00:55.221832+0000","last_clean":"2024-03-26T14:00:55.221832+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T14:00:55.221832+0000","last_undegraded":"2024-03-26T14:00:55.221832+0000","last_fullsized":"2024-03-26T14:00:55.221832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214516+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T14:00:55.214516+0000","last_peered":"2024-03-26T14:00:55.214516+0000","last_clean":"2024-03-26T14:00:55.214516+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T14:00:55.214516+0000","last_undegraded":"2024-03-26T14:00:55.214516+0000","last_fullsized":"2024-03-26T14:00:55.214516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225256+0000","last_change":"2024-03-26T14:00:55.224501+0000","last_active":"2024-03-26T14:00:55.225256+0000","last_peered":"2024-03-26T14:00:55.225256+0000","last_clean":"2024-03-26T14:00:55.225256+0000","last_became_active":"2024-03-26T14:00:55.223843+0000","last_became_peered":"2024-03-26T14:00:55.223843+0000","last_unstale":"2024-03-26T14:00:55.225256+0000","last_undegraded":"2024-03-26T14:00:55.225256+0000","last_fullsized":"2024-03-26T14:00:55.225256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:29:11.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217789+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T14:00:55.217789+0000","last_peered":"2024-03-26T14:00:55.217789+0000","last_clean":"2024-03-26T14:00:55.217789+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T14:00:55.217789+0000","last_undegraded":"2024-03-26T14:00:55.217789+0000","last_fullsized":"2024-03-26T14:00:55.217789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217139+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T14:00:55.217139+0000","last_peered":"2024-03-26T14:00:55.217139+0000","last_clean":"2024-03-26T14:00:55.217139+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T14:00:55.217139+0000","last_undegraded":"2024-03-26T14:00:55.217139+0000","last_fullsized":"2024-03-26T14:00:55.217139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214292+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T14:00:55.214292+0000","last_peered":"2024-03-26T14:00:55.214292+0000","last_clean":"2024-03-26T14:00:55.214292+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T14:00:55.214292+0000","last_undegraded":"2024-03-26T14:00:55.214292+0000","last_fullsized":"2024-03-26T14:00:55.214292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217644+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T14:00:55.217644+0000","last_peered":"2024-03-26T14:00:55.217644+0000","last_clean":"2024-03-26T14:00:55.217644+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T14:00:55.217644+0000","last_undegraded":"2024-03-26T14:00:55.217644+0000","last_fullsized":"2024-03-26T14:00:55.217644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217901+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T14:00:55.217901+0000","last_peered":"2024-03-26T14:00:55.217901+0000","last_clean":"2024-03-26T14:00:55.217901+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T14:00:55.217901+0000","last_undegraded":"2024-03-26T14:00:55.217901+0000","last_fullsized":"2024-03-26T14:00:55.217901+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227219+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T14:00:55.227219+0000","last_peered":"2024-03-26T14:00:55.227219+0000","last_clean":"2024-03-26T14:00:55.227219+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T14:00:55.227219+0000","last_undegraded":"2024-03-26T14:00:55.227219+0000","last_fullsized":"2024-03-26T14:00:55.227219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:25.703123+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T14:02:25.703123+0000","last_peered":"2024-03-26T14:02:25.703123+0000","last_clean":"2024-03-26T14:02:25.703123+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T14:02:25.703123+0000","last_undegraded":"2024-03-26T14:02:25.703123+0000","last_fullsized":"2024-03-26T14:02:25.703123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":157,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214061+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T14:00:55.214061+0000","last_peered":"2024-03-26T14:00:55.214061+0000","last_clean":"2024-03-26T14:00:55.214061+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T14:00:55.214061+0000","last_undegraded":"2024-03-26T14:00:55.214061+0000","last_fullsized":"2024-03-26T14:00:55.214061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207002+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T14:00:55.207002+0000","last_peered":"2024-03-26T14:00:55.207002+0000","last_clean":"2024-03-26T14:00:55.207002+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T14:00:55.207002+0000","last_undegraded":"2024-03-26T14:00:55.207002+0000","last_fullsized":"2024-03-26T14:00:55.207002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209617+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T14:00:55.209617+0000","last_peered":"2024-03-26T14:00:55.209617+0000","last_clean":"2024-03-26T14:00:55.209617+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T14:00:55.209617+0000","last_undegraded":"2024-03-26T14:00:55.209617+0000","last_fullsized":"2024-03-26T14:00:55.209617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213393+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T14:00:55.213393+0000","last_peered":"2024-03-26T14:00:55.213393+0000","last_clean":"2024-03-26T14:00:55.213393+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T14:00:55.213393+0000","last_undegraded":"2024-03-26T14:00:55.213393+0000","last_fullsized":"2024-03-26T14:00:55.213393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228488+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T14:00:55.228488+0000","last_peered":"2024-03-26T14:00:55.228488+0000","last_clean":"2024-03-26T14:00:55.228488+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T14:00:55.228488+0000","last_undegraded":"2024-03-26T14:00:55.228488+0000","last_fullsized":"2024-03-26T14:00:55.228488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216689+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T14:00:55.216689+0000","last_peered":"2024-03-26T14:00:55.216689+0000","last_clean":"2024-03-26T14:00:55.216689+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T14:00:55.216689+0000","last_undegraded":"2024-03-26T14:00:55.216689+0000","last_fullsized":"2024-03-26T14:00:55.216689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.439884+0000","last_change":"2024-03-26T14:00:55.439230+0000","last_active":"2024-03-26T14:00:55.439884+0000","last_peered":"2024-03-26T14:00:55.439884+0000","last_clean":"2024-03-26T14:00:55.439884+0000","last_became_active":"2024-03-26T14:00:55.434988+0000","last_became_peered":"2024-03-26T14:00:55.434988+0000","last_unstale":"2024-03-26T14:00:55.439884+0000","last_undegraded":"2024-03-26T14:00:55.439884+0000","last_fullsized":"2024-03-26T14:00:55.439884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:58.291575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.223104+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T14:00:55.223104+0000","last_peered":"2024-03-26T14:00:55.223104+0000","last_clean":"2024-03-26T14:00:55.223104+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T14:00:55.223104+0000","last_undegraded":"2024-03-26T14:00:55.223104+0000","last_fullsized":"2024-03-26T14:00:55.223104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217076+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T14:00:55.217076+0000","last_peered":"2024-03-26T14:00:55.217076+0000","last_clean":"2024-03-26T14:00:55.217076+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T14:00:55.217076+0000","last_undegraded":"2024-03-26T14:00:55.217076+0000","last_fullsized":"2024-03-26T14:00:55.217076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217337+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T14:00:55.217337+0000","last_peered":"2024-03-26T14:00:55.217337+0000","last_clean":"2024-03-26T14:00:55.217337+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T14:00:55.217337+0000","last_undegraded":"2024-03-26T14:00:55.217337+0000","last_fullsized":"2024-03-26T14:00:55.217337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206756+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T14:00:55.206756+0000","last_peered":"2024-03-26T14:00:55.206756+0000","last_clean":"2024-03-26T14:00:55.206756+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T14:00:55.206756+0000","last_undegraded":"2024-03-26T14:00:55.206756+0000","last_fullsized":"2024-03-26T14:00:55.206756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:20.345895+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T14:02:20.345895+0000","last_peered":"2024-03-26T14:02:20.345895+0000","last_clean":"2024-03-26T14:02:20.345895+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T14:02:20.345895+0000","last_undegraded":"2024-03-26T14:02:20.345895+0000","last_fullsized":"2024-03-26T14:02:20.345895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206944+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T14:00:55.206944+0000","last_peered":"2024-03-26T14:00:55.206944+0000","last_clean":"2024-03-26T14:00:55.206944+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T14:00:55.206944+0000","last_undegraded":"2024-03-26T14:00:55.206944+0000","last_fullsized":"2024-03-26T14:00:55.206944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211722+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T14:00:55.211722+0000","last_peered":"2024-03-26T14:00:55.211722+0000","last_clean":"2024-03-26T14:00:55.211722+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T14:00:55.211722+0000","last_undegraded":"2024-03-26T14:00:55.211722+0000","last_fullsized":"2024-03-26T14:00:55.211722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210428+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T14:00:55.210428+0000","last_peered":"2024-03-26T14:00:55.210428+0000","last_clean":"2024-03-26T14:00:55.210428+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T14:00:55.210428+0000","last_undegraded":"2024-03-26T14:00:55.210428+0000","last_fullsized":"2024-03-26T14:00:55.210428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212631+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T14:00:55.212631+0000","last_peered":"2024-03-26T14:00:55.212631+0000","last_clean":"2024-03-26T14:00:55.212631+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T14:00:55.212631+0000","last_undegraded":"2024-03-26T14:00:55.212631+0000","last_fullsized":"2024-03-26T14:00:55.212631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210150+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T14:00:55.210150+0000","last_peered":"2024-03-26T14:00:55.210150+0000","last_clean":"2024-03-26T14:00:55.210150+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T14:00:55.210150+0000","last_undegraded":"2024-03-26T14:00:55.210150+0000","last_fullsized":"2024-03-26T14:00:55.210150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433840+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T14:00:55.433840+0000","last_peered":"2024-03-26T14:00:55.433840+0000","last_clean":"2024-03-26T14:00:55.433840+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T14:00:55.433840+0000","last_undegraded":"2024-03-26T14:00:55.433840+0000","last_fullsized":"2024-03-26T14:00:55.433840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213281+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T14:00:55.213281+0000","last_peered":"2024-03-26T14:00:55.213281+0000","last_clean":"2024-03-26T14:00:55.213281+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T14:00:55.213281+0000","last_undegraded":"2024-03-26T14:00:55.213281+0000","last_fullsized":"2024-03-26T14:00:55.213281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216737+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T14:00:55.216737+0000","last_peered":"2024-03-26T14:00:55.216737+0000","last_clean":"2024-03-26T14:00:55.216737+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T14:00:55.216737+0000","last_undegraded":"2024-03-26T14:00:55.216737+0000","last_fullsized":"2024-03-26T14:00:55.216737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206794+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T14:00:55.206794+0000","last_peered":"2024-03-26T14:00:55.206794+0000","last_clean":"2024-03-26T14:00:55.206794+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T14:00:55.206794+0000","last_undegraded":"2024-03-26T14:00:55.206794+0000","last_fullsized":"2024-03-26T14:00:55.206794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215526+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T14:00:55.215526+0000","last_peered":"2024-03-26T14:00:55.215526+0000","last_clean":"2024-03-26T14:00:55.215526+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T14:00:55.215526+0000","last_undegraded":"2024-03-26T14:00:55.215526+0000","last_fullsized":"2024-03-26T14:00:55.215526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209311+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T14:00:55.209311+0000","last_peered":"2024-03-26T14:00:55.209311+0000","last_clean":"2024-03-26T14:00:55.209311+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T14:00:55.209311+0000","last_undegraded":"2024-03-26T14:00:55.209311+0000","last_fullsized":"2024-03-26T14:00:55.209311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:34.135298+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T14:01:34.135298+0000","last_peered":"2024-03-26T14:01:34.135298+0000","last_clean":"2024-03-26T14:01:34.135298+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T14:01:34.135298+0000","last_undegraded":"2024-03-26T14:01:34.135298+0000","last_fullsized":"2024-03-26T14:01:34.135298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207377+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T14:00:55.207377+0000","last_peered":"2024-03-26T14:00:55.207377+0000","last_clean":"2024-03-26T14:00:55.207377+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T14:00:55.207377+0000","last_undegraded":"2024-03-26T14:00:55.207377+0000","last_fullsized":"2024-03-26T14:00:55.207377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209832+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T14:00:55.209832+0000","last_peered":"2024-03-26T14:00:55.209832+0000","last_clean":"2024-03-26T14:00:55.209832+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T14:00:55.209832+0000","last_undegraded":"2024-03-26T14:00:55.209832+0000","last_fullsized":"2024-03-26T14:00:55.209832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213280+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T14:00:55.213280+0000","last_peered":"2024-03-26T14:00:55.213280+0000","last_clean":"2024-03-26T14:00:55.213280+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T14:00:55.213280+0000","last_undegraded":"2024-03-26T14:00:55.213280+0000","last_fullsized":"2024-03-26T14:00:55.213280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214151+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T14:00:55.214151+0000","last_peered":"2024-03-26T14:00:55.214151+0000","last_clean":"2024-03-26T14:00:55.214151+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T14:00:55.214151+0000","last_undegraded":"2024-03-26T14:00:55.214151+0000","last_fullsized":"2024-03-26T14:00:55.214151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.432636+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T14:00:55.432636+0000","last_peered":"2024-03-26T14:00:55.432636+0000","last_clean":"2024-03-26T14:00:55.432636+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T14:00:55.432636+0000","last_undegraded":"2024-03-26T14:00:55.432636+0000","last_fullsized":"2024-03-26T14:00:55.432636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212890+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T14:00:55.212890+0000","last_peered":"2024-03-26T14:00:55.212890+0000","last_clean":"2024-03-26T14:00:55.212890+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T14:00:55.212890+0000","last_undegraded":"2024-03-26T14:00:55.212890+0000","last_fullsized":"2024-03-26T14:00:55.212890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214297+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T14:00:55.214297+0000","last_peered":"2024-03-26T14:00:55.214297+0000","last_clean":"2024-03-26T14:00:55.214297+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T14:00:55.214297+0000","last_undegraded":"2024-03-26T14:00:55.214297+0000","last_fullsized":"2024-03-26T14:00:55.214297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210537+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T14:00:55.210537+0000","last_peered":"2024-03-26T14:00:55.210537+0000","last_clean":"2024-03-26T14:00:55.210537+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T14:00:55.210537+0000","last_undegraded":"2024-03-26T14:00:55.210537+0000","last_fullsized":"2024-03-26T14:00:55.210537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228901+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T14:00:55.228901+0000","last_peered":"2024-03-26T14:00:55.228901+0000","last_clean":"2024-03-26T14:00:55.228901+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T14:00:55.228901+0000","last_undegraded":"2024-03-26T14:00:55.228901+0000","last_fullsized":"2024-03-26T14:00:55.228901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214653+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T14:00:55.214653+0000","last_peered":"2024-03-26T14:00:55.214653+0000","last_clean":"2024-03-26T14:00:55.214653+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T14:00:55.214653+0000","last_undegraded":"2024-03-26T14:00:55.214653+0000","last_fullsized":"2024-03-26T14:00:55.214653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225824+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T14:00:55.225824+0000","last_peered":"2024-03-26T14:00:55.225824+0000","last_clean":"2024-03-26T14:00:55.225824+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T14:00:55.225824+0000","last_undegraded":"2024-03-26T14:00:55.225824+0000","last_fullsized":"2024-03-26T14:00:55.225824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217852+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T14:00:55.217852+0000","last_peered":"2024-03-26T14:00:55.217852+0000","last_clean":"2024-03-26T14:00:55.217852+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T14:00:55.217852+0000","last_undegraded":"2024-03-26T14:00:55.217852+0000","last_fullsized":"2024-03-26T14:00:55.217852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214735+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T14:00:55.214735+0000","last_peered":"2024-03-26T14:00:55.214735+0000","last_clean":"2024-03-26T14:00:55.214735+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T14:00:55.214735+0000","last_undegraded":"2024-03-26T14:00:55.214735+0000","last_fullsized":"2024-03-26T14:00:55.214735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215872+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T14:00:55.215872+0000","last_peered":"2024-03-26T14:00:55.215872+0000","last_clean":"2024-03-26T14:00:55.215872+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T14:00:55.215872+0000","last_undegraded":"2024-03-26T14:00:55.215872+0000","last_fullsized":"2024-03-26T14:00:55.215872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212414+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T14:00:55.212414+0000","last_peered":"2024-03-26T14:00:55.212414+0000","last_clean":"2024-03-26T14:00:55.212414+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T14:00:55.212414+0000","last_undegraded":"2024-03-26T14:00:55.212414+0000","last_fullsized":"2024-03-26T14:00:55.212414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213601+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T14:00:55.213601+0000","last_peered":"2024-03-26T14:00:55.213601+0000","last_clean":"2024-03-26T14:00:55.213601+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T14:00:55.213601+0000","last_undegraded":"2024-03-26T14:00:55.213601+0000","last_fullsized":"2024-03-26T14:00:55.213601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213914+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T14:00:55.213914+0000","last_peered":"2024-03-26T14:00:55.213914+0000","last_clean":"2024-03-26T14:00:55.213914+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T14:00:55.213914+0000","last_undegraded":"2024-03-26T14:00:55.213914+0000","last_fullsized":"2024-03-26T14:00:55.213914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217436+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T14:00:55.217436+0000","last_peered":"2024-03-26T14:00:55.217436+0000","last_clean":"2024-03-26T14:00:55.217436+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T14:00:55.217436+0000","last_undegraded":"2024-03-26T14:00:55.217436+0000","last_fullsized":"2024-03-26T14:00:55.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221547+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T14:00:55.221547+0000","last_peered":"2024-03-26T14:00:55.221547+0000","last_clean":"2024-03-26T14:00:55.221547+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T14:00:55.221547+0000","last_undegraded":"2024-03-26T14:00:55.221547+0000","last_fullsized":"2024-03-26T14:00:55.221547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218503+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T14:00:55.218503+0000","last_peered":"2024-03-26T14:00:55.218503+0000","last_clean":"2024-03-26T14:00:55.218503+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T14:00:55.218503+0000","last_undegraded":"2024-03-26T14:00:55.218503+0000","last_fullsized":"2024-03-26T14:00:55.218503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207037+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T14:00:55.207037+0000","last_peered":"2024-03-26T14:00:55.207037+0000","last_clean":"2024-03-26T14:00:55.207037+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T14:00:55.207037+0000","last_undegraded":"2024-03-26T14:00:55.207037+0000","last_fullsized":"2024-03-26T14:00:55.207037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226954+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T14:00:55.226954+0000","last_peered":"2024-03-26T14:00:55.226954+0000","last_clean":"2024-03-26T14:00:55.226954+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T14:00:55.226954+0000","last_undegraded":"2024-03-26T14:00:55.226954+0000","last_fullsized":"2024-03-26T14:00:55.226954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206887+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T14:00:55.206887+0000","last_peered":"2024-03-26T14:00:55.206887+0000","last_clean":"2024-03-26T14:00:55.206887+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T14:00:55.206887+0000","last_undegraded":"2024-03-26T14:00:55.206887+0000","last_fullsized":"2024-03-26T14:00:55.206887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433797+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T14:00:55.433797+0000","last_peered":"2024-03-26T14:00:55.433797+0000","last_clean":"2024-03-26T14:00:55.433797+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T14:00:55.433797+0000","last_undegraded":"2024-03-26T14:00:55.433797+0000","last_fullsized":"2024-03-26T14:00:55.433797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217146+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T14:00:55.217146+0000","last_peered":"2024-03-26T14:00:55.217146+0000","last_clean":"2024-03-26T14:00:55.217146+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T14:00:55.217146+0000","last_undegraded":"2024-03-26T14:00:55.217146+0000","last_fullsized":"2024-03-26T14:00:55.217146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221123+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T14:00:55.221123+0000","last_peered":"2024-03-26T14:00:55.221123+0000","last_clean":"2024-03-26T14:00:55.221123+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T14:00:55.221123+0000","last_undegraded":"2024-03-26T14:00:55.221123+0000","last_fullsized":"2024-03-26T14:00:55.221123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78169,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":964,"num_read_kb":569,"num_write":184,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":237809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":522925,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1188321,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776195,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":523127,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037834,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":523047,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299473,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1120474,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462999,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518303,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":591254,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519691,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":593454,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518497,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523978,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1114646,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:02:28.173 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T14:02:28.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.425+0000 7f99c2200640 1 -- 172.21.15.137:0/191175541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc105490 msgr2=0x7f99bc10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.425+0000 7f99c2200640 1 --2- 172.21.15.137:0/191175541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc105490 0x7f99bc10bd20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f99b4009ca0 tx=0x7f99b402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 -- 172.21.15.137:0/191175541 shutdown_connections 2024-03-26T14:02:28.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 --2- 172.21.15.137:0/191175541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc105490 0x7f99bc10bd20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 --2- 172.21.15.137:0/191175541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99bc104ad0 0x7f99bc104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 --2- 172.21.15.137:0/191175541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99bc1038e0 0x7f99bc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 -- 172.21.15.137:0/191175541 >> 172.21.15.137:0/191175541 conn(0x7f99bc0fee50 msgr2=0x7f99bc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:28.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 -- 172.21.15.137:0/191175541 shutdown_connections 2024-03-26T14:02:28.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.426+0000 7f99c2200640 1 -- 172.21.15.137:0/191175541 wait complete. 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 Processor -- start 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 -- start start 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99bc1038e0 0x7f99bc199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 0x7f99bc199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99bc105490 0x7f99bc1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc10f3f0 con 0x7f99bc104ad0 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc10f270 con 0x7f99bc1038e0 2024-03-26T14:02:28.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99c2200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99bc10f570 con 0x7f99bc105490 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 0x7f99bc199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.427+0000 7f99b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 0x7f99bc199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60786/0 (socket says 172.21.15.137:60786) 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 -- 172.21.15.137:0/2524390027 learned_addr learned my addr 172.21.15.137:0/2524390027 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99bb400640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99bc1038e0 0x7f99bc199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99bbe00640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99bc105490 0x7f99bc1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 -- 172.21.15.137:0/2524390027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99bc105490 msgr2=0x7f99bc1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99bc105490 0x7f99bc1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 -- 172.21.15.137:0/2524390027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99bc1038e0 msgr2=0x7f99bc199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99bc1038e0 0x7f99bc199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 -- 172.21.15.137:0/2524390027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc1a16b0 con 0x7f99bc104ad0 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.428+0000 7f99b3e00640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 0x7f99bc199fd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99a400c890 tx=0x7f99a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.429+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4004220 con 0x7f99bc104ad0 2024-03-26T14:02:28.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.429+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99bc1a19a0 con 0x7f99bc104ad0 2024-03-26T14:02:28.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.429+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f99a40043c0 con 0x7f99bc104ad0 2024-03-26T14:02:28.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.429+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a4004a00 con 0x7f99bc104ad0 2024-03-26T14:02:28.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.429+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99bc1a1f40 con 0x7f99bc104ad0 2024-03-26T14:02:28.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.430+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99bc10ac00 con 0x7f99bc104ad0 2024-03-26T14:02:28.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.432+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f99a4004ba0 con 0x7f99bc104ad0 2024-03-26T14:02:28.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.433+0000 7f99b8c00640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f999c0a2890 0x7f999c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.433+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f99a40cec50 con 0x7f99bc104ad0 2024-03-26T14:02:28.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.434+0000 7f99bb400640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f999c0a2890 0x7f999c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.434+0000 7f99bb400640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f999c0a2890 0x7f999c0a4d50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99ac005fd0 tx=0x7f99ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:28.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.437+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99a4014030 con 0x7f99bc104ad0 2024-03-26T14:02:28.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:28 smithi137 ceph-mon[31539]: pgmap v461: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-26T14:02:28.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:28 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/927882287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:28.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.633+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99bc061ae0 con 0x7f999c0a2890 2024-03-26T14:02:28.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.657+0000 7f99b8c00640 1 -- 172.21.15.137:0/2524390027 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340825 (secure 0 0 0) 0x7f99bc061ae0 con 0x7f999c0a2890 2024-03-26T14:02:28.659 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:28.661 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.664+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f999c0a2890 msgr2=0x7f999c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.664+0000 7f99c2200640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f999c0a2890 0x7f999c0a4d50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99ac005fd0 tx=0x7f99ac00b410 comp rx=0 tx=0).stop 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.664+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 msgr2=0x7f99bc199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.664+0000 7f99c2200640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 0x7f99bc199fd0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99a400c890 tx=0x7f99a400cd50 comp rx=0 tx=0).stop 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 shutdown_connections 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f999c0a2890 0x7f999c0a4d50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99bc105490 0x7f99bc1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f99bc104ad0 0x7f99bc199fd0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 --2- 172.21.15.137:0/2524390027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99bc1038e0 0x7f99bc199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 >> 172.21.15.137:0/2524390027 conn(0x7f99bc0fee50 msgr2=0x7f99bc107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 shutdown_connections 2024-03-26T14:02:28.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.665+0000 7f99c2200640 1 -- 172.21.15.137:0/2524390027 wait complete. 2024-03-26T14:02:28.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:28 smithi179 ceph-mon[35472]: pgmap v461: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-26T14:02:28.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:28 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/927882287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:28.681 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-03-26T14:02:27.155287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668771,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1090,"num_read_kb":678,"num_write":367,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361756,"kb_used_data":12572,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580068,"statfs":{"total":1151940427776,"available":1151569989632,"internally_reserved":0,"allocated":12873728,"data_stored":8257717,"data_compressed":427101,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6184,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":54,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005672"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222603+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T14:00:55.222603+0000","last_peered":"2024-03-26T14:00:55.222603+0000","last_clean":"2024-03-26T14:00:55.222603+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T14:00:55.222603+0000","last_undegraded":"2024-03-26T14:00:55.222603+0000","last_fullsized":"2024-03-26T14:00:55.222603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212530+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T14:00:55.212530+0000","last_peered":"2024-03-26T14:00:55.212530+0000","last_clean":"2024-03-26T14:00:55.212530+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T14:00:55.212530+0000","last_undegraded":"2024-03-26T14:00:55.212530+0000","last_fullsized":"2024-03-26T14:00:55.212530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211035+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T14:00:55.211035+0000","last_peered":"2024-03-26T14:00:55.211035+0000","last_clean":"2024-03-26T14:00:55.211035+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T14:00:55.211035+0000","last_undegraded":"2024-03-26T14:00:55.211035+0000","last_fullsized":"2024-03-26T14:00:55.211035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210071+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T14:00:55.210071+0000","last_peered":"2024-03-26T14:00:55.210071+0000","last_clean":"2024-03-26T14:00:55.210071+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T14:00:55.210071+0000","last_undegraded":"2024-03-26T14:00:55.210071+0000","last_fullsized":"2024-03-26T14:00:55.210071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224530+0000","last_change":"2024-03-26T14:00:55.221738+0000","last_active":"2024-03-26T14:00:55.224530+0000","last_peered":"2024-03-26T14:00:55.224530+0000","last_clean":"2024-03-26T14:00:55.224530+0000","last_became_active":"2024-03-26T14:00:55.220626+0000","last_became_peered":"2024-03-26T14:00:55.220626+0000","last_unstale":"2024-03-26T14:00:55.224530+0000","last_undegraded":"2024-03-26T14:00:55.224530+0000","last_fullsized":"2024-03-26T14:00:55.224530+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:50.208896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217727+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T14:00:55.217727+0000","last_peered":"2024-03-26T14:00:55.217727+0000","last_clean":"2024-03-26T14:00:55.217727+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T14:00:55.217727+0000","last_undegraded":"2024-03-26T14:00:55.217727+0000","last_fullsized":"2024-03-26T14:00:55.217727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213817+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T14:00:55.213817+0000","last_peered":"2024-03-26T14:00:55.213817+0000","last_clean":"2024-03-26T14:00:55.213817+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T14:00:55.213817+0000","last_undegraded":"2024-03-26T14:00:55.213817+0000","last_fullsized":"2024-03-26T14:00:55.213817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206757+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T14:00:55.206757+0000","last_peered":"2024-03-26T14:00:55.206757+0000","last_clean":"2024-03-26T14:00:55.206757+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T14:00:55.206757+0000","last_undegraded":"2024-03-26T14:00:55.206757+0000","last_fullsized":"2024-03-26T14:00:55.206757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434521+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T14:00:55.434521+0000","last_peered":"2024-03-26T14:00:55.434521+0000","last_clean":"2024-03-26T14:00:55.434521+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T14:00:55.434521+0000","last_undegraded":"2024-03-26T14:00:55.434521+0000","last_fullsized":"2024-03-26T14:00:55.434521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218359+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T14:00:55.218359+0000","last_peered":"2024-03-26T14:00:55.218359+0000","last_clean":"2024-03-26T14:00:55.218359+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T14:00:55.218359+0000","last_undegraded":"2024-03-26T14:00:55.218359+0000","last_fullsized":"2024-03-26T14:00:55.218359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214217+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T14:00:55.214217+0000","last_peered":"2024-03-26T14:00:55.214217+0000","last_clean":"2024-03-26T14:00:55.214217+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T14:00:55.214217+0000","last_undegraded":"2024-03-26T14:00:55.214217+0000","last_fullsized":"2024-03-26T14:00:55.214217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209834+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T14:00:55.209834+0000","last_peered":"2024-03-26T14:00:55.209834+0000","last_clean":"2024-03-26T14:00:55.209834+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T14:00:55.209834+0000","last_undegraded":"2024-03-26T14:00:55.209834+0000","last_fullsized":"2024-03-26T14:00:55.209834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217235+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T14:00:55.217235+0000","last_peered":"2024-03-26T14:00:55.217235+0000","last_clean":"2024-03-26T14:00:55.217235+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T14:00:55.217235+0000","last_undegraded":"2024-03-26T14:00:55.217235+0000","last_fullsized":"2024-03-26T14:00:55.217235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210375+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T14:00:55.210375+0000","last_peered":"2024-03-26T14:00:55.210375+0000","last_clean":"2024-03-26T14:00:55.210375+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T14:00:55.210375+0000","last_undegraded":"2024-03-26T14:00:55.210375+0000","last_fullsized":"2024-03-26T14:00:55.210375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214095+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T14:00:55.214095+0000","last_peered":"2024-03-26T14:00:55.214095+0000","last_clean":"2024-03-26T14:00:55.214095+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T14:00:55.214095+0000","last_undegraded":"2024-03-26T14:00:55.214095+0000","last_fullsized":"2024-03-26T14:00:55.214095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433666+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T14:00:55.433666+0000","last_peered":"2024-03-26T14:00:55.433666+0000","last_clean":"2024-03-26T14:00:55.433666+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T14:00:55.433666+0000","last_undegraded":"2024-03-26T14:00:55.433666+0000","last_fullsized":"2024-03-26T14:00:55.433666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222404+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T14:00:55.222404+0000","last_peered":"2024-03-26T14:00:55.222404+0000","last_clean":"2024-03-26T14:00:55.222404+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T14:00:55.222404+0000","last_undegraded":"2024-03-26T14:00:55.222404+0000","last_fullsized":"2024-03-26T14:00:55.222404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212182+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T14:00:55.212182+0000","last_peered":"2024-03-26T14:00:55.212182+0000","last_clean":"2024-03-26T14:00:55.212182+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T14:00:55.212182+0000","last_undegraded":"2024-03-26T14:00:55.212182+0000","last_fullsized":"2024-03-26T14:00:55.212182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215913+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T14:00:55.215913+0000","last_peered":"2024-03-26T14:00:55.215913+0000","last_clean":"2024-03-26T14:00:55.215913+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T14:00:55.215913+0000","last_undegraded":"2024-03-26T14:00:55.215913+0000","last_fullsized":"2024-03-26T14:00:55.215913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212701+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T14:00:55.212701+0000","last_peered":"2024-03-26T14:00:55.212701+0000","last_clean":"2024-03-26T14:00:55.212701+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T14:00:55.212701+0000","last_undegraded":"2024-03-26T14:00:55.212701+0000","last_fullsized":"2024-03-26T14:00:55.212701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212980+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T14:00:55.212980+0000","last_peered":"2024-03-26T14:00:55.212980+0000","last_clean":"2024-03-26T14:00:55.212980+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T14:00:55.212980+0000","last_undegraded":"2024-03-26T14:00:55.212980+0000","last_fullsized":"2024-03-26T14:00:55.212980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207749+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T14:00:55.207749+0000","last_peered":"2024-03-26T14:00:55.207749+0000","last_clean":"2024-03-26T14:00:55.207749+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T14:00:55.207749+0000","last_undegraded":"2024-03-26T14:00:55.207749+0000","last_fullsized":"2024-03-26T14:00:55.207749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209187+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T14:00:55.209187+0000","last_peered":"2024-03-26T14:00:55.209187+0000","last_clean":"2024-03-26T14:00:55.209187+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T14:00:55.209187+0000","last_undegraded":"2024-03-26T14:00:55.209187+0000","last_fullsized":"2024-03-26T14:00:55.209187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207907+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T14:00:55.207907+0000","last_peered":"2024-03-26T14:00:55.207907+0000","last_clean":"2024-03-26T14:00:55.207907+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T14:00:55.207907+0000","last_undegraded":"2024-03-26T14:00:55.207907+0000","last_fullsized":"2024-03-26T14:00:55.207907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211102+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T14:00:55.211102+0000","last_peered":"2024-03-26T14:00:55.211102+0000","last_clean":"2024-03-26T14:00:55.211102+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T14:00:55.211102+0000","last_undegraded":"2024-03-26T14:00:55.211102+0000","last_fullsized":"2024-03-26T14:00:55.211102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209455+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T14:00:55.209455+0000","last_peered":"2024-03-26T14:00:55.209455+0000","last_clean":"2024-03-26T14:00:55.209455+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T14:00:55.209455+0000","last_undegraded":"2024-03-26T14:00:55.209455+0000","last_fullsized":"2024-03-26T14:00:55.209455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213566+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T14:00:55.213566+0000","last_peered":"2024-03-26T14:00:55.213566+0000","last_clean":"2024-03-26T14:00:55.213566+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T14:00:55.213566+0000","last_undegraded":"2024-03-26T14:00:55.213566+0000","last_fullsized":"2024-03-26T14:00:55.213566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433842+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T14:00:55.433842+0000","last_peered":"2024-03-26T14:00:55.433842+0000","last_clean":"2024-03-26T14:00:55.433842+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T14:00:55.433842+0000","last_undegraded":"2024-03-26T14:00:55.433842+0000","last_fullsized":"2024-03-26T14:00:55.433842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209081+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T14:00:55.209081+0000","last_peered":"2024-03-26T14:00:55.209081+0000","last_clean":"2024-03-26T14:00:55.209081+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T14:00:55.209081+0000","last_undegraded":"2024-03-26T14:00:55.209081+0000","last_fullsized":"2024-03-26T14:00:55.209081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214791+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T14:00:55.214791+0000","last_peered":"2024-03-26T14:00:55.214791+0000","last_clean":"2024-03-26T14:00:55.214791+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T14:00:55.214791+0000","last_undegraded":"2024-03-26T14:00:55.214791+0000","last_fullsized":"2024-03-26T14:00:55.214791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228631+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T14:00:55.228631+0000","last_peered":"2024-03-26T14:00:55.228631+0000","last_clean":"2024-03-26T14:00:55.228631+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T14:00:55.228631+0000","last_undegraded":"2024-03-26T14:00:55.228631+0000","last_fullsized":"2024-03-26T14:00:55.228631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214716+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T14:00:55.214716+0000","last_peered":"2024-03-26T14:00:55.214716+0000","last_clean":"2024-03-26T14:00:55.214716+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T14:00:55.214716+0000","last_undegraded":"2024-03-26T14:00:55.214716+0000","last_fullsized":"2024-03-26T14:00:55.214716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":193,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:20.648087+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T14:02:20.648087+0000","last_peered":"2024-03-26T14:02:20.648087+0000","last_clean":"2024-03-26T14:02:20.648087+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T14:02:20.648087+0000","last_undegraded":"2024-03-26T14:02:20.648087+0000","last_fullsized":"2024-03-26T14:02:20.648087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214812+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T14:00:55.214812+0000","last_peered":"2024-03-26T14:00:55.214812+0000","last_clean":"2024-03-26T14:00:55.214812+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T14:00:55.214812+0000","last_undegraded":"2024-03-26T14:00:55.214812+0000","last_fullsized":"2024-03-26T14:00:55.214812+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216341+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T14:00:55.216341+0000","last_peered":"2024-03-26T14:00:55.216341+0000","last_clean":"2024-03-26T14:00:55.216341+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T14:00:55.216341+0000","last_undegraded":"2024-03-26T14:00:55.216341+0000","last_fullsized":"2024-03-26T14:00:55.216341+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217641+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T14:00:55.217641+0000","last_peered":"2024-03-26T14:00:55.217641+0000","last_clean":"2024-03-26T14:00:55.217641+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T14:00:55.217641+0000","last_undegraded":"2024-03-26T14:00:55.217641+0000","last_fullsized":"2024-03-26T14:00:55.217641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227975+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T14:00:55.227975+0000","last_peered":"2024-03-26T14:00:55.227975+0000","last_clean":"2024-03-26T14:00:55.227975+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T14:00:55.227975+0000","last_undegraded":"2024-03-26T14:00:55.227975+0000","last_fullsized":"2024-03-26T14:00:55.227975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222150+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T14:00:55.222150+0000","last_peered":"2024-03-26T14:00:55.222150+0000","last_clean":"2024-03-26T14:00:55.222150+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T14:00:55.222150+0000","last_undegraded":"2024-03-26T14:00:55.222150+0000","last_fullsized":"2024-03-26T14:00:55.222150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217537+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T14:00:55.217537+0000","last_peered":"2024-03-26T14:00:55.217537+0000","last_clean":"2024-03-26T14:00:55.217537+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T14:00:55.217537+0000","last_undegraded":"2024-03-26T14:00:55.217537+0000","last_fullsized":"2024-03-26T14:00:55.217537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210377+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T14:00:55.210377+0000","last_peered":"2024-03-26T14:00:55.210377+0000","last_clean":"2024-03-26T14:00:55.210377+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T14:00:55.210377+0000","last_undegraded":"2024-03-26T14:00:55.210377+0000","last_fullsized":"2024-03-26T14:00:55.210377+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212422+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T14:00:55.212422+0000","last_peered":"2024-03-26T14:00:55.212422+0000","last_clean":"2024-03-26T14:00:55.212422+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T14:00:55.212422+0000","last_undegraded":"2024-03-26T14:00:55.212422+0000","last_fullsized":"2024-03-26T14:00:55.212422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226654+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T14:00:55.226654+0000","last_peered":"2024-03-26T14:00:55.226654+0000","last_clean":"2024-03-26T14:00:55.226654+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T14:00:55.226654+0000","last_undegraded":"2024-03-26T14:00:55.226654+0000","last_fullsized":"2024-03-26T14:00:55.226654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211075+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T14:00:55.211075+0000","last_peered":"2024-03-26T14:00:55.211075+0000","last_clean":"2024-03-26T14:00:55.211075+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T14:00:55.211075+0000","last_undegraded":"2024-03-26T14:00:55.211075+0000","last_fullsized":"2024-03-26T14:00:55.211075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206766+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T14:00:55.206766+0000","last_peered":"2024-03-26T14:00:55.206766+0000","last_clean":"2024-03-26T14:00:55.206766+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T14:00:55.206766+0000","last_undegraded":"2024-03-26T14:00:55.206766+0000","last_fullsized":"2024-03-26T14:00:55.206766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:24.660131+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T14:02:24.660131+0000","last_peered":"2024-03-26T14:02:24.660131+0000","last_clean":"2024-03-26T14:02:24.660131+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T14:02:24.660131+0000","last_undegraded":"2024-03-26T14:02:24.660131+0000","last_fullsized":"2024-03-26T14:02:24.660131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434135+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T14:00:55.434135+0000","last_peered":"2024-03-26T14:00:55.434135+0000","last_clean":"2024-03-26T14:00:55.434135+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T14:00:55.434135+0000","last_undegraded":"2024-03-26T14:00:55.434135+0000","last_fullsized":"2024-03-26T14:00:55.434135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211849+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T14:00:55.211849+0000","last_peered":"2024-03-26T14:00:55.211849+0000","last_clean":"2024-03-26T14:00:55.211849+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T14:00:55.211849+0000","last_undegraded":"2024-03-26T14:00:55.211849+0000","last_fullsized":"2024-03-26T14:00:55.211849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214014+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T14:00:55.214014+0000","last_peered":"2024-03-26T14:00:55.214014+0000","last_clean":"2024-03-26T14:00:55.214014+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T14:00:55.214014+0000","last_undegraded":"2024-03-26T14:00:55.214014+0000","last_fullsized":"2024-03-26T14:00:55.214014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225122+0000","last_change":"2024-03-26T14:00:55.224154+0000","last_active":"2024-03-26T14:00:55.225122+0000","last_peered":"2024-03-26T14:00:55.225122+0000","last_clean":"2024-03-26T14:00:55.225122+0000","last_became_active":"2024-03-26T14:00:55.223674+0000","last_became_peered":"2024-03-26T14:00:55.223674+0000","last_unstale":"2024-03-26T14:00:55.225122+0000","last_undegraded":"2024-03-26T14:00:55.225122+0000","last_fullsized":"2024-03-26T14:00:55.225122+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:48:31.146317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209697+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T14:00:55.209697+0000","last_peered":"2024-03-26T14:00:55.209697+0000","last_clean":"2024-03-26T14:00:55.209697+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T14:00:55.209697+0000","last_undegraded":"2024-03-26T14:00:55.209697+0000","last_fullsized":"2024-03-26T14:00:55.209697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228174+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T14:00:55.228174+0000","last_peered":"2024-03-26T14:00:55.228174+0000","last_clean":"2024-03-26T14:00:55.228174+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T14:00:55.228174+0000","last_undegraded":"2024-03-26T14:00:55.228174+0000","last_fullsized":"2024-03-26T14:00:55.228174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433993+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T14:00:55.433993+0000","last_peered":"2024-03-26T14:00:55.433993+0000","last_clean":"2024-03-26T14:00:55.433993+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T14:00:55.433993+0000","last_undegraded":"2024-03-26T14:00:55.433993+0000","last_fullsized":"2024-03-26T14:00:55.433993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210803+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T14:00:55.210803+0000","last_peered":"2024-03-26T14:00:55.210803+0000","last_clean":"2024-03-26T14:00:55.210803+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T14:00:55.210803+0000","last_undegraded":"2024-03-26T14:00:55.210803+0000","last_fullsized":"2024-03-26T14:00:55.210803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":398,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:25.633621+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T14:02:25.633621+0000","last_peered":"2024-03-26T14:02:25.633621+0000","last_clean":"2024-03-26T14:02:25.633621+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T14:02:25.633621+0000","last_undegraded":"2024-03-26T14:02:25.633621+0000","last_fullsized":"2024-03-26T14:02:25.633621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73041,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":324,"num_read_kb":234,"num_write":43,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217378+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T14:00:55.217378+0000","last_peered":"2024-03-26T14:00:55.217378+0000","last_clean":"2024-03-26T14:00:55.217378+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T14:00:55.217378+0000","last_undegraded":"2024-03-26T14:00:55.217378+0000","last_fullsized":"2024-03-26T14:00:55.217378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.229375+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T14:00:55.229375+0000","last_peered":"2024-03-26T14:00:55.229375+0000","last_clean":"2024-03-26T14:00:55.229375+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T14:00:55.229375+0000","last_undegraded":"2024-03-26T14:00:55.229375+0000","last_fullsized":"2024-03-26T14:00:55.229375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.240886+0000","last_change":"2024-03-26T14:00:55.239852+0000","last_active":"2024-03-26T14:00:55.240886+0000","last_peered":"2024-03-26T14:00:55.240886+0000","last_clean":"2024-03-26T14:00:55.240886+0000","last_became_active":"2024-03-26T14:00:55.221294+0000","last_became_peered":"2024-03-26T14:00:55.221294+0000","last_unstale":"2024-03-26T14:00:55.240886+0000","last_undegraded":"2024-03-26T14:00:55.240886+0000","last_fullsized":"2024-03-26T14:00:55.240886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:14:53.910832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217300+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T14:00:55.217300+0000","last_peered":"2024-03-26T14:00:55.217300+0000","last_clean":"2024-03-26T14:00:55.217300+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T14:00:55.217300+0000","last_undegraded":"2024-03-26T14:00:55.217300+0000","last_fullsized":"2024-03-26T14:00:55.217300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220773+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T14:00:55.220773+0000","last_peered":"2024-03-26T14:00:55.220773+0000","last_clean":"2024-03-26T14:00:55.220773+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T14:00:55.220773+0000","last_undegraded":"2024-03-26T14:00:55.220773+0000","last_fullsized":"2024-03-26T14:00:55.220773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227998+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T14:00:55.227998+0000","last_peered":"2024-03-26T14:00:55.227998+0000","last_clean":"2024-03-26T14:00:55.227998+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T14:00:55.227998+0000","last_undegraded":"2024-03-26T14:00:55.227998+0000","last_fullsized":"2024-03-26T14:00:55.227998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214454+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T14:00:55.214454+0000","last_peered":"2024-03-26T14:00:55.214454+0000","last_clean":"2024-03-26T14:00:55.214454+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T14:00:55.214454+0000","last_undegraded":"2024-03-26T14:00:55.214454+0000","last_fullsized":"2024-03-26T14:00:55.214454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212803+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T14:00:55.212803+0000","last_peered":"2024-03-26T14:00:55.212803+0000","last_clean":"2024-03-26T14:00:55.212803+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T14:00:55.212803+0000","last_undegraded":"2024-03-26T14:00:55.212803+0000","last_fullsized":"2024-03-26T14:00:55.212803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:21.693198+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T14:02:21.693198+0000","last_peered":"2024-03-26T14:02:21.693198+0000","last_clean":"2024-03-26T14:02:21.693198+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T14:02:21.693198+0000","last_undegraded":"2024-03-26T14:02:21.693198+0000","last_fullsized":"2024-03-26T14:02:21.693198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":159,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213552+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T14:00:55.213552+0000","last_peered":"2024-03-26T14:00:55.213552+0000","last_clean":"2024-03-26T14:00:55.213552+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T14:00:55.213552+0000","last_undegraded":"2024-03-26T14:00:55.213552+0000","last_fullsized":"2024-03-26T14:00:55.213552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217932+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T14:00:55.217932+0000","last_peered":"2024-03-26T14:00:55.217932+0000","last_clean":"2024-03-26T14:00:55.217932+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T14:00:55.217932+0000","last_undegraded":"2024-03-26T14:00:55.217932+0000","last_fullsized":"2024-03-26T14:00:55.217932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212507+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T14:00:55.212507+0000","last_peered":"2024-03-26T14:00:55.212507+0000","last_clean":"2024-03-26T14:00:55.212507+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T14:00:55.212507+0000","last_undegraded":"2024-03-26T14:00:55.212507+0000","last_fullsized":"2024-03-26T14:00:55.212507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227066+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T14:00:55.227066+0000","last_peered":"2024-03-26T14:00:55.227066+0000","last_clean":"2024-03-26T14:00:55.227066+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T14:00:55.227066+0000","last_undegraded":"2024-03-26T14:00:55.227066+0000","last_fullsized":"2024-03-26T14:00:55.227066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226862+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T14:00:55.226862+0000","last_peered":"2024-03-26T14:00:55.226862+0000","last_clean":"2024-03-26T14:00:55.226862+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T14:00:55.226862+0000","last_undegraded":"2024-03-26T14:00:55.226862+0000","last_fullsized":"2024-03-26T14:00:55.226862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217173+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T14:00:55.217173+0000","last_peered":"2024-03-26T14:00:55.217173+0000","last_clean":"2024-03-26T14:00:55.217173+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T14:00:55.217173+0000","last_undegraded":"2024-03-26T14:00:55.217173+0000","last_fullsized":"2024-03-26T14:00:55.217173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221832+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T14:00:55.221832+0000","last_peered":"2024-03-26T14:00:55.221832+0000","last_clean":"2024-03-26T14:00:55.221832+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T14:00:55.221832+0000","last_undegraded":"2024-03-26T14:00:55.221832+0000","last_fullsized":"2024-03-26T14:00:55.221832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214516+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T14:00:55.214516+0000","last_peered":"2024-03-26T14:00:55.214516+0000","last_clean":"2024-03-26T14:00:55.214516+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T14:00:55.214516+0000","last_undegraded":"2024-03-26T14:00:55.214516+0000","last_fullsized":"2024-03-26T14:00:55.214516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225256+0000","last_change":"2024-03-26T14:00:55.224501+0000","last_active":"2024-03-26T14:00:55.225256+0000","last_peered":"2024-03-26T14:00:55.225256+0000","last_clean":"2024-03-26T14:00:55.225256+0000","last_became_active":"2024-03-26T14:00:55.223843+0000","last_became_peered":"2024-03-26T14:00:55.223843+0000","last_unstale":"2024-03-26T14:00:55.225256+0000","last_undegraded":"2024-03-26T14:00:55.225256+0000","last_fullsized":"2024-03-26T14:00:55.225256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:29:11.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217789+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T14:00:55.217789+0000","last_peered":"2024-03-26T14:00:55.217789+0000","last_clean":"2024-03-26T14:00:55.217789+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T14:00:55.217789+0000","last_undegraded":"2024-03-26T14:00:55.217789+0000","last_fullsized":"2024-03-26T14:00:55.217789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217139+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T14:00:55.217139+0000","last_peered":"2024-03-26T14:00:55.217139+0000","last_clean":"2024-03-26T14:00:55.217139+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T14:00:55.217139+0000","last_undegraded":"2024-03-26T14:00:55.217139+0000","last_fullsized":"2024-03-26T14:00:55.217139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214292+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T14:00:55.214292+0000","last_peered":"2024-03-26T14:00:55.214292+0000","last_clean":"2024-03-26T14:00:55.214292+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T14:00:55.214292+0000","last_undegraded":"2024-03-26T14:00:55.214292+0000","last_fullsized":"2024-03-26T14:00:55.214292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217644+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T14:00:55.217644+0000","last_peered":"2024-03-26T14:00:55.217644+0000","last_clean":"2024-03-26T14:00:55.217644+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T14:00:55.217644+0000","last_undegraded":"2024-03-26T14:00:55.217644+0000","last_fullsized":"2024-03-26T14:00:55.217644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217901+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T14:00:55.217901+0000","last_peered":"2024-03-26T14:00:55.217901+0000","last_clean":"2024-03-26T14:00:55.217901+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T14:00:55.217901+0000","last_undegraded":"2024-03-26T14:00:55.217901+0000","last_fullsized":"2024-03-26T14:00:55.217901+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227219+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T14:00:55.227219+0000","last_peered":"2024-03-26T14:00:55.227219+0000","last_clean":"2024-03-26T14:00:55.227219+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T14:00:55.227219+0000","last_undegraded":"2024-03-26T14:00:55.227219+0000","last_fullsized":"2024-03-26T14:00:55.227219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:25.703123+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T14:02:25.703123+0000","last_peered":"2024-03-26T14:02:25.703123+0000","last_clean":"2024-03-26T14:02:25.703123+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T14:02:25.703123+0000","last_undegraded":"2024-03-26T14:02:25.703123+0000","last_fullsized":"2024-03-26T14:02:25.703123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":157,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214061+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T14:00:55.214061+0000","last_peered":"2024-03-26T14:00:55.214061+0000","last_clean":"2024-03-26T14:00:55.214061+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T14:00:55.214061+0000","last_undegraded":"2024-03-26T14:00:55.214061+0000","last_fullsized":"2024-03-26T14:00:55.214061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207002+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T14:00:55.207002+0000","last_peered":"2024-03-26T14:00:55.207002+0000","last_clean":"2024-03-26T14:00:55.207002+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T14:00:55.207002+0000","last_undegraded":"2024-03-26T14:00:55.207002+0000","last_fullsized":"2024-03-26T14:00:55.207002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209617+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T14:00:55.209617+0000","last_peered":"2024-03-26T14:00:55.209617+0000","last_clean":"2024-03-26T14:00:55.209617+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T14:00:55.209617+0000","last_undegraded":"2024-03-26T14:00:55.209617+0000","last_fullsized":"2024-03-26T14:00:55.209617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213393+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T14:00:55.213393+0000","last_peered":"2024-03-26T14:00:55.213393+0000","last_clean":"2024-03-26T14:00:55.213393+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T14:00:55.213393+0000","last_undegraded":"2024-03-26T14:00:55.213393+0000","last_fullsized":"2024-03-26T14:00:55.213393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228488+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T14:00:55.228488+0000","last_peered":"2024-03-26T14:00:55.228488+0000","last_clean":"2024-03-26T14:00:55.228488+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T14:00:55.228488+0000","last_undegraded":"2024-03-26T14:00:55.228488+0000","last_fullsized":"2024-03-26T14:00:55.228488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216689+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T14:00:55.216689+0000","last_peered":"2024-03-26T14:00:55.216689+0000","last_clean":"2024-03-26T14:00:55.216689+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T14:00:55.216689+0000","last_undegraded":"2024-03-26T14:00:55.216689+0000","last_fullsized":"2024-03-26T14:00:55.216689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.439884+0000","last_change":"2024-03-26T14:00:55.439230+0000","last_active":"2024-03-26T14:00:55.439884+0000","last_peered":"2024-03-26T14:00:55.439884+0000","last_clean":"2024-03-26T14:00:55.439884+0000","last_became_active":"2024-03-26T14:00:55.434988+0000","last_became_peered":"2024-03-26T14:00:55.434988+0000","last_unstale":"2024-03-26T14:00:55.439884+0000","last_undegraded":"2024-03-26T14:00:55.439884+0000","last_fullsized":"2024-03-26T14:00:55.439884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:58.291575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.223104+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T14:00:55.223104+0000","last_peered":"2024-03-26T14:00:55.223104+0000","last_clean":"2024-03-26T14:00:55.223104+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T14:00:55.223104+0000","last_undegraded":"2024-03-26T14:00:55.223104+0000","last_fullsized":"2024-03-26T14:00:55.223104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217076+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T14:00:55.217076+0000","last_peered":"2024-03-26T14:00:55.217076+0000","last_clean":"2024-03-26T14:00:55.217076+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T14:00:55.217076+0000","last_undegraded":"2024-03-26T14:00:55.217076+0000","last_fullsized":"2024-03-26T14:00:55.217076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217337+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T14:00:55.217337+0000","last_peered":"2024-03-26T14:00:55.217337+0000","last_clean":"2024-03-26T14:00:55.217337+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T14:00:55.217337+0000","last_undegraded":"2024-03-26T14:00:55.217337+0000","last_fullsized":"2024-03-26T14:00:55.217337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206756+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T14:00:55.206756+0000","last_peered":"2024-03-26T14:00:55.206756+0000","last_clean":"2024-03-26T14:00:55.206756+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T14:00:55.206756+0000","last_undegraded":"2024-03-26T14:00:55.206756+0000","last_fullsized":"2024-03-26T14:00:55.206756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:20.345895+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T14:02:20.345895+0000","last_peered":"2024-03-26T14:02:20.345895+0000","last_clean":"2024-03-26T14:02:20.345895+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T14:02:20.345895+0000","last_undegraded":"2024-03-26T14:02:20.345895+0000","last_fullsized":"2024-03-26T14:02:20.345895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206944+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T14:00:55.206944+0000","last_peered":"2024-03-26T14:00:55.206944+0000","last_clean":"2024-03-26T14:00:55.206944+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T14:00:55.206944+0000","last_undegraded":"2024-03-26T14:00:55.206944+0000","last_fullsized":"2024-03-26T14:00:55.206944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211722+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T14:00:55.211722+0000","last_peered":"2024-03-26T14:00:55.211722+0000","last_clean":"2024-03-26T14:00:55.211722+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T14:00:55.211722+0000","last_undegraded":"2024-03-26T14:00:55.211722+0000","last_fullsized":"2024-03-26T14:00:55.211722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210428+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T14:00:55.210428+0000","last_peered":"2024-03-26T14:00:55.210428+0000","last_clean":"2024-03-26T14:00:55.210428+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T14:00:55.210428+0000","last_undegraded":"2024-03-26T14:00:55.210428+0000","last_fullsized":"2024-03-26T14:00:55.210428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212631+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T14:00:55.212631+0000","last_peered":"2024-03-26T14:00:55.212631+0000","last_clean":"2024-03-26T14:00:55.212631+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T14:00:55.212631+0000","last_undegraded":"2024-03-26T14:00:55.212631+0000","last_fullsized":"2024-03-26T14:00:55.212631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210150+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T14:00:55.210150+0000","last_peered":"2024-03-26T14:00:55.210150+0000","last_clean":"2024-03-26T14:00:55.210150+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T14:00:55.210150+0000","last_undegraded":"2024-03-26T14:00:55.210150+0000","last_fullsized":"2024-03-26T14:00:55.210150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433840+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T14:00:55.433840+0000","last_peered":"2024-03-26T14:00:55.433840+0000","last_clean":"2024-03-26T14:00:55.433840+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T14:00:55.433840+0000","last_undegraded":"2024-03-26T14:00:55.433840+0000","last_fullsized":"2024-03-26T14:00:55.433840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213281+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T14:00:55.213281+0000","last_peered":"2024-03-26T14:00:55.213281+0000","last_clean":"2024-03-26T14:00:55.213281+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T14:00:55.213281+0000","last_undegraded":"2024-03-26T14:00:55.213281+0000","last_fullsized":"2024-03-26T14:00:55.213281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216737+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T14:00:55.216737+0000","last_peered":"2024-03-26T14:00:55.216737+0000","last_clean":"2024-03-26T14:00:55.216737+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T14:00:55.216737+0000","last_undegraded":"2024-03-26T14:00:55.216737+0000","last_fullsized":"2024-03-26T14:00:55.216737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206794+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T14:00:55.206794+0000","last_peered":"2024-03-26T14:00:55.206794+0000","last_clean":"2024-03-26T14:00:55.206794+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T14:00:55.206794+0000","last_undegraded":"2024-03-26T14:00:55.206794+0000","last_fullsized":"2024-03-26T14:00:55.206794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215526+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T14:00:55.215526+0000","last_peered":"2024-03-26T14:00:55.215526+0000","last_clean":"2024-03-26T14:00:55.215526+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T14:00:55.215526+0000","last_undegraded":"2024-03-26T14:00:55.215526+0000","last_fullsized":"2024-03-26T14:00:55.215526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209311+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T14:00:55.209311+0000","last_peered":"2024-03-26T14:00:55.209311+0000","last_clean":"2024-03-26T14:00:55.209311+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T14:00:55.209311+0000","last_undegraded":"2024-03-26T14:00:55.209311+0000","last_fullsized":"2024-03-26T14:00:55.209311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:34.135298+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T14:01:34.135298+0000","last_peered":"2024-03-26T14:01:34.135298+0000","last_clean":"2024-03-26T14:01:34.135298+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T14:01:34.135298+0000","last_undegraded":"2024-03-26T14:01:34.135298+0000","last_fullsized":"2024-03-26T14:01:34.135298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207377+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T14:00:55.207377+0000","last_peered":"2024-03-26T14:00:55.207377+0000","last_clean":"2024-03-26T14:00:55.207377+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T14:00:55.207377+0000","last_undegraded":"2024-03-26T14:00:55.207377+0000","last_fullsized":"2024-03-26T14:00:55.207377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209832+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T14:00:55.209832+0000","last_peered":"2024-03-26T14:00:55.209832+0000","last_clean":"2024-03-26T14:00:55.209832+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T14:00:55.209832+0000","last_undegraded":"2024-03-26T14:00:55.209832+0000","last_fullsized":"2024-03-26T14:00:55.209832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213280+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T14:00:55.213280+0000","last_peered":"2024-03-26T14:00:55.213280+0000","last_clean":"2024-03-26T14:00:55.213280+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T14:00:55.213280+0000","last_undegraded":"2024-03-26T14:00:55.213280+0000","last_fullsized":"2024-03-26T14:00:55.213280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214151+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T14:00:55.214151+0000","last_peered":"2024-03-26T14:00:55.214151+0000","last_clean":"2024-03-26T14:00:55.214151+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T14:00:55.214151+0000","last_undegraded":"2024-03-26T14:00:55.214151+0000","last_fullsized":"2024-03-26T14:00:55.214151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.432636+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T14:00:55.432636+0000","last_peered":"2024-03-26T14:00:55.432636+0000","last_clean":"2024-03-26T14:00:55.432636+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T14:00:55.432636+0000","last_undegraded":"2024-03-26T14:00:55.432636+0000","last_fullsized":"2024-03-26T14:00:55.432636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212890+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T14:00:55.212890+0000","last_peered":"2024-03-26T14:00:55.212890+0000","last_clean":"2024-03-26T14:00:55.212890+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T14:00:55.212890+0000","last_undegraded":"2024-03-26T14:00:55.212890+0000","last_fullsized":"2024-03-26T14:00:55.212890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214297+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T14:00:55.214297+0000","last_peered":"2024-03-26T14:00:55.214297+0000","last_clean":"2024-03-26T14:00:55.214297+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T14:00:55.214297+0000","last_undegraded":"2024-03-26T14:00:55.214297+0000","last_fullsized":"2024-03-26T14:00:55.214297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210537+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T14:00:55.210537+0000","last_peered":"2024-03-26T14:00:55.210537+0000","last_clean":"2024-03-26T14:00:55.210537+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T14:00:55.210537+0000","last_undegraded":"2024-03-26T14:00:55.210537+0000","last_fullsized":"2024-03-26T14:00:55.210537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228901+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T14:00:55.228901+0000","last_peered":"2024-03-26T14:00:55.228901+0000","last_clean":"2024-03-26T14:00:55.228901+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T14:00:55.228901+0000","last_undegraded":"2024-03-26T14:00:55.228901+0000","last_fullsized":"2024-03-26T14:00:55.228901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214653+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T14:00:55.214653+0000","last_peered":"2024-03-26T14:00:55.214653+0000","last_clean":"2024-03-26T14:00:55.214653+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T14:00:55.214653+0000","last_undegraded":"2024-03-26T14:00:55.214653+0000","last_fullsized":"2024-03-26T14:00:55.214653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225824+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T14:00:55.225824+0000","last_peered":"2024-03-26T14:00:55.225824+0000","last_clean":"2024-03-26T14:00:55.225824+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T14:00:55.225824+0000","last_undegraded":"2024-03-26T14:00:55.225824+0000","last_fullsized":"2024-03-26T14:00:55.225824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217852+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T14:00:55.217852+0000","last_peered":"2024-03-26T14:00:55.217852+0000","last_clean":"2024-03-26T14:00:55.217852+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T14:00:55.217852+0000","last_undegraded":"2024-03-26T14:00:55.217852+0000","last_fullsized":"2024-03-26T14:00:55.217852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214735+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T14:00:55.214735+0000","last_peered":"2024-03-26T14:00:55.214735+0000","last_clean":"2024-03-26T14:00:55.214735+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T14:00:55.214735+0000","last_undegraded":"2024-03-26T14:00:55.214735+0000","last_fullsized":"2024-03-26T14:00:55.214735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215872+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T14:00:55.215872+0000","last_peered":"2024-03-26T14:00:55.215872+0000","last_clean":"2024-03-26T14:00:55.215872+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T14:00:55.215872+0000","last_undegraded":"2024-03-26T14:00:55.215872+0000","last_fullsized":"2024-03-26T14:00:55.215872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212414+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T14:00:55.212414+0000","last_peered":"2024-03-26T14:00:55.212414+0000","last_clean":"2024-03-26T14:00:55.212414+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T14:00:55.212414+0000","last_undegraded":"2024-03-26T14:00:55.212414+0000","last_fullsized":"2024-03-26T14:00:55.212414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213601+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T14:00:55.213601+0000","last_peered":"2024-03-26T14:00:55.213601+0000","last_clean":"2024-03-26T14:00:55.213601+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T14:00:55.213601+0000","last_undegraded":"2024-03-26T14:00:55.213601+0000","last_fullsized":"2024-03-26T14:00:55.213601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213914+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T14:00:55.213914+0000","last_peered":"2024-03-26T14:00:55.213914+0000","last_clean":"2024-03-26T14:00:55.213914+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T14:00:55.213914+0000","last_undegraded":"2024-03-26T14:00:55.213914+0000","last_fullsized":"2024-03-26T14:00:55.213914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217436+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T14:00:55.217436+0000","last_peered":"2024-03-26T14:00:55.217436+0000","last_clean":"2024-03-26T14:00:55.217436+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T14:00:55.217436+0000","last_undegraded":"2024-03-26T14:00:55.217436+0000","last_fullsized":"2024-03-26T14:00:55.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221547+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T14:00:55.221547+0000","last_peered":"2024-03-26T14:00:55.221547+0000","last_clean":"2024-03-26T14:00:55.221547+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T14:00:55.221547+0000","last_undegraded":"2024-03-26T14:00:55.221547+0000","last_fullsized":"2024-03-26T14:00:55.221547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218503+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T14:00:55.218503+0000","last_peered":"2024-03-26T14:00:55.218503+0000","last_clean":"2024-03-26T14:00:55.218503+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T14:00:55.218503+0000","last_undegraded":"2024-03-26T14:00:55.218503+0000","last_fullsized":"2024-03-26T14:00:55.218503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207037+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T14:00:55.207037+0000","last_peered":"2024-03-26T14:00:55.207037+0000","last_clean":"2024-03-26T14:00:55.207037+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T14:00:55.207037+0000","last_undegraded":"2024-03-26T14:00:55.207037+0000","last_fullsized":"2024-03-26T14:00:55.207037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226954+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T14:00:55.226954+0000","last_peered":"2024-03-26T14:00:55.226954+0000","last_clean":"2024-03-26T14:00:55.226954+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T14:00:55.226954+0000","last_undegraded":"2024-03-26T14:00:55.226954+0000","last_fullsized":"2024-03-26T14:00:55.226954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206887+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T14:00:55.206887+0000","last_peered":"2024-03-26T14:00:55.206887+0000","last_clean":"2024-03-26T14:00:55.206887+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T14:00:55.206887+0000","last_undegraded":"2024-03-26T14:00:55.206887+0000","last_fullsized":"2024-03-26T14:00:55.206887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433797+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T14:00:55.433797+0000","last_peered":"2024-03-26T14:00:55.433797+0000","last_clean":"2024-03-26T14:00:55.433797+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T14:00:55.433797+0000","last_undegraded":"2024-03-26T14:00:55.433797+0000","last_fullsized":"2024-03-26T14:00:55.433797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217146+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T14:00:55.217146+0000","last_peered":"2024-03-26T14:00:55.217146+0000","last_clean":"2024-03-26T14:00:55.217146+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T14:00:55.217146+0000","last_undegraded":"2024-03-26T14:00:55.217146+0000","last_fullsized":"2024-03-26T14:00:55.217146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221123+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T14:00:55.221123+0000","last_peered":"2024-03-26T14:00:55.221123+0000","last_clean":"2024-03-26T14:00:55.221123+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T14:00:55.221123+0000","last_undegraded":"2024-03-26T14:00:55.221123+0000","last_fullsized":"2024-03-26T14:00:55.221123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78169,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":964,"num_read_kb":569,"num_write":184,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":237809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":522925,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1188321,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776195,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":523127,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037834,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":523047,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299473,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1120474,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462999,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518303,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":591254,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519691,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":593454,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518497,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523978,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1114646,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:02:28.685 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-03-26T14:02:28.686 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-26T14:02:28.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:28 smithi159 ceph-mon[35501]: pgmap v461: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-03-26T14:02:28.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:28 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/927882287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:28.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 -- 172.21.15.137:0/2365505674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc0772a0 msgr2=0x7f70cc077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 --2- 172.21.15.137:0/2365505674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc0772a0 0x7f70cc077700 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009900 tx=0x7f70b802bc40 comp rx=0 tx=0).stop 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 -- 172.21.15.137:0/2365505674 shutdown_connections 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 --2- 172.21.15.137:0/2365505674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc077c40 0x7f70cc0784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 --2- 172.21.15.137:0/2365505674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc0772a0 0x7f70cc077700 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 --2- 172.21.15.137:0/2365505674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70cc076060 0x7f70cc076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 -- 172.21.15.137:0/2365505674 >> 172.21.15.137:0/2365505674 conn(0x7f70cc0fee90 msgr2=0x7f70cc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 -- 172.21.15.137:0/2365505674 shutdown_connections 2024-03-26T14:02:28.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.940+0000 7f70d3600640 1 -- 172.21.15.137:0/2365505674 wait complete. 2024-03-26T14:02:28.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 Processor -- start 2024-03-26T14:02:28.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 -- start start 2024-03-26T14:02:28.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 0x7f70cc19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc0772a0 0x7f70cc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70cc077c40 0x7f70cc1a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc113860 con 0x7f70cc076060 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1136e0 con 0x7f70cc0772a0 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.941+0000 7f70d3600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc1139e0 con 0x7f70cc077c40 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 0x7f70cc19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 0x7f70cc19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60790/0 (socket says 172.21.15.137:60790) 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 -- 172.21.15.137:0/2960251936 learned_addr learned my addr 172.21.15.137:0/2960251936 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70c3e00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc0772a0 0x7f70cc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 -- 172.21.15.137:0/2960251936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70cc077c40 msgr2=0x7f70cc1a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d1600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70cc077c40 0x7f70cc1a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70cc077c40 0x7f70cc1a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 -- 172.21.15.137:0/2960251936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc0772a0 msgr2=0x7f70cc19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc0772a0 0x7f70cc19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 -- 172.21.15.137:0/2960251936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc1a5b20 con 0x7f70cc076060 2024-03-26T14:02:28.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70c3e00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc0772a0 0x7f70cc19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.942+0000 7f70d0c00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 0x7f70cc19df00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00d950 tx=0x7f70bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.943+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc017070 con 0x7f70cc076060 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.943+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70bc009c30 con 0x7f70cc076060 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.943+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00f620 con 0x7f70cc076060 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.943+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70cc1a5e10 con 0x7f70cc076060 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.943+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc1a6270 con 0x7f70cc076060 2024-03-26T14:02:28.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.944+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc110480 con 0x7f70cc076060 2024-03-26T14:02:28.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.946+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f70bc0109a0 con 0x7f70cc076060 2024-03-26T14:02:28.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.947+0000 7f70ca000640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70b00a2890 0x7f70b00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:28.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.947+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f70bc0ced00 con 0x7f70cc076060 2024-03-26T14:02:28.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.947+0000 7f70c3e00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70b00a2890 0x7f70b00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:28.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.948+0000 7f70c3e00640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70b00a2890 0x7f70b00a4d50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006010 tx=0x7f70b802c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:28.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:28.952+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f70bc009de0 con 0x7f70cc076060 2024-03-26T14:02:29.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.148+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 --> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70cc061ae0 con 0x7f70b00a2890 2024-03-26T14:02:29.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.177+0000 7f70ca000640 1 -- 172.21.15.137:0/2960251936 <== mgr.14152 v2:172.21.15.137:6800/3167395503 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340825 (secure 0 0 0) 0x7f70cc061ae0 con 0x7f70b00a2890 2024-03-26T14:02:29.179 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:29.181 INFO:teuthology.orchestra.run.smithi137.stderr:dumped all 2024-03-26T14:02:29.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.183+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70b00a2890 msgr2=0x7f70b00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.183+0000 7f70d3600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70b00a2890 0x7f70b00a4d50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006010 tx=0x7f70b802c7f0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 msgr2=0x7f70cc19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 0x7f70cc19df00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00d950 tx=0x7f70bc00de10 comp rx=0 tx=0).stop 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 shutdown_connections 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70b00a2890 0x7f70b00a4d50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70cc0772a0 0x7f70cc19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70cc077c40 0x7f70cc1a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 --2- 172.21.15.137:0/2960251936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f70cc076060 0x7f70cc19df00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 >> 172.21.15.137:0/2960251936 conn(0x7f70cc0fee90 msgr2=0x7f70cc100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 shutdown_connections 2024-03-26T14:02:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.184+0000 7f70d3600640 1 -- 172.21.15.137:0/2960251936 wait complete. 2024-03-26T14:02:29.202 INFO:teuthology.orchestra.run.smithi137.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-03-26T14:02:27.155287+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668771,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":1090,"num_read_kb":678,"num_write":367,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180887,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":798,"ondisk_log_size":798,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361756,"kb_used_data":12572,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580068,"statfs":{"total":1151940427776,"available":1151569989632,"internally_reserved":0,"allocated":12873728,"data_stored":8257717,"data_compressed":427101,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6184,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":54,"num_write":14,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005672"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222603+0000","last_change":"2024-03-26T13:59:23.321621+0000","last_active":"2024-03-26T14:00:55.222603+0000","last_peered":"2024-03-26T14:00:55.222603+0000","last_clean":"2024-03-26T14:00:55.222603+0000","last_became_active":"2024-03-26T13:59:23.321247+0000","last_became_peered":"2024-03-26T13:59:23.321247+0000","last_unstale":"2024-03-26T14:00:55.222603+0000","last_undegraded":"2024-03-26T14:00:55.222603+0000","last_fullsized":"2024-03-26T14:00:55.222603+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:48.113402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212530+0000","last_change":"2024-03-26T13:59:23.303832+0000","last_active":"2024-03-26T14:00:55.212530+0000","last_peered":"2024-03-26T14:00:55.212530+0000","last_clean":"2024-03-26T14:00:55.212530+0000","last_became_active":"2024-03-26T13:59:23.303451+0000","last_became_peered":"2024-03-26T13:59:23.303451+0000","last_unstale":"2024-03-26T14:00:55.212530+0000","last_undegraded":"2024-03-26T14:00:55.212530+0000","last_fullsized":"2024-03-26T14:00:55.212530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:09:41.253717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211035+0000","last_change":"2024-03-26T13:59:23.309391+0000","last_active":"2024-03-26T14:00:55.211035+0000","last_peered":"2024-03-26T14:00:55.211035+0000","last_clean":"2024-03-26T14:00:55.211035+0000","last_became_active":"2024-03-26T13:59:23.309057+0000","last_became_peered":"2024-03-26T13:59:23.309057+0000","last_unstale":"2024-03-26T14:00:55.211035+0000","last_undegraded":"2024-03-26T14:00:55.211035+0000","last_fullsized":"2024-03-26T14:00:55.211035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:50:40.210398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210071+0000","last_change":"2024-03-26T13:59:23.311385+0000","last_active":"2024-03-26T14:00:55.210071+0000","last_peered":"2024-03-26T14:00:55.210071+0000","last_clean":"2024-03-26T14:00:55.210071+0000","last_became_active":"2024-03-26T13:59:23.310607+0000","last_became_peered":"2024-03-26T13:59:23.310607+0000","last_unstale":"2024-03-26T14:00:55.210071+0000","last_undegraded":"2024-03-26T14:00:55.210071+0000","last_fullsized":"2024-03-26T14:00:55.210071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:13:12.215549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.224530+0000","last_change":"2024-03-26T14:00:55.221738+0000","last_active":"2024-03-26T14:00:55.224530+0000","last_peered":"2024-03-26T14:00:55.224530+0000","last_clean":"2024-03-26T14:00:55.224530+0000","last_became_active":"2024-03-26T14:00:55.220626+0000","last_became_peered":"2024-03-26T14:00:55.220626+0000","last_unstale":"2024-03-26T14:00:55.224530+0000","last_undegraded":"2024-03-26T14:00:55.224530+0000","last_fullsized":"2024-03-26T14:00:55.224530+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:47:50.208896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217727+0000","last_change":"2024-03-26T13:59:23.299606+0000","last_active":"2024-03-26T14:00:55.217727+0000","last_peered":"2024-03-26T14:00:55.217727+0000","last_clean":"2024-03-26T14:00:55.217727+0000","last_became_active":"2024-03-26T13:59:23.294395+0000","last_became_peered":"2024-03-26T13:59:23.294395+0000","last_unstale":"2024-03-26T14:00:55.217727+0000","last_undegraded":"2024-03-26T14:00:55.217727+0000","last_fullsized":"2024-03-26T14:00:55.217727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:51:37.512768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213817+0000","last_change":"2024-03-26T13:59:23.299398+0000","last_active":"2024-03-26T14:00:55.213817+0000","last_peered":"2024-03-26T14:00:55.213817+0000","last_clean":"2024-03-26T14:00:55.213817+0000","last_became_active":"2024-03-26T13:59:23.299032+0000","last_became_peered":"2024-03-26T13:59:23.299032+0000","last_unstale":"2024-03-26T14:00:55.213817+0000","last_undegraded":"2024-03-26T14:00:55.213817+0000","last_fullsized":"2024-03-26T14:00:55.213817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:39:57.003639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206757+0000","last_change":"2024-03-26T13:59:23.297387+0000","last_active":"2024-03-26T14:00:55.206757+0000","last_peered":"2024-03-26T14:00:55.206757+0000","last_clean":"2024-03-26T14:00:55.206757+0000","last_became_active":"2024-03-26T13:59:23.297084+0000","last_became_peered":"2024-03-26T13:59:23.297084+0000","last_unstale":"2024-03-26T14:00:55.206757+0000","last_undegraded":"2024-03-26T14:00:55.206757+0000","last_fullsized":"2024-03-26T14:00:55.206757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:22:29.609024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434521+0000","last_change":"2024-03-26T13:59:23.315181+0000","last_active":"2024-03-26T14:00:55.434521+0000","last_peered":"2024-03-26T14:00:55.434521+0000","last_clean":"2024-03-26T14:00:55.434521+0000","last_became_active":"2024-03-26T13:59:23.313620+0000","last_became_peered":"2024-03-26T13:59:23.313620+0000","last_unstale":"2024-03-26T14:00:55.434521+0000","last_undegraded":"2024-03-26T14:00:55.434521+0000","last_fullsized":"2024-03-26T14:00:55.434521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:56:57.822807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218359+0000","last_change":"2024-03-26T13:59:23.307997+0000","last_active":"2024-03-26T14:00:55.218359+0000","last_peered":"2024-03-26T14:00:55.218359+0000","last_clean":"2024-03-26T14:00:55.218359+0000","last_became_active":"2024-03-26T13:59:23.307660+0000","last_became_peered":"2024-03-26T13:59:23.307660+0000","last_unstale":"2024-03-26T14:00:55.218359+0000","last_undegraded":"2024-03-26T14:00:55.218359+0000","last_fullsized":"2024-03-26T14:00:55.218359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:46:28.293664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214217+0000","last_change":"2024-03-26T13:59:23.314340+0000","last_active":"2024-03-26T14:00:55.214217+0000","last_peered":"2024-03-26T14:00:55.214217+0000","last_clean":"2024-03-26T14:00:55.214217+0000","last_became_active":"2024-03-26T13:59:23.313243+0000","last_became_peered":"2024-03-26T13:59:23.313243+0000","last_unstale":"2024-03-26T14:00:55.214217+0000","last_undegraded":"2024-03-26T14:00:55.214217+0000","last_fullsized":"2024-03-26T14:00:55.214217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:23:52.600605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209834+0000","last_change":"2024-03-26T13:59:23.311693+0000","last_active":"2024-03-26T14:00:55.209834+0000","last_peered":"2024-03-26T14:00:55.209834+0000","last_clean":"2024-03-26T14:00:55.209834+0000","last_became_active":"2024-03-26T13:59:23.311080+0000","last_became_peered":"2024-03-26T13:59:23.311080+0000","last_unstale":"2024-03-26T14:00:55.209834+0000","last_undegraded":"2024-03-26T14:00:55.209834+0000","last_fullsized":"2024-03-26T14:00:55.209834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:57:46.718061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217235+0000","last_change":"2024-03-26T13:59:23.308801+0000","last_active":"2024-03-26T14:00:55.217235+0000","last_peered":"2024-03-26T14:00:55.217235+0000","last_clean":"2024-03-26T14:00:55.217235+0000","last_became_active":"2024-03-26T13:59:23.303704+0000","last_became_peered":"2024-03-26T13:59:23.303704+0000","last_unstale":"2024-03-26T14:00:55.217235+0000","last_undegraded":"2024-03-26T14:00:55.217235+0000","last_fullsized":"2024-03-26T14:00:55.217235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:10:51.525978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210375+0000","last_change":"2024-03-26T13:59:23.308365+0000","last_active":"2024-03-26T14:00:55.210375+0000","last_peered":"2024-03-26T14:00:55.210375+0000","last_clean":"2024-03-26T14:00:55.210375+0000","last_became_active":"2024-03-26T13:59:23.308017+0000","last_became_peered":"2024-03-26T13:59:23.308017+0000","last_unstale":"2024-03-26T14:00:55.210375+0000","last_undegraded":"2024-03-26T14:00:55.210375+0000","last_fullsized":"2024-03-26T14:00:55.210375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:18:05.434036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214095+0000","last_change":"2024-03-26T13:59:54.344831+0000","last_active":"2024-03-26T14:00:55.214095+0000","last_peered":"2024-03-26T14:00:55.214095+0000","last_clean":"2024-03-26T14:00:55.214095+0000","last_became_active":"2024-03-26T13:59:54.344257+0000","last_became_peered":"2024-03-26T13:59:54.344257+0000","last_unstale":"2024-03-26T14:00:55.214095+0000","last_undegraded":"2024-03-26T14:00:55.214095+0000","last_fullsized":"2024-03-26T14:00:55.214095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:16:20.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433666+0000","last_change":"2024-03-26T13:59:23.310672+0000","last_active":"2024-03-26T14:00:55.433666+0000","last_peered":"2024-03-26T14:00:55.433666+0000","last_clean":"2024-03-26T14:00:55.433666+0000","last_became_active":"2024-03-26T13:59:23.310318+0000","last_became_peered":"2024-03-26T13:59:23.310318+0000","last_unstale":"2024-03-26T14:00:55.433666+0000","last_undegraded":"2024-03-26T14:00:55.433666+0000","last_fullsized":"2024-03-26T14:00:55.433666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:27:25.708739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222404+0000","last_change":"2024-03-26T13:59:23.321686+0000","last_active":"2024-03-26T14:00:55.222404+0000","last_peered":"2024-03-26T14:00:55.222404+0000","last_clean":"2024-03-26T14:00:55.222404+0000","last_became_active":"2024-03-26T13:59:23.321254+0000","last_became_peered":"2024-03-26T13:59:23.321254+0000","last_unstale":"2024-03-26T14:00:55.222404+0000","last_undegraded":"2024-03-26T14:00:55.222404+0000","last_fullsized":"2024-03-26T14:00:55.222404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:02.378281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212182+0000","last_change":"2024-03-26T13:59:23.301627+0000","last_active":"2024-03-26T14:00:55.212182+0000","last_peered":"2024-03-26T14:00:55.212182+0000","last_clean":"2024-03-26T14:00:55.212182+0000","last_became_active":"2024-03-26T13:59:23.301256+0000","last_became_peered":"2024-03-26T13:59:23.301256+0000","last_unstale":"2024-03-26T14:00:55.212182+0000","last_undegraded":"2024-03-26T14:00:55.212182+0000","last_fullsized":"2024-03-26T14:00:55.212182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:10:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215913+0000","last_change":"2024-03-26T13:59:23.295220+0000","last_active":"2024-03-26T14:00:55.215913+0000","last_peered":"2024-03-26T14:00:55.215913+0000","last_clean":"2024-03-26T14:00:55.215913+0000","last_became_active":"2024-03-26T13:59:23.294956+0000","last_became_peered":"2024-03-26T13:59:23.294956+0000","last_unstale":"2024-03-26T14:00:55.215913+0000","last_undegraded":"2024-03-26T14:00:55.215913+0000","last_fullsized":"2024-03-26T14:00:55.215913+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:40:27.070141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212701+0000","last_change":"2024-03-26T13:59:27.318363+0000","last_active":"2024-03-26T14:00:55.212701+0000","last_peered":"2024-03-26T14:00:55.212701+0000","last_clean":"2024-03-26T14:00:55.212701+0000","last_became_active":"2024-03-26T13:59:27.313617+0000","last_became_peered":"2024-03-26T13:59:27.313617+0000","last_unstale":"2024-03-26T14:00:55.212701+0000","last_undegraded":"2024-03-26T14:00:55.212701+0000","last_fullsized":"2024-03-26T14:00:55.212701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:17:03.135238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212980+0000","last_change":"2024-03-26T13:59:22.282650+0000","last_active":"2024-03-26T14:00:55.212980+0000","last_peered":"2024-03-26T14:00:55.212980+0000","last_clean":"2024-03-26T14:00:55.212980+0000","last_became_active":"2024-03-26T13:59:22.282302+0000","last_became_peered":"2024-03-26T13:59:22.282302+0000","last_unstale":"2024-03-26T14:00:55.212980+0000","last_undegraded":"2024-03-26T14:00:55.212980+0000","last_fullsized":"2024-03-26T14:00:55.212980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:55:22.762827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207749+0000","last_change":"2024-03-26T13:59:23.307274+0000","last_active":"2024-03-26T14:00:55.207749+0000","last_peered":"2024-03-26T14:00:55.207749+0000","last_clean":"2024-03-26T14:00:55.207749+0000","last_became_active":"2024-03-26T13:59:23.306940+0000","last_became_peered":"2024-03-26T13:59:23.306940+0000","last_unstale":"2024-03-26T14:00:55.207749+0000","last_undegraded":"2024-03-26T14:00:55.207749+0000","last_fullsized":"2024-03-26T14:00:55.207749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:55:33.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209187+0000","last_change":"2024-03-26T13:59:27.319896+0000","last_active":"2024-03-26T14:00:55.209187+0000","last_peered":"2024-03-26T14:00:55.209187+0000","last_clean":"2024-03-26T14:00:55.209187+0000","last_became_active":"2024-03-26T13:59:27.319403+0000","last_became_peered":"2024-03-26T13:59:27.319403+0000","last_unstale":"2024-03-26T14:00:55.209187+0000","last_undegraded":"2024-03-26T14:00:55.209187+0000","last_fullsized":"2024-03-26T14:00:55.209187+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:57:24.796564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207907+0000","last_change":"2024-03-26T13:59:22.275638+0000","last_active":"2024-03-26T14:00:55.207907+0000","last_peered":"2024-03-26T14:00:55.207907+0000","last_clean":"2024-03-26T14:00:55.207907+0000","last_became_active":"2024-03-26T13:59:22.275306+0000","last_became_peered":"2024-03-26T13:59:22.275306+0000","last_unstale":"2024-03-26T14:00:55.207907+0000","last_undegraded":"2024-03-26T14:00:55.207907+0000","last_fullsized":"2024-03-26T14:00:55.207907+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:34:21.412251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211102+0000","last_change":"2024-03-26T13:59:23.305903+0000","last_active":"2024-03-26T14:00:55.211102+0000","last_peered":"2024-03-26T14:00:55.211102+0000","last_clean":"2024-03-26T14:00:55.211102+0000","last_became_active":"2024-03-26T13:59:23.305608+0000","last_became_peered":"2024-03-26T13:59:23.305608+0000","last_unstale":"2024-03-26T14:00:55.211102+0000","last_undegraded":"2024-03-26T14:00:55.211102+0000","last_fullsized":"2024-03-26T14:00:55.211102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:20:19.188058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209455+0000","last_change":"2024-03-26T13:59:27.326485+0000","last_active":"2024-03-26T14:00:55.209455+0000","last_peered":"2024-03-26T14:00:55.209455+0000","last_clean":"2024-03-26T14:00:55.209455+0000","last_became_active":"2024-03-26T13:59:27.324381+0000","last_became_peered":"2024-03-26T13:59:27.324381+0000","last_unstale":"2024-03-26T14:00:55.209455+0000","last_undegraded":"2024-03-26T14:00:55.209455+0000","last_fullsized":"2024-03-26T14:00:55.209455+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:30:38.560099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213566+0000","last_change":"2024-03-26T13:59:22.291016+0000","last_active":"2024-03-26T14:00:55.213566+0000","last_peered":"2024-03-26T14:00:55.213566+0000","last_clean":"2024-03-26T14:00:55.213566+0000","last_became_active":"2024-03-26T13:59:22.290532+0000","last_became_peered":"2024-03-26T13:59:22.290532+0000","last_unstale":"2024-03-26T14:00:55.213566+0000","last_undegraded":"2024-03-26T14:00:55.213566+0000","last_fullsized":"2024-03-26T14:00:55.213566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:25:10.810934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433842+0000","last_change":"2024-03-26T13:59:23.300060+0000","last_active":"2024-03-26T14:00:55.433842+0000","last_peered":"2024-03-26T14:00:55.433842+0000","last_clean":"2024-03-26T14:00:55.433842+0000","last_became_active":"2024-03-26T13:59:23.299681+0000","last_became_peered":"2024-03-26T13:59:23.299681+0000","last_unstale":"2024-03-26T14:00:55.433842+0000","last_undegraded":"2024-03-26T14:00:55.433842+0000","last_fullsized":"2024-03-26T14:00:55.433842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:28:38.633072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209081+0000","last_change":"2024-03-26T13:59:27.331308+0000","last_active":"2024-03-26T14:00:55.209081+0000","last_peered":"2024-03-26T14:00:55.209081+0000","last_clean":"2024-03-26T14:00:55.209081+0000","last_became_active":"2024-03-26T13:59:27.330477+0000","last_became_peered":"2024-03-26T13:59:27.330477+0000","last_unstale":"2024-03-26T14:00:55.209081+0000","last_undegraded":"2024-03-26T14:00:55.209081+0000","last_fullsized":"2024-03-26T14:00:55.209081+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:48.541490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214791+0000","last_change":"2024-03-26T13:59:22.287211+0000","last_active":"2024-03-26T14:00:55.214791+0000","last_peered":"2024-03-26T14:00:55.214791+0000","last_clean":"2024-03-26T14:00:55.214791+0000","last_became_active":"2024-03-26T13:59:22.283785+0000","last_became_peered":"2024-03-26T13:59:22.283785+0000","last_unstale":"2024-03-26T14:00:55.214791+0000","last_undegraded":"2024-03-26T14:00:55.214791+0000","last_fullsized":"2024-03-26T14:00:55.214791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:30:21.982334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228631+0000","last_change":"2024-03-26T13:59:23.317519+0000","last_active":"2024-03-26T14:00:55.228631+0000","last_peered":"2024-03-26T14:00:55.228631+0000","last_clean":"2024-03-26T14:00:55.228631+0000","last_became_active":"2024-03-26T13:59:23.316136+0000","last_became_peered":"2024-03-26T13:59:23.316136+0000","last_unstale":"2024-03-26T14:00:55.228631+0000","last_undegraded":"2024-03-26T14:00:55.228631+0000","last_fullsized":"2024-03-26T14:00:55.228631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:25:25.602469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214716+0000","last_change":"2024-03-26T13:59:54.348257+0000","last_active":"2024-03-26T14:00:55.214716+0000","last_peered":"2024-03-26T14:00:55.214716+0000","last_clean":"2024-03-26T14:00:55.214716+0000","last_became_active":"2024-03-26T13:59:54.346581+0000","last_became_peered":"2024-03-26T13:59:54.346581+0000","last_unstale":"2024-03-26T14:00:55.214716+0000","last_undegraded":"2024-03-26T14:00:55.214716+0000","last_fullsized":"2024-03-26T14:00:55.214716+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:54:35.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":193,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:20.648087+0000","last_change":"2024-03-26T13:59:22.283827+0000","last_active":"2024-03-26T14:02:20.648087+0000","last_peered":"2024-03-26T14:02:20.648087+0000","last_clean":"2024-03-26T14:02:20.648087+0000","last_became_active":"2024-03-26T13:59:22.283391+0000","last_became_peered":"2024-03-26T13:59:22.283391+0000","last_unstale":"2024-03-26T14:02:20.648087+0000","last_undegraded":"2024-03-26T14:02:20.648087+0000","last_fullsized":"2024-03-26T14:02:20.648087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:44:56.792348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":155,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214812+0000","last_change":"2024-03-26T13:59:54.346323+0000","last_active":"2024-03-26T14:00:55.214812+0000","last_peered":"2024-03-26T14:00:55.214812+0000","last_clean":"2024-03-26T14:00:55.214812+0000","last_became_active":"2024-03-26T13:59:54.345774+0000","last_became_peered":"2024-03-26T13:59:54.345774+0000","last_unstale":"2024-03-26T14:00:55.214812+0000","last_undegraded":"2024-03-26T14:00:55.214812+0000","last_fullsized":"2024-03-26T14:00:55.214812+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:08.876488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216341+0000","last_change":"2024-03-26T13:59:54.763269+0000","last_active":"2024-03-26T14:00:55.216341+0000","last_peered":"2024-03-26T14:00:55.216341+0000","last_clean":"2024-03-26T14:00:55.216341+0000","last_became_active":"2024-03-26T13:59:54.760848+0000","last_became_peered":"2024-03-26T13:59:54.760848+0000","last_unstale":"2024-03-26T14:00:55.216341+0000","last_undegraded":"2024-03-26T14:00:55.216341+0000","last_fullsized":"2024-03-26T14:00:55.216341+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:35:25.390423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217641+0000","last_change":"2024-03-26T13:59:22.283526+0000","last_active":"2024-03-26T14:00:55.217641+0000","last_peered":"2024-03-26T14:00:55.217641+0000","last_clean":"2024-03-26T14:00:55.217641+0000","last_became_active":"2024-03-26T13:59:22.283031+0000","last_became_peered":"2024-03-26T13:59:22.283031+0000","last_unstale":"2024-03-26T14:00:55.217641+0000","last_undegraded":"2024-03-26T14:00:55.217641+0000","last_fullsized":"2024-03-26T14:00:55.217641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:51.361798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227975+0000","last_change":"2024-03-26T13:59:23.316193+0000","last_active":"2024-03-26T14:00:55.227975+0000","last_peered":"2024-03-26T14:00:55.227975+0000","last_clean":"2024-03-26T14:00:55.227975+0000","last_became_active":"2024-03-26T13:59:23.315822+0000","last_became_peered":"2024-03-26T13:59:23.315822+0000","last_unstale":"2024-03-26T14:00:55.227975+0000","last_undegraded":"2024-03-26T14:00:55.227975+0000","last_fullsized":"2024-03-26T14:00:55.227975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:01.382648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.222150+0000","last_change":"2024-03-26T13:59:27.326691+0000","last_active":"2024-03-26T14:00:55.222150+0000","last_peered":"2024-03-26T14:00:55.222150+0000","last_clean":"2024-03-26T14:00:55.222150+0000","last_became_active":"2024-03-26T13:59:27.326252+0000","last_became_peered":"2024-03-26T13:59:27.326252+0000","last_unstale":"2024-03-26T14:00:55.222150+0000","last_undegraded":"2024-03-26T14:00:55.222150+0000","last_fullsized":"2024-03-26T14:00:55.222150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:44.511363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217537+0000","last_change":"2024-03-26T13:59:22.275880+0000","last_active":"2024-03-26T14:00:55.217537+0000","last_peered":"2024-03-26T14:00:55.217537+0000","last_clean":"2024-03-26T14:00:55.217537+0000","last_became_active":"2024-03-26T13:59:22.275489+0000","last_became_peered":"2024-03-26T13:59:22.275489+0000","last_unstale":"2024-03-26T14:00:55.217537+0000","last_undegraded":"2024-03-26T14:00:55.217537+0000","last_fullsized":"2024-03-26T14:00:55.217537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:17:19.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210377+0000","last_change":"2024-03-26T13:59:54.758628+0000","last_active":"2024-03-26T14:00:55.210377+0000","last_peered":"2024-03-26T14:00:55.210377+0000","last_clean":"2024-03-26T14:00:55.210377+0000","last_became_active":"2024-03-26T13:59:54.757791+0000","last_became_peered":"2024-03-26T13:59:54.757791+0000","last_unstale":"2024-03-26T14:00:55.210377+0000","last_undegraded":"2024-03-26T14:00:55.210377+0000","last_fullsized":"2024-03-26T14:00:55.210377+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:58:36.671569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212422+0000","last_change":"2024-03-26T13:59:27.321898+0000","last_active":"2024-03-26T14:00:55.212422+0000","last_peered":"2024-03-26T14:00:55.212422+0000","last_clean":"2024-03-26T14:00:55.212422+0000","last_became_active":"2024-03-26T13:59:27.321525+0000","last_became_peered":"2024-03-26T13:59:27.321525+0000","last_unstale":"2024-03-26T14:00:55.212422+0000","last_undegraded":"2024-03-26T14:00:55.212422+0000","last_fullsized":"2024-03-26T14:00:55.212422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:32:11.986448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226654+0000","last_change":"2024-03-26T13:59:22.279986+0000","last_active":"2024-03-26T14:00:55.226654+0000","last_peered":"2024-03-26T14:00:55.226654+0000","last_clean":"2024-03-26T14:00:55.226654+0000","last_became_active":"2024-03-26T13:59:22.279627+0000","last_became_peered":"2024-03-26T13:59:22.279627+0000","last_unstale":"2024-03-26T14:00:55.226654+0000","last_undegraded":"2024-03-26T14:00:55.226654+0000","last_fullsized":"2024-03-26T14:00:55.226654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:12:46.159609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211075+0000","last_change":"2024-03-26T13:59:23.295060+0000","last_active":"2024-03-26T14:00:55.211075+0000","last_peered":"2024-03-26T14:00:55.211075+0000","last_clean":"2024-03-26T14:00:55.211075+0000","last_became_active":"2024-03-26T13:59:23.294400+0000","last_became_peered":"2024-03-26T13:59:23.294400+0000","last_unstale":"2024-03-26T14:00:55.211075+0000","last_undegraded":"2024-03-26T14:00:55.211075+0000","last_fullsized":"2024-03-26T14:00:55.211075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:26:20.731473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206766+0000","last_change":"2024-03-26T13:59:27.331090+0000","last_active":"2024-03-26T14:00:55.206766+0000","last_peered":"2024-03-26T14:00:55.206766+0000","last_clean":"2024-03-26T14:00:55.206766+0000","last_became_active":"2024-03-26T13:59:27.330630+0000","last_became_peered":"2024-03-26T13:59:27.330630+0000","last_unstale":"2024-03-26T14:00:55.206766+0000","last_undegraded":"2024-03-26T14:00:55.206766+0000","last_fullsized":"2024-03-26T14:00:55.206766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:34:46.468698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":192,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:24.660131+0000","last_change":"2024-03-26T13:59:22.290302+0000","last_active":"2024-03-26T14:02:24.660131+0000","last_peered":"2024-03-26T14:02:24.660131+0000","last_clean":"2024-03-26T14:02:24.660131+0000","last_became_active":"2024-03-26T13:59:22.289890+0000","last_became_peered":"2024-03-26T13:59:22.289890+0000","last_unstale":"2024-03-26T14:02:24.660131+0000","last_undegraded":"2024-03-26T14:02:24.660131+0000","last_fullsized":"2024-03-26T14:02:24.660131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:16:08.395426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":156,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.434135+0000","last_change":"2024-03-26T13:59:23.296551+0000","last_active":"2024-03-26T14:00:55.434135+0000","last_peered":"2024-03-26T14:00:55.434135+0000","last_clean":"2024-03-26T14:00:55.434135+0000","last_became_active":"2024-03-26T13:59:23.295744+0000","last_became_peered":"2024-03-26T13:59:23.295744+0000","last_unstale":"2024-03-26T14:00:55.434135+0000","last_undegraded":"2024-03-26T14:00:55.434135+0000","last_fullsized":"2024-03-26T14:00:55.434135+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:15:19.370224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211849+0000","last_change":"2024-03-26T13:59:27.315247+0000","last_active":"2024-03-26T14:00:55.211849+0000","last_peered":"2024-03-26T14:00:55.211849+0000","last_clean":"2024-03-26T14:00:55.211849+0000","last_became_active":"2024-03-26T13:59:27.314678+0000","last_became_peered":"2024-03-26T13:59:27.314678+0000","last_unstale":"2024-03-26T14:00:55.211849+0000","last_undegraded":"2024-03-26T14:00:55.211849+0000","last_fullsized":"2024-03-26T14:00:55.211849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:06:47.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214014+0000","last_change":"2024-03-26T13:59:22.289542+0000","last_active":"2024-03-26T14:00:55.214014+0000","last_peered":"2024-03-26T14:00:55.214014+0000","last_clean":"2024-03-26T14:00:55.214014+0000","last_became_active":"2024-03-26T13:59:22.288686+0000","last_became_peered":"2024-03-26T13:59:22.288686+0000","last_unstale":"2024-03-26T14:00:55.214014+0000","last_undegraded":"2024-03-26T14:00:55.214014+0000","last_fullsized":"2024-03-26T14:00:55.214014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:41:40.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225122+0000","last_change":"2024-03-26T14:00:55.224154+0000","last_active":"2024-03-26T14:00:55.225122+0000","last_peered":"2024-03-26T14:00:55.225122+0000","last_clean":"2024-03-26T14:00:55.225122+0000","last_became_active":"2024-03-26T14:00:55.223674+0000","last_became_peered":"2024-03-26T14:00:55.223674+0000","last_unstale":"2024-03-26T14:00:55.225122+0000","last_undegraded":"2024-03-26T14:00:55.225122+0000","last_fullsized":"2024-03-26T14:00:55.225122+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:48:31.146317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,6],"acting":[0,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209697+0000","last_change":"2024-03-26T13:59:27.321797+0000","last_active":"2024-03-26T14:00:55.209697+0000","last_peered":"2024-03-26T14:00:55.209697+0000","last_clean":"2024-03-26T14:00:55.209697+0000","last_became_active":"2024-03-26T13:59:27.319903+0000","last_became_peered":"2024-03-26T13:59:27.319903+0000","last_unstale":"2024-03-26T14:00:55.209697+0000","last_undegraded":"2024-03-26T14:00:55.209697+0000","last_fullsized":"2024-03-26T14:00:55.209697+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:36:09.884329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228174+0000","last_change":"2024-03-26T13:59:22.278063+0000","last_active":"2024-03-26T14:00:55.228174+0000","last_peered":"2024-03-26T14:00:55.228174+0000","last_clean":"2024-03-26T14:00:55.228174+0000","last_became_active":"2024-03-26T13:59:22.277569+0000","last_became_peered":"2024-03-26T13:59:22.277569+0000","last_unstale":"2024-03-26T14:00:55.228174+0000","last_undegraded":"2024-03-26T14:00:55.228174+0000","last_fullsized":"2024-03-26T14:00:55.228174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:34:50.864185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433993+0000","last_change":"2024-03-26T13:59:23.307792+0000","last_active":"2024-03-26T14:00:55.433993+0000","last_peered":"2024-03-26T14:00:55.433993+0000","last_clean":"2024-03-26T14:00:55.433993+0000","last_became_active":"2024-03-26T13:59:23.307353+0000","last_became_peered":"2024-03-26T13:59:23.307353+0000","last_unstale":"2024-03-26T14:00:55.433993+0000","last_undegraded":"2024-03-26T14:00:55.433993+0000","last_fullsized":"2024-03-26T14:00:55.433993+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:39:15.030635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210803+0000","last_change":"2024-03-26T13:59:27.326481+0000","last_active":"2024-03-26T14:00:55.210803+0000","last_peered":"2024-03-26T14:00:55.210803+0000","last_clean":"2024-03-26T14:00:55.210803+0000","last_became_active":"2024-03-26T13:59:27.325403+0000","last_became_peered":"2024-03-26T13:59:27.325403+0000","last_unstale":"2024-03-26T14:00:55.210803+0000","last_undegraded":"2024-03-26T14:00:55.210803+0000","last_fullsized":"2024-03-26T14:00:55.210803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:40:58.897873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":398,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:25.633621+0000","last_change":"2024-03-26T13:59:22.289073+0000","last_active":"2024-03-26T14:02:25.633621+0000","last_peered":"2024-03-26T14:02:25.633621+0000","last_clean":"2024-03-26T14:02:25.633621+0000","last_became_active":"2024-03-26T13:59:22.288670+0000","last_became_peered":"2024-03-26T13:59:22.288670+0000","last_unstale":"2024-03-26T14:02:25.633621+0000","last_undegraded":"2024-03-26T14:02:25.633621+0000","last_fullsized":"2024-03-26T14:02:25.633621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:51:45.744060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":73041,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":324,"num_read_kb":234,"num_write":43,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217378+0000","last_change":"2024-03-26T13:59:23.304968+0000","last_active":"2024-03-26T14:00:55.217378+0000","last_peered":"2024-03-26T14:00:55.217378+0000","last_clean":"2024-03-26T14:00:55.217378+0000","last_became_active":"2024-03-26T13:59:23.304620+0000","last_became_peered":"2024-03-26T13:59:23.304620+0000","last_unstale":"2024-03-26T14:00:55.217378+0000","last_undegraded":"2024-03-26T14:00:55.217378+0000","last_fullsized":"2024-03-26T14:00:55.217378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:31:34.096523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.229375+0000","last_change":"2024-03-26T13:59:27.335336+0000","last_active":"2024-03-26T14:00:55.229375+0000","last_peered":"2024-03-26T14:00:55.229375+0000","last_clean":"2024-03-26T14:00:55.229375+0000","last_became_active":"2024-03-26T13:59:27.334554+0000","last_became_peered":"2024-03-26T13:59:27.334554+0000","last_unstale":"2024-03-26T14:00:55.229375+0000","last_undegraded":"2024-03-26T14:00:55.229375+0000","last_fullsized":"2024-03-26T14:00:55.229375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:15:32.932847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.240886+0000","last_change":"2024-03-26T14:00:55.239852+0000","last_active":"2024-03-26T14:00:55.240886+0000","last_peered":"2024-03-26T14:00:55.240886+0000","last_clean":"2024-03-26T14:00:55.240886+0000","last_became_active":"2024-03-26T14:00:55.221294+0000","last_became_peered":"2024-03-26T14:00:55.221294+0000","last_unstale":"2024-03-26T14:00:55.240886+0000","last_undegraded":"2024-03-26T14:00:55.240886+0000","last_fullsized":"2024-03-26T14:00:55.240886+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:14:53.910832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217300+0000","last_change":"2024-03-26T13:59:23.307561+0000","last_active":"2024-03-26T14:00:55.217300+0000","last_peered":"2024-03-26T14:00:55.217300+0000","last_clean":"2024-03-26T14:00:55.217300+0000","last_became_active":"2024-03-26T13:59:23.307230+0000","last_became_peered":"2024-03-26T13:59:23.307230+0000","last_unstale":"2024-03-26T14:00:55.217300+0000","last_undegraded":"2024-03-26T14:00:55.217300+0000","last_fullsized":"2024-03-26T14:00:55.217300+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:04:10.104519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.220773+0000","last_change":"2024-03-26T13:59:27.325798+0000","last_active":"2024-03-26T14:00:55.220773+0000","last_peered":"2024-03-26T14:00:55.220773+0000","last_clean":"2024-03-26T14:00:55.220773+0000","last_became_active":"2024-03-26T13:59:27.325336+0000","last_became_peered":"2024-03-26T13:59:27.325336+0000","last_unstale":"2024-03-26T14:00:55.220773+0000","last_undegraded":"2024-03-26T14:00:55.220773+0000","last_fullsized":"2024-03-26T14:00:55.220773+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:45:14.246920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227998+0000","last_change":"2024-03-26T13:59:22.275414+0000","last_active":"2024-03-26T14:00:55.227998+0000","last_peered":"2024-03-26T14:00:55.227998+0000","last_clean":"2024-03-26T14:00:55.227998+0000","last_became_active":"2024-03-26T13:59:22.275020+0000","last_became_peered":"2024-03-26T13:59:22.275020+0000","last_unstale":"2024-03-26T14:00:55.227998+0000","last_undegraded":"2024-03-26T14:00:55.227998+0000","last_fullsized":"2024-03-26T14:00:55.227998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:47:21.852237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214454+0000","last_change":"2024-03-26T13:59:23.311986+0000","last_active":"2024-03-26T14:00:55.214454+0000","last_peered":"2024-03-26T14:00:55.214454+0000","last_clean":"2024-03-26T14:00:55.214454+0000","last_became_active":"2024-03-26T13:59:23.311677+0000","last_became_peered":"2024-03-26T13:59:23.311677+0000","last_unstale":"2024-03-26T14:00:55.214454+0000","last_undegraded":"2024-03-26T14:00:55.214454+0000","last_fullsized":"2024-03-26T14:00:55.214454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:09:38.825214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212803+0000","last_change":"2024-03-26T13:59:27.322744+0000","last_active":"2024-03-26T14:00:55.212803+0000","last_peered":"2024-03-26T14:00:55.212803+0000","last_clean":"2024-03-26T14:00:55.212803+0000","last_became_active":"2024-03-26T13:59:27.322260+0000","last_became_peered":"2024-03-26T13:59:27.322260+0000","last_unstale":"2024-03-26T14:00:55.212803+0000","last_undegraded":"2024-03-26T14:00:55.212803+0000","last_fullsized":"2024-03-26T14:00:55.212803+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:41.425890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'6","reported_seq":196,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:21.693198+0000","last_change":"2024-03-26T13:59:22.281795+0000","last_active":"2024-03-26T14:02:21.693198+0000","last_peered":"2024-03-26T14:02:21.693198+0000","last_clean":"2024-03-26T14:02:21.693198+0000","last_became_active":"2024-03-26T13:59:22.281351+0000","last_became_peered":"2024-03-26T13:59:22.281351+0000","last_unstale":"2024-03-26T14:02:21.693198+0000","last_undegraded":"2024-03-26T14:02:21.693198+0000","last_fullsized":"2024-03-26T14:02:21.693198+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:19:57.837312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":159,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213552+0000","last_change":"2024-03-26T13:59:23.298234+0000","last_active":"2024-03-26T14:00:55.213552+0000","last_peered":"2024-03-26T14:00:55.213552+0000","last_clean":"2024-03-26T14:00:55.213552+0000","last_became_active":"2024-03-26T13:59:23.297916+0000","last_became_peered":"2024-03-26T13:59:23.297916+0000","last_unstale":"2024-03-26T14:00:55.213552+0000","last_undegraded":"2024-03-26T14:00:55.213552+0000","last_fullsized":"2024-03-26T14:00:55.213552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:19:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217932+0000","last_change":"2024-03-26T13:59:27.331300+0000","last_active":"2024-03-26T14:00:55.217932+0000","last_peered":"2024-03-26T14:00:55.217932+0000","last_clean":"2024-03-26T14:00:55.217932+0000","last_became_active":"2024-03-26T13:59:27.330125+0000","last_became_peered":"2024-03-26T13:59:27.330125+0000","last_unstale":"2024-03-26T14:00:55.217932+0000","last_undegraded":"2024-03-26T14:00:55.217932+0000","last_fullsized":"2024-03-26T14:00:55.217932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:16:06.915976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212507+0000","last_change":"2024-03-26T13:59:22.291264+0000","last_active":"2024-03-26T14:00:55.212507+0000","last_peered":"2024-03-26T14:00:55.212507+0000","last_clean":"2024-03-26T14:00:55.212507+0000","last_became_active":"2024-03-26T13:59:22.290888+0000","last_became_peered":"2024-03-26T13:59:22.290888+0000","last_unstale":"2024-03-26T14:00:55.212507+0000","last_undegraded":"2024-03-26T14:00:55.212507+0000","last_fullsized":"2024-03-26T14:00:55.212507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:53:10.676537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227066+0000","last_change":"2024-03-26T13:59:23.319170+0000","last_active":"2024-03-26T14:00:55.227066+0000","last_peered":"2024-03-26T14:00:55.227066+0000","last_clean":"2024-03-26T14:00:55.227066+0000","last_became_active":"2024-03-26T13:59:23.318642+0000","last_became_peered":"2024-03-26T13:59:23.318642+0000","last_unstale":"2024-03-26T14:00:55.227066+0000","last_undegraded":"2024-03-26T14:00:55.227066+0000","last_fullsized":"2024-03-26T14:00:55.227066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:20:46.663316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226862+0000","last_change":"2024-03-26T13:59:27.324153+0000","last_active":"2024-03-26T14:00:55.226862+0000","last_peered":"2024-03-26T14:00:55.226862+0000","last_clean":"2024-03-26T14:00:55.226862+0000","last_became_active":"2024-03-26T13:59:27.323673+0000","last_became_peered":"2024-03-26T13:59:27.323673+0000","last_unstale":"2024-03-26T14:00:55.226862+0000","last_undegraded":"2024-03-26T14:00:55.226862+0000","last_fullsized":"2024-03-26T14:00:55.226862+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:25:27.371796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217173+0000","last_change":"2024-03-26T13:59:22.289956+0000","last_active":"2024-03-26T14:00:55.217173+0000","last_peered":"2024-03-26T14:00:55.217173+0000","last_clean":"2024-03-26T14:00:55.217173+0000","last_became_active":"2024-03-26T13:59:22.289464+0000","last_became_peered":"2024-03-26T13:59:22.289464+0000","last_unstale":"2024-03-26T14:00:55.217173+0000","last_undegraded":"2024-03-26T14:00:55.217173+0000","last_fullsized":"2024-03-26T14:00:55.217173+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:45:08.042071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221832+0000","last_change":"2024-03-26T13:59:54.761409+0000","last_active":"2024-03-26T14:00:55.221832+0000","last_peered":"2024-03-26T14:00:55.221832+0000","last_clean":"2024-03-26T14:00:55.221832+0000","last_became_active":"2024-03-26T13:59:54.760324+0000","last_became_peered":"2024-03-26T13:59:54.760324+0000","last_unstale":"2024-03-26T14:00:55.221832+0000","last_undegraded":"2024-03-26T14:00:55.221832+0000","last_fullsized":"2024-03-26T14:00:55.221832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:07:02.339352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214516+0000","last_change":"2024-03-26T13:59:27.324379+0000","last_active":"2024-03-26T14:00:55.214516+0000","last_peered":"2024-03-26T14:00:55.214516+0000","last_clean":"2024-03-26T14:00:55.214516+0000","last_became_active":"2024-03-26T13:59:27.323984+0000","last_became_peered":"2024-03-26T13:59:27.323984+0000","last_unstale":"2024-03-26T14:00:55.214516+0000","last_undegraded":"2024-03-26T14:00:55.214516+0000","last_fullsized":"2024-03-26T14:00:55.214516+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:14.507918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225256+0000","last_change":"2024-03-26T14:00:55.224501+0000","last_active":"2024-03-26T14:00:55.225256+0000","last_peered":"2024-03-26T14:00:55.225256+0000","last_clean":"2024-03-26T14:00:55.225256+0000","last_became_active":"2024-03-26T14:00:55.223843+0000","last_became_peered":"2024-03-26T14:00:55.223843+0000","last_unstale":"2024-03-26T14:00:55.225256+0000","last_undegraded":"2024-03-26T14:00:55.225256+0000","last_fullsized":"2024-03-26T14:00:55.225256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:29:11.310901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217789+0000","last_change":"2024-03-26T13:59:23.288740+0000","last_active":"2024-03-26T14:00:55.217789+0000","last_peered":"2024-03-26T14:00:55.217789+0000","last_clean":"2024-03-26T14:00:55.217789+0000","last_became_active":"2024-03-26T13:59:23.288381+0000","last_became_peered":"2024-03-26T13:59:23.288381+0000","last_unstale":"2024-03-26T14:00:55.217789+0000","last_undegraded":"2024-03-26T14:00:55.217789+0000","last_fullsized":"2024-03-26T14:00:55.217789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:05:27.315890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217139+0000","last_change":"2024-03-26T13:59:27.325884+0000","last_active":"2024-03-26T14:00:55.217139+0000","last_peered":"2024-03-26T14:00:55.217139+0000","last_clean":"2024-03-26T14:00:55.217139+0000","last_became_active":"2024-03-26T13:59:27.324767+0000","last_became_peered":"2024-03-26T13:59:27.324767+0000","last_unstale":"2024-03-26T14:00:55.217139+0000","last_undegraded":"2024-03-26T14:00:55.217139+0000","last_fullsized":"2024-03-26T14:00:55.217139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:28:38.053599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214292+0000","last_change":"2024-03-26T13:59:22.276172+0000","last_active":"2024-03-26T14:00:55.214292+0000","last_peered":"2024-03-26T14:00:55.214292+0000","last_clean":"2024-03-26T14:00:55.214292+0000","last_became_active":"2024-03-26T13:59:22.275626+0000","last_became_peered":"2024-03-26T13:59:22.275626+0000","last_unstale":"2024-03-26T14:00:55.214292+0000","last_undegraded":"2024-03-26T14:00:55.214292+0000","last_fullsized":"2024-03-26T14:00:55.214292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:23:07.388520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217644+0000","last_change":"2024-03-26T13:59:23.317256+0000","last_active":"2024-03-26T14:00:55.217644+0000","last_peered":"2024-03-26T14:00:55.217644+0000","last_clean":"2024-03-26T14:00:55.217644+0000","last_became_active":"2024-03-26T13:59:23.316845+0000","last_became_peered":"2024-03-26T13:59:23.316845+0000","last_unstale":"2024-03-26T14:00:55.217644+0000","last_undegraded":"2024-03-26T14:00:55.217644+0000","last_fullsized":"2024-03-26T14:00:55.217644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:59:12.525195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217901+0000","last_change":"2024-03-26T13:59:27.332992+0000","last_active":"2024-03-26T14:00:55.217901+0000","last_peered":"2024-03-26T14:00:55.217901+0000","last_clean":"2024-03-26T14:00:55.217901+0000","last_became_active":"2024-03-26T13:59:27.332607+0000","last_became_peered":"2024-03-26T13:59:27.332607+0000","last_unstale":"2024-03-26T14:00:55.217901+0000","last_undegraded":"2024-03-26T14:00:55.217901+0000","last_fullsized":"2024-03-26T14:00:55.217901+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:58:41.829902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.227219+0000","last_change":"2024-03-26T13:59:23.316322+0000","last_active":"2024-03-26T14:00:55.227219+0000","last_peered":"2024-03-26T14:00:55.227219+0000","last_clean":"2024-03-26T14:00:55.227219+0000","last_became_active":"2024-03-26T13:59:23.315998+0000","last_became_peered":"2024-03-26T13:59:23.315998+0000","last_unstale":"2024-03-26T14:00:55.227219+0000","last_undegraded":"2024-03-26T14:00:55.227219+0000","last_fullsized":"2024-03-26T14:00:55.227219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:01:25.057246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":195,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:25.703123+0000","last_change":"2024-03-26T13:59:22.260506+0000","last_active":"2024-03-26T14:02:25.703123+0000","last_peered":"2024-03-26T14:02:25.703123+0000","last_clean":"2024-03-26T14:02:25.703123+0000","last_became_active":"2024-03-26T13:59:22.260101+0000","last_became_peered":"2024-03-26T13:59:22.260101+0000","last_unstale":"2024-03-26T14:02:25.703123+0000","last_undegraded":"2024-03-26T14:02:25.703123+0000","last_fullsized":"2024-03-26T14:02:25.703123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:20:57.855046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":157,"num_read_kb":157,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214061+0000","last_change":"2024-03-26T13:59:27.332651+0000","last_active":"2024-03-26T14:00:55.214061+0000","last_peered":"2024-03-26T14:00:55.214061+0000","last_clean":"2024-03-26T14:00:55.214061+0000","last_became_active":"2024-03-26T13:59:27.332160+0000","last_became_peered":"2024-03-26T13:59:27.332160+0000","last_unstale":"2024-03-26T14:00:55.214061+0000","last_undegraded":"2024-03-26T14:00:55.214061+0000","last_fullsized":"2024-03-26T14:00:55.214061+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:26:11.037716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207002+0000","last_change":"2024-03-26T13:59:23.300818+0000","last_active":"2024-03-26T14:00:55.207002+0000","last_peered":"2024-03-26T14:00:55.207002+0000","last_clean":"2024-03-26T14:00:55.207002+0000","last_became_active":"2024-03-26T13:59:23.300478+0000","last_became_peered":"2024-03-26T13:59:23.300478+0000","last_unstale":"2024-03-26T14:00:55.207002+0000","last_undegraded":"2024-03-26T14:00:55.207002+0000","last_fullsized":"2024-03-26T14:00:55.207002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:33:33.408658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209617+0000","last_change":"2024-03-26T13:59:22.276816+0000","last_active":"2024-03-26T14:00:55.209617+0000","last_peered":"2024-03-26T14:00:55.209617+0000","last_clean":"2024-03-26T14:00:55.209617+0000","last_became_active":"2024-03-26T13:59:22.276387+0000","last_became_peered":"2024-03-26T13:59:22.276387+0000","last_unstale":"2024-03-26T14:00:55.209617+0000","last_undegraded":"2024-03-26T14:00:55.209617+0000","last_fullsized":"2024-03-26T14:00:55.209617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:44:24.968652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":357,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213393+0000","last_change":"2024-03-26T13:57:44.288255+0000","last_active":"2024-03-26T14:00:55.213393+0000","last_peered":"2024-03-26T14:00:55.213393+0000","last_clean":"2024-03-26T14:00:55.213393+0000","last_became_active":"2024-03-26T13:57:44.286852+0000","last_became_peered":"2024-03-26T13:57:44.286852+0000","last_unstale":"2024-03-26T14:00:55.213393+0000","last_undegraded":"2024-03-26T14:00:55.213393+0000","last_fullsized":"2024-03-26T14:00:55.213393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:53:05.748699+0000","last_clean_scrub_stamp":"2024-03-26T13:53:05.748699+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:39:55.262061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228488+0000","last_change":"2024-03-26T13:59:27.331795+0000","last_active":"2024-03-26T14:00:55.228488+0000","last_peered":"2024-03-26T14:00:55.228488+0000","last_clean":"2024-03-26T14:00:55.228488+0000","last_became_active":"2024-03-26T13:59:27.329543+0000","last_became_peered":"2024-03-26T13:59:27.329543+0000","last_unstale":"2024-03-26T14:00:55.228488+0000","last_undegraded":"2024-03-26T14:00:55.228488+0000","last_fullsized":"2024-03-26T14:00:55.228488+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:27:15.042972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216689+0000","last_change":"2024-03-26T13:59:23.299408+0000","last_active":"2024-03-26T14:00:55.216689+0000","last_peered":"2024-03-26T14:00:55.216689+0000","last_clean":"2024-03-26T14:00:55.216689+0000","last_became_active":"2024-03-26T13:59:23.295378+0000","last_became_peered":"2024-03-26T13:59:23.295378+0000","last_unstale":"2024-03-26T14:00:55.216689+0000","last_undegraded":"2024-03-26T14:00:55.216689+0000","last_fullsized":"2024-03-26T14:00:55.216689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:30:24.541874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.439884+0000","last_change":"2024-03-26T14:00:55.439230+0000","last_active":"2024-03-26T14:00:55.439884+0000","last_peered":"2024-03-26T14:00:55.439884+0000","last_clean":"2024-03-26T14:00:55.439884+0000","last_became_active":"2024-03-26T14:00:55.434988+0000","last_became_peered":"2024-03-26T14:00:55.434988+0000","last_unstale":"2024-03-26T14:00:55.439884+0000","last_undegraded":"2024-03-26T14:00:55.439884+0000","last_fullsized":"2024-03-26T14:00:55.439884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:38:58.291575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.223104+0000","last_change":"2024-03-26T13:59:27.330413+0000","last_active":"2024-03-26T14:00:55.223104+0000","last_peered":"2024-03-26T14:00:55.223104+0000","last_clean":"2024-03-26T14:00:55.223104+0000","last_became_active":"2024-03-26T13:59:27.329971+0000","last_became_peered":"2024-03-26T13:59:27.329971+0000","last_unstale":"2024-03-26T14:00:55.223104+0000","last_undegraded":"2024-03-26T14:00:55.223104+0000","last_fullsized":"2024-03-26T14:00:55.223104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T19:49:19.632923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217076+0000","last_change":"2024-03-26T13:59:22.279192+0000","last_active":"2024-03-26T14:00:55.217076+0000","last_peered":"2024-03-26T14:00:55.217076+0000","last_clean":"2024-03-26T14:00:55.217076+0000","last_became_active":"2024-03-26T13:59:22.278312+0000","last_became_peered":"2024-03-26T13:59:22.278312+0000","last_unstale":"2024-03-26T14:00:55.217076+0000","last_undegraded":"2024-03-26T14:00:55.217076+0000","last_fullsized":"2024-03-26T14:00:55.217076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:46:49.758496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217337+0000","last_change":"2024-03-26T13:59:23.282761+0000","last_active":"2024-03-26T14:00:55.217337+0000","last_peered":"2024-03-26T14:00:55.217337+0000","last_clean":"2024-03-26T14:00:55.217337+0000","last_became_active":"2024-03-26T13:59:23.282395+0000","last_became_peered":"2024-03-26T13:59:23.282395+0000","last_unstale":"2024-03-26T14:00:55.217337+0000","last_undegraded":"2024-03-26T14:00:55.217337+0000","last_fullsized":"2024-03-26T14:00:55.217337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:03:44.922551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206756+0000","last_change":"2024-03-26T13:59:27.318668+0000","last_active":"2024-03-26T14:00:55.206756+0000","last_peered":"2024-03-26T14:00:55.206756+0000","last_clean":"2024-03-26T14:00:55.206756+0000","last_became_active":"2024-03-26T13:59:27.318257+0000","last_became_peered":"2024-03-26T13:59:27.318257+0000","last_unstale":"2024-03-26T14:00:55.206756+0000","last_undegraded":"2024-03-26T14:00:55.206756+0000","last_fullsized":"2024-03-26T14:00:55.206756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:23:23.854530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:02:20.345895+0000","last_change":"2024-03-26T13:59:22.276904+0000","last_active":"2024-03-26T14:02:20.345895+0000","last_peered":"2024-03-26T14:02:20.345895+0000","last_clean":"2024-03-26T14:02:20.345895+0000","last_became_active":"2024-03-26T13:59:22.276115+0000","last_became_peered":"2024-03-26T13:59:22.276115+0000","last_unstale":"2024-03-26T14:02:20.345895+0000","last_undegraded":"2024-03-26T14:02:20.345895+0000","last_fullsized":"2024-03-26T14:02:20.345895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:00:03.785904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206944+0000","last_change":"2024-03-26T13:59:23.310040+0000","last_active":"2024-03-26T14:00:55.206944+0000","last_peered":"2024-03-26T14:00:55.206944+0000","last_clean":"2024-03-26T14:00:55.206944+0000","last_became_active":"2024-03-26T13:59:23.307584+0000","last_became_peered":"2024-03-26T13:59:23.307584+0000","last_unstale":"2024-03-26T14:00:55.206944+0000","last_undegraded":"2024-03-26T14:00:55.206944+0000","last_fullsized":"2024-03-26T14:00:55.206944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:58:24.416230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.211722+0000","last_change":"2024-03-26T13:59:27.320142+0000","last_active":"2024-03-26T14:00:55.211722+0000","last_peered":"2024-03-26T14:00:55.211722+0000","last_clean":"2024-03-26T14:00:55.211722+0000","last_became_active":"2024-03-26T13:59:27.319655+0000","last_became_peered":"2024-03-26T13:59:27.319655+0000","last_unstale":"2024-03-26T14:00:55.211722+0000","last_undegraded":"2024-03-26T14:00:55.211722+0000","last_fullsized":"2024-03-26T14:00:55.211722+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:47:24.234782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210428+0000","last_change":"2024-03-26T13:59:22.280232+0000","last_active":"2024-03-26T14:00:55.210428+0000","last_peered":"2024-03-26T14:00:55.210428+0000","last_clean":"2024-03-26T14:00:55.210428+0000","last_became_active":"2024-03-26T13:59:22.278482+0000","last_became_peered":"2024-03-26T13:59:22.278482+0000","last_unstale":"2024-03-26T14:00:55.210428+0000","last_undegraded":"2024-03-26T14:00:55.210428+0000","last_fullsized":"2024-03-26T14:00:55.210428+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:34:46.734921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212631+0000","last_change":"2024-03-26T13:59:23.317246+0000","last_active":"2024-03-26T14:00:55.212631+0000","last_peered":"2024-03-26T14:00:55.212631+0000","last_clean":"2024-03-26T14:00:55.212631+0000","last_became_active":"2024-03-26T13:59:23.316932+0000","last_became_peered":"2024-03-26T13:59:23.316932+0000","last_unstale":"2024-03-26T14:00:55.212631+0000","last_undegraded":"2024-03-26T14:00:55.212631+0000","last_fullsized":"2024-03-26T14:00:55.212631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:07:00.148309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210150+0000","last_change":"2024-03-26T13:59:27.330270+0000","last_active":"2024-03-26T14:00:55.210150+0000","last_peered":"2024-03-26T14:00:55.210150+0000","last_clean":"2024-03-26T14:00:55.210150+0000","last_became_active":"2024-03-26T13:59:27.329495+0000","last_became_peered":"2024-03-26T13:59:27.329495+0000","last_unstale":"2024-03-26T14:00:55.210150+0000","last_undegraded":"2024-03-26T14:00:55.210150+0000","last_fullsized":"2024-03-26T14:00:55.210150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:37:40.430011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433840+0000","last_change":"2024-03-26T13:59:22.281070+0000","last_active":"2024-03-26T14:00:55.433840+0000","last_peered":"2024-03-26T14:00:55.433840+0000","last_clean":"2024-03-26T14:00:55.433840+0000","last_became_active":"2024-03-26T13:59:22.279947+0000","last_became_peered":"2024-03-26T13:59:22.279947+0000","last_unstale":"2024-03-26T14:00:55.433840+0000","last_undegraded":"2024-03-26T14:00:55.433840+0000","last_fullsized":"2024-03-26T14:00:55.433840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:45:32.617082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213281+0000","last_change":"2024-03-26T13:59:23.300281+0000","last_active":"2024-03-26T14:00:55.213281+0000","last_peered":"2024-03-26T14:00:55.213281+0000","last_clean":"2024-03-26T14:00:55.213281+0000","last_became_active":"2024-03-26T13:59:23.299975+0000","last_became_peered":"2024-03-26T13:59:23.299975+0000","last_unstale":"2024-03-26T14:00:55.213281+0000","last_undegraded":"2024-03-26T14:00:55.213281+0000","last_fullsized":"2024-03-26T14:00:55.213281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T18:56:08.377830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.216737+0000","last_change":"2024-03-26T13:59:54.763522+0000","last_active":"2024-03-26T14:00:55.216737+0000","last_peered":"2024-03-26T14:00:55.216737+0000","last_clean":"2024-03-26T14:00:55.216737+0000","last_became_active":"2024-03-26T13:59:54.762278+0000","last_became_peered":"2024-03-26T13:59:54.762278+0000","last_unstale":"2024-03-26T14:00:55.216737+0000","last_undegraded":"2024-03-26T14:00:55.216737+0000","last_fullsized":"2024-03-26T14:00:55.216737+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:37.116801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206794+0000","last_change":"2024-03-26T13:59:22.272826+0000","last_active":"2024-03-26T14:00:55.206794+0000","last_peered":"2024-03-26T14:00:55.206794+0000","last_clean":"2024-03-26T14:00:55.206794+0000","last_became_active":"2024-03-26T13:59:22.270619+0000","last_became_peered":"2024-03-26T13:59:22.270619+0000","last_unstale":"2024-03-26T14:00:55.206794+0000","last_undegraded":"2024-03-26T14:00:55.206794+0000","last_fullsized":"2024-03-26T14:00:55.206794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:31:19.817846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215526+0000","last_change":"2024-03-26T13:59:23.317801+0000","last_active":"2024-03-26T14:00:55.215526+0000","last_peered":"2024-03-26T14:00:55.215526+0000","last_clean":"2024-03-26T14:00:55.215526+0000","last_became_active":"2024-03-26T13:59:23.317457+0000","last_became_peered":"2024-03-26T13:59:23.317457+0000","last_unstale":"2024-03-26T14:00:55.215526+0000","last_undegraded":"2024-03-26T14:00:55.215526+0000","last_fullsized":"2024-03-26T14:00:55.215526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:44:11.934045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209311+0000","last_change":"2024-03-26T13:59:54.714081+0000","last_active":"2024-03-26T14:00:55.209311+0000","last_peered":"2024-03-26T14:00:55.209311+0000","last_clean":"2024-03-26T14:00:55.209311+0000","last_became_active":"2024-03-26T13:59:54.705536+0000","last_became_peered":"2024-03-26T13:59:54.705536+0000","last_unstale":"2024-03-26T14:00:55.209311+0000","last_undegraded":"2024-03-26T14:00:55.209311+0000","last_fullsized":"2024-03-26T14:00:55.209311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:06:06.712999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:01:34.135298+0000","last_change":"2024-03-26T13:59:22.290271+0000","last_active":"2024-03-26T14:01:34.135298+0000","last_peered":"2024-03-26T14:01:34.135298+0000","last_clean":"2024-03-26T14:01:34.135298+0000","last_became_active":"2024-03-26T13:59:22.289353+0000","last_became_peered":"2024-03-26T13:59:22.289353+0000","last_unstale":"2024-03-26T14:01:34.135298+0000","last_undegraded":"2024-03-26T14:01:34.135298+0000","last_fullsized":"2024-03-26T14:01:34.135298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:56.195212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207377+0000","last_change":"2024-03-26T13:59:23.310936+0000","last_active":"2024-03-26T14:00:55.207377+0000","last_peered":"2024-03-26T14:00:55.207377+0000","last_clean":"2024-03-26T14:00:55.207377+0000","last_became_active":"2024-03-26T13:59:23.310609+0000","last_became_peered":"2024-03-26T13:59:23.310609+0000","last_unstale":"2024-03-26T14:00:55.207377+0000","last_undegraded":"2024-03-26T14:00:55.207377+0000","last_fullsized":"2024-03-26T14:00:55.207377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:36:35.697016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.209832+0000","last_change":"2024-03-26T13:59:27.320409+0000","last_active":"2024-03-26T14:00:55.209832+0000","last_peered":"2024-03-26T14:00:55.209832+0000","last_clean":"2024-03-26T14:00:55.209832+0000","last_became_active":"2024-03-26T13:59:27.319896+0000","last_became_peered":"2024-03-26T13:59:27.319896+0000","last_unstale":"2024-03-26T14:00:55.209832+0000","last_undegraded":"2024-03-26T14:00:55.209832+0000","last_fullsized":"2024-03-26T14:00:55.209832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.838775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213280+0000","last_change":"2024-03-26T13:59:22.282818+0000","last_active":"2024-03-26T14:00:55.213280+0000","last_peered":"2024-03-26T14:00:55.213280+0000","last_clean":"2024-03-26T14:00:55.213280+0000","last_became_active":"2024-03-26T13:59:22.282315+0000","last_became_peered":"2024-03-26T13:59:22.282315+0000","last_unstale":"2024-03-26T14:00:55.213280+0000","last_undegraded":"2024-03-26T14:00:55.213280+0000","last_fullsized":"2024-03-26T14:00:55.213280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:59:02.900247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214151+0000","last_change":"2024-03-26T13:59:23.306285+0000","last_active":"2024-03-26T14:00:55.214151+0000","last_peered":"2024-03-26T14:00:55.214151+0000","last_clean":"2024-03-26T14:00:55.214151+0000","last_became_active":"2024-03-26T13:59:23.305927+0000","last_became_peered":"2024-03-26T13:59:23.305927+0000","last_unstale":"2024-03-26T14:00:55.214151+0000","last_undegraded":"2024-03-26T14:00:55.214151+0000","last_fullsized":"2024-03-26T14:00:55.214151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:02:18.111359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.432636+0000","last_change":"2024-03-26T13:59:27.331722+0000","last_active":"2024-03-26T14:00:55.432636+0000","last_peered":"2024-03-26T14:00:55.432636+0000","last_clean":"2024-03-26T14:00:55.432636+0000","last_became_active":"2024-03-26T13:59:27.331092+0000","last_became_peered":"2024-03-26T13:59:27.331092+0000","last_unstale":"2024-03-26T14:00:55.432636+0000","last_undegraded":"2024-03-26T14:00:55.432636+0000","last_fullsized":"2024-03-26T14:00:55.432636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:13:25.363893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212890+0000","last_change":"2024-03-26T13:59:22.286597+0000","last_active":"2024-03-26T14:00:55.212890+0000","last_peered":"2024-03-26T14:00:55.212890+0000","last_clean":"2024-03-26T14:00:55.212890+0000","last_became_active":"2024-03-26T13:59:22.286150+0000","last_became_peered":"2024-03-26T13:59:22.286150+0000","last_unstale":"2024-03-26T14:00:55.212890+0000","last_undegraded":"2024-03-26T14:00:55.212890+0000","last_fullsized":"2024-03-26T14:00:55.212890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:07:46.666576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214297+0000","last_change":"2024-03-26T13:59:23.317793+0000","last_active":"2024-03-26T14:00:55.214297+0000","last_peered":"2024-03-26T14:00:55.214297+0000","last_clean":"2024-03-26T14:00:55.214297+0000","last_became_active":"2024-03-26T13:59:23.315944+0000","last_became_peered":"2024-03-26T13:59:23.315944+0000","last_unstale":"2024-03-26T14:00:55.214297+0000","last_undegraded":"2024-03-26T14:00:55.214297+0000","last_fullsized":"2024-03-26T14:00:55.214297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:05:40.498347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.210537+0000","last_change":"2024-03-26T13:59:27.324899+0000","last_active":"2024-03-26T14:00:55.210537+0000","last_peered":"2024-03-26T14:00:55.210537+0000","last_clean":"2024-03-26T14:00:55.210537+0000","last_became_active":"2024-03-26T13:59:27.324482+0000","last_became_peered":"2024-03-26T13:59:27.324482+0000","last_unstale":"2024-03-26T14:00:55.210537+0000","last_undegraded":"2024-03-26T14:00:55.210537+0000","last_fullsized":"2024-03-26T14:00:55.210537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T00:59:33.467314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.228901+0000","last_change":"2024-03-26T13:59:22.285723+0000","last_active":"2024-03-26T14:00:55.228901+0000","last_peered":"2024-03-26T14:00:55.228901+0000","last_clean":"2024-03-26T14:00:55.228901+0000","last_became_active":"2024-03-26T13:59:22.284914+0000","last_became_peered":"2024-03-26T13:59:22.284914+0000","last_unstale":"2024-03-26T14:00:55.228901+0000","last_undegraded":"2024-03-26T14:00:55.228901+0000","last_fullsized":"2024-03-26T14:00:55.228901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:47:19.107695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214653+0000","last_change":"2024-03-26T13:59:23.305972+0000","last_active":"2024-03-26T14:00:55.214653+0000","last_peered":"2024-03-26T14:00:55.214653+0000","last_clean":"2024-03-26T14:00:55.214653+0000","last_became_active":"2024-03-26T13:59:23.305546+0000","last_became_peered":"2024-03-26T13:59:23.305546+0000","last_unstale":"2024-03-26T14:00:55.214653+0000","last_undegraded":"2024-03-26T14:00:55.214653+0000","last_fullsized":"2024-03-26T14:00:55.214653+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:22:28.332705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.225824+0000","last_change":"2024-03-26T13:59:27.333081+0000","last_active":"2024-03-26T14:00:55.225824+0000","last_peered":"2024-03-26T14:00:55.225824+0000","last_clean":"2024-03-26T14:00:55.225824+0000","last_became_active":"2024-03-26T13:59:27.332327+0000","last_became_peered":"2024-03-26T13:59:27.332327+0000","last_unstale":"2024-03-26T14:00:55.225824+0000","last_undegraded":"2024-03-26T14:00:55.225824+0000","last_fullsized":"2024-03-26T14:00:55.225824+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:26.279934+0000","last_clean_scrub_stamp":"2024-03-26T13:59:26.279934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-28T01:14:54.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217852+0000","last_change":"2024-03-26T13:59:22.284440+0000","last_active":"2024-03-26T14:00:55.217852+0000","last_peered":"2024-03-26T14:00:55.217852+0000","last_clean":"2024-03-26T14:00:55.217852+0000","last_became_active":"2024-03-26T13:59:22.283971+0000","last_became_peered":"2024-03-26T13:59:22.283971+0000","last_unstale":"2024-03-26T14:00:55.217852+0000","last_undegraded":"2024-03-26T14:00:55.217852+0000","last_fullsized":"2024-03-26T14:00:55.217852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:21.232458+0000","last_clean_scrub_stamp":"2024-03-26T13:59:21.232458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:21:16.626658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.214735+0000","last_change":"2024-03-26T13:59:23.304793+0000","last_active":"2024-03-26T14:00:55.214735+0000","last_peered":"2024-03-26T14:00:55.214735+0000","last_clean":"2024-03-26T14:00:55.214735+0000","last_became_active":"2024-03-26T13:59:23.304290+0000","last_became_peered":"2024-03-26T13:59:23.304290+0000","last_unstale":"2024-03-26T14:00:55.214735+0000","last_undegraded":"2024-03-26T14:00:55.214735+0000","last_fullsized":"2024-03-26T14:00:55.214735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:23:51.179460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.215872+0000","last_change":"2024-03-26T13:59:23.310824+0000","last_active":"2024-03-26T14:00:55.215872+0000","last_peered":"2024-03-26T14:00:55.215872+0000","last_clean":"2024-03-26T14:00:55.215872+0000","last_became_active":"2024-03-26T13:59:23.307541+0000","last_became_peered":"2024-03-26T13:59:23.307541+0000","last_unstale":"2024-03-26T14:00:55.215872+0000","last_undegraded":"2024-03-26T14:00:55.215872+0000","last_fullsized":"2024-03-26T14:00:55.215872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:24:57.836724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.212414+0000","last_change":"2024-03-26T13:59:23.300714+0000","last_active":"2024-03-26T14:00:55.212414+0000","last_peered":"2024-03-26T14:00:55.212414+0000","last_clean":"2024-03-26T14:00:55.212414+0000","last_became_active":"2024-03-26T13:59:23.299603+0000","last_became_peered":"2024-03-26T13:59:23.299603+0000","last_unstale":"2024-03-26T14:00:55.212414+0000","last_undegraded":"2024-03-26T14:00:55.212414+0000","last_fullsized":"2024-03-26T14:00:55.212414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:14:40.474487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213601+0000","last_change":"2024-03-26T13:59:23.310112+0000","last_active":"2024-03-26T14:00:55.213601+0000","last_peered":"2024-03-26T14:00:55.213601+0000","last_clean":"2024-03-26T14:00:55.213601+0000","last_became_active":"2024-03-26T13:59:23.309728+0000","last_became_peered":"2024-03-26T13:59:23.309728+0000","last_unstale":"2024-03-26T14:00:55.213601+0000","last_undegraded":"2024-03-26T14:00:55.213601+0000","last_fullsized":"2024-03-26T14:00:55.213601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T14:46:55.283632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.213914+0000","last_change":"2024-03-26T13:59:23.318197+0000","last_active":"2024-03-26T14:00:55.213914+0000","last_peered":"2024-03-26T14:00:55.213914+0000","last_clean":"2024-03-26T14:00:55.213914+0000","last_became_active":"2024-03-26T13:59:23.317885+0000","last_became_peered":"2024-03-26T13:59:23.317885+0000","last_unstale":"2024-03-26T14:00:55.213914+0000","last_undegraded":"2024-03-26T14:00:55.213914+0000","last_fullsized":"2024-03-26T14:00:55.213914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:54:04.582387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217436+0000","last_change":"2024-03-26T13:59:23.314758+0000","last_active":"2024-03-26T14:00:55.217436+0000","last_peered":"2024-03-26T14:00:55.217436+0000","last_clean":"2024-03-26T14:00:55.217436+0000","last_became_active":"2024-03-26T13:59:23.311822+0000","last_became_peered":"2024-03-26T13:59:23.311822+0000","last_unstale":"2024-03-26T14:00:55.217436+0000","last_undegraded":"2024-03-26T14:00:55.217436+0000","last_fullsized":"2024-03-26T14:00:55.217436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:10:15.557257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221547+0000","last_change":"2024-03-26T13:59:23.312774+0000","last_active":"2024-03-26T14:00:55.221547+0000","last_peered":"2024-03-26T14:00:55.221547+0000","last_clean":"2024-03-26T14:00:55.221547+0000","last_became_active":"2024-03-26T13:59:23.312260+0000","last_became_peered":"2024-03-26T13:59:23.312260+0000","last_unstale":"2024-03-26T14:00:55.221547+0000","last_undegraded":"2024-03-26T14:00:55.221547+0000","last_fullsized":"2024-03-26T14:00:55.221547+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T23:36:51.981065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.218503+0000","last_change":"2024-03-26T13:59:23.299559+0000","last_active":"2024-03-26T14:00:55.218503+0000","last_peered":"2024-03-26T14:00:55.218503+0000","last_clean":"2024-03-26T14:00:55.218503+0000","last_became_active":"2024-03-26T13:59:23.299102+0000","last_became_peered":"2024-03-26T13:59:23.299102+0000","last_unstale":"2024-03-26T14:00:55.218503+0000","last_undegraded":"2024-03-26T14:00:55.218503+0000","last_fullsized":"2024-03-26T14:00:55.218503+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T15:40:24.427060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.207037+0000","last_change":"2024-03-26T13:59:23.307083+0000","last_active":"2024-03-26T14:00:55.207037+0000","last_peered":"2024-03-26T14:00:55.207037+0000","last_clean":"2024-03-26T14:00:55.207037+0000","last_became_active":"2024-03-26T13:59:23.306573+0000","last_became_peered":"2024-03-26T13:59:23.306573+0000","last_unstale":"2024-03-26T14:00:55.207037+0000","last_undegraded":"2024-03-26T14:00:55.207037+0000","last_fullsized":"2024-03-26T14:00:55.207037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:14:56.768749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.226954+0000","last_change":"2024-03-26T13:59:23.307267+0000","last_active":"2024-03-26T14:00:55.226954+0000","last_peered":"2024-03-26T14:00:55.226954+0000","last_clean":"2024-03-26T14:00:55.226954+0000","last_became_active":"2024-03-26T13:59:23.306880+0000","last_became_peered":"2024-03-26T13:59:23.306880+0000","last_unstale":"2024-03-26T14:00:55.226954+0000","last_undegraded":"2024-03-26T14:00:55.226954+0000","last_fullsized":"2024-03-26T14:00:55.226954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T17:40:53.383126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.206887+0000","last_change":"2024-03-26T13:59:23.306261+0000","last_active":"2024-03-26T14:00:55.206887+0000","last_peered":"2024-03-26T14:00:55.206887+0000","last_clean":"2024-03-26T14:00:55.206887+0000","last_became_active":"2024-03-26T13:59:23.305914+0000","last_became_peered":"2024-03-26T13:59:23.305914+0000","last_unstale":"2024-03-26T14:00:55.206887+0000","last_undegraded":"2024-03-26T14:00:55.206887+0000","last_fullsized":"2024-03-26T14:00:55.206887+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T16:29:49.048881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.433797+0000","last_change":"2024-03-26T13:59:23.305194+0000","last_active":"2024-03-26T14:00:55.433797+0000","last_peered":"2024-03-26T14:00:55.433797+0000","last_clean":"2024-03-26T14:00:55.433797+0000","last_became_active":"2024-03-26T13:59:23.304883+0000","last_became_peered":"2024-03-26T13:59:23.304883+0000","last_unstale":"2024-03-26T14:00:55.433797+0000","last_undegraded":"2024-03-26T14:00:55.433797+0000","last_fullsized":"2024-03-26T14:00:55.433797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T21:00:21.853559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.217146+0000","last_change":"2024-03-26T13:59:23.316194+0000","last_active":"2024-03-26T14:00:55.217146+0000","last_peered":"2024-03-26T14:00:55.217146+0000","last_clean":"2024-03-26T14:00:55.217146+0000","last_became_active":"2024-03-26T13:59:23.315752+0000","last_became_peered":"2024-03-26T13:59:23.315752+0000","last_unstale":"2024-03-26T14:00:55.217146+0000","last_undegraded":"2024-03-26T14:00:55.217146+0000","last_fullsized":"2024-03-26T14:00:55.217146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T22:19:29.161004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-03-26T14:00:55.221123+0000","last_change":"2024-03-26T13:59:54.346428+0000","last_active":"2024-03-26T14:00:55.221123+0000","last_peered":"2024-03-26T14:00:55.221123+0000","last_clean":"2024-03-26T14:00:55.221123+0000","last_became_active":"2024-03-26T13:59:54.345901+0000","last_became_peered":"2024-03-26T13:59:54.345901+0000","last_unstale":"2024-03-26T14:00:55.221123+0000","last_undegraded":"2024-03-26T14:00:55.221123+0000","last_fullsized":"2024-03-26T14:00:55.221123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-26T13:59:22.235940+0000","last_clean_scrub_stamp":"2024-03-26T13:59:22.235940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-27T20:31:17.041517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280308,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78169,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":964,"num_read_kb":569,"num_write":184,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":458752,"data_stored":237809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351030,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":522925,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612668,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1188321,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776195,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":523127,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037834,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":523047,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299473,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1093632,"data_stored":1120474,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462999,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":518303,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":591254,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":519691,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":593454,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":518497,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":523978,"data_compressed":26678,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724637,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1101824,"data_stored":1114646,"data_compressed":62333,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34011,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35655,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":75561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":75241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":75644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-26T14:02:29.207 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.458+0000 7f2963c00640 1 -- 172.21.15.137:0/157522319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c077c40 msgr2=0x7f295c0784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.458+0000 7f2963c00640 1 --2- 172.21.15.137:0/157522319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c077c40 0x7f295c0784e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2954009ca0 tx=0x7f295402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 -- 172.21.15.137:0/157522319 shutdown_connections 2024-03-26T14:02:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 --2- 172.21.15.137:0/157522319 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c077c40 0x7f295c0784e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 --2- 172.21.15.137:0/157522319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 0x7f295c077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 --2- 172.21.15.137:0/157522319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f295c076060 0x7f295c076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 -- 172.21.15.137:0/157522319 >> 172.21.15.137:0/157522319 conn(0x7f295c0fee50 msgr2=0x7f295c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 -- 172.21.15.137:0/157522319 shutdown_connections 2024-03-26T14:02:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.459+0000 7f2963c00640 1 -- 172.21.15.137:0/157522319 wait complete. 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 Processor -- start 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 -- start start 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 0x7f295c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 0x7f295c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f295c077c40 0x7f295c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c113820 con 0x7f295c076060 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1136a0 con 0x7f295c077c40 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2963c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1139a0 con 0x7f295c0772a0 2024-03-26T14:02:29.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.460+0000 7f2961200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 0x7f295c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 0x7f295c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60808/0 (socket says 172.21.15.137:60808) 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 -- 172.21.15.137:0/3914598977 learned_addr learned my addr 172.21.15.137:0/3914598977 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f295c077c40 0x7f295c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2953e00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 0x7f295c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 -- 172.21.15.137:0/3914598977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 msgr2=0x7f295c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 0x7f295c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 -- 172.21.15.137:0/3914598977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f295c077c40 msgr2=0x7f295c1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f295c077c40 0x7f295c1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 -- 172.21.15.137:0/3914598977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c1a5ad0 con 0x7f295c076060 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2953e00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 0x7f295c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f2961200640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 0x7f295c19deb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f294c00d950 tx=0x7f294c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.461+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c017030 con 0x7f295c076060 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.462+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f294c004020 con 0x7f295c076060 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.462+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c1a5dc0 con 0x7f295c076060 2024-03-26T14:02:29.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.462+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c0047a0 con 0x7f295c076060 2024-03-26T14:02:29.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.462+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1a6280 con 0x7f295c076060 2024-03-26T14:02:29.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.463+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c1103d0 con 0x7f295c076060 2024-03-26T14:02:29.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.465+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f294c0041d0 con 0x7f295c076060 2024-03-26T14:02:29.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.466+0000 7f295a000640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29400a2910 0x7f29400a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.466+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f294c0cf130 con 0x7f295c076060 2024-03-26T14:02:29.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.466+0000 7f2953e00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29400a2910 0x7f29400a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.467+0000 7f2953e00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29400a2910 0x7f29400a4dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2944005fd0 tx=0x7f29440094d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:29.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.469+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f294c09bce0 con 0x7f295c076060 2024-03-26T14:02:29.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:29 smithi137 ceph-mon[31539]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:29.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.669+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f295c02d320 con 0x7f295c076060 2024-03-26T14:02:29.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.673+0000 7f295a000640 1 -- 172.21.15.137:0/3914598977 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f294c09b5e0 con 0x7f295c076060 2024-03-26T14:02:29.671 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:29.671 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:29.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:29 smithi179 ceph-mon[35472]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:29.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29400a2910 msgr2=0x7f29400a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29400a2910 0x7f29400a4dd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2944005fd0 tx=0x7f29440094d0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 msgr2=0x7f295c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 0x7f295c19deb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f294c00d950 tx=0x7f294c00de10 comp rx=0 tx=0).stop 2024-03-26T14:02:29.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 shutdown_connections 2024-03-26T14:02:29.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29400a2910 0x7f29400a4dd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f295c077c40 0x7f295c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f295c0772a0 0x7f295c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 --2- 172.21.15.137:0/3914598977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f295c076060 0x7f295c19deb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 >> 172.21.15.137:0/3914598977 conn(0x7f295c0fee50 msgr2=0x7f295c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:29.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 shutdown_connections 2024-03-26T14:02:29.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.676+0000 7f2963c00640 1 -- 172.21.15.137:0/3914598977 wait complete. 2024-03-26T14:02:29.692 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:02:29.693 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:02:29.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:29 smithi159 ceph-mon[35501]: from='client.15612 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 -- 172.21.15.137:0/945569072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a00772a0 msgr2=0x7f07a0077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 --2- 172.21.15.137:0/945569072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a00772a0 0x7f07a0077700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0788009900 tx=0x7f078802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:29.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 -- 172.21.15.137:0/945569072 shutdown_connections 2024-03-26T14:02:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 --2- 172.21.15.137:0/945569072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a0077c40 0x7f07a00784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 --2- 172.21.15.137:0/945569072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a00772a0 0x7f07a0077700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 --2- 172.21.15.137:0/945569072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a0076060 0x7f07a0076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 -- 172.21.15.137:0/945569072 >> 172.21.15.137:0/945569072 conn(0x7f07a00fee50 msgr2=0x7f07a0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 -- 172.21.15.137:0/945569072 shutdown_connections 2024-03-26T14:02:29.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.910+0000 7f07a7400640 1 -- 172.21.15.137:0/945569072 wait complete. 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 Processor -- start 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 -- start start 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 0x7f07a019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a00772a0 0x7f07a019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a0077c40 0x7f07a01a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a0113860 con 0x7f07a0076060 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01136e0 con 0x7f07a00772a0 2024-03-26T14:02:29.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a7400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a01139e0 con 0x7f07a0077c40 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 0x7f07a019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 0x7f07a019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60836/0 (socket says 172.21.15.137:60836) 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.911+0000 7f07a4a00640 1 -- 172.21.15.137:0/3329916599 learned_addr learned my addr 172.21.15.137:0/3329916599 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f0797e00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a00772a0 0x7f07a019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a4a00640 1 -- 172.21.15.137:0/3329916599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a0077c40 msgr2=0x7f07a01a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a4a00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a0077c40 0x7f07a01a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a4a00640 1 -- 172.21.15.137:0/3329916599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a00772a0 msgr2=0x7f07a019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a4a00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a00772a0 0x7f07a019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a4a00640 1 -- 172.21.15.137:0/3329916599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a01a5b10 con 0x7f07a0076060 2024-03-26T14:02:29.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f0797e00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a00772a0 0x7f07a019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a4a00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 0x7f07a019def0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f079000d770 tx=0x7f079000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790017070 con 0x7f07a0076060 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07900054d0 con 0x7f07a0076060 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.912+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a01a5e00 con 0x7f07a0076060 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.913+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790004cf0 con 0x7f07a0076060 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.913+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a01a6270 con 0x7f07a0076060 2024-03-26T14:02:29.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.914+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a0110250 con 0x7f07a0076060 2024-03-26T14:02:29.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.916+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f0790004e90 con 0x7f07a0076060 2024-03-26T14:02:29.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.917+0000 7f079e000640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07840a2910 0x7f07840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:29.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.917+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f07900cea90 con 0x7f07a0076060 2024-03-26T14:02:29.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.917+0000 7f0797e00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07840a2910 0x7f07840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:29.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.918+0000 7f0797e00640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07840a2910 0x7f07840a4dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0788036040 tx=0x7f0788005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:29.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:29.922+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f079009b5c0 con 0x7f07a0076060 2024-03-26T14:02:30.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.119+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07a002d300 con 0x7f07a0076060 2024-03-26T14:02:30.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.120+0000 7f079e000640 1 -- 172.21.15.137:0/3329916599 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f079009aec0 con 0x7f07a0076060 2024-03-26T14:02:30.118 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:02:30.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07840a2910 msgr2=0x7f07840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07840a2910 0x7f07840a4dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0788036040 tx=0x7f0788005fd0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 msgr2=0x7f07a019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 0x7f07a019def0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f079000d770 tx=0x7f079000dc30 comp rx=0 tx=0).stop 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 shutdown_connections 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07840a2910 0x7f07840a4dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07a00772a0 0x7f07a019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07a0077c40 0x7f07a01a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 --2- 172.21.15.137:0/3329916599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07a0076060 0x7f07a019def0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 >> 172.21.15.137:0/3329916599 conn(0x7f07a00fee50 msgr2=0x7f07a0100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 shutdown_connections 2024-03-26T14:02:30.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.123+0000 7f07a7400640 1 -- 172.21.15.137:0/3329916599 wait complete. 2024-03-26T14:02:30.138 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.358+0000 7f1f18200640 1 -- 172.21.15.137:0/247436008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 msgr2=0x7f1f1010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.358+0000 7f1f18200640 1 --2- 172.21.15.137:0/247436008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f1010b4d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009900 tx=0x7f1efc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 -- 172.21.15.137:0/247436008 shutdown_connections 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 --2- 172.21.15.137:0/247436008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 0x7f1f10112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 --2- 172.21.15.137:0/247436008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f1010b4d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 --2- 172.21.15.137:0/247436008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f10109e70 0x7f1f1010a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 -- 172.21.15.137:0/247436008 >> 172.21.15.137:0/247436008 conn(0x7f1f10070650 msgr2=0x7f1f10072a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:30.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 -- 172.21.15.137:0/247436008 shutdown_connections 2024-03-26T14:02:30.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.359+0000 7f1f18200640 1 -- 172.21.15.137:0/247436008 wait complete. 2024-03-26T14:02:30.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 Processor -- start 2024-03-26T14:02:30.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 -- start start 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f10109e70 0x7f1f101a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f101a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 0x7f1f101a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10115a70 con 0x7f1f1010b050 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f101158f0 con 0x7f1f1010ba10 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f18200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10115bf0 con 0x7f1f10109e70 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f14e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f101a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f14e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f101a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60866/0 (socket says 172.21.15.137:60866) 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.360+0000 7f1f14e00640 1 -- 172.21.15.137:0/2080950977 learned_addr learned my addr 172.21.15.137:0/2080950977 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f16200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 0x7f1f101a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f15800640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f10109e70 0x7f1f101a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f14e00640 1 -- 172.21.15.137:0/2080950977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f10109e70 msgr2=0x7f1f101a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f14e00640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f10109e70 0x7f1f101a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f14e00640 1 -- 172.21.15.137:0/2080950977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 msgr2=0x7f1f101a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f14e00640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 0x7f1f101a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f14e00640 1 -- 172.21.15.137:0/2080950977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f101a7d60 con 0x7f1f1010b050 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f16200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 0x7f1f101a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.361+0000 7f1f14e00640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f101a0680 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1efc000c00 tx=0x7f1efc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.362+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc043020 con 0x7f1f1010b050 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.362+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1efc0054c0 con 0x7f1f1010b050 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.362+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc003d40 con 0x7f1f1010b050 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.362+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f101a7ff0 con 0x7f1f1010b050 2024-03-26T14:02:30.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.362+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f101a8460 con 0x7f1f1010b050 2024-03-26T14:02:30.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.363+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f10111260 con 0x7f1f1010b050 2024-03-26T14:02:30.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.365+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f1efc046020 con 0x7f1f1010b050 2024-03-26T14:02:30.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.366+0000 7f1f0e000640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1ef80a2890 0x7f1ef80a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.366+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1efc0f1390 con 0x7f1f1010b050 2024-03-26T14:02:30.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.367+0000 7f1f15800640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1ef80a2890 0x7f1ef80a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.367+0000 7f1f15800640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1ef80a2890 0x7f1ef80a4d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f04005fd0 tx=0x7f1f04005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:30.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.371+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1efc04d020 con 0x7f1f1010b050 2024-03-26T14:02:30.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:30 smithi137 ceph-mon[31539]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:30.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:30 smithi137 ceph-mon[31539]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:30.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:30 smithi137 ceph-mon[31539]: pgmap v462: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-26T14:02:30.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3914598977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:30.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3329916599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.568+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f1002d290 con 0x7f1f1010b050 2024-03-26T14:02:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.569+0000 7f1f0e000640 1 -- 172.21.15.137:0/2080950977 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1efc0be0f0 con 0x7f1f1010b050 2024-03-26T14:02:30.567 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1ef80a2890 msgr2=0x7f1ef80a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1ef80a2890 0x7f1ef80a4d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1f04005fd0 tx=0x7f1f04005850 comp rx=0 tx=0).stop 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 msgr2=0x7f1f101a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f101a0680 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1efc000c00 tx=0x7f1efc0042f0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 shutdown_connections 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1ef80a2890 0x7f1ef80a4d50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f1010ba10 0x7f1f101a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1f1010b050 0x7f1f101a0680 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 --2- 172.21.15.137:0/2080950977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f10109e70 0x7f1f101a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 >> 172.21.15.137:0/2080950977 conn(0x7f1f10070650 msgr2=0x7f1f1010e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 shutdown_connections 2024-03-26T14:02:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.572+0000 7f1f18200640 1 -- 172.21.15.137:0/2080950977 wait complete. 2024-03-26T14:02:30.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:02:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:30 smithi179 ceph-mon[35472]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:30.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:30 smithi179 ceph-mon[35472]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:30.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:30 smithi179 ceph-mon[35472]: pgmap v462: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-26T14:02:30.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3914598977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:30.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3329916599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:30 smithi159 ceph-mon[35501]: from='client.15618 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:30 smithi159 ceph-mon[35501]: from='client.15624 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-26T14:02:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:30 smithi159 ceph-mon[35501]: pgmap v462: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-26T14:02:30.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3914598977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:30.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3329916599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:02:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.808+0000 7f08c6c00640 1 -- 172.21.15.137:0/769198935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0109870 msgr2=0x7f08c0110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.808+0000 7f08c6c00640 1 --2- 172.21.15.137:0/769198935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0109870 0x7f08c0110100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009ca0 tx=0x7f08b802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 -- 172.21.15.137:0/769198935 shutdown_connections 2024-03-26T14:02:30.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 --2- 172.21.15.137:0/769198935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0108eb0 0x7f08c0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 --2- 172.21.15.137:0/769198935 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0109870 0x7f08c0110100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 --2- 172.21.15.137:0/769198935 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0107cc0 0x7f08c01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 -- 172.21.15.137:0/769198935 >> 172.21.15.137:0/769198935 conn(0x7f08c0076c60 msgr2=0x7f08c00790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 -- 172.21.15.137:0/769198935 shutdown_connections 2024-03-26T14:02:30.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.809+0000 7f08c6c00640 1 -- 172.21.15.137:0/769198935 wait complete. 2024-03-26T14:02:30.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 Processor -- start 2024-03-26T14:02:30.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 -- start start 2024-03-26T14:02:30.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 0x7f08c019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0108eb0 0x7f08c019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0109870 0x7f08c01a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01137f0 con 0x7f08c0107cc0 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0113670 con 0x7f08c0109870 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08c6c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0113970 con 0x7f08c0108eb0 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.810+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 0x7f08c019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 0x7f08c019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60884/0 (socket says 172.21.15.137:60884) 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 -- 172.21.15.137:0/2770001011 learned_addr learned my addr 172.21.15.137:0/2770001011 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08c4c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0109870 0x7f08c01a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08b7400640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0108eb0 0x7f08c019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 -- 172.21.15.137:0/2770001011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0108eb0 msgr2=0x7f08c019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0108eb0 0x7f08c019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 -- 172.21.15.137:0/2770001011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0109870 msgr2=0x7f08c01a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0109870 0x7f08c01a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 -- 172.21.15.137:0/2770001011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c01a5ab0 con 0x7f08c0107cc0 2024-03-26T14:02:30.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08b7400640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0108eb0 0x7f08c019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08c4c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0109870 0x7f08c01a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.811+0000 7f08bfe00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 0x7f08c019de90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f08b000d950 tx=0x7f08b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.812+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0017070 con 0x7f08c0107cc0 2024-03-26T14:02:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.812+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08b0009c30 con 0x7f08c0107cc0 2024-03-26T14:02:30.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.812+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c01a5da0 con 0x7f08c0107cc0 2024-03-26T14:02:30.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.812+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000f620 con 0x7f08c0107cc0 2024-03-26T14:02:30.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.812+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c01a6210 con 0x7f08c0107cc0 2024-03-26T14:02:30.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.813+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c010efe0 con 0x7f08c0107cc0 2024-03-26T14:02:30.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.815+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f08b001a070 con 0x7f08c0107cc0 2024-03-26T14:02:30.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.816+0000 7f08bd600640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08a40a2910 0x7f08a40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:30.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.816+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f08b00ceaa0 con 0x7f08c0107cc0 2024-03-26T14:02:30.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.816+0000 7f08b7400640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08a40a2910 0x7f08a40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:30.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.817+0000 7f08b7400640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08a40a2910 0x7f08a40a4dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08c019f3a0 tx=0x7f08a8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:30.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:30.821+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f08b0009de0 con 0x7f08c0107cc0 2024-03-26T14:02:31.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.018+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08c002d070 con 0x7f08c0107cc0 2024-03-26T14:02:31.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.018+0000 7f08bd600640 1 -- 172.21.15.137:0/2770001011 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08c002d070 con 0x7f08c0107cc0 2024-03-26T14:02:31.016 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:02:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08a40a2910 msgr2=0x7f08a40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08a40a2910 0x7f08a40a4dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08c019f3a0 tx=0x7f08a8005e50 comp rx=0 tx=0).stop 2024-03-26T14:02:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 msgr2=0x7f08c019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 0x7f08c019de90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f08b000d950 tx=0x7f08b000de10 comp rx=0 tx=0).stop 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 shutdown_connections 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f08a40a2910 0x7f08a40a4dd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f08c0109870 0x7f08c01a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f08c0108eb0 0x7f08c019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 --2- 172.21.15.137:0/2770001011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f08c0107cc0 0x7f08c019de90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 >> 172.21.15.137:0/2770001011 conn(0x7f08c0076c60 msgr2=0x7f08c010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 shutdown_connections 2024-03-26T14:02:31.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.022+0000 7f08c6c00640 1 -- 172.21.15.137:0/2770001011 wait complete. 2024-03-26T14:02:31.040 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 -- 172.21.15.137:0/341291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581038e0 msgr2=0x7f1858103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 --2- 172.21.15.137:0/341291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581038e0 0x7f1858103d00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 -- 172.21.15.137:0/341291107 shutdown_connections 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 --2- 172.21.15.137:0/341291107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105490 0x7f185810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 --2- 172.21.15.137:0/341291107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1858104ad0 0x7f1858104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 --2- 172.21.15.137:0/341291107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581038e0 0x7f1858103d00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 -- 172.21.15.137:0/341291107 >> 172.21.15.137:0/341291107 conn(0x7f18580fee50 msgr2=0x7f1858101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 -- 172.21.15.137:0/341291107 shutdown_connections 2024-03-26T14:02:31.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.255+0000 7f185ea00640 1 -- 172.21.15.137:0/341291107 wait complete. 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.256+0000 7f185ea00640 1 Processor -- start 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.256+0000 7f185ea00640 1 -- start start 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18581038e0 0x7f1858199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858104ad0 0x7f185819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 0x7f18581a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185810f430 con 0x7f1858105490 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185810f2b0 con 0x7f1858104ad0 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185810f5b0 con 0x7f18581038e0 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 0x7f18581a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 0x7f18581a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60900/0 (socket says 172.21.15.137:60900) 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 -- 172.21.15.137:0/1192660918 learned_addr learned my addr 172.21.15.137:0/1192660918 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:31.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f1857e00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18581038e0 0x7f1858199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f184f400640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858104ad0 0x7f185819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 -- 172.21.15.137:0/1192660918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18581038e0 msgr2=0x7f1858199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18581038e0 0x7f1858199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 -- 172.21.15.137:0/1192660918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858104ad0 msgr2=0x7f185819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858104ad0 0x7f185819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.257+0000 7f185ca00640 1 -- 172.21.15.137:0/1192660918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18581a1770 con 0x7f1858105490 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.258+0000 7f185ca00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 0x7f18581a1100 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f185000eaa0 tx=0x7f185000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.258+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185000cdb0 con 0x7f1858105490 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.258+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1850005020 con 0x7f1858105490 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.258+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18581a1a60 con 0x7f1858105490 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.258+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500116d0 con 0x7f1858105490 2024-03-26T14:02:31.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.258+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18581a1fb0 con 0x7f1858105490 2024-03-26T14:02:31.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.259+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185810ad20 con 0x7f1858105490 2024-03-26T14:02:31.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.261+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f18500051d0 con 0x7f1858105490 2024-03-26T14:02:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.263+0000 7f1855600640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f183c0a2910 0x7f183c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.263+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f18500cf5d0 con 0x7f1858105490 2024-03-26T14:02:31.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.263+0000 7f1857e00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f183c0a2910 0x7f183c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.264+0000 7f1857e00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f183c0a2910 0x7f183c0a4dd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f184802c240 tx=0x7f1848005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:31.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.266+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1850016030 con 0x7f1858105490 2024-03-26T14:02:31.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.471+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f185802d2c0 con 0x7f1858105490 2024-03-26T14:02:31.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.472+0000 7f1855600640 1 -- 172.21.15.137:0/1192660918 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7f185009c180 con 0x7f1858105490 2024-03-26T14:02:31.470 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:02:31.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f183c0a2910 msgr2=0x7f183c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f183c0a2910 0x7f183c0a4dd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f184802c240 tx=0x7f1848005fd0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 msgr2=0x7f18581a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 0x7f18581a1100 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f185000eaa0 tx=0x7f185000ef60 comp rx=0 tx=0).stop 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 shutdown_connections 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f183c0a2910 0x7f183c0a4dd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858104ad0 0x7f185819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1858105490 0x7f18581a1100 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 --2- 172.21.15.137:0/1192660918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f18581038e0 0x7f1858199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 >> 172.21.15.137:0/1192660918 conn(0x7f18580fee50 msgr2=0x7f1858107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 shutdown_connections 2024-03-26T14:02:31.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.475+0000 7f185ea00640 1 -- 172.21.15.137:0/1192660918 wait complete. 2024-03-26T14:02:31.491 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:02:31.566 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:31 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2080950977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:31.566 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:31 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2770001011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:31.566 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:31 smithi137 ceph-mon[31539]: pgmap v463: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-26T14:02:31.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:31 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2080950977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:31.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:31 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2770001011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:31.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:31 smithi179 ceph-mon[35472]: pgmap v463: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-26T14:02:31.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:31 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2080950977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:02:31.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:31 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2770001011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:02:31.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:31 smithi159 ceph-mon[35501]: pgmap v463: 129 pgs: 129 active+clean; 653 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 -- 172.21.15.137:0/4120469257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db80756e0 msgr2=0x7f0db8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 --2- 172.21.15.137:0/4120469257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db80756e0 0x7f0db8073b30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009900 tx=0x7f0da802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 -- 172.21.15.137:0/4120469257 shutdown_connections 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 --2- 172.21.15.137:0/4120469257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0db8109570 0x7f0db810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 --2- 172.21.15.137:0/4120469257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db8074100 0x7f0db8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 --2- 172.21.15.137:0/4120469257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db80756e0 0x7f0db8073b30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 -- 172.21.15.137:0/4120469257 >> 172.21.15.137:0/4120469257 conn(0x7f0db80fcc80 msgr2=0x7f0db80ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 -- 172.21.15.137:0/4120469257 shutdown_connections 2024-03-26T14:02:31.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.712+0000 7f0dbd200640 1 -- 172.21.15.137:0/4120469257 wait complete. 2024-03-26T14:02:31.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.713+0000 7f0dbd200640 1 Processor -- start 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.713+0000 7f0dbd200640 1 -- start start 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 0x7f0db8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0db80756e0 0x7f0db819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0dbd200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db8109570 0x7f0db81a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0dbd200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810f410 con 0x7f0db8074100 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0dbd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810f290 con 0x7f0db80756e0 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0dbd200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db810f590 con 0x7f0db8109570 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 0x7f0db8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 0x7f0db8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60932/0 (socket says 172.21.15.137:60932) 2024-03-26T14:02:31.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 -- 172.21.15.137:0/3189811221 learned_addr learned my addr 172.21.15.137:0/3189811221 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5400640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0db80756e0 0x7f0db819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db6800640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db8109570 0x7f0db81a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 -- 172.21.15.137:0/3189811221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db8109570 msgr2=0x7f0db81a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db8109570 0x7f0db81a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 -- 172.21.15.137:0/3189811221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0db80756e0 msgr2=0x7f0db819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0db80756e0 0x7f0db819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.714+0000 7f0db5e00640 1 -- 172.21.15.137:0/3189811221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db81a1750 con 0x7f0db8074100 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.715+0000 7f0db5e00640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 0x7f0db8199b30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0da8000c00 tx=0x7f0da80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.715+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8043020 con 0x7f0db8074100 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.715+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0da80054c0 con 0x7f0db8074100 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.715+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8003d70 con 0x7f0db8074100 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.715+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db81a19e0 con 0x7f0db8074100 2024-03-26T14:02:31.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.715+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db81a1ee0 con 0x7f0db8074100 2024-03-26T14:02:31.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.716+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db81082a0 con 0x7f0db8074100 2024-03-26T14:02:31.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.718+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f0da8046020 con 0x7f0db8074100 2024-03-26T14:02:31.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.721+0000 7f0daea00640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0d9c0a2910 0x7f0d9c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:31.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.721+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0da80f1520 con 0x7f0db8074100 2024-03-26T14:02:31.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.721+0000 7f0db5400640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0d9c0a2910 0x7f0d9c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:31.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.722+0000 7f0db5400640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0d9c0a2910 0x7f0d9c0a4dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0db819b040 tx=0x7f0da400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:31.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.725+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0da804d020 con 0x7f0db8074100 2024-03-26T14:02:31.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.962+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0db802d070 con 0x7f0db8074100 2024-03-26T14:02:31.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.963+0000 7f0daea00640 1 -- 172.21.15.137:0/3189811221 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f0da80be280 con 0x7f0db8074100 2024-03-26T14:02:31.961 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:31.961 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:02:31.961 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0d9c0a2910 msgr2=0x7f0d9c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0d9c0a2910 0x7f0d9c0a4dd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0db819b040 tx=0x7f0da400b390 comp rx=0 tx=0).stop 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 msgr2=0x7f0db8199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 0x7f0db8199b30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0da8000c00 tx=0x7f0da80042f0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 shutdown_connections 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0d9c0a2910 0x7f0d9c0a4dd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0db80756e0 0x7f0db819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0db8109570 0x7f0db81a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 --2- 172.21.15.137:0/3189811221 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0db8074100 0x7f0db8199b30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:31.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.966+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 >> 172.21.15.137:0/3189811221 conn(0x7f0db80fcc80 msgr2=0x7f0db8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:31.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.967+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 shutdown_connections 2024-03-26T14:02:31.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:31.967+0000 7f0dbd200640 1 -- 172.21.15.137:0/3189811221 wait complete. 2024-03-26T14:02:31.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:02:32.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.201+0000 7fc4c9a00640 1 -- 172.21.15.137:0/606182306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c4107b00 msgr2=0x7fc4c4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.201+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/606182306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c4107b00 0x7fc4c4107f20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009900 tx=0x7fc4b402bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:32.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 -- 172.21.15.137:0/606182306 shutdown_connections 2024-03-26T14:02:32.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/606182306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4c4108cf0 0x7fc4c4109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/606182306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c41096b0 0x7fc4c410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/606182306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c4107b00 0x7fc4c4107f20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 -- 172.21.15.137:0/606182306 >> 172.21.15.137:0/606182306 conn(0x7fc4c4076ab0 msgr2=0x7fc4c4078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 -- 172.21.15.137:0/606182306 shutdown_connections 2024-03-26T14:02:32.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.202+0000 7fc4c9a00640 1 -- 172.21.15.137:0/606182306 wait complete. 2024-03-26T14:02:32.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 Processor -- start 2024-03-26T14:02:32.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 -- start start 2024-03-26T14:02:32.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4c4107b00 0x7fc4c419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4108cf0 0x7fc4c419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 0x7fc4c41a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4113680 con 0x7fc4c41096b0 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4113500 con 0x7fc4c4107b00 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4113800 con 0x7fc4c4108cf0 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.203+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 0x7fc4c41a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 0x7fc4c41a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60954/0 (socket says 172.21.15.137:60954) 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 -- 172.21.15.137:0/446860937 learned_addr learned my addr 172.21.15.137:0/446860937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c3e00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4c4107b00 0x7fc4c419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4bb400640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4108cf0 0x7fc4c419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 -- 172.21.15.137:0/446860937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4108cf0 msgr2=0x7fc4c419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4108cf0 0x7fc4c419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 -- 172.21.15.137:0/446860937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4c4107b00 msgr2=0x7fc4c419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4c4107b00 0x7fc4c419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 -- 172.21.15.137:0/446860937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c41a5950 con 0x7fc4c41096b0 2024-03-26T14:02:32.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.204+0000 7fc4c9000640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 0x7fc4c41a52b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00e970 tx=0x7fc4bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.205+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00ccc0 con 0x7fc4c41096b0 2024-03-26T14:02:32.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.205+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc4bc00ce60 con 0x7fc4c41096b0 2024-03-26T14:02:32.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.205+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc010430 con 0x7fc4c41096b0 2024-03-26T14:02:32.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.205+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c41a5c40 con 0x7fc4c41096b0 2024-03-26T14:02:32.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.205+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c41a61b0 con 0x7fc4c41096b0 2024-03-26T14:02:32.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.206+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c410ee20 con 0x7fc4c41096b0 2024-03-26T14:02:32.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.208+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fc4bc004f90 con 0x7fc4c41096b0 2024-03-26T14:02:32.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.209+0000 7fc4c1600640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4a80a2910 0x7fc4a80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.209+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc4bc0cf380 con 0x7fc4c41096b0 2024-03-26T14:02:32.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.210+0000 7fc4c3e00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4a80a2910 0x7fc4a80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.210+0000 7fc4c3e00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4a80a2910 0x7fc4a80a4dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc4b402c240 tx=0x7fc4b402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.213+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc4bc09bfb0 con 0x7fc4c41096b0 2024-03-26T14:02:32.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.417+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4c402d2c0 con 0x7fc4c41096b0 2024-03-26T14:02:32.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.421+0000 7fc4c1600640 1 -- 172.21.15.137:0/446860937 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7fc4bc09b8b0 con 0x7fc4c41096b0 2024-03-26T14:02:32.420 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:02:32.420 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4a80a2910 msgr2=0x7fc4a80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4a80a2910 0x7fc4a80a4dd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc4b402c240 tx=0x7fc4b402c7e0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 msgr2=0x7fc4c41a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 0x7fc4c41a52b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00e970 tx=0x7fc4bc00ed70 comp rx=0 tx=0).stop 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 shutdown_connections 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc4a80a2910 0x7fc4a80a4dd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4c41096b0 0x7fc4c41a52b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4c4108cf0 0x7fc4c419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 --2- 172.21.15.137:0/446860937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4c4107b00 0x7fc4c419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 >> 172.21.15.137:0/446860937 conn(0x7fc4c4076ab0 msgr2=0x7fc4c410bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 shutdown_connections 2024-03-26T14:02:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.424+0000 7fc4c9a00640 1 -- 172.21.15.137:0/446860937 wait complete. 2024-03-26T14:02:32.442 DEBUG:tasks.fwd_scrub:Yielding 2024-03-26T14:02:32.442 INFO:teuthology.run_tasks:Running task workunit... 2024-03-26T14:02:32.454 INFO:tasks.workunit:Pulling workunits from ref 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T14:02:32.454 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-26T14:02:32.454 INFO:tasks.workunit:timeout=3h 2024-03-26T14:02:32.454 INFO:tasks.workunit:cleanup=True 2024-03-26T14:02:32.455 DEBUG:teuthology.orchestra.run.smithi137:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:32.455 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-03-26T14:02:32.455 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:02:32.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:02:32.455 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-03-26 13:59:51.251212145 +0000 2024-03-26T14:02:32.500 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-03-26 13:59:51.251212145 +0000 2024-03-26T14:02:32.501 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-03-26 14:02:21.251427707 +0000 2024-03-26T14:02:32.501 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-03-26T14:02:32.501 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-26T14:02:32.501 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-26T14:02:32.608 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1192660918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:32.608 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3189811221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:32.634 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3be9a5c9bc793e11e2800a8c0c696e8b46742033 2024-03-26T14:02:32.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.658+0000 7f9ad6c00640 1 -- 172.21.15.137:0/3946370138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0104ad0 msgr2=0x7f9ad0104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.658+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/3946370138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0104ad0 0x7f9ad0104f50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009900 tx=0x7f9ab802bc30 comp rx=0 tx=0).stop 2024-03-26T14:02:32.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 -- 172.21.15.137:0/3946370138 shutdown_connections 2024-03-26T14:02:32.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/3946370138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad0105490 0x7f9ad010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/3946370138 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0104ad0 0x7f9ad0104f50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/3946370138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad01038e0 0x7f9ad0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 -- 172.21.15.137:0/3946370138 >> 172.21.15.137:0/3946370138 conn(0x7f9ad00fee50 msgr2=0x7f9ad0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 -- 172.21.15.137:0/3946370138 shutdown_connections 2024-03-26T14:02:32.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.660+0000 7f9ad6c00640 1 -- 172.21.15.137:0/3946370138 wait complete. 2024-03-26T14:02:32.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 Processor -- start 2024-03-26T14:02:32.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 -- start start 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 0x7f9ad0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad0104ad0 0x7f9ad019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad0105490 0x7f9ad01a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad010f440 con 0x7f9ad01038e0 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad010f2c0 con 0x7f9ad0105490 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9ad6c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad010f5c0 con 0x7f9ad0104ad0 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.661+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 0x7f9ad0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 0x7f9ad0199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60960/0 (socket says 172.21.15.137:60960) 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 -- 172.21.15.137:0/1244477861 learned_addr learned my addr 172.21.15.137:0/1244477861 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad0105490 0x7f9ad01a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9ac7400640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad0104ad0 0x7f9ad019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 -- 172.21.15.137:0/1244477861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad0104ad0 msgr2=0x7f9ad019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad0104ad0 0x7f9ad019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 -- 172.21.15.137:0/1244477861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad0105490 msgr2=0x7f9ad01a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad0105490 0x7f9ad01a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 -- 172.21.15.137:0/1244477861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad01a1740 con 0x7f9ad01038e0 2024-03-26T14:02:32.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad0105490 0x7f9ad01a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:32.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 0x7f9ad0199b20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000ebc0 tx=0x7f9ac000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.662+0000 7f9acd600640 1 -- 172.21.15.137:0/1244477861 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0003b70 con 0x7f9ad01038e0 2024-03-26T14:02:32.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.663+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad01a1a30 con 0x7f9ad01038e0 2024-03-26T14:02:32.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.663+0000 7f9acd600640 1 -- 172.21.15.137:0/1244477861 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ac0003d10 con 0x7f9ad01038e0 2024-03-26T14:02:32.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.663+0000 7f9acd600640 1 -- 172.21.15.137:0/1244477861 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0019820 con 0x7f9ad01038e0 2024-03-26T14:02:32.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.663+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad01a1fd0 con 0x7f9ad01038e0 2024-03-26T14:02:32.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 Processor -- start 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 -- start start 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c2c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c31590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b7b91eb0 con 0x5628b7c31c00 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b7b3aa60 con 0x5628b7c2c0a0 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9ad9969740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b7b8df50 con 0x5628b7c2ca90 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.667+0000 7f9acd600640 1 -- 172.21.15.137:0/1244477861 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f9ac00199c0 con 0x7f9ad01038e0 2024-03-26T14:02:32.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ac7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ad4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c2c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ad4c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c2c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35484/0 (socket says 172.21.15.137:35484) 2024-03-26T14:02:32.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ad4c00640 1 -- 172.21.15.137:0/3122024097 learned_addr learned my addr 172.21.15.137:0/3122024097 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c31590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ac7400640 1 -- 172.21.15.137:0/3122024097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 msgr2=0x5628b7c31590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ac7400640 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c31590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ac7400640 1 -- 172.21.15.137:0/3122024097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 msgr2=0x5628b7c2c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ac7400640 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c2c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9ac7400640 1 -- 172.21.15.137:0/3122024097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b796e7e0 con 0x5628b7c31c00 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.668+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c31590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9ac7400640 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c34090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9ab802c170 tx=0x7f9ab80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705a80f04a261283 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9ac5200640 1 -- 172.21.15.137:0/3122024097 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8043020 con 0x5628b7c31c00 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9ac5200640 1 -- 172.21.15.137:0/3122024097 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ab8004040 con 0x5628b7c31c00 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9ac5200640 1 -- 172.21.15.137:0/3122024097 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80055f0 con 0x5628b7c31c00 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9acd600640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ab40a2910 0x7f9ab40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9acd600640 1 -- 172.21.15.137:0/1244477861 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9ac00cf920 con 0x7f9ad01038e0 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.669+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ab40a2910 0x7f9ab40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 -- 172.21.15.137:0/3122024097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 msgr2=0x5628b7c34090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c34090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9ab802c170 tx=0x7f9ab80095e0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 -- 172.21.15.137:0/3122024097 shutdown_connections 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c34090 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c31590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3122024097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c2c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 -- 172.21.15.137:0/3122024097 >> 172.21.15.137:0/3122024097 conn(0x5628b7c27610 msgr2=0x5628b7c27a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 -- 172.21.15.137:0/3122024097 shutdown_connections 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ab40a2910 0x7f9ab40a4dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005fd0 tx=0x7f9ac800b5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.670+0000 7f9ad9969740 1 -- 172.21.15.137:0/3122024097 wait complete. 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 -- start start 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 Processor -- start 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c36730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c3d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b7c34b50 con 0x5628b7c31c00 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b7b4deb0 con 0x5628b7c2c0a0 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.671+0000 7f9ad9969740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628b7c34cd0 con 0x5628b7c2ca90 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad9969740 10 client.0 _ll_register_callbacks cb 0x5628b7b42150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c3d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c3d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60976/0 (socket says 172.21.15.137:60976) 2024-03-26T14:02:32.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 -- 172.21.15.137:0/3012241234 learned_addr learned my addr 172.21.15.137:0/3012241234 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ac7400640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c36730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 -- 172.21.15.137:0/3012241234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 msgr2=0x5628b7c36730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c36730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 -- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 msgr2=0x5628b7c361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c361f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 -- 172.21.15.137:0/3012241234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b79cef70 con 0x5628b7c31c00 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c3d770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80107f0 tx=0x7f9ac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a2292a13f92e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.672+0000 7f9abd600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9abd600640 1 -- 172.21.15.137:0/3012241234 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800fa10 con 0x5628b7c31c00 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9abd600640 1 -- 172.21.15.137:0/3012241234 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9ac800fbb0 con 0x5628b7c31c00 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ab8004040 con 0x5628b7c31c00 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9abd600640 1 -- 172.21.15.137:0/3012241234 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ac8030050 con 0x5628b7c31c00 2024-03-26T14:02:32.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9ad9969740 10 client.15678 fetch_fsmap learned FSMap version 24 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5628b7866170 con 0x5628b7c31c00 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.673+0000 7f9abd600640 1 -- 172.21.15.137:0/3012241234 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ab8004040 con 0x5628b7c31c00 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9abd600640 1 -- 172.21.15.137:0/3012241234 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9ac801f3f0 con 0x5628b7c31c00 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9ad9969740 10 client.15678 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9ad9969740 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9ad9969740 20 client.15678 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c42040 0x5628b7c444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9ad9969740 4 client.15678 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.674+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628b7b9f3f0 con 0x5628b7c42040 2024-03-26T14:02:32.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.675+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c42040 0x5628b7c444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1192660918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:32.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3189811221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.675+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c42040 0x5628b7c444d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9148608f085776e7 server_cookie=14df6dfa5ddd50f9 in_seq=0 out_seq=0 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.675+0000 7f9abd600640 10 client.15678 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.677+0000 7f9abd600640 1 -- 172.21.15.137:0/3012241234 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5628b7b9f3f0 con 0x5628b7c42040 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.677+0000 7f9abd600640 10 client.15678 handle_command_reply: tid=0 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 client.15678 shutdown 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 msgr2=0x5628b7c3d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c3d770 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80107f0 tx=0x7f9ac80046c0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 shutdown_connections 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c42040 0x5628b7c444d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c31c00 0x5628b7c3d770 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c36730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3012241234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c2c0a0 0x5628b7c361f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 >> 172.21.15.137:0/3012241234 conn(0x5628b7c27610 msgr2=0x5628b7c26830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9abd600640 0 client.15678 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 shutdown_connections 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 1 -- 172.21.15.137:0/3012241234 wait complete. 2024-03-26T14:02:32.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.678+0000 7f9ad9969740 20 client.15678 trim_cache size 0 max 16384 2024-03-26T14:02:32.681 INFO:tasks.workunit.client.0.smithi137.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-26T14:02:32.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 Processor -- start 2024-03-26T14:02:32.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 -- start start 2024-03-26T14:02:32.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 0x5628b7c26610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c3d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c48dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01ae870 con 0x5628b7c2c0a0 2024-03-26T14:02:32.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01ac370 con 0x5628b7c31c00 2024-03-26T14:02:32.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad9969740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01aa3f0 con 0x5628b7c2ca90 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 0x5628b7c26610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9acfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 0x5628b7c26610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60982/0 (socket says 172.21.15.137:60982) 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9acfe00640 1 -- 172.21.15.137:0/3365665154 learned_addr learned my addr 172.21.15.137:0/3365665154 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.696+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c48dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9ac7400640 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c3d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9acfe00640 1 -- 172.21.15.137:0/3365665154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 msgr2=0x5628b7c3d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c3d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9acfe00640 1 -- 172.21.15.137:0/3365665154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 msgr2=0x5628b7c48dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c48dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9acfe00640 1 -- 172.21.15.137:0/3365665154 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628b7c26f60 con 0x5628b7c2c0a0 2024-03-26T14:02:32.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9acfe00640 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 0x5628b7c26610 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0004820 tx=0x7f9ac00d1db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de20f8ef150654b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9abfe00640 1 -- 172.21.15.137:0/3365665154 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00b1030 con 0x5628b7c2c0a0 2024-03-26T14:02:32.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9abfe00640 1 -- 172.21.15.137:0/3365665154 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ac00b4e50 con 0x5628b7c2c0a0 2024-03-26T14:02:32.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.697+0000 7f9abfe00640 1 -- 172.21.15.137:0/3365665154 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0098400 con 0x5628b7c2c0a0 2024-03-26T14:02:32.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 -- 172.21.15.137:0/3365665154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 msgr2=0x5628b7c26610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 0x5628b7c26610 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0004820 tx=0x7f9ac00d1db0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 -- 172.21.15.137:0/3365665154 shutdown_connections 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c48dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2ca90 0x5628b7c3d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 --2- 172.21.15.137:0/3365665154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2c0a0 0x5628b7c26610 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 -- 172.21.15.137:0/3365665154 >> 172.21.15.137:0/3365665154 conn(0x5628b7c27610 msgr2=0x5628b7c2b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 -- 172.21.15.137:0/3365665154 shutdown_connections 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.698+0000 7f9ad9969740 1 -- 172.21.15.137:0/3365665154 wait complete. 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 -- start start 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 Processor -- start 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2c0a0 0x5628b7c3c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 0x5628b7c3cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c9da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01aa9f0 con 0x5628b7c2ca90 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01abd10 con 0x5628b7c31c00 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.699+0000 7f9ad9969740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad01ab050 con 0x5628b7c2c0a0 2024-03-26T14:02:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ad9969740 10 client.0 _ll_register_callbacks cb 0x7f9ac800bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 0x5628b7c3cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 0x5628b7c3cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60996/0 (socket says 172.21.15.137:60996) 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 -- 172.21.15.137:0/1192141605 learned_addr learned my addr 172.21.15.137:0/1192141605 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ad4c00640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c9da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2c0a0 0x5628b7c3c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 -- 172.21.15.137:0/1192141605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2c0a0 msgr2=0x5628b7c3c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2c0a0 0x5628b7c3c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 -- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 msgr2=0x5628b7c9da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c9da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 -- 172.21.15.137:0/1192141605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628b7c22c20 con 0x5628b7c2ca90 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2c0a0 0x5628b7c3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9ac7400640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 0x5628b7c3cdf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8005c40 tx=0x7f9ab8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeecb0b02ed7e37e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.700+0000 7f9abd600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:32.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9abd600640 1 -- 172.21.15.137:0/1192141605 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab804c070 con 0x5628b7c2ca90 2024-03-26T14:02:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ac00b4e50 con 0x5628b7c2ca90 2024-03-26T14:02:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9abd600640 1 -- 172.21.15.137:0/1192141605 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9ab8005240 con 0x5628b7c2ca90 2024-03-26T14:02:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9abd600640 1 -- 172.21.15.137:0/1192141605 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab8057050 con 0x5628b7c2ca90 2024-03-26T14:02:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9ad9969740 10 client.15684 fetch_fsmap learned FSMap version 24 2024-03-26T14:02:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5628b7c9f210 con 0x5628b7c2ca90 2024-03-26T14:02:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9abd600640 1 -- 172.21.15.137:0/1192141605 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ac00b4e50 con 0x5628b7c2ca90 2024-03-26T14:02:32.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1192660918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:02:32.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3189811221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:02:32.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.701+0000 7f9abd600640 1 -- 172.21.15.137:0/1192141605 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9ab80035f0 con 0x5628b7c2ca90 2024-03-26T14:02:32.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9ad9969740 10 client.15684 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:02:32.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9ad9969740 10 client.15684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9ad9969740 20 client.15684 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9ad9969740 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c9f7b0 0x5628b7ca1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9ad9969740 4 client.15684 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5628b7b99740 con 0x5628b7c9f7b0 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.702+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c9f7b0 0x5628b7ca1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.703+0000 7f9acfe00640 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c9f7b0 0x5628b7ca1bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1ead94e9b2bfb9 server_cookie=dd7b63d729c59678 in_seq=0 out_seq=0 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.703+0000 7f9abd600640 10 client.15684 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9abd600640 1 -- 172.21.15.137:0/1192141605 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5628b7b99740 con 0x5628b7c9f7b0 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9abd600640 10 client.15684 handle_command_reply: tid=0 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 client.15684 shutdown 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 msgr2=0x5628b7c3cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 0x5628b7c3cdf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8005c40 tx=0x7f9ab8004270 comp rx=0 tx=0).stop 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 shutdown_connections 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5628b7c9f7b0 0x5628b7ca1bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628b7c31c00 0x5628b7c9da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5628b7c2ca90 0x5628b7c3cdf0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 --2- 172.21.15.137:0/1192141605 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5628b7c2c0a0 0x5628b7c3c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9abd600640 0 client.15684 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.705+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 >> 172.21.15.137:0/1192141605 conn(0x5628b7c27610 msgr2=0x5628b7c2d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.706+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 shutdown_connections 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.706+0000 7f9ad9969740 1 -- 172.21.15.137:0/1192141605 wait complete. 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.706+0000 7f9ad9969740 20 client.15684 trim_cache size 0 max 16384 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:02:32.705 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "c7c07a52-7149-47b7-a9eb-9480efd56dc4", 2024-03-26T14:02:32.706 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:02:32.706 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:02:32.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ab40a2910 msgr2=0x7f9ab40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ab40a2910 0x7f9ab40a4dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005fd0 tx=0x7f9ac800b5e0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 msgr2=0x7f9ad0199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 0x7f9ad0199b20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000ebc0 tx=0x7f9ac000c6a0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 shutdown_connections 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9ab40a2910 0x7f9ab40a4dd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ad0105490 0x7f9ad01a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ad0104ad0 0x7f9ad019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 --2- 172.21.15.137:0/1244477861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad01038e0 0x7f9ad0199b20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 >> 172.21.15.137:0/1244477861 conn(0x7f9ad00fee50 msgr2=0x7f9ad0107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 shutdown_connections 2024-03-26T14:02:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.707+0000 7f9ad6c00640 1 -- 172.21.15.137:0/1244477861 wait complete. 2024-03-26T14:02:32.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:02:32.728 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:02:32.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.970+0000 7f6abec00640 1 -- 172.21.15.137:0/2947501286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab8100910 msgr2=0x7f6ab8100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.970+0000 7f6abec00640 1 --2- 172.21.15.137:0/2947501286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab8100910 0x7f6ab8100d70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009900 tx=0x7f6aa402bcd0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 -- 172.21.15.137:0/2947501286 shutdown_connections 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 --2- 172.21.15.137:0/2947501286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab81012b0 0x7f6ab8101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 --2- 172.21.15.137:0/2947501286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab8100910 0x7f6ab8100d70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 --2- 172.21.15.137:0/2947501286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab80fffb0 0x7f6ab81003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 -- 172.21.15.137:0/2947501286 >> 172.21.15.137:0/2947501286 conn(0x7f6ab80fbb30 msgr2=0x7f6ab80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 -- 172.21.15.137:0/2947501286 shutdown_connections 2024-03-26T14:02:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.971+0000 7f6abec00640 1 -- 172.21.15.137:0/2947501286 wait complete. 2024-03-26T14:02:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 Processor -- start 2024-03-26T14:02:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 -- start start 2024-03-26T14:02:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 0x7f6ab8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab8100910 0x7f6ab819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab81012b0 0x7f6ab81a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab810f440 con 0x7f6ab80fffb0 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab810f2c0 con 0x7f6ab8100910 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.972+0000 7f6abec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab810f5c0 con 0x7f6ab81012b0 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 0x7f6ab8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 0x7f6ab8199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32786/0 (socket says 172.21.15.137:32786) 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 -- 172.21.15.137:0/692029953 learned_addr learned my addr 172.21.15.137:0/692029953 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7400640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab8100910 0x7f6ab819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 -- 172.21.15.137:0/692029953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab81012b0 msgr2=0x7f6ab81a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab81012b0 0x7f6ab81a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 -- 172.21.15.137:0/692029953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab8100910 msgr2=0x7f6ab819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab8100910 0x7f6ab819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 -- 172.21.15.137:0/692029953 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab81a1790 con 0x7f6ab80fffb0 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7400640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab8100910 0x7f6ab819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.973+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 0x7f6ab8199b70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800d770 tx=0x7f6aa800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.974+0000 7f6ab4c00640 1 -- 172.21.15.137:0/692029953 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8017070 con 0x7f6ab80fffb0 2024-03-26T14:02:32.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.974+0000 7f6ab4c00640 1 -- 172.21.15.137:0/692029953 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6aa80054d0 con 0x7f6ab80fffb0 2024-03-26T14:02:32.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.974+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab81a1a80 con 0x7f6ab80fffb0 2024-03-26T14:02:32.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.974+0000 7f6ab4c00640 1 -- 172.21.15.137:0/692029953 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8004cf0 con 0x7f6ab80fffb0 2024-03-26T14:02:32.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.974+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab81a1fa0 con 0x7f6ab80fffb0 2024-03-26T14:02:32.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.977+0000 7f6ac1980740 1 Processor -- start 2024-03-26T14:02:32.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.977+0000 7f6ab4c00640 1 -- 172.21.15.137:0/692029953 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6aa8004e90 con 0x7f6ab80fffb0 2024-03-26T14:02:32.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.977+0000 7f6ac1980740 1 -- start start 2024-03-26T14:02:32.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb872a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb8c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 0x564dbfb8ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ac1980740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbfaec250 con 0x564dbfb8c9e0 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ac1980740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbfa94a30 con 0x564dbfb87870 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ac1980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbfae82f0 con 0x564dbfb86e80 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 0x564dbfb8ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 0x564dbfb8ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32792/0 (socket says 172.21.15.137:32792) 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ab7400640 1 -- 172.21.15.137:0/163674261 learned_addr learned my addr 172.21.15.137:0/163674261 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb8c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.978+0000 7f6abcc00640 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb872a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab7400640 1 -- 172.21.15.137:0/163674261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 msgr2=0x564dbfb872a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab7400640 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb872a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab7400640 1 -- 172.21.15.137:0/163674261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 msgr2=0x564dbfb8c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab7400640 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb8c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab7400640 1 -- 172.21.15.137:0/163674261 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dbfa7f690 con 0x564dbfb8c9e0 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab4c00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a9c0a2910 0x7f6a9c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab7400640 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 0x564dbfb8ee70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4000c00 tx=0x7f6aa4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6424322378215e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6ab4c00640 1 -- 172.21.15.137:0/692029953 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6aa80cea90 con 0x7f6ab80fffb0 2024-03-26T14:02:32.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6abcc00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a9c0a2910 0x7f6a9c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6aad200640 1 -- 172.21.15.137:0/163674261 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4004400 con 0x564dbfb8c9e0 2024-03-26T14:02:32.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6aad200640 1 -- 172.21.15.137:0/163674261 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6aa402ce30 con 0x564dbfb8c9e0 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.979+0000 7f6aad200640 1 -- 172.21.15.137:0/163674261 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40048e0 con 0x564dbfb8c9e0 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 -- 172.21.15.137:0/163674261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 msgr2=0x564dbfb8ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6abcc00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a9c0a2910 0x7f6a9c0a4dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009910 tx=0x7f6ab00098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 0x564dbfb8ee70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4000c00 tx=0x7f6aa4005e80 comp rx=0 tx=0).stop 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 -- 172.21.15.137:0/163674261 shutdown_connections 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb8c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb8c9e0 0x564dbfb8ee70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 --2- 172.21.15.137:0/163674261 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb872a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 -- 172.21.15.137:0/163674261 >> 172.21.15.137:0/163674261 conn(0x564dbfb4f200 msgr2=0x564dbfb827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.980+0000 7f6ac1980740 1 -- 172.21.15.137:0/163674261 shutdown_connections 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.981+0000 7f6ac1980740 1 -- 172.21.15.137:0/163674261 wait complete. 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.981+0000 7f6ac1980740 1 -- start start 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.981+0000 7f6ac1980740 1 Processor -- start 2024-03-26T14:02:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 0x564dbfb90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb8c9e0 0x564dbfb98310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbfb8f980 con 0x564dbfb86e80 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbfaa7e80 con 0x564dbfb87870 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dbfb8fb00 con 0x564dbfb8c9e0 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ac1980740 10 client.0 _ll_register_callbacks cb 0x564dbfaba130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 0x564dbfb90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 0x564dbfb90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32798/0 (socket says 172.21.15.137:32798) 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 -- 172.21.15.137:0/443945691 learned_addr learned my addr 172.21.15.137:0/443945691 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7400640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6abcc00640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb8c9e0 0x564dbfb98310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 -- 172.21.15.137:0/443945691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb8c9e0 msgr2=0x564dbfb98310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb8c9e0 0x564dbfb98310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 -- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 msgr2=0x564dbfb912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.982+0000 7f6ab7e00640 1 -- 172.21.15.137:0/443945691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dbf8ca7d0 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 0x564dbfb90d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80cc060 tx=0x7f6aa800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4121006d36c32fe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6aa1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6aa1600640 1 -- 172.21.15.137:0/443945691 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8020070 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6aa1600640 1 -- 172.21.15.137:0/443945691 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6aa809a310 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aa402ce30 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6aa1600640 1 -- 172.21.15.137:0/443945691 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aa80d7070 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6ac1980740 10 client.15696 fetch_fsmap learned FSMap version 24 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564dbf8003f0 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6aa1600640 1 -- 172.21.15.137:0/443945691 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6aa402ce30 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.983+0000 7f6aa1600640 1 -- 172.21.15.137:0/443945691 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6aa80b1050 con 0x564dbfb86e80 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ac1980740 10 client.15696 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ac1980740 10 client.15696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:02:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ac1980740 20 client.15696 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ac1980740 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfb9cb70 0x564dbfb9f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ac1980740 4 client.15696 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:02:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dbfaf9790 con 0x564dbfb9cb70 2024-03-26T14:02:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.984+0000 7f6ab7400640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfb9cb70 0x564dbfb9f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.985+0000 7f6ab7400640 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfb9cb70 0x564dbfb9f000 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5aa43c1c3e6f5ad server_cookie=5ea9a84a2e2a2cbc in_seq=0 out_seq=0 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.985+0000 7f6aa1600640 10 client.15696 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.986+0000 7f6aa1600640 1 -- 172.21.15.137:0/443945691 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564dbfaf9790 con 0x564dbfb9cb70 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.986+0000 7f6aa1600640 10 client.15696 handle_command_reply: tid=0 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.986+0000 7f6ac1980740 1 client.15696 shutdown 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 msgr2=0x564dbfb90d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 0x564dbfb90d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80cc060 tx=0x7f6aa800df70 comp rx=0 tx=0).stop 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 shutdown_connections 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfb9cb70 0x564dbfb9f000 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb87870 0x564dbfb912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb8c9e0 0x564dbfb98310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6aa1600640 0 client.15696 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 --2- 172.21.15.137:0/443945691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb86e80 0x564dbfb90d90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 >> 172.21.15.137:0/443945691 conn(0x564dbfb4f200 msgr2=0x564dbfb81a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 shutdown_connections 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 1 -- 172.21.15.137:0/443945691 wait complete. 2024-03-26T14:02:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:32.987+0000 7f6ac1980740 20 client.15696 trim_cache size 0 max 16384 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.004+0000 7f6ac1980740 1 Processor -- start 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.004+0000 7f6ac1980740 1 -- start start 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.004+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb91230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb982d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb81910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ac1980740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81ae8d0 con 0x564dbfb87870 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ac1980740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81ac3d0 con 0x564dbfb8c9e0 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ac1980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81aa450 con 0x564dbfb86e80 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32810/0 (socket says 172.21.15.137:32810) 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 -- 172.21.15.137:0/1231567105 learned_addr learned my addr 172.21.15.137:0/1231567105 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:33.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6abcc00640 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb81910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb91230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 -- 172.21.15.137:0/1231567105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 msgr2=0x564dbfb91230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb91230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 -- 172.21.15.137:0/1231567105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 msgr2=0x564dbfb81910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb81910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.005+0000 7f6ab7400640 1 -- 172.21.15.137:0/1231567105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dbf7c00f0 con 0x564dbfb87870 2024-03-26T14:02:33.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.006+0000 7f6ab7400640 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb982d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009230 tx=0x7f6aa4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b110830aa21762 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:33.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.006+0000 7f6aa3e00640 1 -- 172.21.15.137:0/1231567105 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4054280 con 0x564dbfb87870 2024-03-26T14:02:33.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.006+0000 7f6aa3e00640 1 -- 172.21.15.137:0/1231567105 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6aa402ce30 con 0x564dbfb87870 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.006+0000 7f6aa3e00640 1 -- 172.21.15.137:0/1231567105 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40046c0 con 0x564dbfb87870 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 -- 172.21.15.137:0/1231567105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 msgr2=0x564dbfb982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb982d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009230 tx=0x7f6aa4009260 comp rx=0 tx=0).stop 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 -- 172.21.15.137:0/1231567105 shutdown_connections 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb81910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb982d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 --2- 172.21.15.137:0/1231567105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb91230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 -- 172.21.15.137:0/1231567105 >> 172.21.15.137:0/1231567105 conn(0x564dbfb4f200 msgr2=0x564dbfb81200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 -- 172.21.15.137:0/1231567105 shutdown_connections 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 -- 172.21.15.137:0/1231567105 wait complete. 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 -- start start 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.007+0000 7f6ac1980740 1 Processor -- start 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb95f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb96ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81aaa50 con 0x564dbfb87870 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81abd70 con 0x564dbfb8c9e0 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab81ab0b0 con 0x564dbfb86e80 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ac1980740 10 client.0 _ll_register_callbacks cb 0x7f6aa8009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb95f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb95f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32822/0 (socket says 172.21.15.137:32822) 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 -- 172.21.15.137:0/4008644965 learned_addr learned my addr 172.21.15.137:0/4008644965 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6abcc00640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb96ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 -- 172.21.15.137:0/4008644965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 msgr2=0x564dbfb95a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 -- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 msgr2=0x564dbfb96ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb96ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6ab7400640 1 -- 172.21.15.137:0/4008644965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dbfb974e0 con 0x564dbfb87870 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.008+0000 7f6abcc00640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb96ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6ab7400640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb95f50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40059d0 tx=0x7f6aa40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f0f912e51e09e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:02:33.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6aa1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6aa1600640 1 -- 172.21.15.137:0/4008644965 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40542b0 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6aa1600640 1 -- 172.21.15.137:0/4008644965 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6aa402c980 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6aa402ce30 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6aa1600640 1 -- 172.21.15.137:0/4008644965 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6aa4065050 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6ac1980740 10 client.15702 fetch_fsmap learned FSMap version 24 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564dbfbf8490 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.009+0000 7f6aa1600640 1 -- 172.21.15.137:0/4008644965 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6aa402ce30 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6aa1600640 1 -- 172.21.15.137:0/4008644965 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6aa405b030 con 0x564dbfb87870 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ac1980740 10 client.15702 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ac1980740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ac1980740 20 client.15702 populate_metadata read hostname 'smithi137' 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ac1980740 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfbf8980 0x564dbfbfad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ac1980740 4 client.15702 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:02:33.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564dbfaf3ae0 con 0x564dbfbf8980 2024-03-26T14:02:33.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.010+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfbf8980 0x564dbfbfad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.011+0000 7f6ab7e00640 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfbf8980 0x564dbfbfad70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d3278794461461 server_cookie=4119a373700e619d in_seq=0 out_seq=0 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.011+0000 7f6aa1600640 10 client.15702 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6aa1600640 1 -- 172.21.15.137:0/4008644965 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564dbfaf3ae0 con 0x564dbfbf8980 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6aa1600640 10 client.15702 handle_command_reply: tid=0 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 client.15702 shutdown 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 msgr2=0x564dbfb95f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb95f50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40059d0 tx=0x7f6aa40043b0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 shutdown_connections 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564dbfbf8980 0x564dbfbfad70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564dbfb8c9e0 0x564dbfb96ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564dbfb87870 0x564dbfb95f50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 --2- 172.21.15.137:0/4008644965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564dbfb86e80 0x564dbfb95a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 >> 172.21.15.137:0/4008644965 conn(0x564dbfb4f200 msgr2=0x564dbfb910b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6aa1600640 0 client.15702 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 shutdown_connections 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 1 -- 172.21.15.137:0/4008644965 wait complete. 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.012+0000 7f6ac1980740 20 client.15702 trim_cache size 0 max 16384 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:02:33.013 INFO:teuthology.orchestra.run.smithi137.stdout: "c7c07a52-7149-47b7-a9eb-9480efd56dc4": { 2024-03-26T14:02:33.014 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:02:33.014 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "c7c07a52-7149-47b7-a9eb-9480efd56dc4", 2024-03-26T14:02:33.014 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:02:33.014 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:02:33.014 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:02:33.014 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a9c0a2910 msgr2=0x7f6a9c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a9c0a2910 0x7f6a9c0a4dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0009910 tx=0x7f6ab00098a0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 msgr2=0x7f6ab8199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 0x7f6ab8199b70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800d770 tx=0x7f6aa800dc30 comp rx=0 tx=0).stop 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 shutdown_connections 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6a9c0a2910 0x7f6a9c0a4dd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ab8100910 0x7f6ab819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ab81012b0 0x7f6ab81a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 --2- 172.21.15.137:0/692029953 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6ab80fffb0 0x7f6ab8199b70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 >> 172.21.15.137:0/692029953 conn(0x7f6ab80fbb30 msgr2=0x7f6ab80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 shutdown_connections 2024-03-26T14:02:33.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:02:33.013+0000 7f6abec00640 1 -- 172.21.15.137:0/692029953 wait complete. 2024-03-26T14:02:33.031 INFO:tasks.cephfs.filesystem:scrub status for tag:c7c07a52-7149-47b7-a9eb-9480efd56dc4 - {'path': '/', 'tag': 'c7c07a52-7149-47b7-a9eb-9480efd56dc4', 'options': 'recursive,force'} 2024-03-26T14:02:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:33 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/446860937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:33 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:02:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:33 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:02:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:33 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:02:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:33 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:02:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:33 smithi179 ceph-mon[35472]: pgmap v464: 129 pgs: 129 active+clean; 656 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-26T14:02:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:33 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/446860937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:33 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:02:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:33 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:02:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:33 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:02:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:33 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:02:33.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:33 smithi159 ceph-mon[35501]: pgmap v464: 129 pgs: 129 active+clean; 656 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-26T14:02:33.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:33 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/446860937' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:02:33.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:33 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:02:33.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:33 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:02:33.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:33 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:02:33.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:33 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:02:33.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:33 smithi137 ceph-mon[31539]: pgmap v464: 129 pgs: 129 active+clean; 656 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-26T14:02:36.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:36 smithi137 ceph-mon[31539]: pgmap v465: 129 pgs: 129 active+clean; 656 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-26T14:02:36.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:36 smithi179 ceph-mon[35472]: pgmap v465: 129 pgs: 129 active+clean; 656 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-26T14:02:36.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:36 smithi159 ceph-mon[35501]: pgmap v465: 129 pgs: 129 active+clean; 656 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-03-26T14:02:38.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:38 smithi137 ceph-mon[31539]: pgmap v466: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-26T14:02:38.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:38 smithi179 ceph-mon[35472]: pgmap v466: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-26T14:02:38.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:38 smithi159 ceph-mon[35501]: pgmap v466: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-03-26T14:02:40.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:40 smithi137 ceph-mon[31539]: pgmap v467: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 936 B/s wr, 6 op/s 2024-03-26T14:02:40.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:40 smithi179 ceph-mon[35472]: pgmap v467: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 936 B/s wr, 6 op/s 2024-03-26T14:02:40.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:40 smithi159 ceph-mon[35501]: pgmap v467: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 936 B/s wr, 6 op/s 2024-03-26T14:02:42.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:42 smithi137 ceph-mon[31539]: pgmap v468: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-26T14:02:42.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:42 smithi179 ceph-mon[35472]: pgmap v468: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-26T14:02:42.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:42 smithi159 ceph-mon[35501]: pgmap v468: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 5 op/s 2024-03-26T14:02:44.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:44 smithi137 ceph-mon[31539]: pgmap v469: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-03-26T14:02:44.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:44 smithi179 ceph-mon[35472]: pgmap v469: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-03-26T14:02:44.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:44 smithi159 ceph-mon[35501]: pgmap v469: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 7 op/s 2024-03-26T14:02:46.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:46 smithi137 ceph-mon[31539]: pgmap v470: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:46.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:46 smithi179 ceph-mon[35472]: pgmap v470: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:46.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:46 smithi159 ceph-mon[35501]: pgmap v470: 129 pgs: 129 active+clean; 661 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:47.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:47 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:02:47.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:47 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:02:47.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:47 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:02:48.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:48 smithi137 ceph-mon[31539]: pgmap v471: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-26T14:02:48.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:48 smithi179 ceph-mon[35472]: pgmap v471: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-26T14:02:48.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:48 smithi159 ceph-mon[35501]: pgmap v471: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-03-26T14:02:49.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:49 smithi179 ceph-mon[35472]: pgmap v472: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:49.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:49 smithi159 ceph-mon[35501]: pgmap v472: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:49.782 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:49 smithi137 ceph-mon[31539]: pgmap v472: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:02:52.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:52 smithi137 ceph-mon[31539]: pgmap v473: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:02:52.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:52 smithi179 ceph-mon[35472]: pgmap v473: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:02:52.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:52 smithi159 ceph-mon[35501]: pgmap v473: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:02:54.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:54 smithi137 ceph-mon[31539]: pgmap v474: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:02:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:54 smithi179 ceph-mon[35472]: pgmap v474: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:02:54.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:54 smithi159 ceph-mon[35501]: pgmap v474: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:02:56.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:56 smithi137 ceph-mon[31539]: pgmap v475: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T14:02:56.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:56 smithi179 ceph-mon[35472]: pgmap v475: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T14:02:56.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:56 smithi159 ceph-mon[35501]: pgmap v475: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-03-26T14:02:58.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:02:58 smithi137 ceph-mon[31539]: pgmap v476: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-26T14:02:58.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:02:58 smithi179 ceph-mon[35472]: pgmap v476: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-26T14:02:58.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:02:58 smithi159 ceph-mon[35501]: pgmap v476: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-03-26T14:03:00.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:00 smithi137 ceph-mon[31539]: pgmap v477: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:00.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:00 smithi179 ceph-mon[35472]: pgmap v477: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:00.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:00 smithi159 ceph-mon[35501]: pgmap v477: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:02.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:02 smithi137 ceph-mon[31539]: pgmap v478: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:03:02.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:02 smithi179 ceph-mon[35472]: pgmap v478: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:03:02.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:02 smithi159 ceph-mon[35501]: pgmap v478: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:03:03.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:03:03.032 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:03:03.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.247+0000 7f6b16e00640 1 -- 172.21.15.137:0/4190251565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 msgr2=0x7f6b10104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.247+0000 7f6b16e00640 1 --2- 172.21.15.137:0/4190251565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b10104f30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009900 tx=0x7f6af802bc30 comp rx=0 tx=0).stop 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.247+0000 7f6b16e00640 1 -- 172.21.15.137:0/4190251565 shutdown_connections 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.247+0000 7f6b16e00640 1 --2- 172.21.15.137:0/4190251565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 0x7f6b1010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.247+0000 7f6b16e00640 1 --2- 172.21.15.137:0/4190251565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b10104f30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.247+0000 7f6b16e00640 1 --2- 172.21.15.137:0/4190251565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b101038c0 0x7f6b10103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.248+0000 7f6b16e00640 1 -- 172.21.15.137:0/4190251565 >> 172.21.15.137:0/4190251565 conn(0x7f6b100fee50 msgr2=0x7f6b10101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.248+0000 7f6b16e00640 1 -- 172.21.15.137:0/4190251565 shutdown_connections 2024-03-26T14:03:03.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.248+0000 7f6b16e00640 1 -- 172.21.15.137:0/4190251565 wait complete. 2024-03-26T14:03:03.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.248+0000 7f6b16e00640 1 Processor -- start 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 -- start start 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b101038c0 0x7f6b10199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b1019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 0x7f6b101a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1010f440 con 0x7f6b10104ab0 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1010f2c0 con 0x7f6b10105470 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b16e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1010f5c0 con 0x7f6b101038c0 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b07400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b1019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b07400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b1019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54102/0 (socket says 172.21.15.137:54102) 2024-03-26T14:03:03.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b07400640 1 -- 172.21.15.137:0/1632908043 learned_addr learned my addr 172.21.15.137:0/1632908043 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.249+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 0x7f6b101a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b101038c0 0x7f6b10199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b07400640 1 -- 172.21.15.137:0/1632908043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b101038c0 msgr2=0x7f6b10199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b07400640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b101038c0 0x7f6b10199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b07400640 1 -- 172.21.15.137:0/1632908043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 msgr2=0x7f6b101a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b07400640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 0x7f6b101a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b07400640 1 -- 172.21.15.137:0/1632908043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b101a1720 con 0x7f6b10104ab0 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 0x7f6b101a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b07400640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b1019a040 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000c00 tx=0x7f6af80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b0d600640 1 -- 172.21.15.137:0/1632908043 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8043020 con 0x7f6b10104ab0 2024-03-26T14:03:03.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b0d600640 1 -- 172.21.15.137:0/1632908043 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6af8004aa0 con 0x7f6b10104ab0 2024-03-26T14:03:03.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b101a19b0 con 0x7f6b10104ab0 2024-03-26T14:03:03.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b0d600640 1 -- 172.21.15.137:0/1632908043 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8003d00 con 0x7f6b10104ab0 2024-03-26T14:03:03.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.250+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b101a1f50 con 0x7f6b10104ab0 2024-03-26T14:03:03.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.253+0000 7f6b0d600640 1 -- 172.21.15.137:0/1632908043 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6af8046020 con 0x7f6b10104ab0 2024-03-26T14:03:03.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 Processor -- start 2024-03-26T14:03:03.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 -- start start 2024-03-26T14:03:03.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc94082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc9408870 0x564fc940d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc940d9e0 0x564fc940fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fc936d250 con 0x564fc9407e80 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fc9315a30 con 0x564fc940d9e0 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.254+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fc93692f0 con 0x564fc9408870 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc94082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc94082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54104/0 (socket says 172.21.15.137:54104) 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 -- 172.21.15.137:0/2141456126 learned_addr learned my addr 172.21.15.137:0/2141456126 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b0d600640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af40a2910 0x7f6af40a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b07400640 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc940d9e0 0x564fc940fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc9408870 0x564fc940d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af40a2910 0x7f6af40a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b0d600640 1 -- 172.21.15.137:0/1632908043 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6af80f1270 con 0x7f6b10104ab0 2024-03-26T14:03:03.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 -- 172.21.15.137:0/2141456126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc9408870 msgr2=0x564fc940d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc9408870 0x564fc940d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 -- 172.21.15.137:0/2141456126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc940d9e0 msgr2=0x564fc940fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc940d9e0 0x564fc940fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.255+0000 7f6b14e00640 1 -- 172.21.15.137:0/2141456126 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fc9300690 con 0x564fc9407e80 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.256+0000 7f6b14e00640 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc94082a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6b080137a0 tx=0x7f6b08013c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d66e11de51d0f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.256+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af40a2910 0x7f6af40a4dd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6b0801e000 tx=0x7f6b08020000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.256+0000 7f6b05600640 1 -- 172.21.15.137:0/2141456126 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0801f850 con 0x564fc9407e80 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.256+0000 7f6b05600640 1 -- 172.21.15.137:0/2141456126 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b08023070 con 0x564fc9407e80 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.256+0000 7f6b05600640 1 -- 172.21.15.137:0/2141456126 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08011850 con 0x564fc9407e80 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 -- 172.21.15.137:0/2141456126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 msgr2=0x564fc94082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc94082a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6b080137a0 tx=0x7f6b08013c60 comp rx=0 tx=0).stop 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 -- 172.21.15.137:0/2141456126 shutdown_connections 2024-03-26T14:03:03.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc940d9e0 0x564fc940fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc9408870 0x564fc940d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/2141456126 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc94082a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 -- 172.21.15.137:0/2141456126 >> 172.21.15.137:0/2141456126 conn(0x564fc93d0200 msgr2=0x564fc94037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 -- 172.21.15.137:0/2141456126 shutdown_connections 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.257+0000 7f6b19bcd740 1 -- 172.21.15.137:0/2141456126 wait complete. 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.258+0000 7f6b19bcd740 1 -- start start 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.258+0000 7f6b19bcd740 1 Processor -- start 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.258+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.258+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.258+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc9421c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fc9410930 con 0x564fc9407e80 2024-03-26T14:03:03.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fc9328e80 con 0x564fc9408870 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564fc9410ab0 con 0x564fc940d9e0 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b19bcd740 10 client.0 _ll_register_callbacks cb 0x564fc933b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b07400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54112/0 (socket says 172.21.15.137:54112) 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 -- 172.21.15.137:0/3214148336 learned_addr learned my addr 172.21.15.137:0/3214148336 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 -- 172.21.15.137:0/3214148336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 msgr2=0x564fc9421c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b14e00640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc9421c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc9421c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 -- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 msgr2=0x564fc941ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 -- 172.21.15.137:0/3214148336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fc914b7d0 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b07400640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941a720 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009fd0 tx=0x7f6b00004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f026939b2af62 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.259+0000 7f6afe000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6afe000640 1 -- 172.21.15.137:0/3214148336 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00012980 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6afe000640 1 -- 172.21.15.137:0/3214148336 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6b00012b20 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b08023070 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6afe000640 1 -- 172.21.15.137:0/3214148336 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b00024050 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6b19bcd740 10 client.15714 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564fc90813f0 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6afe000640 1 -- 172.21.15.137:0/3214148336 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b08023070 con 0x564fc9407e80 2024-03-26T14:03:03.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.260+0000 7f6afe000640 1 -- 172.21.15.137:0/3214148336 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6b00019020 con 0x564fc9407e80 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b19bcd740 10 client.15714 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b19bcd740 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b19bcd740 20 client.15714 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9426550 0x564fc94289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b19bcd740 4 client.15714 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564fc937a790 con 0x564fc9426550 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.261+0000 7f6b07400640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9426550 0x564fc94289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.262+0000 7f6b07400640 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9426550 0x564fc94289f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118127ea26c29b3f server_cookie=5261cf52dd5ec218 in_seq=0 out_seq=0 2024-03-26T14:03:03.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.262+0000 7f6afe000640 10 client.15714 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:03.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.262+0000 7f6afe000640 1 -- 172.21.15.137:0/3214148336 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564fc937a790 con 0x564fc9426550 2024-03-26T14:03:03.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.262+0000 7f6afe000640 10 client.15714 handle_command_reply: tid=0 2024-03-26T14:03:03.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.262+0000 7f6b19bcd740 1 client.15714 shutdown 2024-03-26T14:03:03.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 msgr2=0x564fc941a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941a720 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009fd0 tx=0x7f6b00004410 comp rx=0 tx=0).stop 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 shutdown_connections 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9426550 0x564fc94289f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc9421c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/3214148336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941a720 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6afe000640 0 client.15714 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 >> 172.21.15.137:0/3214148336 conn(0x564fc93d0200 msgr2=0x564fc9402a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 shutdown_connections 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 1 -- 172.21.15.137:0/3214148336 wait complete. 2024-03-26T14:03:03.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.263+0000 7f6b19bcd740 20 client.15714 trim_cache size 0 max 16384 2024-03-26T14:03:03.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 Processor -- start 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 -- start start 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc9406a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc9421bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc942d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101aa490 con 0x564fc9407e80 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101a7f90 con 0x564fc9408870 2024-03-26T14:03:03.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101a6010 con 0x564fc940d9e0 2024-03-26T14:03:03.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc9406a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc9406a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54122/0 (socket says 172.21.15.137:54122) 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.281+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1539042830 learned_addr learned my addr 172.21.15.137:0/1539042830 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc942d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b07400640 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc9421bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1539042830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 msgr2=0x564fc942d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc942d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1539042830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 msgr2=0x564fc9421bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc9421bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1539042830 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564fc90410f0 con 0x564fc9407e80 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc942d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.282+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc9406a30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b00003b40 tx=0x7f6b00003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa19cccf7fad46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.283+0000 7f6b04c00640 1 -- 172.21.15.137:0/1539042830 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000120d0 con 0x564fc9407e80 2024-03-26T14:03:03.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.283+0000 7f6b04c00640 1 -- 172.21.15.137:0/1539042830 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b0002d070 con 0x564fc9407e80 2024-03-26T14:03:03.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.283+0000 7f6b04c00640 1 -- 172.21.15.137:0/1539042830 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00004c40 con 0x564fc9407e80 2024-03-26T14:03:03.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.283+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1539042830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 msgr2=0x564fc9406a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.283+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc9406a30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b00003b40 tx=0x7f6b00003e70 comp rx=0 tx=0).stop 2024-03-26T14:03:03.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1539042830 shutdown_connections 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc9421bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc942d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1539042830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc9406a30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1539042830 >> 172.21.15.137:0/1539042830 conn(0x564fc93d0200 msgr2=0x564fc9402a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1539042830 shutdown_connections 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1539042830 wait complete. 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 -- start start 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.284+0000 7f6b19bcd740 1 Processor -- start 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc94206a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101a6610 con 0x564fc9407e80 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101a7930 con 0x564fc9408870 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b101a6c70 con 0x564fc940d9e0 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b19bcd740 10 client.0 _ll_register_callbacks cb 0x7f6b00009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54124/0 (socket says 172.21.15.137:54124) 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1837454096 learned_addr learned my addr 172.21.15.137:0/1837454096 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b07400640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc94206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1837454096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 msgr2=0x564fc94206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc94206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 msgr2=0x564fc941f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b0fe00640 1 -- 172.21.15.137:0/1837454096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564fc9420c90 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.285+0000 7f6b14e00640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc94206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6b0fe00640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941f150 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005fd0 tx=0x7f6b00003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c575aa7d4a449a2a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6afe000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6afe000640 1 -- 172.21.15.137:0/1837454096 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00035070 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6afe000640 1 -- 172.21.15.137:0/1837454096 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6b00004d90 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b0002d070 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6afe000640 1 -- 172.21.15.137:0/1837454096 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b000108c0 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6b19bcd740 10 client.15720 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564fc9481e80 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.286+0000 7f6afe000640 1 -- 172.21.15.137:0/1837454096 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b0002d070 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6afe000640 1 -- 172.21.15.137:0/1837454096 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6b00019040 con 0x564fc9407e80 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b19bcd740 10 client.15720 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b19bcd740 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b19bcd740 20 client.15720 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:03.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9482620 0x564fc9484a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b19bcd740 4 client.15720 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:03:03.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564fc9374ae0 con 0x564fc9482620 2024-03-26T14:03:03.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.287+0000 7f6b07400640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9482620 0x564fc9484a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.288+0000 7f6b07400640 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9482620 0x564fc9484a40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c23910e25825df0a server_cookie=a0386d9db069f990 in_seq=0 out_seq=0 2024-03-26T14:03:03.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.288+0000 7f6afe000640 10 client.15720 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:03.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.288+0000 7f6afe000640 1 -- 172.21.15.137:0/1837454096 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564fc9374ae0 con 0x564fc9482620 2024-03-26T14:03:03.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.288+0000 7f6afe000640 10 client.15720 handle_command_reply: tid=0 2024-03-26T14:03:03.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.288+0000 7f6b19bcd740 1 client.15720 shutdown 2024-03-26T14:03:03.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 msgr2=0x564fc941f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941f150 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6b00005fd0 tx=0x7f6b00003b80 comp rx=0 tx=0).stop 2024-03-26T14:03:03.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 shutdown_connections 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564fc9482620 0x564fc9484a40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fc9408870 0x564fc941f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564fc940d9e0 0x564fc94206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 --2- 172.21.15.137:0/1837454096 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564fc9407e80 0x564fc941f150 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6afe000640 0 client.15720 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 >> 172.21.15.137:0/1837454096 conn(0x564fc93d0200 msgr2=0x564fc9409100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 shutdown_connections 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 1 -- 172.21.15.137:0/1837454096 wait complete. 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.289+0000 7f6b19bcd740 20 client.15720 trim_cache size 0 max 16384 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:03:03.291 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:03:03.292 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:03:03.292 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:03:03.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af40a2910 msgr2=0x7f6af40a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af40a2910 0x7f6af40a4dd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6b0801e000 tx=0x7f6b08020000 comp rx=0 tx=0).stop 2024-03-26T14:03:03.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 msgr2=0x7f6b1019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b1019a040 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6af8000c00 tx=0x7f6af80042f0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 shutdown_connections 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af40a2910 0x7f6af40a4dd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b10105470 0x7f6b101a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b10104ab0 0x7f6b1019a040 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 --2- 172.21.15.137:0/1632908043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b101038c0 0x7f6b10199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 >> 172.21.15.137:0/1632908043 conn(0x7f6b100fee50 msgr2=0x7f6b10107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 shutdown_connections 2024-03-26T14:03:03.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.290+0000 7f6b16e00640 1 -- 172.21.15.137:0/1632908043 wait complete. 2024-03-26T14:03:03.305 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:03:03.305 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:03:03.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 -- 172.21.15.137:0/3409322598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 msgr2=0x7fa0b0074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3409322598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0074500 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009900 tx=0x7fa09c02bcd0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 -- 172.21.15.137:0/3409322598 shutdown_connections 2024-03-26T14:03:03.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3409322598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0b0109550 0x7fa0b010b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3409322598 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0074500 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3409322598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0b0075680 0x7fa0b0073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 -- 172.21.15.137:0/3409322598 >> 172.21.15.137:0/3409322598 conn(0x7fa0b00fcc50 msgr2=0x7fa0b00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 -- 172.21.15.137:0/3409322598 shutdown_connections 2024-03-26T14:03:03.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.509+0000 7fa0b8000640 1 -- 172.21.15.137:0/3409322598 wait complete. 2024-03-26T14:03:03.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.510+0000 7fa0b8000640 1 Processor -- start 2024-03-26T14:03:03.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 -- start start 2024-03-26T14:03:03.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0b0075680 0x7fa0b019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0b0109550 0x7fa0b01a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010f440 con 0x7fa0b00740a0 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010f2c0 con 0x7fa0b0075680 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b8000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b010f5c0 con 0x7fa0b0109550 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b5600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b5600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54140/0 (socket says 172.21.15.137:54140) 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b5600640 1 -- 172.21.15.137:0/3853697877 learned_addr learned my addr 172.21.15.137:0/3853697877 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.511+0000 7fa0b4c00640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0b0075680 0x7fa0b019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b6000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0b0109550 0x7fa0b01a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b5600640 1 -- 172.21.15.137:0/3853697877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0b0109550 msgr2=0x7fa0b01a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b5600640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0b0109550 0x7fa0b01a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b5600640 1 -- 172.21.15.137:0/3853697877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0b0075680 msgr2=0x7fa0b019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b5600640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0b0075680 0x7fa0b019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b5600640 1 -- 172.21.15.137:0/3853697877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b01a1720 con 0x7fa0b00740a0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0b5600640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0199b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400e9f0 tx=0x7fa0a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a400cd80 con 0x7fa0b00740a0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.512+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa0a40108e0 con 0x7fa0b00740a0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.513+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b01a1a10 con 0x7fa0b00740a0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.513+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a4018920 con 0x7fa0b00740a0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.513+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b01a1f30 con 0x7fa0b00740a0 2024-03-26T14:03:03.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.514+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0108310 con 0x7fa0b00740a0 2024-03-26T14:03:03.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.516+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fa0a4010a90 con 0x7fa0b00740a0 2024-03-26T14:03:03.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.517+0000 7fa0ae000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0980a2890 0x7fa0980a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.517+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa0a40cf3a0 con 0x7fa0b00740a0 2024-03-26T14:03:03.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.518+0000 7fa0b4c00640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0980a2890 0x7fa0980a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.518+0000 7fa0b4c00640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0980a2890 0x7fa0980a4d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c02c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.522+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa0a409bed0 con 0x7fa0b00740a0 2024-03-26T14:03:03.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.762+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0b00622a0 con 0x7fa0b00740a0 2024-03-26T14:03:03.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.764+0000 7fa0ae000640 1 -- 172.21.15.137:0/3853697877 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fa0a409b7d0 con 0x7fa0b00740a0 2024-03-26T14:03:03.762 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:03:03.762 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:03:03.762 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:03:03.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.766+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0980a2890 msgr2=0x7fa0980a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.766+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0980a2890 0x7fa0980a4d50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c02c880 comp rx=0 tx=0).stop 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.766+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 msgr2=0x7fa0b0199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.766+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0199b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400e9f0 tx=0x7fa0a400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 shutdown_connections 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0980a2890 0x7fa0980a4d50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0b0075680 0x7fa0b019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0b0109550 0x7fa0b01a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 --2- 172.21.15.137:0/3853697877 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa0b00740a0 0x7fa0b0199b00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 >> 172.21.15.137:0/3853697877 conn(0x7fa0b00fcc50 msgr2=0x7fa0b01054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 shutdown_connections 2024-03-26T14:03:03.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.767+0000 7fa0b8000640 1 -- 172.21.15.137:0/3853697877 wait complete. 2024-03-26T14:03:03.784 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:03:03.784 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 -- 172.21.15.137:0/3691092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac108eb0 msgr2=0x7f4aac109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 --2- 172.21.15.137:0/3691092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac108eb0 0x7f4aac109330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009900 tx=0x7f4a9c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 -- 172.21.15.137:0/3691092622 shutdown_connections 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 --2- 172.21.15.137:0/3691092622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac109870 0x7f4aac110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 --2- 172.21.15.137:0/3691092622 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac108eb0 0x7f4aac109330 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 --2- 172.21.15.137:0/3691092622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aac107cc0 0x7f4aac1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 -- 172.21.15.137:0/3691092622 >> 172.21.15.137:0/3691092622 conn(0x7f4aac076c60 msgr2=0x7f4aac0790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:03.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 -- 172.21.15.137:0/3691092622 shutdown_connections 2024-03-26T14:03:03.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.996+0000 7f4ab4800640 1 -- 172.21.15.137:0/3691092622 wait complete. 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.997+0000 7f4ab4800640 1 Processor -- start 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.997+0000 7f4ab4800640 1 -- start start 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab4800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 0x7f4aac19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab4800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac108eb0 0x7f4aac19e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab4800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aac109870 0x7f4aac1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab4800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac113820 con 0x7f4aac107cc0 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab4800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1136a0 con 0x7f4aac108eb0 2024-03-26T14:03:03.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab4800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1139a0 con 0x7f4aac109870 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 0x7f4aac19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 0x7f4aac19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54156/0 (socket says 172.21.15.137:54156) 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab1e00640 1 -- 172.21.15.137:0/4027817542 learned_addr learned my addr 172.21.15.137:0/4027817542 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab1400640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac108eb0 0x7f4aac19e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.998+0000 7f4ab2800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aac109870 0x7f4aac1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1e00640 1 -- 172.21.15.137:0/4027817542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aac109870 msgr2=0x7f4aac1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aac109870 0x7f4aac1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1e00640 1 -- 172.21.15.137:0/4027817542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac108eb0 msgr2=0x7f4aac19e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac108eb0 0x7f4aac19e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1e00640 1 -- 172.21.15.137:0/4027817542 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac1a5a50 con 0x7f4aac107cc0 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1400640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac108eb0 0x7f4aac19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 0x7f4aac19de30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c970 tx=0x7f4aa000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4aa6a00640 1 -- 172.21.15.137:0/4027817542 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0007c10 con 0x7f4aac107cc0 2024-03-26T14:03:03.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4aa6a00640 1 -- 172.21.15.137:0/4027817542 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4aa0007db0 con 0x7f4aac107cc0 2024-03-26T14:03:03.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac1a5d40 con 0x7f4aac107cc0 2024-03-26T14:03:03.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4aa6a00640 1 -- 172.21.15.137:0/4027817542 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000f870 con 0x7f4aac107cc0 2024-03-26T14:03:03.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:03.999+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac1a61f0 con 0x7f4aac107cc0 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4aa6a00640 1 -- 172.21.15.137:0/4027817542 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f4aa000fa10 con 0x7f4aac107cc0 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 Processor -- start 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 -- start start 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 0x5596c2f38e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f393a0 0x5596c2f3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596c2e9c2c0 con 0x5596c2f389e0 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596c2e44c90 con 0x5596c2f37810 2024-03-26T14:03:04.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.003+0000 7f4ab7591740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596c2e98360 con 0x5596c2f393a0 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 0x5596c2f38e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 0x5596c2f38e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54172/0 (socket says 172.21.15.137:54172) 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 -- 172.21.15.137:0/210876762 learned_addr learned my addr 172.21.15.137:0/210876762 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab2800640 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1400640 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f393a0 0x5596c2f3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 -- 172.21.15.137:0/210876762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f393a0 msgr2=0x5596c2f3fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f393a0 0x5596c2f3fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 -- 172.21.15.137:0/210876762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 msgr2=0x5596c2f37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4ab1e00640 1 -- 172.21.15.137:0/210876762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596c2e69080 con 0x5596c2f389e0 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.004+0000 7f4aa6a00640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a940a2910 0x7f4a940a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.005+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 0x5596c2f38e60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00d7560 tx=0x7f4aa00d7650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba02192b577f5510 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.005+0000 7f4ab2800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a940a2910 0x7f4a940a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.005+0000 7f4aa6a00640 1 -- 172.21.15.137:0/4027817542 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4aa00cf3a0 con 0x7f4aac107cc0 2024-03-26T14:03:04.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.005+0000 7f4a9b000640 1 -- 172.21.15.137:0/210876762 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0097b00 con 0x5596c2f389e0 2024-03-26T14:03:04.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.005+0000 7f4a9b000640 1 -- 172.21.15.137:0/210876762 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4aa0018b50 con 0x5596c2f389e0 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.005+0000 7f4a9b000640 1 -- 172.21.15.137:0/210876762 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa009b1c0 con 0x5596c2f389e0 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 -- 172.21.15.137:0/210876762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 msgr2=0x5596c2f38e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 0x5596c2f38e60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00d7560 tx=0x7f4aa00d7650 comp rx=0 tx=0).stop 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab2800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a940a2910 0x7f4a940a4dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5596c2f388e0 tx=0x7f4aa8009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 -- 172.21.15.137:0/210876762 shutdown_connections 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f393a0 0x5596c2f3fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f389e0 0x5596c2f38e60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 --2- 172.21.15.137:0/210876762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 -- 172.21.15.137:0/210876762 >> 172.21.15.137:0/210876762 conn(0x5596c2f32c00 msgr2=0x5596c2f33000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 -- 172.21.15.137:0/210876762 shutdown_connections 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 -- 172.21.15.137:0/210876762 wait complete. 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.006+0000 7f4ab7591740 1 -- start start 2024-03-26T14:03:04.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 Processor -- start 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f41a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f389e0 0x5596c2f41f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f48fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596c2f403b0 con 0x5596c2f393a0 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596c2e580d0 con 0x5596c2f37810 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596c2f40530 con 0x5596c2f389e0 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab7591740 10 client.0 _ll_register_callbacks cb 0x5596c2e5ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f48fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f48fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54180/0 (socket says 172.21.15.137:54180) 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab2800640 1 -- 172.21.15.137:0/918348570 learned_addr learned my addr 172.21.15.137:0/918348570 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.007+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f41a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab2800640 1 -- 172.21.15.137:0/918348570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f389e0 msgr2=0x5596c2f41f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab1400640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f389e0 0x5596c2f41f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab2800640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f389e0 0x5596c2f41f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab2800640 1 -- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 msgr2=0x5596c2f41a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab2800640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f41a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab2800640 1 -- 172.21.15.137:0/918348570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596c2bb03f0 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab2800640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f48fb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4aa801e8c0 tx=0x7f4aa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903a0cf5e56c0098 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4a8f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4a8f400640 1 -- 172.21.15.137:0/918348570 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8010d60 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4a8f400640 1 -- 172.21.15.137:0/918348570 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4aa801c450 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.008+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596c2ea9800 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4a8f400640 1 -- 172.21.15.137:0/918348570 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4aa801ee40 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 10 client.15738 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5596c2c7a7d0 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4a8f400640 1 -- 172.21.15.137:0/918348570 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5596c2ea9800 con 0x5596c2f393a0 2024-03-26T14:03:04.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4a8f400640 1 -- 172.21.15.137:0/918348570 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4aa800f040 con 0x5596c2f393a0 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 10 client.15738 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 10 client.15738 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 20 client.15738 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2f4d500 0x5596c2f4f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 4 client.15738 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.009+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596c2ea3b50 con 0x5596c2f4d500 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.010+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2f4d500 0x5596c2f4f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.011+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2f4d500 0x5596c2f4f970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfd09d9e72f92162 server_cookie=47a120642828dfa in_seq=0 out_seq=0 2024-03-26T14:03:04.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.011+0000 7f4a8f400640 10 client.15738 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:04.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4a8f400640 1 -- 172.21.15.137:0/918348570 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5596c2ea3b50 con 0x5596c2f4d500 2024-03-26T14:03:04.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4a8f400640 10 client.15738 handle_command_reply: tid=0 2024-03-26T14:03:04.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 client.15738 shutdown 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 msgr2=0x5596c2f48fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f48fb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4aa801e8c0 tx=0x7f4aa8004a40 comp rx=0 tx=0).stop 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 shutdown_connections 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2f4d500 0x5596c2f4f970 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f48fb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f389e0 0x5596c2f41f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 --2- 172.21.15.137:0/918348570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f37810 0x5596c2f41a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4a8f400640 0 client.15738 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 >> 172.21.15.137:0/918348570 conn(0x5596c2f32c00 msgr2=0x5596c2f31eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 shutdown_connections 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.013+0000 7f4ab7591740 1 -- 172.21.15.137:0/918348570 wait complete. 2024-03-26T14:03:04.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.014+0000 7f4ab7591740 20 client.15738 trim_cache size 0 max 16384 2024-03-26T14:03:04.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.028+0000 7f4ab7591740 1 Processor -- start 2024-03-26T14:03:04.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.028+0000 7f4ab7591740 1 -- start start 2024-03-26T14:03:04.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.028+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f48ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1b2ca0 con 0x5596c2f393a0 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab7591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1b07a0 con 0x5596c2f389e0 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab7591740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1ae820 con 0x5596c2f37810 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f54230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54190/0 (socket says 172.21.15.137:54190) 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 -- 172.21.15.137:0/3363270362 learned_addr learned my addr 172.21.15.137:0/3363270362 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab1400640 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f48ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 -- 172.21.15.137:0/3363270362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 msgr2=0x5596c2f31d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f31d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 -- 172.21.15.137:0/3363270362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 msgr2=0x5596c2f48ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f48ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab2800640 1 -- 172.21.15.137:0/3363270362 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596c2f38460 con 0x5596c2f393a0 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.029+0000 7f4ab1400640 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f48ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.030+0000 7f4ab2800640 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f54230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8028890 tx=0x7f4aa8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d405feba5b8eb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.030+0000 7f4a9a600640 1 -- 172.21.15.137:0/3363270362 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8039030 con 0x5596c2f393a0 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.030+0000 7f4a9a600640 1 -- 172.21.15.137:0/3363270362 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4aa8028a80 con 0x5596c2f393a0 2024-03-26T14:03:04.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.030+0000 7f4a9a600640 1 -- 172.21.15.137:0/3363270362 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8010fc0 con 0x5596c2f393a0 2024-03-26T14:03:04.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 -- 172.21.15.137:0/3363270362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 msgr2=0x5596c2f54230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f54230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8028890 tx=0x7f4aa8009a90 comp rx=0 tx=0).stop 2024-03-26T14:03:04.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 -- 172.21.15.137:0/3363270362 shutdown_connections 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f48ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f54230 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3363270362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f31d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 -- 172.21.15.137:0/3363270362 >> 172.21.15.137:0/3363270362 conn(0x5596c2f32c00 msgr2=0x5596c2f31620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 -- 172.21.15.137:0/3363270362 shutdown_connections 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 -- 172.21.15.137:0/3363270362 wait complete. 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 -- start start 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.031+0000 7f4ab7591740 1 Processor -- start 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f48dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f45870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f45db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1aee20 con 0x5596c2f393a0 2024-03-26T14:03:04.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1b0140 con 0x5596c2f389e0 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac1af480 con 0x5596c2f37810 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab7591740 10 client.0 _ll_register_callbacks cb 0x7f4aa8009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f45db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f45db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54194/0 (socket says 172.21.15.137:54194) 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab2800640 1 -- 172.21.15.137:0/3142141092 learned_addr learned my addr 172.21.15.137:0/3142141092 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f48dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.032+0000 7f4ab1400640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f45870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab2800640 1 -- 172.21.15.137:0/3142141092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 msgr2=0x5596c2f48dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab2800640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f48dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab2800640 1 -- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 msgr2=0x5596c2f45870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab2800640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f45870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab2800640 1 -- 172.21.15.137:0/3142141092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596c2fa8e00 con 0x5596c2f393a0 2024-03-26T14:03:04.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab2800640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f45db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8010ba0 tx=0x7f4aa8004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e514b3b94ff75b97 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4a8f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4a8f400640 1 -- 172.21.15.137:0/3142141092 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8042070 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4a8f400640 1 -- 172.21.15.137:0/3142141092 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4aa8011150 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4aa8028a80 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4a8f400640 1 -- 172.21.15.137:0/3142141092 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4aa804e050 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab7591740 10 client.15744 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.033+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5596c2faa020 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4a8f400640 1 -- 172.21.15.137:0/3142141092 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4aa8028a80 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4a8f400640 1 -- 172.21.15.137:0/3142141092 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4aa8005090 con 0x5596c2f393a0 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4ab7591740 10 client.15744 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4ab7591740 10 client.15744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4ab7591740 20 client.15744 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2faa920 0x5596c2facd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4ab7591740 4 client.15744 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:03:04.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.034+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596c2e21500 con 0x5596c2faa920 2024-03-26T14:03:04.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.035+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2faa920 0x5596c2facd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.035+0000 7f4ab1e00640 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2faa920 0x5596c2facd40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4abd9a4e20c5d7cf server_cookie=659d1f95c6965230 in_seq=0 out_seq=0 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.035+0000 7f4a8f400640 10 client.15744 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4a8f400640 1 -- 172.21.15.137:0/3142141092 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596c2e21500 con 0x5596c2faa920 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4a8f400640 10 client.15744 handle_command_reply: tid=0 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 client.15744 shutdown 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 msgr2=0x5596c2f45db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f45db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4aa8010ba0 tx=0x7f4aa8004c90 comp rx=0 tx=0).stop 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 shutdown_connections 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5596c2faa920 0x5596c2facd40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596c2f389e0 0x5596c2f45870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5596c2f393a0 0x5596c2f45db0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 --2- 172.21.15.137:0/3142141092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5596c2f37810 0x5596c2f48dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4a8f400640 0 client.15744 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.036+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 >> 172.21.15.137:0/3142141092 conn(0x5596c2f32c00 msgr2=0x5596c2f31eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 shutdown_connections 2024-03-26T14:03:04.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab7591740 1 -- 172.21.15.137:0/3142141092 wait complete. 2024-03-26T14:03:04.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab7591740 20 client.15744 trim_cache size 0 max 16384 2024-03-26T14:03:04.039 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a940a2910 msgr2=0x7f4a940a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a940a2910 0x7f4a940a4dd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5596c2f388e0 tx=0x7f4aa8009740 comp rx=0 tx=0).stop 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 msgr2=0x7f4aac19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 0x7f4aac19de30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c970 tx=0x7f4aa000ce30 comp rx=0 tx=0).stop 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 shutdown_connections 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a940a2910 0x7f4a940a4dd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aac108eb0 0x7f4aac19e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aac109870 0x7f4aac1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 --2- 172.21.15.137:0/4027817542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aac107cc0 0x7f4aac19de30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.037+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 >> 172.21.15.137:0/4027817542 conn(0x7f4aac076c60 msgr2=0x7f4aac10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.038+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 shutdown_connections 2024-03-26T14:03:04.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.038+0000 7f4ab4800640 1 -- 172.21.15.137:0/4027817542 wait complete. 2024-03-26T14:03:04.051 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:03:04.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:03:04.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.260+0000 7f6fbc200640 1 -- 172.21.15.137:0/3589483874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4109870 msgr2=0x7f6fb4110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.260+0000 7f6fbc200640 1 --2- 172.21.15.137:0/3589483874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4109870 0x7f6fb4110100 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009ca0 tx=0x7f6fac02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 -- 172.21.15.137:0/3589483874 shutdown_connections 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 --2- 172.21.15.137:0/3589483874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4108eb0 0x7f6fb4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 --2- 172.21.15.137:0/3589483874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4109870 0x7f6fb4110100 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 --2- 172.21.15.137:0/3589483874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb4107cc0 0x7f6fb41080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 -- 172.21.15.137:0/3589483874 >> 172.21.15.137:0/3589483874 conn(0x7f6fb4076c60 msgr2=0x7f6fb40790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 -- 172.21.15.137:0/3589483874 shutdown_connections 2024-03-26T14:03:04.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.261+0000 7f6fbc200640 1 -- 172.21.15.137:0/3589483874 wait complete. 2024-03-26T14:03:04.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 Processor -- start 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 -- start start 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 0x7f6fb419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb4108eb0 0x7f6fb419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4109870 0x7f6fb41a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41137d0 con 0x7f6fb4107cc0 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4113650 con 0x7f6fb4109870 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fbc200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb4113950 con 0x7f6fb4108eb0 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 0x7f6fb419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 0x7f6fb419dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54206/0 (socket says 172.21.15.137:54206) 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.262+0000 7f6fb9800640 1 -- 172.21.15.137:0/633512730 learned_addr learned my addr 172.21.15.137:0/633512730 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fba200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4109870 0x7f6fb41a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb4108eb0 0x7f6fb419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb9800640 1 -- 172.21.15.137:0/633512730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb4108eb0 msgr2=0x7f6fb419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb9800640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb4108eb0 0x7f6fb419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb9800640 1 -- 172.21.15.137:0/633512730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4109870 msgr2=0x7f6fb41a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb9800640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4109870 0x7f6fb41a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb9800640 1 -- 172.21.15.137:0/633512730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb41a5ac0 con 0x7f6fb4107cc0 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fba200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4109870 0x7f6fb41a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb9800640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 0x7f6fb419dea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800c970 tx=0x7f6fa800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb2000640 1 -- 172.21.15.137:0/633512730 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8004220 con 0x7f6fb4107cc0 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fb2000640 1 -- 172.21.15.137:0/633512730 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fa80043c0 con 0x7f6fb4107cc0 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.263+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb41a5db0 con 0x7f6fb4107cc0 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.264+0000 7f6fb2000640 1 -- 172.21.15.137:0/633512730 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800f450 con 0x7f6fb4107cc0 2024-03-26T14:03:04.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.264+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb41a6270 con 0x7f6fb4107cc0 2024-03-26T14:03:04.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 Processor -- start 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fb2000640 1 -- 172.21.15.137:0/633512730 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6fa80079b0 con 0x7f6fb4107cc0 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 -- start start 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ff8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd8ff9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9000c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd8f5d2c0 con 0x55bdd8ff99e0 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd8f05c90 con 0x55bdd8ff8810 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd8f59360 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.267+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd8ff9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd8ff9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54222/0 (socket says 172.21.15.137:54222) 2024-03-26T14:03:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 -- 172.21.15.137:0/1484174819 learned_addr learned my addr 172.21.15.137:0/1484174819 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fba200640 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ff8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9000c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 -- 172.21.15.137:0/1484174819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 msgr2=0x55bdd9000c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9000c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 -- 172.21.15.137:0/1484174819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 msgr2=0x55bdd8ff8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ff8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 -- 172.21.15.137:0/1484174819 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd8f2a080 con 0x55bdd8ff99e0 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb2000640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f9c0a2890 0x7f6f9c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.268+0000 7f6fb9800640 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd8ff9e60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80d2f30 tx=0x7f6fa80d7570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18468ac69d680b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.269+0000 7f6fb2000640 1 -- 172.21.15.137:0/633512730 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6fa80cee50 con 0x7f6fb4107cc0 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.269+0000 7f6fba200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f9c0a2890 0x7f6f9c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.269+0000 7f6fa6a00640 1 -- 172.21.15.137:0/1484174819 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8097520 con 0x55bdd8ff99e0 2024-03-26T14:03:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.269+0000 7f6fa6a00640 1 -- 172.21.15.137:0/1484174819 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fa809b980 con 0x55bdd8ff99e0 2024-03-26T14:03:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.269+0000 7f6fa6a00640 1 -- 172.21.15.137:0/1484174819 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa80d29f0 con 0x55bdd8ff99e0 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 -- 172.21.15.137:0/1484174819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 msgr2=0x55bdd8ff9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fba200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f9c0a2890 0x7f6f9c0a4d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6fac02c630 tx=0x7f6fac009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd8ff9e60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80d2f30 tx=0x7f6fa80d7570 comp rx=0 tx=0).stop 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 -- 172.21.15.137:0/1484174819 shutdown_connections 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9000c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd8ff9e60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/1484174819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ff8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 -- 172.21.15.137:0/1484174819 >> 172.21.15.137:0/1484174819 conn(0x55bdd8ff3c00 msgr2=0x55bdd8ff4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 -- 172.21.15.137:0/1484174819 shutdown_connections 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 -- 172.21.15.137:0/1484174819 wait complete. 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.270+0000 7f6fbef7f740 1 -- start start 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 Processor -- start 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd9002a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9002f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9009fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd9001350 con 0x55bdd8ff99e0 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd8f190d0 con 0x55bdd8ff8810 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd90014d0 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fbef7f740 10 client.0 _ll_register_callbacks cb 0x55bdd8f1de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9002f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9002f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54228/0 (socket says 172.21.15.137:54228) 2024-03-26T14:03:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3850522960 learned_addr learned my addr 172.21.15.137:0/3850522960 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.271+0000 7f6fb9800640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd9002a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fba200640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9009fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3850522960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 msgr2=0x55bdd9009fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9009fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 msgr2=0x55bdd9002a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd9002a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3850522960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdd8c713f0 con 0x55bdd8ff99e0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9002f60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00136e0 tx=0x7f6fa0013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50a87bcb883ba87 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6f9b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6f9b400640 1 -- 172.21.15.137:0/3850522960 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0020060 con 0x55bdd8ff99e0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6f9b400640 1 -- 172.21.15.137:0/3850522960 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6fa0011d70 con 0x55bdd8ff99e0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.272+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fa809b980 con 0x55bdd8ff99e0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6f9b400640 1 -- 172.21.15.137:0/3850522960 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fa0027050 con 0x55bdd8ff99e0 2024-03-26T14:03:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 10 client.15756 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bdd8d3b7d0 con 0x55bdd8ff99e0 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6f9b400640 1 -- 172.21.15.137:0/3850522960 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fa809b980 con 0x55bdd8ff99e0 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6f9b400640 1 -- 172.21.15.137:0/3850522960 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6fa001a030 con 0x55bdd8ff99e0 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 10 client.15756 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 10 client.15756 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 20 client.15756 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd900e560 0x55bdd90109c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 4 client.15756 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.273+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdd8f6a800 con 0x55bdd900e560 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.274+0000 7f6fb9800640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd900e560 0x55bdd90109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.275+0000 7f6f9b400640 10 client.15756 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.275+0000 7f6fb9800640 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd900e560 0x55bdd90109c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aaf2a76415cf4b72 server_cookie=7b5fde21386320fb in_seq=0 out_seq=0 2024-03-26T14:03:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.276+0000 7f6f9b400640 1 -- 172.21.15.137:0/3850522960 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bdd8f6a800 con 0x55bdd900e560 2024-03-26T14:03:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.276+0000 7f6f9b400640 10 client.15756 handle_command_reply: tid=0 2024-03-26T14:03:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.276+0000 7f6fbef7f740 1 client.15756 shutdown 2024-03-26T14:03:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 msgr2=0x55bdd9002f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9002f60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00136e0 tx=0x7f6fa0013ba0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 shutdown_connections 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd900e560 0x55bdd90109c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9009fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9002f60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3850522960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd9002a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6f9b400640 0 client.15756 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 >> 172.21.15.137:0/3850522960 conn(0x55bdd8ff3c00 msgr2=0x55bdd8ff2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 shutdown_connections 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3850522960 wait complete. 2024-03-26T14:03:04.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.277+0000 7f6fbef7f740 20 client.15756 trim_cache size 0 max 16384 2024-03-26T14:03:04.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 Processor -- start 2024-03-26T14:03:04.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 -- start start 2024-03-26T14:03:04.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ffcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 0x55bdd8ffd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41b2e10 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41b0910 con 0x55bdd8ff8810 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41ae990 con 0x55bdd8ff99e0 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fba200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 0x55bdd8ffd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fba200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 0x55bdd8ffd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54236/0 (socket says 172.21.15.137:54236) 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.291+0000 7f6fba200640 1 -- 172.21.15.137:0/2789500508 learned_addr learned my addr 172.21.15.137:0/2789500508 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fb9800640 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ffcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fba200640 1 -- 172.21.15.137:0/2789500508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff99e0 msgr2=0x55bdd9009f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fba200640 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fba200640 1 -- 172.21.15.137:0/2789500508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 msgr2=0x55bdd8ffcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fba200640 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ffcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fba200640 1 -- 172.21.15.137:0/2789500508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd900e340 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.292+0000 7f6fba200640 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 0x55bdd8ffd150 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009ca0 tx=0x7f6fac040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1817fbf73cded669 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fa6000640 1 -- 172.21.15.137:0/2789500508 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac046070 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fa6000640 1 -- 172.21.15.137:0/2789500508 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fac052070 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fa6000640 1 -- 172.21.15.137:0/2789500508 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac039040 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 -- 172.21.15.137:0/2789500508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 msgr2=0x55bdd8ffd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 0x55bdd8ffd150 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009ca0 tx=0x7f6fac040000 comp rx=0 tx=0).stop 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 -- 172.21.15.137:0/2789500508 shutdown_connections 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ffa3a0 0x55bdd8ffd150 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/2789500508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ff8810 0x55bdd8ffcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 -- 172.21.15.137:0/2789500508 >> 172.21.15.137:0/2789500508 conn(0x55bdd8ff3c00 msgr2=0x55bdd8ff76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.293+0000 7f6fbef7f740 1 -- 172.21.15.137:0/2789500508 shutdown_connections 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 -- 172.21.15.137:0/2789500508 wait complete. 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 -- start start 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 Processor -- start 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff8810 0x55bdd9009900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9007f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41aef90 con 0x55bdd8ff99e0 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41b02b0 con 0x55bdd8ffa3a0 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41af5f0 con 0x55bdd8ff8810 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.294+0000 7f6fbef7f740 10 client.0 _ll_register_callbacks cb 0x7f6fa00095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54238/0 (socket says 172.21.15.137:54238) 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3911236895 learned_addr learned my addr 172.21.15.137:0/3911236895 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fba200640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9007f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb9800640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff8810 0x55bdd9009900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3911236895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff8810 msgr2=0x55bdd9009900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff8810 0x55bdd9009900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ffa3a0 msgr2=0x55bdd9007f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9007f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 -- 172.21.15.137:0/3911236895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdd9008620 con 0x55bdd8ff99e0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6fb8e00640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009e40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00136e0 tx=0x7f6fa0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d574984a60854a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.295+0000 7f6f9b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6f9b400640 1 -- 172.21.15.137:0/3911236895 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa001bdb0 con 0x55bdd8ff99e0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6f9b400640 1 -- 172.21.15.137:0/3911236895 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6fa0011b70 con 0x55bdd8ff99e0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fac052070 con 0x55bdd8ff99e0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6f9b400640 1 -- 172.21.15.137:0/3911236895 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fa0038050 con 0x55bdd8ff99e0 2024-03-26T14:03:04.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 10 client.15762 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bdd906a8f0 con 0x55bdd8ff99e0 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6f9b400640 1 -- 172.21.15.137:0/3911236895 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6fac052070 con 0x55bdd8ff99e0 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6f9b400640 1 -- 172.21.15.137:0/3911236895 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6fa0033030 con 0x55bdd8ff99e0 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 10 client.15762 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 10 client.15762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 20 client.15762 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd906b2a0 0x55bdd906d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 4 client.15762 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:03:04.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.296+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bdd8f64b50 con 0x55bdd906b2a0 2024-03-26T14:03:04.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.297+0000 7f6fb9800640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd906b2a0 0x55bdd906d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fb9800640 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd906b2a0 0x55bdd906d6c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21ff956e1d5921f8 server_cookie=a6786cc43e7a749a in_seq=0 out_seq=0 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6f9b400640 10 client.15762 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6f9b400640 1 -- 172.21.15.137:0/3911236895 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bdd8f64b50 con 0x55bdd906b2a0 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6f9b400640 10 client.15762 handle_command_reply: tid=0 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 client.15762 shutdown 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 msgr2=0x55bdd9009e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009e40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00136e0 tx=0x7f6fa0009ca0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 shutdown_connections 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bdd906b2a0 0x55bdd906d6c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdd8ffa3a0 0x55bdd9007f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bdd8ff99e0 0x55bdd9009e40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 --2- 172.21.15.137:0/3911236895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdd8ff8810 0x55bdd9009900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 >> 172.21.15.137:0/3911236895 conn(0x55bdd8ff3c00 msgr2=0x55bdd8ff2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6f9b400640 0 client.15762 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:04.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 shutdown_connections 2024-03-26T14:03:04.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 1 -- 172.21.15.137:0/3911236895 wait complete. 2024-03-26T14:03:04.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.298+0000 7f6fbef7f740 20 client.15762 trim_cache size 0 max 16384 2024-03-26T14:03:04.300 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f9c0a2890 msgr2=0x7f6f9c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f9c0a2890 0x7f6f9c0a4d50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6fac02c630 tx=0x7f6fac009360 comp rx=0 tx=0).stop 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 msgr2=0x7f6fb419dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 0x7f6fb419dea0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800c970 tx=0x7f6fa800ce30 comp rx=0 tx=0).stop 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 shutdown_connections 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f9c0a2890 0x7f6f9c0a4d50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb4109870 0x7f6fb41a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6fb4108eb0 0x7f6fb419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 --2- 172.21.15.137:0/633512730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6fb4107cc0 0x7f6fb419dea0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 >> 172.21.15.137:0/633512730 conn(0x7f6fb4076c60 msgr2=0x7f6fb410c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 shutdown_connections 2024-03-26T14:03:04.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.299+0000 7f6fbc200640 1 -- 172.21.15.137:0/633512730 wait complete. 2024-03-26T14:03:04.310 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:04 smithi137 ceph-mon[31539]: pgmap v479: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:03:04.310 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:04 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3853697877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:03:04.314 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:03:04.314 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:03:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 -- 172.21.15.137:0/3042474727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec1038e0 msgr2=0x7f37ec103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 --2- 172.21.15.137:0/3042474727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec1038e0 0x7f37ec103d00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009900 tx=0x7f37dc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:03:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 -- 172.21.15.137:0/3042474727 shutdown_connections 2024-03-26T14:03:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 --2- 172.21.15.137:0/3042474727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec104ad0 0x7f37ec104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 --2- 172.21.15.137:0/3042474727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37ec105490 0x7f37ec10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 --2- 172.21.15.137:0/3042474727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec1038e0 0x7f37ec103d00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 -- 172.21.15.137:0/3042474727 >> 172.21.15.137:0/3042474727 conn(0x7f37ec0fee50 msgr2=0x7f37ec101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 -- 172.21.15.137:0/3042474727 shutdown_connections 2024-03-26T14:03:04.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.515+0000 7f37f3000640 1 -- 172.21.15.137:0/3042474727 wait complete. 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.516+0000 7f37f3000640 1 Processor -- start 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.516+0000 7f37f3000640 1 -- start start 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f3000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec1038e0 0x7f37ec199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f3000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 0x7f37ec19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f3000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37ec105490 0x7f37ec1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f3000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec10f430 con 0x7f37ec104ad0 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f3000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec10f2b0 con 0x7f37ec1038e0 2024-03-26T14:03:04.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f3000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec10f5b0 con 0x7f37ec105490 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 0x7f37ec19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 0x7f37ec19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54256/0 (socket says 172.21.15.137:54256) 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37e3400640 1 -- 172.21.15.137:0/148315500 learned_addr learned my addr 172.21.15.137:0/148315500 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37ebe00640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec1038e0 0x7f37ec199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37f1000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37ec105490 0x7f37ec1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37e3400640 1 -- 172.21.15.137:0/148315500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37ec105490 msgr2=0x7f37ec1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37e3400640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37ec105490 0x7f37ec1a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.517+0000 7f37e3400640 1 -- 172.21.15.137:0/148315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec1038e0 msgr2=0x7f37ec199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37e3400640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec1038e0 0x7f37ec199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37e3400640 1 -- 172.21.15.137:0/148315500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec1a1750 con 0x7f37ec104ad0 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37ebe00640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec1038e0 0x7f37ec199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37e3400640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 0x7f37ec19a070 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f37d400d950 tx=0x7f37d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37e9600640 1 -- 172.21.15.137:0/148315500 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4017030 con 0x7f37ec104ad0 2024-03-26T14:03:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37e9600640 1 -- 172.21.15.137:0/148315500 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37d4004020 con 0x7f37ec104ad0 2024-03-26T14:03:04.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec1a1a40 con 0x7f37ec104ad0 2024-03-26T14:03:04.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37e9600640 1 -- 172.21.15.137:0/148315500 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40047a0 con 0x7f37ec104ad0 2024-03-26T14:03:04.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.518+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec1a1f90 con 0x7f37ec104ad0 2024-03-26T14:03:04.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.521+0000 7f37f5dd4740 1 Processor -- start 2024-03-26T14:03:04.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.521+0000 7f37e9600640 1 -- 172.21.15.137:0/148315500 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f37d40041d0 con 0x7f37ec104ad0 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 -- start start 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076516810 0x55c076516c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076517e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c07651ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07647b2c0 con 0x55c0765179e0 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c076423c90 con 0x55c076516810 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c076477360 con 0x55c0765183a0 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076517e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076517e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54272/0 (socket says 172.21.15.137:54272) 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37ebe00640 1 -- 172.21.15.137:0/97215594 learned_addr learned my addr 172.21.15.137:0/97215594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.522+0000 7f37f1000640 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076516810 0x55c076516c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e3400640 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c07651ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37ebe00640 1 -- 172.21.15.137:0/97215594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 msgr2=0x55c07651ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37ebe00640 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c07651ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37ebe00640 1 -- 172.21.15.137:0/97215594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076516810 msgr2=0x55c076516c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37ebe00640 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076516810 0x55c076516c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37ebe00640 1 -- 172.21.15.137:0/97215594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c076448080 con 0x55c0765179e0 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e3400640 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c07651ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e9600640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37d00a2890 0x7f37d00a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37ebe00640 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076517e60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009a20 tx=0x7f37dc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522175a5dce14ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e9600640 1 -- 172.21.15.137:0/148315500 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f37d40cebb0 con 0x7f37ec104ad0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e1600640 1 -- 172.21.15.137:0/97215594 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc043020 con 0x55c0765179e0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37f1000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37d00a2890 0x7f37d00a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e1600640 1 -- 172.21.15.137:0/97215594 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37dc02ca60 con 0x55c0765179e0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.523+0000 7f37e1600640 1 -- 172.21.15.137:0/97215594 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0046c0 con 0x55c0765179e0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 -- 172.21.15.137:0/97215594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 msgr2=0x55c076517e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076517e60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009a20 tx=0x7f37dc009650 comp rx=0 tx=0).stop 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f1000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37d00a2890 0x7f37d00a4d50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55c0765178e0 tx=0x7f37e4009440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 -- 172.21.15.137:0/97215594 shutdown_connections 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c07651ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076517e60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/97215594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076516810 0x55c076516c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 -- 172.21.15.137:0/97215594 >> 172.21.15.137:0/97215594 conn(0x55c076511c00 msgr2=0x55c076512000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 -- 172.21.15.137:0/97215594 shutdown_connections 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.524+0000 7f37f5dd4740 1 -- 172.21.15.137:0/97215594 wait complete. 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 -- start start 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 Processor -- start 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076520a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0765179e0 0x55c076520f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c076527f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07651f350 con 0x55c076516810 2024-03-26T14:03:04.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0764370d0 con 0x55c0765179e0 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c07651f4d0 con 0x55c0765183a0 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.525+0000 7f37f5dd4740 10 client.0 _ll_register_callbacks cb 0x55c07643be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076520a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076520a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54278/0 (socket says 172.21.15.137:54278) 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 -- 172.21.15.137:0/867330746 learned_addr learned my addr 172.21.15.137:0/867330746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37e3400640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0765179e0 0x55c076520f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37f1000640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c076527f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 -- 172.21.15.137:0/867330746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 msgr2=0x55c076527f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c076527f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 -- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0765179e0 msgr2=0x55c076520f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0765179e0 0x55c076520f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 -- 172.21.15.137:0/867330746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c07618f3f0 con 0x55c076516810 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37f1000640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c076527f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37e3400640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0765179e0 0x55c076520f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:04.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37ebe00640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076520a00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005c20 tx=0x7f37dc02c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc40b2e8c64464c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.526+0000 7f37da000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37da000640 1 -- 172.21.15.137:0/867330746 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc043030 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37da000640 1 -- 172.21.15.137:0/867330746 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f37dc003a00 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37dc02ca60 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37da000640 1 -- 172.21.15.137:0/867330746 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37dc058050 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37f5dd4740 10 client.15774 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c0762597d0 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37da000640 1 -- 172.21.15.137:0/867330746 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f37dc02ca60 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37da000640 1 -- 172.21.15.137:0/867330746 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f37dc04abb0 con 0x55c076516810 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37f5dd4740 10 client.15774 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.527+0000 7f37f5dd4740 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.528+0000 7f37f5dd4740 20 client.15774 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.528+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c07652c050 0x55c07652e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.528+0000 7f37f5dd4740 4 client.15774 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:04.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.528+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c076488800 con 0x55c07652c050 2024-03-26T14:03:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.528+0000 7f37e3400640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c07652c050 0x55c07652e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.529+0000 7f37e3400640 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c07652c050 0x55c07652e440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ba06b8913817bb server_cookie=e1002b20a021d1d3 in_seq=0 out_seq=0 2024-03-26T14:03:04.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.529+0000 7f37da000640 10 client.15774 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.531+0000 7f37da000640 1 -- 172.21.15.137:0/867330746 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x7f37d4079e50 con 0x55c07652c050 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.531+0000 7f37da000640 10 client.15774 handle_command_reply: tid=0 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.531+0000 7f37f5dd4740 1 client.15774 shutdown 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.531+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 msgr2=0x55c076520a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.531+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076520a00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f37dc005c20 tx=0x7f37dc02c330 comp rx=0 tx=0).stop 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 shutdown_connections 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c07652c050 0x55c07652e440 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0765179e0 0x55c076520f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765183a0 0x55c076527f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/867330746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076520a00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 >> 172.21.15.137:0/867330746 conn(0x55c076511c00 msgr2=0x55c076510eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37da000640 0 client.15774 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 shutdown_connections 2024-03-26T14:03:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 1 -- 172.21.15.137:0/867330746 wait complete. 2024-03-26T14:03:04.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.532+0000 7f37f5dd4740 20 client.15774 trim_cache size 0 max 16384 2024-03-26T14:03:04.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.545+0000 7f37f5dd4740 1 Processor -- start 2024-03-26T14:03:04.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.545+0000 7f37f5dd4740 1 -- start start 2024-03-26T14:03:04.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.545+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c07651aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.545+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076527f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765179e0 0x55c07651afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1ae9d0 con 0x55c076516810 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1ac4d0 con 0x55c076511c00 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1aa550 con 0x55c0765179e0 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076527f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076527f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54282/0 (socket says 172.21.15.137:54282) 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 -- 172.21.15.137:0/3251660818 learned_addr learned my addr 172.21.15.137:0/3251660818 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37ebe00640 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c07651aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 -- 172.21.15.137:0/3251660818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765179e0 msgr2=0x55c07651afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765179e0 0x55c07651afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 -- 172.21.15.137:0/3251660818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 msgr2=0x55c07651aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c07651aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 -- 172.21.15.137:0/3251660818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c076517460 con 0x55c076516810 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.546+0000 7f37e3400640 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076527f40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f37d409ace0 tx=0x7f37d40d1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85948a479fefdee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37e0c00640 1 -- 172.21.15.137:0/3251660818 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40a3780 con 0x55c076516810 2024-03-26T14:03:04.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37e0c00640 1 -- 172.21.15.137:0/3251660818 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55c076488800 con 0x55c076516810 2024-03-26T14:03:04.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37e0c00640 1 -- 172.21.15.137:0/3251660818 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40a32f0 con 0x55c076516810 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 -- 172.21.15.137:0/3251660818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 msgr2=0x55c076527f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076527f40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f37d409ace0 tx=0x7f37d40d1000 comp rx=0 tx=0).stop 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 -- 172.21.15.137:0/3251660818 shutdown_connections 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0765179e0 0x55c07651afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c076516810 0x55c076527f40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/3251660818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c07651aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 -- 172.21.15.137:0/3251660818 >> 172.21.15.137:0/3251660818 conn(0x55c07651fc10 msgr2=0x55c0765156f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 -- 172.21.15.137:0/3251660818 shutdown_connections 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.547+0000 7f37f5dd4740 1 -- 172.21.15.137:0/3251660818 wait complete. 2024-03-26T14:03:04.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 -- start start 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 Processor -- start 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c0765238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c076516810 0x55c076523e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076526df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1aab50 con 0x55c0765179e0 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1abe70 con 0x55c076511c00 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1ab1b0 con 0x55c076516810 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.548+0000 7f37f5dd4740 10 client.0 _ll_register_callbacks cb 0x7f37dc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076526df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076526df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54286/0 (socket says 172.21.15.137:54286) 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 -- 172.21.15.137:0/596750731 learned_addr learned my addr 172.21.15.137:0/596750731 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:04.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37ebe00640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c0765238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37e3400640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c076516810 0x55c076523e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 -- 172.21.15.137:0/596750731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c076516810 msgr2=0x55c076523e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c076516810 0x55c076523e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 -- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 msgr2=0x55c0765238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c0765238e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 -- 172.21.15.137:0/596750731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0765273e0 con 0x55c0765179e0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37f1000640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076526df0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f37e4012700 tx=0x7f37e4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de315f1b7df404da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37da000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.549+0000 7f37da000640 1 -- 172.21.15.137:0/596750731 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e4011ae0 con 0x55c0765179e0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c076488800 con 0x55c0765179e0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37da000640 1 -- 172.21.15.137:0/596750731 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f37e4011c80 con 0x55c0765179e0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37da000640 1 -- 172.21.15.137:0/596750731 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37e401f420 con 0x55c0765179e0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 10 client.15780 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c076587a60 con 0x55c0765179e0 2024-03-26T14:03:04.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37da000640 1 -- 172.21.15.137:0/596750731 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55c076488800 con 0x55c0765179e0 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37da000640 1 -- 172.21.15.137:0/596750731 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f37e400f040 con 0x55c0765179e0 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 10 client.15780 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 20 client.15780 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c0765183a0 0x55c07658a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 4 client.15780 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.550+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c076482b50 con 0x55c0765183a0 2024-03-26T14:03:04.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.551+0000 7f37ebe00640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c0765183a0 0x55c07658a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:04.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37ebe00640 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c0765183a0 0x55c07658a3f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e75072a287ee09e1 server_cookie=d84f19a080fdf843 in_seq=0 out_seq=0 2024-03-26T14:03:04.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37da000640 10 client.15780 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:04.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37da000640 1 -- 172.21.15.137:0/596750731 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c076482b50 con 0x55c0765183a0 2024-03-26T14:03:04.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37da000640 10 client.15780 handle_command_reply: tid=0 2024-03-26T14:03:04.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37f5dd4740 1 client.15780 shutdown 2024-03-26T14:03:04.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 msgr2=0x55c076526df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.552+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076526df0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f37e4012700 tx=0x7f37e4012bc0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 shutdown_connections 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c0765183a0 0x55c07658a3f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0765179e0 0x55c076526df0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c076516810 0x55c076523e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 --2- 172.21.15.137:0/596750731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c076511c00 0x55c0765238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37da000640 0 client.15780 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 >> 172.21.15.137:0/596750731 conn(0x55c07651fc10 msgr2=0x55c076510620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 shutdown_connections 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 1 -- 172.21.15.137:0/596750731 wait complete. 2024-03-26T14:03:04.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f5dd4740 20 client.15780 trim_cache size 0 max 16384 2024-03-26T14:03:04.555 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:03:04.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37d00a2890 msgr2=0x7f37d00a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f3000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37d00a2890 0x7f37d00a4d50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55c0765178e0 tx=0x7f37e4009440 comp rx=0 tx=0).stop 2024-03-26T14:03:04.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 msgr2=0x7f37ec19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:04.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.553+0000 7f37f3000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 0x7f37ec19a070 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f37d400d950 tx=0x7f37d400de10 comp rx=0 tx=0).stop 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 shutdown_connections 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37d00a2890 0x7f37d00a4d50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37ec105490 0x7f37ec1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37ec104ad0 0x7f37ec19a070 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 --2- 172.21.15.137:0/148315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37ec1038e0 0x7f37ec199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 >> 172.21.15.137:0/148315500 conn(0x7f37ec0fee50 msgr2=0x7f37ec107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 shutdown_connections 2024-03-26T14:03:04.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:04.554+0000 7f37f3000640 1 -- 172.21.15.137:0/148315500 wait complete. 2024-03-26T14:03:04.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:04 smithi179 ceph-mon[35472]: pgmap v479: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:03:04.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:04 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3853697877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:03:04.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:04 smithi159 ceph-mon[35501]: pgmap v479: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:03:04.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:04 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3853697877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:03:05.568 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:03:05.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:03:05.569 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:03:05.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 -- 172.21.15.137:0/2578313519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6070104ad0 msgr2=0x7f6070104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 --2- 172.21.15.137:0/2578313519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6070104ad0 0x7f6070104f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6058009900 tx=0x7f605802bc30 comp rx=0 tx=0).stop 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 -- 172.21.15.137:0/2578313519 shutdown_connections 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 --2- 172.21.15.137:0/2578313519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6070104ad0 0x7f6070104f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 --2- 172.21.15.137:0/2578313519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f607010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 --2- 172.21.15.137:0/2578313519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60701038e0 0x7f6070103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 -- 172.21.15.137:0/2578313519 >> 172.21.15.137:0/2578313519 conn(0x7f60700fee50 msgr2=0x7f6070101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 -- 172.21.15.137:0/2578313519 shutdown_connections 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.798+0000 7f6076600640 1 -- 172.21.15.137:0/2578313519 wait complete. 2024-03-26T14:03:05.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.799+0000 7f6076600640 1 Processor -- start 2024-03-26T14:03:05.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.799+0000 7f6076600640 1 -- start start 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.799+0000 7f6076600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60701038e0 0x7f6070199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f6076600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6070104ad0 0x7f607019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f6076600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f60701a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f6076600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010f410 con 0x7f6070105490 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f6076600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010f290 con 0x7f60701038e0 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f6076600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607010f590 con 0x7f6070104ad0 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f60701a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f60701a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54296/0 (socket says 172.21.15.137:54296) 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 -- 172.21.15.137:0/2049918474 learned_addr learned my addr 172.21.15.137:0/2049918474 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606f400640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60701038e0 0x7f6070199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f6067e00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6070104ad0 0x7f607019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 -- 172.21.15.137:0/2049918474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6070104ad0 msgr2=0x7f607019a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6070104ad0 0x7f607019a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 -- 172.21.15.137:0/2049918474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60701038e0 msgr2=0x7f6070199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.800+0000 7f606fe00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60701038e0 0x7f6070199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f606fe00640 1 -- 172.21.15.137:0/2049918474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60701a17d0 con 0x7f6070105490 2024-03-26T14:03:05.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f606fe00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f60701a1160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f606800efd0 tx=0x7f606800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:05.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f606cc00640 1 -- 172.21.15.137:0/2049918474 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068003b70 con 0x7f6070105490 2024-03-26T14:03:05.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f606cc00640 1 -- 172.21.15.137:0/2049918474 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6068003d10 con 0x7f6070105490 2024-03-26T14:03:05.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60701a1ac0 con 0x7f6070105490 2024-03-26T14:03:05.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f606cc00640 1 -- 172.21.15.137:0/2049918474 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068010980 con 0x7f6070105490 2024-03-26T14:03:05.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.801+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60701a1f90 con 0x7f6070105490 2024-03-26T14:03:05.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.804+0000 7f606cc00640 1 -- 172.21.15.137:0/2049918474 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6068004020 con 0x7f6070105490 2024-03-26T14:03:05.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.805+0000 7f60791ea740 1 Processor -- start 2024-03-26T14:03:05.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.805+0000 7f60791ea740 1 -- start start 2024-03-26T14:03:05.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f60791ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f60791ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb6a90 0x55a99cebb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f60791ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99cebbc00 0x55a99cebe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f60791ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ce1beb0 con 0x55a99ceb60a0 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f60791ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99cdc4a60 con 0x55a99cebbc00 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f60791ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99ce17f50 con 0x55a99ceb6a90 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54308/0 (socket says 172.21.15.137:54308) 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 -- 172.21.15.137:0/916977209 learned_addr learned my addr 172.21.15.137:0/916977209 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f606f400640 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb6a90 0x55a99cebb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f606cc00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2890 0x7f60540a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 -- 172.21.15.137:0/916977209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb6a90 msgr2=0x55a99cebb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb6a90 0x55a99cebb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 -- 172.21.15.137:0/916977209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99cebbc00 msgr2=0x55a99cebe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99cebbc00 0x55a99cebe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f6067e00640 1 -- 172.21.15.137:0/916977209 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a99cbf87e0 con 0x55a99ceb60a0 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.806+0000 7f606f400640 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb6a90 0x55a99cebb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.807+0000 7f606cc00640 1 -- 172.21.15.137:0/2049918474 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f60680cf460 con 0x7f6070105490 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.807+0000 7f606fe00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2890 0x7f60540a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.807+0000 7f6067e00640 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb64c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6058009900 tx=0x7f6058005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc136e384f0793f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.807+0000 7f6064c00640 1 -- 172.21.15.137:0/916977209 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058044020 con 0x55a99ceb60a0 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.807+0000 7f6064c00640 1 -- 172.21.15.137:0/916977209 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6058004030 con 0x55a99ceb60a0 2024-03-26T14:03:05.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.807+0000 7f6064c00640 1 -- 172.21.15.137:0/916977209 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058005600 con 0x55a99ceb60a0 2024-03-26T14:03:05.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 -- 172.21.15.137:0/916977209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 msgr2=0x55a99ceb64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb64c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6058009900 tx=0x7f6058005d50 comp rx=0 tx=0).stop 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f606fe00640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2890 0x7f60540a4d50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6068005c40 tx=0x7f6068007850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 -- 172.21.15.137:0/916977209 shutdown_connections 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99cebbc00 0x55a99cebe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb6a90 0x55a99cebb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 --2- 172.21.15.137:0/916977209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb64c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 -- 172.21.15.137:0/916977209 >> 172.21.15.137:0/916977209 conn(0x55a99ceb1610 msgr2=0x55a99ceb1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 -- 172.21.15.137:0/916977209 shutdown_connections 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 -- 172.21.15.137:0/916977209 wait complete. 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.808+0000 7f60791ea740 1 -- start start 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 Processor -- start 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99cec02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 0x55a99cec7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99cebeb50 con 0x55a99ceb60a0 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99cdd7eb0 con 0x55a99ceb6a90 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a99cebecd0 con 0x55a99cebbc00 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f60791ea740 10 client.0 _ll_register_callbacks cb 0x55a99cdcc150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99cec02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99cec02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54318/0 (socket says 172.21.15.137:54318) 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.809+0000 7f6067e00640 1 -- 172.21.15.137:0/174330461 learned_addr learned my addr 172.21.15.137:0/174330461 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f606f400640 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f6067e00640 1 -- 172.21.15.137:0/174330461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 msgr2=0x55a99cec7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f6067e00640 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 0x55a99cec7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f6067e00640 1 -- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 msgr2=0x55a99cec07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f6067e00640 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f6067e00640 1 -- 172.21.15.137:0/174330461 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a99cc58f70 con 0x55a99ceb60a0 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f6067e00640 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99cec02b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6058036040 tx=0x7f60580056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f398069d374b56 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:05.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f605d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f605d600640 1 -- 172.21.15.137:0/174330461 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058044030 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f605d600640 1 -- 172.21.15.137:0/174330461 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f60580035f0 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.810+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6058004030 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f605d600640 1 -- 172.21.15.137:0/174330461 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60580037f0 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 10 client.15792 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a99caf0170 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f605d600640 1 -- 172.21.15.137:0/174330461 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6058004030 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f605d600640 1 -- 172.21.15.137:0/174330461 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6058004be0 con 0x55a99ceb60a0 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 10 client.15792 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 20 client.15792 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cecbc90 0x55a99cece080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 4 client.15792 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.811+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a99ce293f0 con 0x55a99cecbc90 2024-03-26T14:03:05.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.812+0000 7f606f400640 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cecbc90 0x55a99cece080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.813+0000 7f605d600640 10 client.15792 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.813+0000 7f606f400640 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cecbc90 0x55a99cece080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db76331dec569246 server_cookie=c90827b048ade792 in_seq=0 out_seq=0 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f605d600640 1 -- 172.21.15.137:0/174330461 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a99ce293f0 con 0x55a99cecbc90 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f605d600640 10 client.15792 handle_command_reply: tid=0 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 client.15792 shutdown 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 msgr2=0x55a99cec02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99cec02b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6058036040 tx=0x7f60580056a0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 shutdown_connections 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cecbc90 0x55a99cece080 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 0x55a99cec7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f605d600640 0 client.15792 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:05.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 --2- 172.21.15.137:0/174330461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99cec02b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.814+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 >> 172.21.15.137:0/174330461 conn(0x55a99ceb1610 msgr2=0x55a99ceb0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:05.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.815+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 shutdown_connections 2024-03-26T14:03:05.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.815+0000 7f60791ea740 1 -- 172.21.15.137:0/174330461 wait complete. 2024-03-26T14:03:05.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.815+0000 7f60791ea740 20 client.15792 trim_cache size 0 max 16384 2024-03-26T14:03:05.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.832+0000 7f60791ea740 1 Processor -- start 2024-03-26T14:03:05.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.832+0000 7f60791ea740 1 -- start start 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.832+0000 7f60791ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f60791ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f60791ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 0x55a99ceaffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f60791ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701ae850 con 0x55a99ceb60a0 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f60791ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701ac350 con 0x55a99ceb6a90 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f60791ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701aa3d0 con 0x55a99cebbc00 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54330/0 (socket says 172.21.15.137:54330) 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 -- 172.21.15.137:0/3379793871 learned_addr learned my addr 172.21.15.137:0/3379793871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:05.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f606f400640 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 -- 172.21.15.137:0/3379793871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 msgr2=0x55a99ceaffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 0x55a99ceaffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 -- 172.21.15.137:0/3379793871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 msgr2=0x55a99cec77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f6067e00640 1 -- 172.21.15.137:0/3379793871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a99ceacc20 con 0x55a99ceb60a0 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.833+0000 7f606f400640 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.834+0000 7f6067e00640 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb4ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6058005d00 tx=0x7f605804c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27ba1ce3b8ce9ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.834+0000 7f605fe00640 1 -- 172.21.15.137:0/3379793871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058044070 con 0x55a99ceb60a0 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.834+0000 7f605fe00640 1 -- 172.21.15.137:0/3379793871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f605804c620 con 0x55a99ceb60a0 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.834+0000 7f605fe00640 1 -- 172.21.15.137:0/3379793871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605804cda0 con 0x55a99ceb60a0 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 -- 172.21.15.137:0/3379793871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 msgr2=0x55a99ceb4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb4ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6058005d00 tx=0x7f605804c4a0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 -- 172.21.15.137:0/3379793871 shutdown_connections 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99cebbc00 0x55a99ceaffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 --2- 172.21.15.137:0/3379793871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99ceb60a0 0x55a99ceb4ee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 -- 172.21.15.137:0/3379793871 >> 172.21.15.137:0/3379793871 conn(0x55a99ceb1610 msgr2=0x55a99ceb7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 -- 172.21.15.137:0/3379793871 shutdown_connections 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 -- 172.21.15.137:0/3379793871 wait complete. 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 -- start start 2024-03-26T14:03:05.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.835+0000 7f60791ea740 1 Processor -- start 2024-03-26T14:03:05.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb60a0 0x55a99cec58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 0x55a99cec6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701aa9d0 con 0x55a99cebbc00 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701abcf0 con 0x55a99ceb6a90 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60701ab030 con 0x55a99ceb60a0 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f60791ea740 10 client.0 _ll_register_callbacks cb 0x7f6058009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 0x55a99cec6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f6067e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 0x55a99cec6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54338/0 (socket says 172.21.15.137:54338) 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f606f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f6067e00640 1 -- 172.21.15.137:0/3316106637 learned_addr learned my addr 172.21.15.137:0/3316106637 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.836+0000 7f6067e00640 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb60a0 0x55a99cec58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f6067e00640 1 -- 172.21.15.137:0/3316106637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb60a0 msgr2=0x55a99cec58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f6067e00640 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb60a0 0x55a99cec58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f6067e00640 1 -- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 msgr2=0x55a99cec5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f6067e00640 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f6067e00640 1 -- 172.21.15.137:0/3316106637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a99cec7350 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f6067e00640 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 0x55a99cec6cb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55a99cec66e0 tx=0x7f605802c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f0b489cdf72713 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f605d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f605d600640 1 -- 172.21.15.137:0/3316106637 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605806f060 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f605d600640 1 -- 172.21.15.137:0/3316106637 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6058003990 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f605804c620 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f605d600640 1 -- 172.21.15.137:0/3316106637 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6058044040 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f60791ea740 10 client.15798 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.837+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a99cf27d20 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f605d600640 1 -- 172.21.15.137:0/3316106637 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f605804c620 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f605d600640 1 -- 172.21.15.137:0/3316106637 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f605806a440 con 0x55a99cebbc00 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f60791ea740 10 client.15798 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:05.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f60791ea740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f60791ea740 20 client.15798 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f60791ea740 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cf28400 0x55a99cf2a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f60791ea740 4 client.15798 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.838+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a99ce23740 con 0x55a99cf28400 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.839+0000 7f606f400640 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cf28400 0x55a99cf2a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.840+0000 7f606f400640 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cf28400 0x55a99cf2a820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5e14ac17c8072e server_cookie=4345f755bfcb8ad1 in_seq=0 out_seq=0 2024-03-26T14:03:05.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.840+0000 7f605d600640 10 client.15798 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:05.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f605d600640 1 -- 172.21.15.137:0/3316106637 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a99ce23740 con 0x55a99cf28400 2024-03-26T14:03:05.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f605d600640 10 client.15798 handle_command_reply: tid=0 2024-03-26T14:03:05.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 client.15798 shutdown 2024-03-26T14:03:05.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 msgr2=0x55a99cec6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 0x55a99cec6cb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55a99cec66e0 tx=0x7f605802c3b0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 shutdown_connections 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a99cf28400 0x55a99cf2a820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a99ceb6a90 0x55a99cec5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a99cebbc00 0x55a99cec6cb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 --2- 172.21.15.137:0/3316106637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a99ceb60a0 0x55a99cec58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f605d600640 0 client.15798 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 >> 172.21.15.137:0/3316106637 conn(0x55a99ceb1610 msgr2=0x55a99ced2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 shutdown_connections 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.843+0000 7f60791ea740 1 -- 172.21.15.137:0/3316106637 wait complete. 2024-03-26T14:03:05.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f60791ea740 20 client.15798 trim_cache size 0 max 16384 2024-03-26T14:03:05.843 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:03:05.843 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:03:05.843 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "26f0eaa8-53fc-44e3-9241-af47b2c0bd58", 2024-03-26T14:03:05.843 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:03:05.843 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:03:05.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2890 msgr2=0x7f60540a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2890 0x7f60540a4d50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6068005c40 tx=0x7f6068007850 comp rx=0 tx=0).stop 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 msgr2=0x7f60701a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f60701a1160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f606800efd0 tx=0x7f606800c6a0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 shutdown_connections 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2890 0x7f60540a4d50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6070105490 0x7f60701a1160 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6070104ad0 0x7f607019a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 --2- 172.21.15.137:0/2049918474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60701038e0 0x7f6070199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 >> 172.21.15.137:0/2049918474 conn(0x7f60700fee50 msgr2=0x7f6070107f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 shutdown_connections 2024-03-26T14:03:05.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:05.844+0000 7f6076600640 1 -- 172.21.15.137:0/2049918474 wait complete. 2024-03-26T14:03:05.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:03:05.861 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:03:06.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 -- 172.21.15.137:0/1520304875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c109770 msgr2=0x7fca5c110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 --2- 172.21.15.137:0/1520304875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c109770 0x7fca5c110000 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fca54009ca0 tx=0x7fca5402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 -- 172.21.15.137:0/1520304875 shutdown_connections 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 --2- 172.21.15.137:0/1520304875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c109770 0x7fca5c110000 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 --2- 172.21.15.137:0/1520304875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca5c108db0 0x7fca5c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 --2- 172.21.15.137:0/1520304875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca5c107bc0 0x7fca5c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.078+0000 7fca5b200640 1 -- 172.21.15.137:0/1520304875 >> 172.21.15.137:0/1520304875 conn(0x7fca5c076b70 msgr2=0x7fca5c078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.079+0000 7fca5b200640 1 -- 172.21.15.137:0/1520304875 shutdown_connections 2024-03-26T14:03:06.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.079+0000 7fca5b200640 1 -- 172.21.15.137:0/1520304875 wait complete. 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.079+0000 7fca5b200640 1 Processor -- start 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 -- start start 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 0x7fca5c19ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca5c108db0 0x7fca5c19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca5c109770 0x7fca5c1a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1136f0 con 0x7fca5c107bc0 2024-03-26T14:03:06.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c113570 con 0x7fca5c109770 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca5b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c113870 con 0x7fca5c108db0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca59e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 0x7fca5c19ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca59e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 0x7fca5c19ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54354/0 (socket says 172.21.15.137:54354) 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.080+0000 7fca59e00640 1 -- 172.21.15.137:0/3906835240 learned_addr learned my addr 172.21.15.137:0/3906835240 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca5a800640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca5c109770 0x7fca5c1a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca51400640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca5c108db0 0x7fca5c19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca59e00640 1 -- 172.21.15.137:0/3906835240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca5c108db0 msgr2=0x7fca5c19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca59e00640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca5c108db0 0x7fca5c19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca59e00640 1 -- 172.21.15.137:0/3906835240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca5c109770 msgr2=0x7fca5c1a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca59e00640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca5c109770 0x7fca5c1a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca59e00640 1 -- 172.21.15.137:0/3906835240 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c1a5990 con 0x7fca5c107bc0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca59e00640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 0x7fca5c19ddc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9c0 tx=0x7fca4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca53400640 1 -- 172.21.15.137:0/3906835240 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00cd80 con 0x7fca5c107bc0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca53400640 1 -- 172.21.15.137:0/3906835240 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca4c0035f0 con 0x7fca5c107bc0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c1a5c80 con 0x7fca5c107bc0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca53400640 1 -- 172.21.15.137:0/3906835240 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c010690 con 0x7fca5c107bc0 2024-03-26T14:03:06.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.081+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c1a61d0 con 0x7fca5c107bc0 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca53400640 1 -- 172.21.15.137:0/3906835240 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fca4c0037a0 con 0x7fca5c107bc0 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 Processor -- start 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 -- start start 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b59880540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59883040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b597e0250 con 0x564b5987ba40 2024-03-26T14:03:06.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b59788a30 con 0x564b5987b050 2024-03-26T14:03:06.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca63be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b597dc2f0 con 0x564b59880bb0 2024-03-26T14:03:06.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.085+0000 7fca59e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b59880540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b59880540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54370/0 (socket says 172.21.15.137:54370) 2024-03-26T14:03:06.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 -- 172.21.15.137:0/4122286542 learned_addr learned my addr 172.21.15.137:0/4122286542 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:06.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca5a800640 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca51400640 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59883040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 -- 172.21.15.137:0/4122286542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 msgr2=0x564b59883040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59883040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 -- 172.21.15.137:0/4122286542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 msgr2=0x564b5987b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca59e00640 1 -- 172.21.15.137:0/4122286542 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b59773690 con 0x564b5987ba40 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca53400640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca400a2910 0x7fca400a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca5a800640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca400a2910 0x7fca400a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.086+0000 7fca53400640 1 -- 172.21.15.137:0/3906835240 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fca4c0cf360 con 0x7fca5c107bc0 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.087+0000 7fca59e00640 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b59880540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0cc970 tx=0x7fca4c0d5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823f79ef21c0dd3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.087+0000 7fca4b400640 1 -- 172.21.15.137:0/4122286542 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c09b660 con 0x564b5987ba40 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.087+0000 7fca4b400640 1 -- 172.21.15.137:0/4122286542 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca4c003b30 con 0x564b5987ba40 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.087+0000 7fca4b400640 1 -- 172.21.15.137:0/4122286542 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c09b1f0 con 0x564b5987ba40 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 -- 172.21.15.137:0/4122286542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 msgr2=0x564b59880540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b59880540 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0cc970 tx=0x7fca4c0d5510 comp rx=0 tx=0).stop 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca5a800640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca400a2910 0x7fca400a4dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca5402c630 tx=0x7fca54005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 -- 172.21.15.137:0/4122286542 shutdown_connections 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59883040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b59880540 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 --2- 172.21.15.137:0/4122286542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 -- 172.21.15.137:0/4122286542 >> 172.21.15.137:0/4122286542 conn(0x564b59843200 msgr2=0x564b598769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 -- 172.21.15.137:0/4122286542 shutdown_connections 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 -- 172.21.15.137:0/4122286542 wait complete. 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 -- start start 2024-03-26T14:03:06.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.088+0000 7fca63be9740 1 Processor -- start 2024-03-26T14:03:06.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987b050 0x564b59884ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987ba40 0x564b59885530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b59883b60 con 0x564b59880bb0 2024-03-26T14:03:06.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5979be80 con 0x564b5987ba40 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b59883ce0 con 0x564b5987b050 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca63be9740 10 client.0 _ll_register_callbacks cb 0x564b597ae130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca5a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca5a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54372/0 (socket says 172.21.15.137:54372) 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca5a800640 1 -- 172.21.15.137:0/3705994328 learned_addr learned my addr 172.21.15.137:0/3705994328 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca51400640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987ba40 0x564b59885530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.089+0000 7fca59e00640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987b050 0x564b59884ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca5a800640 1 -- 172.21.15.137:0/3705994328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987b050 msgr2=0x564b59884ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca5a800640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987b050 0x564b59884ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca5a800640 1 -- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987ba40 msgr2=0x564b59885530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca5a800640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987ba40 0x564b59885530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca5a800640 1 -- 172.21.15.137:0/3705994328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b595be7d0 con 0x564b59880bb0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca5a800640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988c570 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fca54004af0 tx=0x7fca540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b3a3319072ef94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:06.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca3fe00640 1 -- 172.21.15.137:0/3705994328 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54039040 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca3fe00640 1 -- 172.21.15.137:0/3705994328 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fca5404c660 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.090+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b597ed790 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca3fe00640 1 -- 172.21.15.137:0/3705994328 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca5405a050 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 10 client.15810 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564b594f43f0 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca3fe00640 1 -- 172.21.15.137:0/3705994328 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564b597ed790 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca3fe00640 1 -- 172.21.15.137:0/3705994328 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fca54040960 con 0x564b59880bb0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 10 client.15810 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 10 client.15810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 20 client.15810 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b59890f10 0x564b59893370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 4 client.15810 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.091+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b597e7ae0 con 0x564b59890f10 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.092+0000 7fca59e00640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b59890f10 0x564b59893370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.093+0000 7fca59e00640 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b59890f10 0x564b59893370 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc814ea148a07fcc server_cookie=dadef5865c1a3d60 in_seq=0 out_seq=0 2024-03-26T14:03:06.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.093+0000 7fca3fe00640 10 client.15810 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:06.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.094+0000 7fca3fe00640 1 -- 172.21.15.137:0/3705994328 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564b597e7ae0 con 0x564b59890f10 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.094+0000 7fca3fe00640 10 client.15810 handle_command_reply: tid=0 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.094+0000 7fca63be9740 1 client.15810 shutdown 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 msgr2=0x564b5988c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988c570 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fca54004af0 tx=0x7fca540095c0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 shutdown_connections 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b59890f10 0x564b59893370 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987ba40 0x564b59885530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988c570 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 --2- 172.21.15.137:0/3705994328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987b050 0x564b59884ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca3fe00640 0 client.15810 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 >> 172.21.15.137:0/3705994328 conn(0x564b59843200 msgr2=0x564b59875bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 shutdown_connections 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 1 -- 172.21.15.137:0/3705994328 wait complete. 2024-03-26T14:03:06.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.095+0000 7fca63be9740 20 client.15810 trim_cache size 0 max 16384 2024-03-26T14:03:06.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.112+0000 7fca63be9740 1 Processor -- start 2024-03-26T14:03:06.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.112+0000 7fca63be9740 1 -- start start 2024-03-26T14:03:06.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca63be9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca63be9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b5988c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca63be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59879c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca63be9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1b2ad0 con 0x564b5987ba40 2024-03-26T14:03:06.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca63be9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1b05d0 con 0x564b5987b050 2024-03-26T14:03:06.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca63be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1ae650 con 0x564b59880bb0 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca51400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b5988c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca51400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b5988c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54386/0 (socket says 172.21.15.137:54386) 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca59e00640 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca51400640 1 -- 172.21.15.137:0/368766959 learned_addr learned my addr 172.21.15.137:0/368766959 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.113+0000 7fca5a800640 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59879c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca51400640 1 -- 172.21.15.137:0/368766959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 msgr2=0x564b59879c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca51400640 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59879c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca51400640 1 -- 172.21.15.137:0/368766959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 msgr2=0x564b5987cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca51400640 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca51400640 1 -- 172.21.15.137:0/368766959 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b594b40f0 con 0x564b5987ba40 2024-03-26T14:03:06.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca51400640 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b5988c530 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fca4400fc60 tx=0x7fca44013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1a8a55f477c667 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:06.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca4aa00640 1 -- 172.21.15.137:0/368766959 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44005220 con 0x564b5987ba40 2024-03-26T14:03:06.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca4aa00640 1 -- 172.21.15.137:0/368766959 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca4400fe10 con 0x564b5987ba40 2024-03-26T14:03:06.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.114+0000 7fca4aa00640 1 -- 172.21.15.137:0/368766959 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca44011810 con 0x564b5987ba40 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 -- 172.21.15.137:0/368766959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 msgr2=0x564b5988c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b5988c530 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fca4400fc60 tx=0x7fca44013830 comp rx=0 tx=0).stop 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 -- 172.21.15.137:0/368766959 shutdown_connections 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b59880bb0 0x564b59879c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b5987ba40 0x564b5988c530 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 --2- 172.21.15.137:0/368766959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b5987cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 -- 172.21.15.137:0/368766959 >> 172.21.15.137:0/368766959 conn(0x564b59843200 msgr2=0x564b5987c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 -- 172.21.15.137:0/368766959 shutdown_connections 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 -- 172.21.15.137:0/368766959 wait complete. 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.115+0000 7fca63be9740 1 -- start start 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 Processor -- start 2024-03-26T14:03:06.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b59889e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987ba40 0x564b5988a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1aec50 con 0x564b59880bb0 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1aff70 con 0x564b5987b050 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c1af2b0 con 0x564b5987ba40 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca63be9740 10 client.0 _ll_register_callbacks cb 0x7fca4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca5a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca5a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54394/0 (socket says 172.21.15.137:54394) 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.116+0000 7fca5a800640 1 -- 172.21.15.137:0/2345161661 learned_addr learned my addr 172.21.15.137:0/2345161661 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca51400640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987ba40 0x564b5988a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca59e00640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b59889e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca5a800640 1 -- 172.21.15.137:0/2345161661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987ba40 msgr2=0x564b5988a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca5a800640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987ba40 0x564b5988a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca5a800640 1 -- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 msgr2=0x564b59889e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca5a800640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b59889e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca5a800640 1 -- 172.21.15.137:0/2345161661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b5988b970 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca59e00640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b59889e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca5a800640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988b380 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fca54033000 tx=0x7fca54034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0dd213028142a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca3fe00640 1 -- 172.21.15.137:0/2345161661 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54062020 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca3fe00640 1 -- 172.21.15.137:0/2345161661 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fca54040870 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca4400fe10 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca3fe00640 1 -- 172.21.15.137:0/2345161661 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca5406d050 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca63be9740 10 client.15816 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.117+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564b598ec9f0 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca3fe00640 1 -- 172.21.15.137:0/2345161661 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca4400fe10 con 0x564b59880bb0 2024-03-26T14:03:06.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca3fe00640 1 -- 172.21.15.137:0/2345161661 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fca5404c460 con 0x564b59880bb0 2024-03-26T14:03:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca63be9740 10 client.15816 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca63be9740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca63be9740 20 client.15816 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca63be9740 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b598ed360 0x564b598ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca63be9740 4 client.15816 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:03:06.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.118+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564b597653a0 con 0x564b598ed360 2024-03-26T14:03:06.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.119+0000 7fca59e00640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b598ed360 0x564b598ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:06.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.119+0000 7fca59e00640 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b598ed360 0x564b598ef780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eaa0a7bc0869814 server_cookie=d011a7b4adef3d97 in_seq=0 out_seq=0 2024-03-26T14:03:06.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.119+0000 7fca3fe00640 10 client.15816 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:06.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca3fe00640 1 -- 172.21.15.137:0/2345161661 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564b597653a0 con 0x564b598ed360 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca3fe00640 10 client.15816 handle_command_reply: tid=0 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 client.15816 shutdown 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 msgr2=0x564b5988b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988b380 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fca54033000 tx=0x7fca54034670 comp rx=0 tx=0).stop 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 shutdown_connections 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564b598ed360 0x564b598ef780 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564b59880bb0 0x564b5988b380 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca3fe00640 0 client.15816 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564b5987ba40 0x564b5988a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 --2- 172.21.15.137:0/2345161661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564b5987b050 0x564b59889e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 >> 172.21.15.137:0/2345161661 conn(0x564b59843200 msgr2=0x564b5987cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 shutdown_connections 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.120+0000 7fca63be9740 1 -- 172.21.15.137:0/2345161661 wait complete. 2024-03-26T14:03:06.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca63be9740 20 client.15816 trim_cache size 0 max 16384 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: "26f0eaa8-53fc-44e3-9241-af47b2c0bd58": { 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "26f0eaa8-53fc-44e3-9241-af47b2c0bd58", 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:03:06.122 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca400a2910 msgr2=0x7fca400a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca400a2910 0x7fca400a4dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca5402c630 tx=0x7fca54005d80 comp rx=0 tx=0).stop 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 msgr2=0x7fca5c19ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 0x7fca5c19ddc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00e9c0 tx=0x7fca4c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 shutdown_connections 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca400a2910 0x7fca400a4dd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fca5c109770 0x7fca5c1a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca5c108db0 0x7fca5c19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 --2- 172.21.15.137:0/3906835240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fca5c107bc0 0x7fca5c19ddc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 >> 172.21.15.137:0/3906835240 conn(0x7fca5c076b70 msgr2=0x7fca5c10c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 shutdown_connections 2024-03-26T14:03:06.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:06.121+0000 7fca5b200640 1 -- 172.21.15.137:0/3906835240 wait complete. 2024-03-26T14:03:06.135 INFO:tasks.cephfs.filesystem:scrub status for tag:26f0eaa8-53fc-44e3-9241-af47b2c0bd58 - {'path': '/', 'tag': '26f0eaa8-53fc-44e3-9241-af47b2c0bd58', 'options': 'recursive,force'} 2024-03-26T14:03:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:06 smithi179 ceph-mon[35472]: pgmap v480: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:06 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:03:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:06 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:06 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:03:06.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:06 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:06.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:06 smithi159 ceph-mon[35501]: pgmap v480: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:06.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:06 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:03:06.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:06 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:06.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:06 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:03:06.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:06 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:06.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:06 smithi137 ceph-mon[31539]: pgmap v480: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:06.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:06 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:03:06.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:06 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:06.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:06 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:03:06.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:06 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:07.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:07 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:03:07.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:07 smithi179 ceph-mon[35472]: pgmap v481: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:03:07.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:07 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:03:07.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:07 smithi159 ceph-mon[35501]: pgmap v481: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:03:07.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:07 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:03:07.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:07 smithi137 ceph-mon[31539]: pgmap v481: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:03:10.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:10 smithi137 ceph-mon[31539]: pgmap v482: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:10.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:10 smithi179 ceph-mon[35472]: pgmap v482: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:10.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:10 smithi159 ceph-mon[35501]: pgmap v482: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:12.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:12 smithi137 ceph-mon[31539]: pgmap v483: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:12.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:12 smithi179 ceph-mon[35472]: pgmap v483: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:12.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:12 smithi159 ceph-mon[35501]: pgmap v483: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:14.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:14 smithi137 ceph-mon[31539]: pgmap v484: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:03:14.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:14 smithi179 ceph-mon[35472]: pgmap v484: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:03:14.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:14 smithi159 ceph-mon[35501]: pgmap v484: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:03:16.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:16 smithi137 ceph-mon[31539]: pgmap v485: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:16.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:16 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:03:16.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:16 smithi179 ceph-mon[35472]: pgmap v485: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:16.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:16 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:03:16.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:16 smithi159 ceph-mon[35501]: pgmap v485: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:16.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:16 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:03:18.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:18 smithi137 ceph-mon[31539]: pgmap v486: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:03:18.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:03:18.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:03:18.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:03:18.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:18 smithi179 ceph-mon[35472]: pgmap v486: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:03:18.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:03:18.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:03:18.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:03:18.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:18 smithi159 ceph-mon[35501]: pgmap v486: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:03:18.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:03:18.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:03:18.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:03:20.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:20 smithi137 ceph-mon[31539]: pgmap v487: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:03:20.671 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:20 smithi179 ceph-mon[35472]: pgmap v487: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:03:20.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:20 smithi159 ceph-mon[35501]: pgmap v487: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:03:22.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:22 smithi179 ceph-mon[35472]: pgmap v488: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-26T14:03:22.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:22 smithi159 ceph-mon[35501]: pgmap v488: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-26T14:03:22.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:22 smithi137 ceph-mon[31539]: pgmap v488: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-26T14:03:23.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:23 smithi179 ceph-mon[35472]: pgmap v489: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:03:23.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:23 smithi159 ceph-mon[35501]: pgmap v489: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:03:23.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:23 smithi137 ceph-mon[31539]: pgmap v489: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:03:25.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:25 smithi179 ceph-mon[35472]: mgrmap e20: x(active, since 13m), standbys: y, z 2024-03-26T14:03:25.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:25 smithi179 ceph-mon[35472]: pgmap v490: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:03:25.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:25 smithi159 ceph-mon[35501]: mgrmap e20: x(active, since 13m), standbys: y, z 2024-03-26T14:03:25.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:25 smithi159 ceph-mon[35501]: pgmap v490: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:03:25.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:25 smithi137 ceph-mon[31539]: mgrmap e20: x(active, since 13m), standbys: y, z 2024-03-26T14:03:25.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:25 smithi137 ceph-mon[31539]: pgmap v490: 129 pgs: 129 active+clean; 666 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:03:28.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:28 smithi137 ceph-mon[31539]: pgmap v491: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-26T14:03:28.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:28 smithi179 ceph-mon[35472]: pgmap v491: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-26T14:03:28.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:28 smithi159 ceph-mon[35501]: pgmap v491: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-26T14:03:30.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:30 smithi137 ceph-mon[31539]: pgmap v492: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:03:30.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:30 smithi179 ceph-mon[35472]: pgmap v492: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:03:30.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:30 smithi159 ceph-mon[35501]: pgmap v492: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:03:32.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:32 smithi137 ceph-mon[31539]: pgmap v493: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T14:03:32.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:32 smithi179 ceph-mon[35472]: pgmap v493: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T14:03:32.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:32 smithi159 ceph-mon[35501]: pgmap v493: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T14:03:34.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:34 smithi137 ceph-mon[31539]: pgmap v494: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T14:03:34.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:34 smithi179 ceph-mon[35472]: pgmap v494: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T14:03:34.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:34 smithi159 ceph-mon[35501]: pgmap v494: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T14:03:36.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:03:36.136 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.395+0000 7f3c4be00640 1 -- 172.21.15.137:0/4019893340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 msgr2=0x7f3c4c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.395+0000 7f3c4be00640 1 --2- 172.21.15.137:0/4019893340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c106d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009900 tx=0x7f3c3802bc30 comp rx=0 tx=0).stop 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 -- 172.21.15.137:0/4019893340 shutdown_connections 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 --2- 172.21.15.137:0/4019893340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c4c107260 0x7f3c4c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 --2- 172.21.15.137:0/4019893340 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c106d20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 --2- 172.21.15.137:0/4019893340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c4c102000 0x7f3c4c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 -- 172.21.15.137:0/4019893340 >> 172.21.15.137:0/4019893340 conn(0x7f3c4c0fba10 msgr2=0x7f3c4c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 -- 172.21.15.137:0/4019893340 shutdown_connections 2024-03-26T14:03:36.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.396+0000 7f3c4be00640 1 -- 172.21.15.137:0/4019893340 wait complete. 2024-03-26T14:03:36.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.397+0000 7f3c4be00640 1 Processor -- start 2024-03-26T14:03:36.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.397+0000 7f3c4be00640 1 -- start start 2024-03-26T14:03:36.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c4c102000 0x7f3c4c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c4c107260 0x7f3c4c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4be00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c10d110 con 0x7f3c4c104930 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c10cf90 con 0x7f3c4c107260 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c10d290 con 0x7f3c4c102000 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.398+0000 7f3c4a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c4c107260 0x7f3c4c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c0fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58896/0 (socket says 172.21.15.137:58896) 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c4c102000 0x7f3c4c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 -- 172.21.15.137:0/2348385655 learned_addr learned my addr 172.21.15.137:0/2348385655 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:36.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 -- 172.21.15.137:0/2348385655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c4c102000 msgr2=0x7f3c4c101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c4c102000 0x7f3c4c101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 -- 172.21.15.137:0/2348385655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c4c107260 msgr2=0x7f3c4c1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c4c107260 0x7f3c4c1004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.399+0000 7f3c4a000640 1 -- 172.21.15.137:0/2348385655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c100d80 con 0x7f3c4c104930 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.400+0000 7f3c4a000640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c0fffb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c380040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.400+0000 7f3c43400640 1 -- 172.21.15.137:0/2348385655 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38043020 con 0x7f3c4c104930 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.400+0000 7f3c43400640 1 -- 172.21.15.137:0/2348385655 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c380044c0 con 0x7f3c4c104930 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.400+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c1a1910 con 0x7f3c4c104930 2024-03-26T14:03:36.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.400+0000 7f3c43400640 1 -- 172.21.15.137:0/2348385655 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38003c60 con 0x7f3c4c104930 2024-03-26T14:03:36.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.401+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c1a1eb0 con 0x7f3c4c104930 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.406+0000 7f3c54618740 1 Processor -- start 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 -- start start 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 0x55cc7506dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506e250 0x55cc75072d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc75075850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc74fca250 con 0x55cc7506e250 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc74f72a30 con 0x55cc750733c0 2024-03-26T14:03:36.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c54618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc74fc62f0 con 0x55cc7506d860 2024-03-26T14:03:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.407+0000 7f3c43400640 1 -- 172.21.15.137:0/2348385655 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3c38046020 con 0x7f3c4c104930 2024-03-26T14:03:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc75075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506e250 0x55cc75072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 0x55cc7506dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 0x55cc7506dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:59706/0 (socket says 172.21.15.137:59706) 2024-03-26T14:03:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4b400640 1 -- 172.21.15.137:0/2625298052 learned_addr learned my addr 172.21.15.137:0/2625298052 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4b400640 1 -- 172.21.15.137:0/2625298052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 msgr2=0x55cc75075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.408+0000 7f3c4b400640 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc75075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.409+0000 7f3c4b400640 1 -- 172.21.15.137:0/2625298052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506e250 msgr2=0x55cc75072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.409+0000 7f3c4b400640 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506e250 0x55cc75072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.409+0000 7f3c4b400640 1 -- 172.21.15.137:0/2625298052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc74f5d690 con 0x55cc7506d860 2024-03-26T14:03:36.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.409+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506e250 0x55cc75072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:36.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.409+0000 7f3c4b400640 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 0x55cc7506dc80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c44010370 tx=0x7f3c44010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a0a18f111daae8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.410+0000 7f3c37e00640 1 -- 172.21.15.137:0/2625298052 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44016070 con 0x55cc7506d860 2024-03-26T14:03:36.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.410+0000 7f3c37e00640 1 -- 172.21.15.137:0/2625298052 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c44004d20 con 0x55cc7506d860 2024-03-26T14:03:36.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.410+0000 7f3c37e00640 1 -- 172.21.15.137:0/2625298052 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44005310 con 0x55cc7506d860 2024-03-26T14:03:36.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.410+0000 7f3c43400640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c300a2900 0x7f3c300a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.410+0000 7f3c43400640 1 -- 172.21.15.137:0/2348385655 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3c380f0f80 con 0x7f3c4c104930 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.410+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c300a2900 0x7f3c300a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 -- 172.21.15.137:0/2625298052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 msgr2=0x55cc7506dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 0x55cc7506dc80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c44010370 tx=0x7f3c44010830 comp rx=0 tx=0).stop 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 -- 172.21.15.137:0/2625298052 shutdown_connections 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc75075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506e250 0x55cc75072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 --2- 172.21.15.137:0/2625298052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506d860 0x55cc7506dc80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.411+0000 7f3c54618740 1 -- 172.21.15.137:0/2625298052 >> 172.21.15.137:0/2625298052 conn(0x55cc7502d200 msgr2=0x55cc750609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.412+0000 7f3c54618740 1 -- 172.21.15.137:0/2625298052 shutdown_connections 2024-03-26T14:03:36.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.412+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c300a2900 0x7f3c300a4dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55cc7506e650 tx=0x7f3c3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.412+0000 7f3c54618740 1 -- 172.21.15.137:0/2625298052 wait complete. 2024-03-26T14:03:36.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.412+0000 7f3c54618740 1 -- start start 2024-03-26T14:03:36.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.413+0000 7f3c54618740 1 Processor -- start 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.413+0000 7f3c54618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 0x55cc75077840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c54618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc75077d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c54618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c54618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc75076370 con 0x55cc7506d860 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c54618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc74f85e80 con 0x55cc750733c0 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c54618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc750764f0 con 0x55cc7506e250 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c54618740 10 client.0 _ll_register_callbacks cb 0x55cc74f98130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 0x55cc75077840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 0x55cc75077840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58906/0 (socket says 172.21.15.137:58906) 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c4aa00640 1 -- 172.21.15.137:0/1993385498 learned_addr learned my addr 172.21.15.137:0/1993385498 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:36.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.414+0000 7f3c4a000640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc75077d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4b400640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4aa00640 1 -- 172.21.15.137:0/1993385498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 msgr2=0x55cc75077d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc75077d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4aa00640 1 -- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 msgr2=0x55cc7507edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4aa00640 1 -- 172.21.15.137:0/1993385498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc74da87d0 con 0x55cc7506d860 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 0x55cc75077840 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c004560 tx=0x7f3c3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17d9cdd3597c5d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c34c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.415+0000 7f3c34c00640 1 -- 172.21.15.137:0/1993385498 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c005020 con 0x55cc7506d860 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.416+0000 7f3c34c00640 1 -- 172.21.15.137:0/1993385498 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3c3c0051c0 con 0x55cc7506d860 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.416+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c44004d20 con 0x55cc7506d860 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.416+0000 7f3c34c00640 1 -- 172.21.15.137:0/1993385498 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c3c0053c0 con 0x55cc7506d860 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.416+0000 7f3c54618740 10 client.15828 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:36.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.416+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cc74cde3f0 con 0x55cc7506d860 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.416+0000 7f3c34c00640 1 -- 172.21.15.137:0/1993385498 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c44004d20 con 0x55cc7506d860 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c34c00640 1 -- 172.21.15.137:0/1993385498 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3c3c027070 con 0x55cc7506d860 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c54618740 10 client.15828 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c54618740 10 client.15828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c54618740 20 client.15828 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c54618740 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750836b0 0x55cc75085b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c54618740 4 client.15828 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.417+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc74fd7790 con 0x55cc750836b0 2024-03-26T14:03:36.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.418+0000 7f3c4a000640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750836b0 0x55cc75085b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.419+0000 7f3c4a000640 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750836b0 0x55cc75085b30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9caef8055380ee5a server_cookie=971a38f498bbfab6 in_seq=0 out_seq=0 2024-03-26T14:03:36.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.419+0000 7f3c34c00640 10 client.15828 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:36.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c34c00640 1 -- 172.21.15.137:0/1993385498 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55cc74fd7790 con 0x55cc750836b0 2024-03-26T14:03:36.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c34c00640 10 client.15828 handle_command_reply: tid=0 2024-03-26T14:03:36.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 client.15828 shutdown 2024-03-26T14:03:36.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 msgr2=0x55cc75077840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 0x55cc75077840 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c004560 tx=0x7f3c3c004730 comp rx=0 tx=0).stop 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 shutdown_connections 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750836b0 0x55cc75085b30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc75077d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 --2- 172.21.15.137:0/1993385498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7506d860 0x55cc75077840 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 >> 172.21.15.137:0/1993385498 conn(0x55cc7502d200 msgr2=0x55cc7505fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.421+0000 7f3c34c00640 0 client.15828 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.422+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 shutdown_connections 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.422+0000 7f3c54618740 1 -- 172.21.15.137:0/1993385498 wait complete. 2024-03-26T14:03:36.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.422+0000 7f3c54618740 20 client.15828 trim_cache size 0 max 16384 2024-03-26T14:03:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.454+0000 7f3c54618740 1 Processor -- start 2024-03-26T14:03:36.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 -- start start 2024-03-26T14:03:36.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc7506e250 0x55cc7506f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc750733c0 0x55cc7507ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7505f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1aa3a0 con 0x55cc7505f330 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1a7ea0 con 0x55cc7506e250 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c54618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1a5f20 con 0x55cc750733c0 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.455+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7505f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc7506e250 0x55cc7506f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7505f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58916/0 (socket says 172.21.15.137:58916) 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 -- 172.21.15.137:0/876602511 learned_addr learned my addr 172.21.15.137:0/876602511 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:36.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4a000640 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc750733c0 0x55cc7507ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 -- 172.21.15.137:0/876602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc750733c0 msgr2=0x55cc7507ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc750733c0 0x55cc7507ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 -- 172.21.15.137:0/876602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc7506e250 msgr2=0x55cc7506f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc7506e250 0x55cc7506f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.456+0000 7f3c4b400640 1 -- 172.21.15.137:0/876602511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc74c9e0f0 con 0x55cc7505f330 2024-03-26T14:03:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.457+0000 7f3c4b400640 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7505f7b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3c44019000 tx=0x7f3c44009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf46f43539b16bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.457+0000 7f3c37400640 1 -- 172.21.15.137:0/876602511 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44016070 con 0x55cc7505f330 2024-03-26T14:03:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.457+0000 7f3c37400640 1 -- 172.21.15.137:0/876602511 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c440054b0 con 0x55cc7505f330 2024-03-26T14:03:36.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.457+0000 7f3c37400640 1 -- 172.21.15.137:0/876602511 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44005310 con 0x55cc7505f330 2024-03-26T14:03:36.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 -- 172.21.15.137:0/876602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 msgr2=0x55cc7505f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7505f7b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3c44019000 tx=0x7f3c44009a20 comp rx=0 tx=0).stop 2024-03-26T14:03:36.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 -- 172.21.15.137:0/876602511 shutdown_connections 2024-03-26T14:03:36.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7505f7b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc750733c0 0x55cc7507ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 --2- 172.21.15.137:0/876602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc7506e250 0x55cc7506f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.458+0000 7f3c54618740 1 -- 172.21.15.137:0/876602511 >> 172.21.15.137:0/876602511 conn(0x55cc7502d200 msgr2=0x55cc75077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.459+0000 7f3c54618740 1 -- 172.21.15.137:0/876602511 shutdown_connections 2024-03-26T14:03:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.459+0000 7f3c54618740 1 -- 172.21.15.137:0/876602511 wait complete. 2024-03-26T14:03:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.459+0000 7f3c54618740 1 -- start start 2024-03-26T14:03:36.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.459+0000 7f3c54618740 1 Processor -- start 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7507cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc7507d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1a6520 con 0x55cc7505f330 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1a7840 con 0x55cc750733c0 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c1a6b80 con 0x55cc7506e250 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c54618740 10 client.0 _ll_register_callbacks cb 0x7f3c3c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:36.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7507cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c4aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7507cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58920/0 (socket says 172.21.15.137:58920) 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.460+0000 7f3c4aa00640 1 -- 172.21.15.137:0/440399382 learned_addr learned my addr 172.21.15.137:0/440399382 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4b400640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4a000640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc7507d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4aa00640 1 -- 172.21.15.137:0/440399382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 msgr2=0x55cc7507d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc7507d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4aa00640 1 -- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 msgr2=0x55cc7507e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4aa00640 1 -- 172.21.15.137:0/440399382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc7507e680 con 0x55cc7505f330 2024-03-26T14:03:36.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4a000640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc7507d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c4aa00640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7507cb40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c01fa40 tx=0x7f3c3c003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5b94792d76efb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.461+0000 7f3c34c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.462+0000 7f3c34c00640 1 -- 172.21.15.137:0/440399382 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c03a020 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.462+0000 7f3c34c00640 1 -- 172.21.15.137:0/440399382 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3c3c036650 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.462+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c440054b0 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.462+0000 7f3c34c00640 1 -- 172.21.15.137:0/440399382 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c3c045050 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.462+0000 7f3c54618740 10 client.15834 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.462+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55cc750df5e0 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c34c00640 1 -- 172.21.15.137:0/440399382 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c440054b0 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c34c00640 1 -- 172.21.15.137:0/440399382 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3c3c027050 con 0x55cc7505f330 2024-03-26T14:03:36.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c54618740 10 client.15834 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:36.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c54618740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:36.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c54618740 20 client.15834 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:36.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c54618740 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750dff80 0x55cc750e23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c54618740 4 client.15834 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:03:36.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.463+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc74fd1ae0 con 0x55cc750dff80 2024-03-26T14:03:36.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.464+0000 7f3c4a000640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750dff80 0x55cc750e23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.464+0000 7f3c4a000640 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750dff80 0x55cc750e23a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566e1b49d3cc20a5 server_cookie=9ef56876876785ff in_seq=0 out_seq=0 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.464+0000 7f3c34c00640 10 client.15834 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c34c00640 1 -- 172.21.15.137:0/440399382 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cc74fd1ae0 con 0x55cc750dff80 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c34c00640 10 client.15834 handle_command_reply: tid=0 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 client.15834 shutdown 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 msgr2=0x55cc7507cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7507cb40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c01fa40 tx=0x7f3c3c003ec0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 shutdown_connections 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55cc750dff80 0x55cc750e23a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cc750733c0 0x55cc7507e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7506e250 0x55cc7507d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c34c00640 0 client.15834 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 --2- 172.21.15.137:0/440399382 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55cc7505f330 0x55cc7507cb40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.465+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 >> 172.21.15.137:0/440399382 conn(0x55cc7502d200 msgr2=0x55cc75063c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.466+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 shutdown_connections 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.466+0000 7f3c54618740 1 -- 172.21.15.137:0/440399382 wait complete. 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.466+0000 7f3c54618740 20 client.15834 trim_cache size 0 max 16384 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:03:36.467 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:03:36.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c300a2900 msgr2=0x7f3c300a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c300a2900 0x7f3c300a4dc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x55cc7506e650 tx=0x7f3c3c005f90 comp rx=0 tx=0).stop 2024-03-26T14:03:36.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 msgr2=0x7f3c4c0fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c0fffb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c380040c0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 shutdown_connections 2024-03-26T14:03:36.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c300a2900 0x7f3c300a4dc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c4c107260 0x7f3c4c1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c4c104930 0x7f3c4c0fffb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 --2- 172.21.15.137:0/2348385655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c4c102000 0x7f3c4c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 >> 172.21.15.137:0/2348385655 conn(0x7f3c4c0fba10 msgr2=0x7f3c4c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 shutdown_connections 2024-03-26T14:03:36.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.467+0000 7f3c4be00640 1 -- 172.21.15.137:0/2348385655 wait complete. 2024-03-26T14:03:36.485 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:36 smithi137 ceph-mon[31539]: pgmap v495: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:03:36.488 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:03:36.488 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:03:36.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.673+0000 7f941de00640 1 -- 172.21.15.137:0/2891810357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181048a0 msgr2=0x7f9418106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.673+0000 7f941de00640 1 --2- 172.21.15.137:0/2891810357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181048a0 0x7f9418106c90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402bc40 comp rx=0 tx=0).stop 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 -- 172.21.15.137:0/2891810357 shutdown_connections 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 --2- 172.21.15.137:0/2891810357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181071d0 0x7f9418109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 --2- 172.21.15.137:0/2891810357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181048a0 0x7f9418106c90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 --2- 172.21.15.137:0/2891810357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9418101f70 0x7f9418104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 -- 172.21.15.137:0/2891810357 >> 172.21.15.137:0/2891810357 conn(0x7f94180fb980 msgr2=0x7f94180fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 -- 172.21.15.137:0/2891810357 shutdown_connections 2024-03-26T14:03:36.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.674+0000 7f941de00640 1 -- 172.21.15.137:0/2891810357 wait complete. 2024-03-26T14:03:36.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:36 smithi179 ceph-mon[35472]: pgmap v495: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:03:36.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 Processor -- start 2024-03-26T14:03:36.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 -- start start 2024-03-26T14:03:36.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9418101f70 0x7f94181998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 0x7f9418199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181071d0 0x7f94181a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941810cff0 con 0x7f94181071d0 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941810ce70 con 0x7f94181048a0 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.675+0000 7f941de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941810d170 con 0x7f9418101f70 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f9417e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 0x7f9418199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f9417e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 0x7f9418199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36016/0 (socket says 172.21.15.137:36016) 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f9417e00640 1 -- 172.21.15.137:0/464415924 learned_addr learned my addr 172.21.15.137:0/464415924 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f941d400640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181071d0 0x7f94181a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f941ca00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9418101f70 0x7f94181998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f9417e00640 1 -- 172.21.15.137:0/464415924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9418101f70 msgr2=0x7f94181998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f9417e00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9418101f70 0x7f94181998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.676+0000 7f9417e00640 1 -- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181071d0 msgr2=0x7f94181a0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.677+0000 7f9417e00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181071d0 0x7f94181a0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.677+0000 7f9417e00640 1 -- 172.21.15.137:0/464415924 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94181a1500 con 0x7f94181048a0 2024-03-26T14:03:36.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.677+0000 7f941d400640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181071d0 0x7f94181a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:36.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.677+0000 7f9417e00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 0x7f9418199e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9404000c00 tx=0x7f9404004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.678+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404043020 con 0x7f94181048a0 2024-03-26T14:03:36.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.678+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94181a1790 con 0x7f94181048a0 2024-03-26T14:03:36.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.678+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9404004aa0 con 0x7f94181048a0 2024-03-26T14:03:36.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.678+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404003d00 con 0x7f94181048a0 2024-03-26T14:03:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.678+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94181a1d30 con 0x7f94181048a0 2024-03-26T14:03:36.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.679+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9418108d20 con 0x7f94181048a0 2024-03-26T14:03:36.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.683+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9404046020 con 0x7f94181048a0 2024-03-26T14:03:36.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.685+0000 7f9415600640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94000a2800 0x7f94000a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:36.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.685+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f94040f1790 con 0x7f94181048a0 2024-03-26T14:03:36.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.686+0000 7f941ca00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94000a2800 0x7f94000a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:36.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.686+0000 7f941ca00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94000a2800 0x7f94000a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f940c005d00 tx=0x7f940c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:36.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.691+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f940404d020 con 0x7f94181048a0 2024-03-26T14:03:36.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:36 smithi159 ceph-mon[35501]: pgmap v495: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:03:36.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.926+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f941802d070 con 0x7f94181048a0 2024-03-26T14:03:36.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.928+0000 7f9415600640 1 -- 172.21.15.137:0/464415924 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f94040be420 con 0x7f94181048a0 2024-03-26T14:03:36.926 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:03:36.926 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:03:36.926 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94000a2800 msgr2=0x7f94000a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94000a2800 0x7f94000a4cc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f940c005d00 tx=0x7f940c00b410 comp rx=0 tx=0).stop 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 msgr2=0x7f9418199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 0x7f9418199e20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9404000c00 tx=0x7f9404004270 comp rx=0 tx=0).stop 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 shutdown_connections 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94000a2800 0x7f94000a4cc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94181048a0 0x7f9418199e20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94181071d0 0x7f94181a0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 --2- 172.21.15.137:0/464415924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9418101f70 0x7f94181998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.930+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 >> 172.21.15.137:0/464415924 conn(0x7f94180fb980 msgr2=0x7f9418101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.931+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 shutdown_connections 2024-03-26T14:03:36.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:36.931+0000 7f941de00640 1 -- 172.21.15.137:0/464415924 wait complete. 2024-03-26T14:03:36.946 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:03:36.946 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.142+0000 7f6846400640 1 -- 172.21.15.137:0/2798373042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840107260 msgr2=0x7f68401096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.142+0000 7f6846400640 1 --2- 172.21.15.137:0/2798373042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840107260 0x7f68401096b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6838009ca0 tx=0x7f683802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 -- 172.21.15.137:0/2798373042 shutdown_connections 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 --2- 172.21.15.137:0/2798373042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840107260 0x7f68401096b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 --2- 172.21.15.137:0/2798373042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840104930 0x7f6840106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 --2- 172.21.15.137:0/2798373042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 0x7f68401043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 -- 172.21.15.137:0/2798373042 >> 172.21.15.137:0/2798373042 conn(0x7f68400fba10 msgr2=0x7f68400fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 -- 172.21.15.137:0/2798373042 shutdown_connections 2024-03-26T14:03:37.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.143+0000 7f6846400640 1 -- 172.21.15.137:0/2798373042 wait complete. 2024-03-26T14:03:37.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.144+0000 7f6846400640 1 Processor -- start 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.144+0000 7f6846400640 1 -- start start 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6846400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 0x7f6840197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6846400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840104930 0x7f6840197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6846400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840107260 0x7f684019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6846400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684010d0c0 con 0x7f6840104930 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6846400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684010cf40 con 0x7f6840102000 2024-03-26T14:03:37.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6846400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684010d240 con 0x7f6840107260 2024-03-26T14:03:37.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6845000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 0x7f6840197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f683fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840104930 0x7f6840197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6845000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 0x7f6840197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36038/0 (socket says 172.21.15.137:36038) 2024-03-26T14:03:37.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6845000640 1 -- 172.21.15.137:0/1477952834 learned_addr learned my addr 172.21.15.137:0/1477952834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.145+0000 7f6845a00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840107260 0x7f684019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683fe00640 1 -- 172.21.15.137:0/1477952834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840107260 msgr2=0x7f684019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683fe00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840107260 0x7f684019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683fe00640 1 -- 172.21.15.137:0/1477952834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 msgr2=0x7f6840197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683fe00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 0x7f6840197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683fe00640 1 -- 172.21.15.137:0/1477952834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684019f330 con 0x7f6840104930 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f6845a00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840107260 0x7f684019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683fe00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840104930 0x7f6840197c50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f682c00d6a0 tx=0x7f682c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683d600640 1 -- 172.21.15.137:0/1477952834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c017070 con 0x7f6840104930 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684019f620 con 0x7f6840104930 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683d600640 1 -- 172.21.15.137:0/1477952834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f682c009820 con 0x7f6840104930 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f683d600640 1 -- 172.21.15.137:0/1477952834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c004c40 con 0x7f6840104930 2024-03-26T14:03:37.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.146+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684019fbc0 con 0x7f6840104930 2024-03-26T14:03:37.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.151+0000 7f683d600640 1 -- 172.21.15.137:0/1477952834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f682c0099d0 con 0x7f6840104930 2024-03-26T14:03:37.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.151+0000 7f684a61c740 1 Processor -- start 2024-03-26T14:03:37.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.151+0000 7f684a61c740 1 -- start start 2024-03-26T14:03:37.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.151+0000 7f684a61c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9c4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.151+0000 7f684a61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 0x55813f9c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f684a61c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f684a61c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813f9242c0 con 0x55813f9c5990 2024-03-26T14:03:37.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f684a61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813f8ccc90 con 0x55813f9c4fd0 2024-03-26T14:03:37.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f684a61c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813f920360 con 0x55813f9c3e00 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f6845000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 0x55813f9c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f683fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f6845000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 0x55813f9c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36050/0 (socket says 172.21.15.137:36050) 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f6845000640 1 -- 172.21.15.137:0/755305432 learned_addr learned my addr 172.21.15.137:0/755305432 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.152+0000 7f6845a00640 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6845000640 1 -- 172.21.15.137:0/755305432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 msgr2=0x55813f9c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6845000640 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6845000640 1 -- 172.21.15.137:0/755305432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 msgr2=0x55813f9cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6845000640 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6845000640 1 -- 172.21.15.137:0/755305432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55813f8f1080 con 0x55813f9c4fd0 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f683fe00640 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f683d600640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68280a2880 0x7f68280a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f683d600640 1 -- 172.21.15.137:0/1477952834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f682c0ce6d0 con 0x7f6840104930 2024-03-26T14:03:37.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6845000640 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 0x55813f9c5450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6834009dc0 tx=0x7f6834011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cf3d02ea20299f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6832000640 1 -- 172.21.15.137:0/755305432 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834005220 con 0x55813f9c4fd0 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6832000640 1 -- 172.21.15.137:0/755305432 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6834004c50 con 0x55813f9c4fd0 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.153+0000 7f6832000640 1 -- 172.21.15.137:0/755305432 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68340197b0 con 0x55813f9c4fd0 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.154+0000 7f6845a00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68280a2880 0x7f68280a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.154+0000 7f684a61c740 1 -- 172.21.15.137:0/755305432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 msgr2=0x55813f9c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.154+0000 7f684a61c740 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 0x55813f9c5450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6834009dc0 tx=0x7f6834011810 comp rx=0 tx=0).stop 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.154+0000 7f6845a00640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68280a2880 0x7f68280a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f683802c630 tx=0x7f6838005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 -- 172.21.15.137:0/755305432 shutdown_connections 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c4fd0 0x55813f9c5450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 --2- 172.21.15.137:0/755305432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 -- 172.21.15.137:0/755305432 >> 172.21.15.137:0/755305432 conn(0x55813f9bade0 msgr2=0x55813f9bb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 -- 172.21.15.137:0/755305432 shutdown_connections 2024-03-26T14:03:37.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 -- 172.21.15.137:0/755305432 wait complete. 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.155+0000 7f684a61c740 1 -- start start 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 Processor -- start 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 0x55813f9ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813f9cc9a0 con 0x55813f9c5990 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813f8e00d0 con 0x55813f9c3e00 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55813f9ccb20 con 0x55813f9c4fd0 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f684a61c740 10 client.0 _ll_register_callbacks cb 0x55813f8e4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.156+0000 7f6845000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 0x55813f9ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 0x55813f9ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36066/0 (socket says 172.21.15.137:36066) 2024-03-26T14:03:37.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 -- 172.21.15.137:0/1698247502 learned_addr learned my addr 172.21.15.137:0/1698247502 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845a00640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f683fe00640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 -- 172.21.15.137:0/1698247502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 msgr2=0x55813f9ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 -- 172.21.15.137:0/1698247502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 msgr2=0x55813f9d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845000640 1 -- 172.21.15.137:0/1698247502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55813f6383f0 con 0x55813f9c3e00 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f683fe00640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.157+0000 7f6845a00640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.158+0000 7f6845000640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 0x55813f9ce040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f683401b040 tx=0x7f6834004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1736fa8fd81c7717 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.158+0000 7f6826a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.158+0000 7f6826a00640 1 -- 172.21.15.137:0/1698247502 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834027070 con 0x55813f9c3e00 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.158+0000 7f6826a00640 1 -- 172.21.15.137:0/1698247502 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6834005000 con 0x55813f9c3e00 2024-03-26T14:03:37.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.158+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6834004c50 con 0x55813f9c3e00 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f6826a00640 1 -- 172.21.15.137:0/1698247502 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6834031050 con 0x55813f9c3e00 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 10 client.35549 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55813f7027d0 con 0x55813f9c3e00 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f6826a00640 1 -- 172.21.15.137:0/1698247502 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6834004c50 con 0x55813f9c3e00 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f6826a00640 1 -- 172.21.15.137:0/1698247502 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6834019530 con 0x55813f9c3e00 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 10 client.35549 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 10 client.35549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 20 client.35549 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813f9d98d0 0x55813f9dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 4 client.35549 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.159+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55813f931800 con 0x55813f9d98d0 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.160+0000 7f683fe00640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813f9d98d0 0x55813f9dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.161+0000 7f6826a00640 10 client.35549 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:37.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.161+0000 7f683fe00640 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813f9d98d0 0x55813f9dbd00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd0a12bd6c83c616 server_cookie=96d8bf3f84872d21 in_seq=0 out_seq=0 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f6826a00640 1 -- 172.21.15.137:0/1698247502 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55813f931800 con 0x55813f9d98d0 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f6826a00640 10 client.35549 handle_command_reply: tid=0 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 client.35549 shutdown 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 msgr2=0x55813f9ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 0x55813f9ce040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f683401b040 tx=0x7f6834004270 comp rx=0 tx=0).stop 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 shutdown_connections 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813f9d98d0 0x55813f9dbd00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c5990 0x55813f9d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 --2- 172.21.15.137:0/1698247502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c3e00 0x55813f9ce040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 >> 172.21.15.137:0/1698247502 conn(0x55813f9bade0 msgr2=0x55813f9ba160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f6826a00640 0 client.35549 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 shutdown_connections 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 1 -- 172.21.15.137:0/1698247502 wait complete. 2024-03-26T14:03:37.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.162+0000 7f684a61c740 20 client.35549 trim_cache size 0 max 16384 2024-03-26T14:03:37.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.187+0000 7f684a61c740 1 Processor -- start 2024-03-26T14:03:37.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.187+0000 7f684a61c740 1 -- start start 2024-03-26T14:03:37.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f684a61c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f684a61c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c4fd0 0x55813f9d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f684a61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f684a61c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68401aa400 con 0x55813f9c4fd0 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f684a61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68401a7f00 con 0x55813f9c5990 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f684a61c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68401a5f80 con 0x55813f9c3e00 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f6845a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f6845a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36070/0 (socket says 172.21.15.137:36070) 2024-03-26T14:03:37.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f6845a00640 1 -- 172.21.15.137:0/1902786260 learned_addr learned my addr 172.21.15.137:0/1902786260 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f683fe00640 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c4fd0 0x55813f9d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.188+0000 7f6845000640 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6845a00640 1 -- 172.21.15.137:0/1902786260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 msgr2=0x55813f9e05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6845a00640 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9e05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6845a00640 1 -- 172.21.15.137:0/1902786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c4fd0 msgr2=0x55813f9d5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6845a00640 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c4fd0 0x55813f9d5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6845a00640 1 -- 172.21.15.137:0/1902786260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55813f9bbd50 con 0x55813f9c5990 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6845a00640 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9e0b00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6838036000 tx=0x7f6838040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7162610e9e1c363d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6831600640 1 -- 172.21.15.137:0/1902786260 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838039040 con 0x55813f9c5990 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6831600640 1 -- 172.21.15.137:0/1902786260 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6838052070 con 0x55813f9c5990 2024-03-26T14:03:37.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.189+0000 7f6831600640 1 -- 172.21.15.137:0/1902786260 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683804da60 con 0x55813f9c5990 2024-03-26T14:03:37.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 -- 172.21.15.137:0/1902786260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 msgr2=0x55813f9e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9e0b00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6838036000 tx=0x7f6838040000 comp rx=0 tx=0).stop 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 -- 172.21.15.137:0/1902786260 shutdown_connections 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9e0b00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c4fd0 0x55813f9d5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 --2- 172.21.15.137:0/1902786260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c3e00 0x55813f9e05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 -- 172.21.15.137:0/1902786260 >> 172.21.15.137:0/1902786260 conn(0x55813f9bade0 msgr2=0x55813f9be8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 -- 172.21.15.137:0/1902786260 shutdown_connections 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.190+0000 7f684a61c740 1 -- 172.21.15.137:0/1902786260 wait complete. 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.191+0000 7f684a61c740 1 -- start start 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.191+0000 7f684a61c740 1 Processor -- start 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.191+0000 7f684a61c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 0x55813f9d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f684a61c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f684a61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9d4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f684a61c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68401a6580 con 0x55813f9c3e00 2024-03-26T14:03:37.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f684a61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68401a78a0 con 0x55813f9c5990 2024-03-26T14:03:37.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f684a61c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68401a6be0 con 0x55813f9c4fd0 2024-03-26T14:03:37.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 0x55813f9d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 0x55813f9d11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58974/0 (socket says 172.21.15.137:58974) 2024-03-26T14:03:37.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 -- 172.21.15.137:0/1169029159 learned_addr learned my addr 172.21.15.137:0/1169029159 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f684a61c740 10 client.0 _ll_register_callbacks cb 0x7f682c0d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:37.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845a00640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9d4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f683fe00640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 -- 172.21.15.137:0/1169029159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 msgr2=0x55813f9d1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9d1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 -- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 msgr2=0x55813f9d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9d4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.192+0000 7f6845000640 1 -- 172.21.15.137:0/1169029159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55813f9d4d20 con 0x55813f9c3e00 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f6845000640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 0x55813f9d11d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6834019920 tx=0x7f68340095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73cb6d1e91d0770 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f6826a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f6826a00640 1 -- 172.21.15.137:0/1169029159 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834027050 con 0x55813f9c3e00 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6838052070 con 0x55813f9c3e00 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f6826a00640 1 -- 172.21.15.137:0/1169029159 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6834019630 con 0x55813f9c3e00 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f6826a00640 1 -- 172.21.15.137:0/1169029159 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6834044050 con 0x55813f9c3e00 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f684a61c740 10 client.15858 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.193+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55813fa356a0 con 0x55813f9c3e00 2024-03-26T14:03:37.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f6826a00640 1 -- 172.21.15.137:0/1169029159 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6838052070 con 0x55813f9c3e00 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f6826a00640 1 -- 172.21.15.137:0/1169029159 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f68340035f0 con 0x55813f9c3e00 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f684a61c740 10 client.15858 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f684a61c740 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f684a61c740 20 client.15858 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f684a61c740 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813fa36070 0x55813fa38490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f684a61c740 4 client.15858 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.194+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55813f92bb50 con 0x55813fa36070 2024-03-26T14:03:37.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.195+0000 7f683fe00640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813fa36070 0x55813fa38490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.195+0000 7f683fe00640 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813fa36070 0x55813fa38490 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61bdc3905bf1b65 server_cookie=91b28d207bd8db2 in_seq=0 out_seq=0 2024-03-26T14:03:37.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.195+0000 7f6826a00640 10 client.15858 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:37.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f6826a00640 1 -- 172.21.15.137:0/1169029159 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55813f92bb50 con 0x55813fa36070 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f6826a00640 10 client.15858 handle_command_reply: tid=0 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 client.15858 shutdown 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 msgr2=0x55813f9d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 0x55813f9d11d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6834019920 tx=0x7f68340095a0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 shutdown_connections 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55813fa36070 0x55813fa38490 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55813f9c5990 0x55813f9d4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f6826a00640 0 client.15858 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.196+0000 7f684a61c740 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55813f9c4fd0 0x55813f9d1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.197+0000 7f684a61c740 1 --2- 172.21.15.137:0/1169029159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55813f9c3e00 0x55813f9d11d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.197+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 >> 172.21.15.137:0/1169029159 conn(0x55813f9bade0 msgr2=0x55813f9c8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.197+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 shutdown_connections 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.197+0000 7f684a61c740 1 -- 172.21.15.137:0/1169029159 wait complete. 2024-03-26T14:03:37.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.197+0000 7f684a61c740 20 client.15858 trim_cache size 0 max 16384 2024-03-26T14:03:37.197 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:03:37.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68280a2880 msgr2=0x7f68280a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68280a2880 0x7f68280a4d40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f683802c630 tx=0x7f6838005d80 comp rx=0 tx=0).stop 2024-03-26T14:03:37.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840104930 msgr2=0x7f6840197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840104930 0x7f6840197c50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f682c00d6a0 tx=0x7f682c00db60 comp rx=0 tx=0).stop 2024-03-26T14:03:37.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 shutdown_connections 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68280a2880 0x7f68280a4d40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6840107260 0x7f684019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6840104930 0x7f6840197c50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 --2- 172.21.15.137:0/1477952834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6840102000 0x7f6840197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 >> 172.21.15.137:0/1477952834 conn(0x7f68400fba10 msgr2=0x7f6840102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 shutdown_connections 2024-03-26T14:03:37.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.198+0000 7f6846400640 1 -- 172.21.15.137:0/1477952834 wait complete. 2024-03-26T14:03:37.214 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:03:37.215 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.470+0000 7f483e200640 1 -- 172.21.15.137:0/1741659116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838109430 msgr2=0x7f483810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.470+0000 7f483e200640 1 --2- 172.21.15.137:0/1741659116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838109430 0x7f483810b8d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4830009ca0 tx=0x7f483002c180 comp rx=0 tx=0).stop 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 -- 172.21.15.137:0/1741659116 shutdown_connections 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 --2- 172.21.15.137:0/1741659116 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838109430 0x7f483810b8d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 --2- 172.21.15.137:0/1741659116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48380740a0 0x7f4838074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 --2- 172.21.15.137:0/1741659116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838075680 0x7f4838073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 -- 172.21.15.137:0/1741659116 >> 172.21.15.137:0/1741659116 conn(0x7f48380fcb10 msgr2=0x7f48380fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 -- 172.21.15.137:0/1741659116 shutdown_connections 2024-03-26T14:03:37.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.471+0000 7f483e200640 1 -- 172.21.15.137:0/1741659116 wait complete. 2024-03-26T14:03:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 Processor -- start 2024-03-26T14:03:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 -- start start 2024-03-26T14:03:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48380740a0 0x7f4838199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 0x7f4838199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 0x7f48381a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483810f2d0 con 0x7f4838075680 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483810f150 con 0x7f4838109430 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.472+0000 7f483e200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483810f450 con 0x7f48380740a0 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f4837e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 0x7f4838199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 0x7f48381a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f4837e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 0x7f4838199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58994/0 (socket says 172.21.15.137:58994) 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f4837e00640 1 -- 172.21.15.137:0/3765681759 learned_addr learned my addr 172.21.15.137:0/3765681759 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 0x7f48381a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36104/0 (socket says 172.21.15.137:36104) 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483ce00640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48380740a0 0x7f4838199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 -- 172.21.15.137:0/3765681759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48380740a0 msgr2=0x7f4838199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48380740a0 0x7f4838199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 -- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 msgr2=0x7f4838199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 0x7f4838199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f483d800640 1 -- 172.21.15.137:0/3765681759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48381a1590 con 0x7f4838109430 2024-03-26T14:03:37.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.473+0000 7f4837e00640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 0x7f4838199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:37.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.474+0000 7f483d800640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 0x7f48381a0f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4830037040 tx=0x7f4830003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.474+0000 7f4835600640 1 -- 172.21.15.137:0/3765681759 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830039040 con 0x7f4838109430 2024-03-26T14:03:37.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.474+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48381a1820 con 0x7f4838109430 2024-03-26T14:03:37.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.474+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48381a1d30 con 0x7f4838109430 2024-03-26T14:03:37.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.474+0000 7f4835600640 1 -- 172.21.15.137:0/3765681759 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4830044070 con 0x7f4838109430 2024-03-26T14:03:37.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.474+0000 7f4835600640 1 -- 172.21.15.137:0/3765681759 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300042a0 con 0x7f4838109430 2024-03-26T14:03:37.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.478+0000 7f48425fe740 1 Processor -- start 2024-03-26T14:03:37.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f4835600640 1 -- 172.21.15.137:0/3765681759 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4830004440 con 0x7f4838109430 2024-03-26T14:03:37.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 -- start start 2024-03-26T14:03:37.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1836620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 0x559fd1837850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd183e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd17922c0 con 0x559fd18373d0 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd173ac90 con 0x559fd1837d90 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.479+0000 7f48425fe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd178e360 con 0x559fd1836200 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.480+0000 7f483ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 0x559fd1837850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.480+0000 7f483ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 0x559fd1837850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58996/0 (socket says 172.21.15.137:58996) 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.480+0000 7f483ce00640 1 -- 172.21.15.137:0/1674355997 learned_addr learned my addr 172.21.15.137:0/1674355997 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.480+0000 7f4837e00640 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd183e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.480+0000 7f4837e00640 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1836620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.480+0000 7f4835600640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f48200a2880 0x7f48200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f4835600640 1 -- 172.21.15.137:0/3765681759 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f48300f1260 con 0x7f4838109430 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483ce00640 1 -- 172.21.15.137:0/1674355997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 msgr2=0x559fd1836620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483ce00640 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1836620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483ce00640 1 -- 172.21.15.137:0/1674355997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 msgr2=0x559fd183e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483ce00640 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd183e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483ce00640 1 -- 172.21.15.137:0/1674355997 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd175f080 con 0x559fd18373d0 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f4837e00640 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd183e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:37.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483d800640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f48200a2880 0x7f48200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.481+0000 7f483ce00640 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 0x559fd1837850 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f482c009cf0 tx=0x7f482c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04db6013a9ce7ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.482+0000 7f483d800640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f48200a2880 0x7f48200a4d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48300ba200 tx=0x7f4830003f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.482+0000 7f482a000640 1 -- 172.21.15.137:0/1674355997 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c004230 con 0x559fd18373d0 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.482+0000 7f482a000640 1 -- 172.21.15.137:0/1674355997 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f482c0043d0 con 0x559fd18373d0 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.482+0000 7f482a000640 1 -- 172.21.15.137:0/1674355997 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0197c0 con 0x559fd18373d0 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.482+0000 7f48425fe740 1 -- 172.21.15.137:0/1674355997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 msgr2=0x559fd1837850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.482+0000 7f48425fe740 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 0x559fd1837850 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f482c009cf0 tx=0x7f482c011810 comp rx=0 tx=0).stop 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 -- 172.21.15.137:0/1674355997 shutdown_connections 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd183e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd18373d0 0x559fd1837850 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 --2- 172.21.15.137:0/1674355997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1836620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 -- 172.21.15.137:0/1674355997 >> 172.21.15.137:0/1674355997 conn(0x559fd18315f0 msgr2=0x559fd18319f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 -- 172.21.15.137:0/1674355997 shutdown_connections 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 -- 172.21.15.137:0/1674355997 wait complete. 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.483+0000 7f48425fe740 1 -- start start 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 Processor -- start 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1840470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 0x559fd18409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd18479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd183eda0 con 0x559fd1837d90 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd174e0d0 con 0x559fd18373d0 2024-03-26T14:03:37.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd183ef20 con 0x559fd1836200 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.484+0000 7f48425fe740 10 client.0 _ll_register_callbacks cb 0x559fd1752e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 0x559fd18409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 0x559fd18409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36122/0 (socket says 172.21.15.137:36122) 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f4837e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd18479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 -- 172.21.15.137:0/2300299542 learned_addr learned my addr 172.21.15.137:0/2300299542 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f4837e00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1840470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 -- 172.21.15.137:0/2300299542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 msgr2=0x559fd1840470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1840470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 -- 172.21.15.137:0/2300299542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 msgr2=0x559fd18479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.485+0000 7f483ce00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd18479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.486+0000 7f483ce00640 1 -- 172.21.15.137:0/2300299542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fd14a63f0 con 0x559fd18373d0 2024-03-26T14:03:37.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.486+0000 7f4837e00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd18479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:37.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.486+0000 7f483ce00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 0x559fd18409b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f482c0096e0 tx=0x7f482c003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=987c9c4157fd1764 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.486+0000 7f481ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:03:37.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.486+0000 7f481ea00640 1 -- 172.21.15.137:0/2300299542 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c027070 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.487+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f482c0043d0 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.487+0000 7f481ea00640 1 -- 172.21.15.137:0/2300299542 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f482c005240 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.487+0000 7f481ea00640 1 -- 172.21.15.137:0/2300299542 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f482c033050 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.487+0000 7f48425fe740 10 client.35567 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.487+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559fd15707d0 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.487+0000 7f481ea00640 1 -- 172.21.15.137:0/2300299542 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f482c0043d0 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f481ea00640 1 -- 172.21.15.137:0/2300299542 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f482c019500 con 0x559fd18373d0 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f48425fe740 10 client.35567 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f48425fe740 10 client.35567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f48425fe740 20 client.35567 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f48425fe740 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd184bcc0 0x559fd184e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f48425fe740 4 client.35567 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:37.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.488+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fd179f800 con 0x559fd184bcc0 2024-03-26T14:03:37.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.489+0000 7f4837e00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd184bcc0 0x559fd184e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.490+0000 7f4837e00640 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd184bcc0 0x559fd184e0b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c10fc99862bc6475 server_cookie=9a74b4825e1d5f33 in_seq=0 out_seq=0 2024-03-26T14:03:37.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.490+0000 7f481ea00640 10 client.35567 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:37.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f481ea00640 1 -- 172.21.15.137:0/2300299542 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559fd179f800 con 0x559fd184bcc0 2024-03-26T14:03:37.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f481ea00640 10 client.35567 handle_command_reply: tid=0 2024-03-26T14:03:37.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 client.35567 shutdown 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 msgr2=0x559fd18409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 0x559fd18409b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f482c0096e0 tx=0x7f482c003820 comp rx=0 tx=0).stop 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 shutdown_connections 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd184bcc0 0x559fd184e0b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd18373d0 0x559fd18409b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd18479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 --2- 172.21.15.137:0/2300299542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd1836200 0x559fd1840470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.491+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 >> 172.21.15.137:0/2300299542 conn(0x559fd18315f0 msgr2=0x559fd1834850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.492+0000 7f481ea00640 0 client.35567 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:37.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.492+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 shutdown_connections 2024-03-26T14:03:37.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.492+0000 7f48425fe740 1 -- 172.21.15.137:0/2300299542 wait complete. 2024-03-26T14:03:37.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.492+0000 7f48425fe740 20 client.35567 trim_cache size 0 max 16384 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 Processor -- start 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 -- start start 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 0x559fd1828160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd18479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd1834f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381aa260 con 0x559fd1836200 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a7d60 con 0x559fd1837d90 2024-03-26T14:03:37.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f48425fe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a5de0 con 0x559fd18373d0 2024-03-26T14:03:37.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f4837e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 0x559fd1828160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f4837e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 0x559fd1828160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59016/0 (socket says 172.21.15.137:59016) 2024-03-26T14:03:37.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.517+0000 7f4837e00640 1 -- 172.21.15.137:0/620107627 learned_addr learned my addr 172.21.15.137:0/620107627 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd1834f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f483ce00640 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd18479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 -- 172.21.15.137:0/620107627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 msgr2=0x559fd18479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd18479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 -- 172.21.15.137:0/620107627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 msgr2=0x559fd1834f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd1834f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 -- 172.21.15.137:0/620107627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd184e730 con 0x559fd1836200 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f483ce00640 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd18479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4837e00640 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 0x559fd1828160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4824014010 tx=0x7f482400b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f8ff7bc4b6a609 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.518+0000 7f4829600640 1 -- 172.21.15.137:0/620107627 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824005020 con 0x559fd1836200 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.519+0000 7f4829600640 1 -- 172.21.15.137:0/620107627 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f48240047a0 con 0x559fd1836200 2024-03-26T14:03:37.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.519+0000 7f4829600640 1 -- 172.21.15.137:0/620107627 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824010750 con 0x559fd1836200 2024-03-26T14:03:37.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.519+0000 7f48425fe740 1 -- 172.21.15.137:0/620107627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 msgr2=0x559fd1828160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.519+0000 7f48425fe740 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 0x559fd1828160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4824014010 tx=0x7f482400b410 comp rx=0 tx=0).stop 2024-03-26T14:03:37.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 -- 172.21.15.137:0/620107627 shutdown_connections 2024-03-26T14:03:37.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1837d90 0x559fd1834f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd18479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 --2- 172.21.15.137:0/620107627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1836200 0x559fd1828160 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 -- 172.21.15.137:0/620107627 >> 172.21.15.137:0/620107627 conn(0x559fd18315f0 msgr2=0x559fd1834850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 -- 172.21.15.137:0/620107627 shutdown_connections 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 -- 172.21.15.137:0/620107627 wait complete. 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 -- start start 2024-03-26T14:03:37.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.520+0000 7f48425fe740 1 Processor -- start 2024-03-26T14:03:37.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1836200 0x559fd18457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd1845cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd1846d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a63e0 con 0x559fd1837d90 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a7700 con 0x559fd1836200 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48381a6a40 con 0x559fd18373d0 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f48425fe740 10 client.0 _ll_register_callbacks cb 0x7f482400bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f4837e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd1846d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f4837e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1836200 0x559fd18457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f4837e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1836200 0x559fd18457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36134/0 (socket says 172.21.15.137:36134) 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.521+0000 7f4837e00640 1 -- 172.21.15.137:0/1061897564 learned_addr learned my addr 172.21.15.137:0/1061897564 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f4837e00640 1 -- 172.21.15.137:0/1061897564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 msgr2=0x559fd1845cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f4837e00640 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd1845cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f4837e00640 1 -- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1836200 msgr2=0x559fd18457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f4837e00640 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1836200 0x559fd18457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f4837e00640 1 -- 172.21.15.137:0/1061897564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fd1832060 con 0x559fd1837d90 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f4837e00640 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd1846d00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4824012450 tx=0x7f4824004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b529d0b1c3e43079 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f481ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:37.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f481ea00640 1 -- 172.21.15.137:0/1061897564 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824032070 con 0x559fd1837d90 2024-03-26T14:03:37.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48240047a0 con 0x559fd1837d90 2024-03-26T14:03:37.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.522+0000 7f481ea00640 1 -- 172.21.15.137:0/1061897564 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f48240051f0 con 0x559fd1837d90 2024-03-26T14:03:37.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f481ea00640 1 -- 172.21.15.137:0/1061897564 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4824041050 con 0x559fd1837d90 2024-03-26T14:03:37.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f48425fe740 10 client.15876 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:37.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559fd183f0c0 con 0x559fd1837d90 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f481ea00640 1 -- 172.21.15.137:0/1061897564 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f48240047a0 con 0x559fd1837d90 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f481ea00640 1 -- 172.21.15.137:0/1061897564 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4824010ce0 con 0x559fd1837d90 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f48425fe740 10 client.15876 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f48425fe740 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.523+0000 7f48425fe740 20 client.15876 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.524+0000 7f48425fe740 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd18a8180 0x559fd18aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.524+0000 7f48425fe740 4 client.15876 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.524+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559fd1840260 con 0x559fd18a8180 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.524+0000 7f483ce00640 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd18a8180 0x559fd18aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f481ea00640 10 client.15876 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:37.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f483ce00640 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd18a8180 0x559fd18aa5a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=432a17c07453aaa3 server_cookie=543f740dd2bc1310 in_seq=0 out_seq=0 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f481ea00640 1 -- 172.21.15.137:0/1061897564 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559fd1840260 con 0x559fd18a8180 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f481ea00640 10 client.15876 handle_command_reply: tid=0 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 client.15876 shutdown 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 msgr2=0x559fd1846d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd1846d00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4824012450 tx=0x7f4824004730 comp rx=0 tx=0).stop 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 shutdown_connections 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559fd18a8180 0x559fd18aa5a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559fd1837d90 0x559fd1846d00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559fd18373d0 0x559fd1845cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 --2- 172.21.15.137:0/1061897564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fd1836200 0x559fd18457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f481ea00640 0 client.15876 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 >> 172.21.15.137:0/1061897564 conn(0x559fd18315f0 msgr2=0x559fd18350e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.525+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 shutdown_connections 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.526+0000 7f48425fe740 1 -- 172.21.15.137:0/1061897564 wait complete. 2024-03-26T14:03:37.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.526+0000 7f48425fe740 20 client.15876 trim_cache size 0 max 16384 2024-03-26T14:03:37.526 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:03:37.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.526+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f48200a2880 msgr2=0x7f48200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.526+0000 7f483e200640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f48200a2880 0x7f48200a4d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48300ba200 tx=0x7f4830003f90 comp rx=0 tx=0).stop 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.526+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 msgr2=0x7f48381a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.526+0000 7f483e200640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 0x7f48381a0f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4830037040 tx=0x7f4830003ba0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 shutdown_connections 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f48200a2880 0x7f48200a4d40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4838109430 0x7f48381a0f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4838075680 0x7f4838199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 --2- 172.21.15.137:0/3765681759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f48380740a0 0x7f4838199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 >> 172.21.15.137:0/3765681759 conn(0x7f48380fcb10 msgr2=0x7f48381053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 shutdown_connections 2024-03-26T14:03:37.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.527+0000 7f483e200640 1 -- 172.21.15.137:0/3765681759 wait complete. 2024-03-26T14:03:37.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:37 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/464415924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:03:37.546 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:03:37.546 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:03:37.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:37 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/464415924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:03:37.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:37 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/464415924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:03:37.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.727+0000 7f2f7f200640 1 -- 172.21.15.137:0/3451200205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80102000 msgr2=0x7f2f801043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.727+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3451200205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80102000 0x7f2f801043f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2f70009900 tx=0x7f2f7002bc30 comp rx=0 tx=0).stop 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.727+0000 7f2f7f200640 1 -- 172.21.15.137:0/3451200205 shutdown_connections 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.727+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3451200205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80104930 0x7f2f80106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.727+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3451200205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f80107260 0x7f2f801096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.727+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3451200205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80102000 0x7f2f801043f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.728+0000 7f2f7f200640 1 -- 172.21.15.137:0/3451200205 >> 172.21.15.137:0/3451200205 conn(0x7f2f800fba10 msgr2=0x7f2f800fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.728+0000 7f2f7f200640 1 -- 172.21.15.137:0/3451200205 shutdown_connections 2024-03-26T14:03:37.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.728+0000 7f2f7f200640 1 -- 172.21.15.137:0/3451200205 wait complete. 2024-03-26T14:03:37.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.729+0000 7f2f7f200640 1 Processor -- start 2024-03-26T14:03:37.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.729+0000 7f2f7f200640 1 -- start start 2024-03-26T14:03:37.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.729+0000 7f2f7f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80102000 0x7f2f801977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 0x7f2f80197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f80107260 0x7f2f8019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8010d110 con 0x7f2f80104930 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8010cf90 con 0x7f2f80102000 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8010d290 con 0x7f2f80107260 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 0x7f2f80197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 0x7f2f80197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59032/0 (socket says 172.21.15.137:59032) 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 -- 172.21.15.137:0/3322009426 learned_addr learned my addr 172.21.15.137:0/3322009426 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7de00640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80102000 0x7f2f801977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7e800640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f80107260 0x7f2f8019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 -- 172.21.15.137:0/3322009426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f80107260 msgr2=0x7f2f8019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f80107260 0x7f2f8019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 -- 172.21.15.137:0/3322009426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80102000 msgr2=0x7f2f801977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80102000 0x7f2f801977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.730+0000 7f2f7d400640 1 -- 172.21.15.137:0/3322009426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f800ffdf0 con 0x7f2f80104930 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.731+0000 7f2f7d400640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 0x7f2f80197cf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9c0 tx=0x7f2f6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.731+0000 7f2f76a00640 1 -- 172.21.15.137:0/3322009426 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00cd80 con 0x7f2f80104930 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.731+0000 7f2f76a00640 1 -- 172.21.15.137:0/3322009426 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c0054d0 con 0x7f2f80104930 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.731+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f801000e0 con 0x7f2f80104930 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.731+0000 7f2f76a00640 1 -- 172.21.15.137:0/3322009426 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c010430 con 0x7f2f80104930 2024-03-26T14:03:37.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.731+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f80100630 con 0x7f2f80104930 2024-03-26T14:03:37.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.735+0000 7f2f76a00640 1 -- 172.21.15.137:0/3322009426 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2f6c004fd0 con 0x7f2f80104930 2024-03-26T14:03:37.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.735+0000 7f2f877da740 1 Processor -- start 2024-03-26T14:03:37.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 -- start start 2024-03-26T14:03:37.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce69f0 0x556374ce6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374ce8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f76a00640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f640a2800 0x7f2f640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374ceee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556374c4b2c0 con 0x556374ce7bc0 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556374bf3c90 con 0x556374ce8580 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f877da740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556374c47360 con 0x556374ce69f0 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f76a00640 1 -- 172.21.15.137:0/3322009426 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2f6c0cf2f0 con 0x7f2f80104930 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f7de00640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f640a2800 0x7f2f640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.736+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374ce8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374ce8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59040/0 (socket says 172.21.15.137:59040) 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 -- 172.21.15.137:0/1992713233 learned_addr learned my addr 172.21.15.137:0/1992713233 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7e800640 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374ceee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7e800640 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce69f0 0x556374ce6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 -- 172.21.15.137:0/1992713233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce69f0 msgr2=0x556374ce6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce69f0 0x556374ce6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 -- 172.21.15.137:0/1992713233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 msgr2=0x556374ceee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374ceee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 -- 172.21.15.137:0/1992713233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556374c18080 con 0x556374ce7bc0 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7e800640 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374ceee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.737+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374ce8040 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c09b810 tx=0x7f2f6c097c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e87f5f36668296 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.738+0000 7f2f7de00640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f640a2800 0x7f2f640a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2f70000c00 tx=0x7f2f7002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.738+0000 7f2f6b000640 1 -- 172.21.15.137:0/1992713233 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0736b0 con 0x556374ce7bc0 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.738+0000 7f2f6b000640 1 -- 172.21.15.137:0/1992713233 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c09bf80 con 0x556374ce7bc0 2024-03-26T14:03:37.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.738+0000 7f2f6b000640 1 -- 172.21.15.137:0/1992713233 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0ccaf0 con 0x556374ce7bc0 2024-03-26T14:03:37.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 -- 172.21.15.137:0/1992713233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 msgr2=0x556374ce8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374ce8040 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c09b810 tx=0x7f2f6c097c50 comp rx=0 tx=0).stop 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 -- 172.21.15.137:0/1992713233 shutdown_connections 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374ceee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374ce8040 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 --2- 172.21.15.137:0/1992713233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce69f0 0x556374ce6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 -- 172.21.15.137:0/1992713233 >> 172.21.15.137:0/1992713233 conn(0x556374ce1de0 msgr2=0x556374ce21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 -- 172.21.15.137:0/1992713233 shutdown_connections 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.739+0000 7f2f877da740 1 -- 172.21.15.137:0/1992713233 wait complete. 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.740+0000 7f2f877da740 1 -- start start 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.740+0000 7f2f877da740 1 Processor -- start 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cf0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374cf81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556374cef590 con 0x556374ce7bc0 2024-03-26T14:03:37.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556374c070d0 con 0x556374ce69f0 2024-03-26T14:03:37.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556374cef710 con 0x556374ce8580 2024-03-26T14:03:37.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f877da740 10 client.0 _ll_register_callbacks cb 0x556374c0be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:37.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59042/0 (socket says 172.21.15.137:59042) 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 -- 172.21.15.137:0/1948236041 learned_addr learned my addr 172.21.15.137:0/1948236041 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7de00640 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cf0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 -- 172.21.15.137:0/1948236041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 msgr2=0x556374cf81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374cf81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 -- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 msgr2=0x556374cf0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cf0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.741+0000 7f2f7d400640 1 -- 172.21.15.137:0/1948236041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55637495f3f0 con 0x556374ce7bc0 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf1170 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0048f0 tx=0x7f2f6c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7935c6cf8e496b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f5f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f5f400640 1 -- 172.21.15.137:0/1948236041 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c073030 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f5f400640 1 -- 172.21.15.137:0/1948236041 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2f6c076040 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f6c09bf80 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f5f400640 1 -- 172.21.15.137:0/1948236041 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f6c076240 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f877da740 10 client.15888 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556374a297d0 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.742+0000 7f2f5f400640 1 -- 172.21.15.137:0/1948236041 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c09bf80 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f5f400640 1 -- 172.21.15.137:0/1948236041 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2f6c09cbb0 con 0x556374ce7bc0 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f877da740 10 client.15888 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f877da740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f877da740 20 client.15888 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f877da740 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374cfc310 0x556374cfe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f877da740 4 client.15888 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:37.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556374c58800 con 0x556374cfc310 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.743+0000 7f2f7de00640 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374cfc310 0x556374cfe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.744+0000 7f2f7de00640 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374cfc310 0x556374cfe700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34478c18b9600327 server_cookie=77e585b29bc4c053 in_seq=0 out_seq=0 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.744+0000 7f2f5f400640 10 client.15888 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.745+0000 7f2f5f400640 1 -- 172.21.15.137:0/1948236041 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556374c58800 con 0x556374cfc310 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.745+0000 7f2f5f400640 10 client.15888 handle_command_reply: tid=0 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.745+0000 7f2f877da740 1 client.15888 shutdown 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.745+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 msgr2=0x556374cf1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.745+0000 7f2f877da740 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf1170 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0048f0 tx=0x7f2f6c005ae0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 shutdown_connections 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374cfc310 0x556374cfe700 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374cf81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf1170 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 --2- 172.21.15.137:0/1948236041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cf0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 >> 172.21.15.137:0/1948236041 conn(0x556374ce1de0 msgr2=0x556374ce1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f5f400640 0 client.15888 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 shutdown_connections 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 1 -- 172.21.15.137:0/1948236041 wait complete. 2024-03-26T14:03:37.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.746+0000 7f2f877da740 20 client.15888 trim_cache size 0 max 16384 2024-03-26T14:03:37.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 Processor -- start 2024-03-26T14:03:37.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 -- start start 2024-03-26T14:03:37.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cdcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374ce5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801ae7a0 con 0x556374ce7bc0 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801ac2a0 con 0x556374ce69f0 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.772+0000 7f2f877da740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801aa320 con 0x556374ce8580 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cdcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cdcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36178/0 (socket says 172.21.15.137:36178) 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7de00640 1 -- 172.21.15.137:0/1918361015 learned_addr learned my addr 172.21.15.137:0/1918361015 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7e800640 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374ce5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 -- 172.21.15.137:0/1918361015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 msgr2=0x556374ce5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374ce5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 -- 172.21.15.137:0/1918361015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 msgr2=0x556374cdcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cdcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 -- 172.21.15.137:0/1918361015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556374cfece0 con 0x556374ce7bc0 2024-03-26T14:03:37.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7de00640 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cdcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.773+0000 7f2f7d400640 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf8170 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c098250 tx=0x7f2f6c099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c8693cf7ca66c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.774+0000 7f2f6a600640 1 -- 172.21.15.137:0/1918361015 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c073040 con 0x556374ce7bc0 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.774+0000 7f2f6a600640 1 -- 172.21.15.137:0/1918361015 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c09bf80 con 0x556374ce7bc0 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.774+0000 7f2f6a600640 1 -- 172.21.15.137:0/1918361015 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c09cbb0 con 0x556374ce7bc0 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 -- 172.21.15.137:0/1918361015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 msgr2=0x556374cf8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf8170 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c098250 tx=0x7f2f6c099d90 comp rx=0 tx=0).stop 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 -- 172.21.15.137:0/1918361015 shutdown_connections 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce8580 0x556374ce5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce7bc0 0x556374cf8170 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 --2- 172.21.15.137:0/1918361015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce69f0 0x556374cdcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 -- 172.21.15.137:0/1918361015 >> 172.21.15.137:0/1918361015 conn(0x556374ce1de0 msgr2=0x556374ce5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 -- 172.21.15.137:0/1918361015 shutdown_connections 2024-03-26T14:03:37.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 -- 172.21.15.137:0/1918361015 wait complete. 2024-03-26T14:03:37.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.775+0000 7f2f877da740 1 -- start start 2024-03-26T14:03:37.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 Processor -- start 2024-03-26T14:03:37.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 0x556374cf3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce7bc0 0x556374cf4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374cf70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801aa920 con 0x556374ce69f0 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801abc40 con 0x556374ce8580 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f801aaf80 con 0x556374ce7bc0 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.776+0000 7f2f877da740 10 client.0 _ll_register_callbacks cb 0x7f2f6c097b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 0x556374cf3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374cf70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 0x556374cf3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59060/0 (socket says 172.21.15.137:59060) 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 -- 172.21.15.137:0/530410132 learned_addr learned my addr 172.21.15.137:0/530410132 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374cf70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36184/0 (socket says 172.21.15.137:36184) 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7d400640 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce7bc0 0x556374cf4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 -- 172.21.15.137:0/530410132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce7bc0 msgr2=0x556374cf4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce7bc0 0x556374cf4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 -- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 msgr2=0x556374cf70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:37.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374cf70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.777+0000 7f2f7de00640 1 -- 172.21.15.137:0/530410132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556374cf76c0 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f7de00640 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 0x556374cf3be0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2f7002c170 tx=0x7f2f70009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da3d60e07e6c2c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f5f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f5f400640 1 -- 172.21.15.137:0/530410132 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70005180 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f5f400640 1 -- 172.21.15.137:0/530410132 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2f70005320 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f6c09bf80 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f5f400640 1 -- 172.21.15.137:0/530410132 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f70005520 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f877da740 10 client.15894 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.778+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556374d57e50 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f5f400640 1 -- 172.21.15.137:0/530410132 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f6c09bf80 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f5f400640 1 -- 172.21.15.137:0/530410132 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2f70004b50 con 0x556374ce69f0 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f877da740 10 client.15894 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f877da740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f877da740 20 client.15894 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:37.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f877da740 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374d58730 0x556374d5abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:37.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f877da740 4 client.15894 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:03:37.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.779+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556374c52b50 con 0x556374d58730 2024-03-26T14:03:37.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.780+0000 7f2f7d400640 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374d58730 0x556374d5abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.780+0000 7f2f7d400640 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374d58730 0x556374d5abf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a219b74a045e668 server_cookie=265f22c146497972 in_seq=0 out_seq=0 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.780+0000 7f2f5f400640 10 client.15894 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f5f400640 1 -- 172.21.15.137:0/530410132 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556374c52b50 con 0x556374d58730 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f5f400640 10 client.15894 handle_command_reply: tid=0 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 client.15894 shutdown 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 msgr2=0x556374cf3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 0x556374cf3be0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2f7002c170 tx=0x7f2f70009260 comp rx=0 tx=0).stop 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 shutdown_connections 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556374d58730 0x556374d5abf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556374ce8580 0x556374cf70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556374ce7bc0 0x556374cf4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 --2- 172.21.15.137:0/530410132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556374ce69f0 0x556374cf3be0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f5f400640 0 client.15894 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:37.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 >> 172.21.15.137:0/530410132 conn(0x556374ce1de0 msgr2=0x556374ce58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 shutdown_connections 2024-03-26T14:03:37.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 1 -- 172.21.15.137:0/530410132 wait complete. 2024-03-26T14:03:37.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.781+0000 7f2f877da740 20 client.15894 trim_cache size 0 max 16384 2024-03-26T14:03:37.782 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f640a2800 msgr2=0x7f2f640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f640a2800 0x7f2f640a4cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2f70000c00 tx=0x7f2f7002c7e0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 msgr2=0x7f2f80197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 0x7f2f80197cf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00e9c0 tx=0x7f2f6c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 shutdown_connections 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f640a2800 0x7f2f640a4cc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f80107260 0x7f2f8019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f80104930 0x7f2f80197cf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 --2- 172.21.15.137:0/3322009426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f80102000 0x7f2f801977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.782+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 >> 172.21.15.137:0/3322009426 conn(0x7f2f800fba10 msgr2=0x7f2f801028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.784+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 shutdown_connections 2024-03-26T14:03:37.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:37.784+0000 7f2f7f200640 1 -- 172.21.15.137:0/3322009426 wait complete. 2024-03-26T14:03:38.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:38 smithi179 ceph-mon[35472]: pgmap v496: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T14:03:38.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:38 smithi159 ceph-mon[35501]: pgmap v496: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T14:03:38.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:38 smithi137 ceph-mon[31539]: pgmap v496: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-03-26T14:03:38.804 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:03:38.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:03:38.805 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:03:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 -- 172.21.15.137:0/3351576232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 msgr2=0x7fc090105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 --2- 172.21.15.137:0/3351576232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 0x7fc090105590 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc088009ca0 tx=0x7fc08802c100 comp rx=0 tx=0).stop 2024-03-26T14:03:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 -- 172.21.15.137:0/3351576232 shutdown_connections 2024-03-26T14:03:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 --2- 172.21.15.137:0/3351576232 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 0x7fc090105590 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 --2- 172.21.15.137:0/3351576232 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc090102760 0x7fc090104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 --2- 172.21.15.137:0/3351576232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc090102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 -- 172.21.15.137:0/3351576232 >> 172.21.15.137:0/3351576232 conn(0x7fc0900fba10 msgr2=0x7fc0900fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 -- 172.21.15.137:0/3351576232 shutdown_connections 2024-03-26T14:03:38.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.995+0000 7fc095a00640 1 -- 172.21.15.137:0/3351576232 wait complete. 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.996+0000 7fc095a00640 1 Processor -- start 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.996+0000 7fc095a00640 1 -- start start 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.996+0000 7fc095a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc09019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.996+0000 7fc095a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc090102760 0x7fc09019e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.996+0000 7fc095a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 0x7fc0901a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc095a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0901136a0 con 0x7fc090105110 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc095a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc090113520 con 0x7fc0900ffdf0 2024-03-26T14:03:38.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc095a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc090113820 con 0x7fc090102760 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc095000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 0x7fc0901a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc09019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc09019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35368/0 (socket says 172.21.15.137:35368) 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 -- 172.21.15.137:0/756640489 learned_addr learned my addr 172.21.15.137:0/756640489 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08f400640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc090102760 0x7fc09019e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 -- 172.21.15.137:0/756640489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc090102760 msgr2=0x7fc09019e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc090102760 0x7fc09019e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 -- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 msgr2=0x7fc0901a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 0x7fc0901a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.997+0000 7fc08fe00640 1 -- 172.21.15.137:0/756640489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0901a5970 con 0x7fc0900ffdf0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.998+0000 7fc08fe00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc09019dd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e9f0 tx=0x7fc08000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.998+0000 7fc08cc00640 1 -- 172.21.15.137:0/756640489 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08000cd80 con 0x7fc0900ffdf0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.998+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0901a5c60 con 0x7fc0900ffdf0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.998+0000 7fc08cc00640 1 -- 172.21.15.137:0/756640489 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc0800108e0 con 0x7fc0900ffdf0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.998+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0901a61b0 con 0x7fc0900ffdf0 2024-03-26T14:03:38.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:38.998+0000 7fc08cc00640 1 -- 172.21.15.137:0/756640489 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800188e0 con 0x7fc0900ffdf0 2024-03-26T14:03:39.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.002+0000 7fc099d75740 1 Processor -- start 2024-03-26T14:03:39.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.002+0000 7fc099d75740 1 -- start start 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc099d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753b2b0 0x55742753b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc099d75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753bca0 0x5574275407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc099d75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x5574275432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc099d75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574274a0eb0 con 0x55742753bca0 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc099d75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557427449a60 con 0x557427540e10 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc099d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742749cf50 con 0x55742753b2b0 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc08cc00640 1 -- 172.21.15.137:0/756640489 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc080018a80 con 0x7fc0900ffdf0 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc08f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x5574275432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc095000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753b2b0 0x55742753b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc08f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x5574275432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35370/0 (socket says 172.21.15.137:35370) 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.003+0000 7fc08f400640 1 -- 172.21.15.137:0/2643498033 learned_addr learned my addr 172.21.15.137:0/2643498033 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08fe00640 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753bca0 0x5574275407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08f400640 1 -- 172.21.15.137:0/2643498033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753b2b0 msgr2=0x55742753b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08f400640 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753b2b0 0x55742753b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08f400640 1 -- 172.21.15.137:0/2643498033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753bca0 msgr2=0x5574275407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08f400640 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753bca0 0x5574275407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08f400640 1 -- 172.21.15.137:0/2643498033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55742727d7e0 con 0x557427540e10 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.004+0000 7fc08f400640 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x5574275432a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009c30 tx=0x7fc07c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f29b195d7fee92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.005+0000 7fc085600640 1 -- 172.21.15.137:0/2643498033 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c012450 con 0x557427540e10 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.005+0000 7fc085600640 1 -- 172.21.15.137:0/2643498033 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc07c012aa0 con 0x557427540e10 2024-03-26T14:03:39.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.005+0000 7fc085600640 1 -- 172.21.15.137:0/2643498033 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00fb10 con 0x557427540e10 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.005+0000 7fc08cc00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc0740a2880 0x7fc0740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc08cc00640 1 -- 172.21.15.137:0/756640489 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc0800cf170 con 0x7fc0900ffdf0 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc095000640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc0740a2880 0x7fc0740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 -- 172.21.15.137:0/2643498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 msgr2=0x5574275432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x5574275432a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009c30 tx=0x7fc07c0117b0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 -- 172.21.15.137:0/2643498033 shutdown_connections 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x5574275432a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753bca0 0x5574275407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 --2- 172.21.15.137:0/2643498033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753b2b0 0x55742753b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 -- 172.21.15.137:0/2643498033 >> 172.21.15.137:0/2643498033 conn(0x557427536820 msgr2=0x557427536c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 -- 172.21.15.137:0/2643498033 shutdown_connections 2024-03-26T14:03:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.006+0000 7fc099d75740 1 -- 172.21.15.137:0/2643498033 wait complete. 2024-03-26T14:03:39.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.007+0000 7fc099d75740 1 -- start start 2024-03-26T14:03:39.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.007+0000 7fc095000640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc0740a2880 0x7fc0740a4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc088032380 tx=0x7fc0880324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.007+0000 7fc099d75740 1 Processor -- start 2024-03-26T14:03:39.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x557427545460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x5574275459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557427543dc0 con 0x55742753b2b0 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55742745ceb0 con 0x557427540e10 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557427543f40 con 0x55742753bca0 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc099d75740 10 client.0 _ll_register_callbacks cb 0x557427451150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc095000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x557427545460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc095000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35372/0 (socket says 172.21.15.137:35372) 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc095000640 1 -- 172.21.15.137:0/1412323389 learned_addr learned my addr 172.21.15.137:0/1412323389 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x557427545460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33198/0 (socket says 172.21.15.137:33198) 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.008+0000 7fc08f400640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x5574275459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc095000640 1 -- 172.21.15.137:0/1412323389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 msgr2=0x5574275459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc095000640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x5574275459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc095000640 1 -- 172.21.15.137:0/1412323389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 msgr2=0x557427545460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc095000640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x557427545460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc095000640 1 -- 172.21.15.137:0/1412323389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574272ddf70 con 0x557427540e10 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc08fe00640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x557427545460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc095000640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754c9e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc088009ca0 tx=0x7fc0880095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d37e7e32dd5313 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc07a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.009+0000 7fc07a000640 1 -- 172.21.15.137:0/1412323389 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088051020 con 0x557427540e10 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.010+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc07c012aa0 con 0x557427540e10 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.010+0000 7fc07a000640 1 -- 172.21.15.137:0/1412323389 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc088003f80 con 0x557427540e10 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.010+0000 7fc07a000640 1 -- 172.21.15.137:0/1412323389 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc08805c050 con 0x557427540e10 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.010+0000 7fc099d75740 10 client.35603 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.010+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557427175170 con 0x557427540e10 2024-03-26T14:03:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.010+0000 7fc07a000640 1 -- 172.21.15.137:0/1412323389 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc07c012aa0 con 0x557427540e10 2024-03-26T14:03:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc07a000640 1 -- 172.21.15.137:0/1412323389 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc08804d8c0 con 0x557427540e10 2024-03-26T14:03:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc099d75740 10 client.35603 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc099d75740 10 client.35603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc099d75740 20 client.35603 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc099d75740 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275537f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc099d75740 4 client.35603 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574274ae3f0 con 0x557427551370 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.011+0000 7fc08fe00640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.012+0000 7fc08fe00640 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275537f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91995ff19270b133 server_cookie=a7ab385315537a2b in_seq=0 out_seq=0 2024-03-26T14:03:39.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.012+0000 7fc07a000640 10 client.35603 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.014+0000 7fc07a000640 1 -- 172.21.15.137:0/1412323389 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5574274ae3f0 con 0x557427551370 2024-03-26T14:03:39.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.014+0000 7fc07a000640 10 client.35603 handle_command_reply: tid=0 2024-03-26T14:03:39.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.014+0000 7fc099d75740 1 client.35603 shutdown 2024-03-26T14:03:39.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.014+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 msgr2=0x55742754c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.014+0000 7fc099d75740 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754c9e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc088009ca0 tx=0x7fc0880095c0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 shutdown_connections 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275537f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc07a000640 0 client.35603 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754c9e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x5574275459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 --2- 172.21.15.137:0/1412323389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x557427545460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 >> 172.21.15.137:0/1412323389 conn(0x557427536820 msgr2=0x557427535a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 shutdown_connections 2024-03-26T14:03:39.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 1 -- 172.21.15.137:0/1412323389 wait complete. 2024-03-26T14:03:39.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.015+0000 7fc099d75740 20 client.35603 trim_cache size 0 max 16384 2024-03-26T14:03:39.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.047+0000 7fc099d75740 1 Processor -- start 2024-03-26T14:03:39.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.047+0000 7fc099d75740 1 -- start start 2024-03-26T14:03:39.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc099d75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 0x557427535900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc099d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc099d75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557427540e10 0x5574275580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc099d75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0700120e0 con 0x557427540e10 2024-03-26T14:03:39.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc099d75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000fbe0 con 0x55742753b2b0 2024-03-26T14:03:39.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc099d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000dc60 con 0x55742753bca0 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.048+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 0x557427535900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 0x557427535900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35384/0 (socket says 172.21.15.137:35384) 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08f400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 -- 172.21.15.137:0/2066845744 learned_addr learned my addr 172.21.15.137:0/2066845744 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc095000640 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557427540e10 0x5574275580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 -- 172.21.15.137:0/2066845744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 msgr2=0x55742754c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 -- 172.21.15.137:0/2066845744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557427540e10 msgr2=0x5574275580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557427540e10 0x5574275580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc08fe00640 1 -- 172.21.15.137:0/2066845744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55742753d750 con 0x55742753b2b0 2024-03-26T14:03:39.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.049+0000 7fc095000640 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557427540e10 0x5574275580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:03:39.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.050+0000 7fc08fe00640 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 0x557427535900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc080097b60 tx=0x7fc0800d15c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4728f27085e6f59c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.050+0000 7fc084c00640 1 -- 172.21.15.137:0/2066845744 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08009b700 con 0x55742753b2b0 2024-03-26T14:03:39.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.050+0000 7fc084c00640 1 -- 172.21.15.137:0/2066845744 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc08009f850 con 0x55742753b2b0 2024-03-26T14:03:39.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.050+0000 7fc084c00640 1 -- 172.21.15.137:0/2066845744 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800d10b0 con 0x55742753b2b0 2024-03-26T14:03:39.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 -- 172.21.15.137:0/2066845744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 msgr2=0x557427535900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 0x557427535900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc080097b60 tx=0x7fc0800d15c0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 -- 172.21.15.137:0/2066845744 shutdown_connections 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557427540e10 0x5574275580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 --2- 172.21.15.137:0/2066845744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55742753b2b0 0x557427535900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 -- 172.21.15.137:0/2066845744 >> 172.21.15.137:0/2066845744 conn(0x557427536820 msgr2=0x5574275351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 -- 172.21.15.137:0/2066845744 shutdown_connections 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.051+0000 7fc099d75740 1 -- 172.21.15.137:0/2066845744 wait complete. 2024-03-26T14:03:39.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.052+0000 7fc099d75740 1 -- start start 2024-03-26T14:03:39.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.052+0000 7fc099d75740 1 Processor -- start 2024-03-26T14:03:39.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x55742754c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000e260 con 0x55742753b2b0 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000f580 con 0x557427540e10 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000e8c0 con 0x55742753bca0 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc099d75740 10 client.0 _ll_register_callbacks cb 0x7fc088044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x55742754c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc08fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x55742754c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33214/0 (socket says 172.21.15.137:33214) 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc08fe00640 1 -- 172.21.15.137:0/1742344799 learned_addr learned my addr 172.21.15.137:0/1742344799 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc095000640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.053+0000 7fc08f400640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc08fe00640 1 -- 172.21.15.137:0/1742344799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 msgr2=0x55742754aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc08fe00640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc08fe00640 1 -- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 msgr2=0x55742754b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc08fe00640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc08fe00640 1 -- 172.21.15.137:0/1742344799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55742754bb50 con 0x55742753b2b0 2024-03-26T14:03:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc08fe00640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x55742754c7f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc080009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab6af03f4606fde server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc07a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc07a000640 1 -- 172.21.15.137:0/1742344799 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08009b700 con 0x55742753b2b0 2024-03-26T14:03:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.054+0000 7fc07a000640 1 -- 172.21.15.137:0/1742344799 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc080010b00 con 0x55742753b2b0 2024-03-26T14:03:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.055+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc08009f850 con 0x55742753b2b0 2024-03-26T14:03:39.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.055+0000 7fc07a000640 1 -- 172.21.15.137:0/1742344799 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0800a3ea0 con 0x55742753b2b0 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.055+0000 7fc099d75740 10 client.15912 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.055+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5574275add10 con 0x55742753b2b0 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.055+0000 7fc07a000640 1 -- 172.21.15.137:0/1742344799 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc08009f850 con 0x55742753b2b0 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.055+0000 7fc07a000640 1 -- 172.21.15.137:0/1742344799 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc08009ac70 con 0x55742753b2b0 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc099d75740 10 client.15912 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc099d75740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc099d75740 20 client.15912 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc099d75740 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc099d75740 4 client.15912 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5574274a8740 con 0x557427551370 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.056+0000 7fc08f400640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.057+0000 7fc08f400640 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275b0690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e49587ce97c3de9 server_cookie=edf393586cd3adf in_seq=0 out_seq=0 2024-03-26T14:03:39.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.057+0000 7fc07a000640 10 client.15912 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.059+0000 7fc07a000640 1 -- 172.21.15.137:0/1742344799 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5574274a8740 con 0x557427551370 2024-03-26T14:03:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.059+0000 7fc07a000640 10 client.15912 handle_command_reply: tid=0 2024-03-26T14:03:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.059+0000 7fc099d75740 1 client.15912 shutdown 2024-03-26T14:03:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.059+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 msgr2=0x55742754c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.059+0000 7fc099d75740 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x55742754c7f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc080000c00 tx=0x7fc080009700 comp rx=0 tx=0).stop 2024-03-26T14:03:39.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 shutdown_connections 2024-03-26T14:03:39.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557427551370 0x5574275b0690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557427540e10 0x55742754b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55742753bca0 0x55742754aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 --2- 172.21.15.137:0/1742344799 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55742753b2b0 0x55742754c7f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 >> 172.21.15.137:0/1742344799 conn(0x557427536820 msgr2=0x557427535a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc07a000640 0 client.15912 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 shutdown_connections 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 1 -- 172.21.15.137:0/1742344799 wait complete. 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.060+0000 7fc099d75740 20 client.15912 trim_cache size 0 max 16384 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "9ec57acc-473b-46e6-ab89-289d674661a3", 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:03:39.059 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc0740a2880 msgr2=0x7fc0740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc0740a2880 0x7fc0740a4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc088032380 tx=0x7fc0880324c0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 msgr2=0x7fc09019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc09019dd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc08000e9f0 tx=0x7fc08000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 shutdown_connections 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc0740a2880 0x7fc0740a4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc090105110 0x7fc0901a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc090102760 0x7fc09019e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 --2- 172.21.15.137:0/756640489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0900ffdf0 0x7fc09019dd50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 >> 172.21.15.137:0/756640489 conn(0x7fc0900fba10 msgr2=0x7fc090100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 shutdown_connections 2024-03-26T14:03:39.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.061+0000 7fc095a00640 1 -- 172.21.15.137:0/756640489 wait complete. 2024-03-26T14:03:39.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:03:39.076 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:03:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 -- 172.21.15.137:0/356360345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a41010f0 msgr2=0x7ff9a4101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 --2- 172.21.15.137:0/356360345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a41010f0 0x7ff9a4101570 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009ca0 tx=0x7ff99c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 -- 172.21.15.137:0/356360345 shutdown_connections 2024-03-26T14:03:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 --2- 172.21.15.137:0/356360345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 0x7ff9a4100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 --2- 172.21.15.137:0/356360345 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a41010f0 0x7ff9a4101570 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 --2- 172.21.15.137:0/356360345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 -- 172.21.15.137:0/356360345 >> 172.21.15.137:0/356360345 conn(0x7ff9a40fba10 msgr2=0x7ff9a40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 -- 172.21.15.137:0/356360345 shutdown_connections 2024-03-26T14:03:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.255+0000 7ff9a9200640 1 -- 172.21.15.137:0/356360345 wait complete. 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.256+0000 7ff9a9200640 1 Processor -- start 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.256+0000 7ff9a9200640 1 -- start start 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 0x7ff9a4199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a9200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a41010f0 0x7ff9a41a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a9200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a410f2a0 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a9200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a410f120 con 0x7ff9a4100750 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a9200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a410f420 con 0x7ff9a41010f0 2024-03-26T14:03:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33238/0 (socket says 172.21.15.137:33238) 2024-03-26T14:03:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a3400640 1 -- 172.21.15.137:0/1600315510 learned_addr learned my addr 172.21.15.137:0/1600315510 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 0x7ff9a4199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.257+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a41010f0 0x7ff9a41a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a3400640 1 -- 172.21.15.137:0/1600315510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a41010f0 msgr2=0x7ff9a41a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a3400640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a41010f0 0x7ff9a41a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a3400640 1 -- 172.21.15.137:0/1600315510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 msgr2=0x7ff9a4199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a3400640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 0x7ff9a4199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a3400640 1 -- 172.21.15.137:0/1600315510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a41a1560 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 0x7ff9a4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a3400640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4199940 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d770 tx=0x7ff99400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff99be00640 1 -- 172.21.15.137:0/1600315510 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994017070 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff99be00640 1 -- 172.21.15.137:0/1600315510 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9940054d0 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a41a1850 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.258+0000 7ff99be00640 1 -- 172.21.15.137:0/1600315510 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994004cf0 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.259+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a41a1d50 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.262+0000 7ff9ad3f0740 1 Processor -- start 2024-03-26T14:03:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.262+0000 7ff9ad3f0740 1 -- start start 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c524a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c524f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524fbb0 0x5562c5252040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c51af250 con 0x5562c524a050 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c5157a30 con 0x5562c524aa40 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c51ab2f0 con 0x5562c524fbb0 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c524a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c524a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33252/0 (socket says 172.21.15.137:33252) 2024-03-26T14:03:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 -- 172.21.15.137:0/2390886574 learned_addr learned my addr 172.21.15.137:0/2390886574 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 -- 172.21.15.137:0/2390886574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524fbb0 msgr2=0x5562c5252040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524fbb0 0x5562c5252040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 -- 172.21.15.137:0/2390886574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 msgr2=0x5562c524f540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T14:03:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c524f540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.263+0000 7ff9a3e00640 1 -- 172.21.15.137:0/2390886574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562c5142690 con 0x5562c524a050 2024-03-26T14:03:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.264+0000 7ff99be00640 1 -- 172.21.15.137:0/1600315510 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff994004e90 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.264+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c524a470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005e30 tx=0x7ff99c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e03a9872ceb7a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.264+0000 7ff998c00640 1 -- 172.21.15.137:0/2390886574 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c043020 con 0x5562c524a050 2024-03-26T14:03:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.264+0000 7ff998c00640 1 -- 172.21.15.137:0/2390886574 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff99c032db0 con 0x5562c524a050 2024-03-26T14:03:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.264+0000 7ff998c00640 1 -- 172.21.15.137:0/2390886574 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c039040 con 0x5562c524a050 2024-03-26T14:03:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.265+0000 7ff99be00640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9880a2800 0x7ff9880a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.265+0000 7ff99be00640 1 -- 172.21.15.137:0/1600315510 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff9940ce980 con 0x7ff9a40ffdf0 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/2390886574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 msgr2=0x5562c524a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c524a470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005e30 tx=0x7ff99c005f80 comp rx=0 tx=0).stop 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9a3400640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9880a2800 0x7ff9880a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/2390886574 shutdown_connections 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c524f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524fbb0 0x5562c5252040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/2390886574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c524a470 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.266+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/2390886574 >> 172.21.15.137:0/2390886574 conn(0x5562c5212200 msgr2=0x5562c52459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.267+0000 7ff9a3400640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9880a2800 0x7ff9880a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff994009610 tx=0x7ff994009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.267+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/2390886574 shutdown_connections 2024-03-26T14:03:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.267+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/2390886574 wait complete. 2024-03-26T14:03:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.267+0000 7ff9ad3f0740 1 -- start start 2024-03-26T14:03:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 Processor -- start 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c5253fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5254510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524fbb0 0x5562c525b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c5252b00 con 0x5562c524a050 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c516ae80 con 0x5562c524fbb0 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.268+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562c5252c80 con 0x5562c524aa40 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9ad3f0740 10 client.0 _ll_register_callbacks cb 0x5562c517d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c5253fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c5253fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33264/0 (socket says 172.21.15.137:33264) 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 -- 172.21.15.137:0/3891741142 learned_addr learned my addr 172.21.15.137:0/3891741142 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524fbb0 0x5562c525b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5254510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 -- 172.21.15.137:0/3891741142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 msgr2=0x5562c5254510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5254510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 -- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524fbb0 msgr2=0x5562c525b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524fbb0 0x5562c525b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.269+0000 7ff9a3e00640 1 -- 172.21.15.137:0/3891741142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562c4f8d7d0 con 0x5562c524a050 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c5253fd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff99c032550 tx=0x7ff99c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddefac53cc7ede2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff98d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff98d600640 1 -- 172.21.15.137:0/3891741142 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c043030 con 0x5562c524a050 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff98d600640 1 -- 172.21.15.137:0/3891741142 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff99c004000 con 0x5562c524a050 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff99c032db0 con 0x5562c524a050 2024-03-26T14:03:39.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff98d600640 1 -- 172.21.15.137:0/3891741142 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff99c057050 con 0x5562c524a050 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff9ad3f0740 10 client.15924 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.270+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5562c4ec33f0 con 0x5562c524a050 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff98d600640 1 -- 172.21.15.137:0/3891741142 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff99c032db0 con 0x5562c524a050 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff98d600640 1 -- 172.21.15.137:0/3891741142 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff99c03fa80 con 0x5562c524a050 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff9ad3f0740 10 client.15924 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff9ad3f0740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff9ad3f0740 20 client.15924 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c525fa40 0x5562c5261e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff9ad3f0740 4 client.15924 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:03:39.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.271+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562c51bc790 con 0x5562c525fa40 2024-03-26T14:03:39.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.272+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c525fa40 0x5562c5261e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.273+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c525fa40 0x5562c5261e30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96388fe45dfdeb2f server_cookie=67e88e720a21016 in_seq=0 out_seq=0 2024-03-26T14:03:39.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.273+0000 7ff98d600640 10 client.15924 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff98d600640 1 -- 172.21.15.137:0/3891741142 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5562c51bc790 con 0x5562c525fa40 2024-03-26T14:03:39.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff98d600640 10 client.15924 handle_command_reply: tid=0 2024-03-26T14:03:39.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 client.15924 shutdown 2024-03-26T14:03:39.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 msgr2=0x5562c5253fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c5253fd0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff99c032550 tx=0x7ff99c0095c0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 shutdown_connections 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c525fa40 0x5562c5261e30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524fbb0 0x5562c525b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5254510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/3891741142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524a050 0x5562c5253fd0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 >> 172.21.15.137:0/3891741142 conn(0x5562c5212200 msgr2=0x5562c5244bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff98d600640 0 client.15924 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 shutdown_connections 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.274+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/3891741142 wait complete. 2024-03-26T14:03:39.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.275+0000 7ff9ad3f0740 20 client.15924 trim_cache size 0 max 16384 2024-03-26T14:03:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.308+0000 7ff9ad3f0740 1 Processor -- start 2024-03-26T14:03:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.308+0000 7ff9ad3f0740 1 -- start start 2024-03-26T14:03:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524a050 0x5562c5249310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c525b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c5244330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41ae6b0 con 0x5562c524fbb0 2024-03-26T14:03:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41ac1b0 con 0x5562c524aa40 2024-03-26T14:03:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9ad3f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41aa230 con 0x5562c524a050 2024-03-26T14:03:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c525b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9a2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c525b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35426/0 (socket says 172.21.15.137:35426) 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.309+0000 7ff9a2a00640 1 -- 172.21.15.137:0/405698356 learned_addr learned my addr 172.21.15.137:0/405698356 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c5244330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524a050 0x5562c5249310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a2a00640 1 -- 172.21.15.137:0/405698356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524a050 msgr2=0x5562c5249310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524a050 0x5562c5249310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a2a00640 1 -- 172.21.15.137:0/405698356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 msgr2=0x5562c5244330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c5244330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.310+0000 7ff9a2a00640 1 -- 172.21.15.137:0/405698356 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562c4e830f0 con 0x5562c524aa40 2024-03-26T14:03:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.311+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c525b4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c00 tx=0x7ff99001b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e710681db008f02d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.311+0000 7ff98fe00640 1 -- 172.21.15.137:0/405698356 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990005030 con 0x5562c524aa40 2024-03-26T14:03:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.311+0000 7ff98fe00640 1 -- 172.21.15.137:0/405698356 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff99001f020 con 0x5562c524aa40 2024-03-26T14:03:39.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.311+0000 7ff98fe00640 1 -- 172.21.15.137:0/405698356 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900126b0 con 0x5562c524aa40 2024-03-26T14:03:39.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/405698356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 msgr2=0x5562c525b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c525b4e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c00 tx=0x7ff99001b960 comp rx=0 tx=0).stop 2024-03-26T14:03:39.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/405698356 shutdown_connections 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524aa40 0x5562c525b4e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c5244330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/405698356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524a050 0x5562c5249310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.312+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/405698356 >> 172.21.15.137:0/405698356 conn(0x5562c5212200 msgr2=0x5562c5248c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.313+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/405698356 shutdown_connections 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.313+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/405698356 wait complete. 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.313+0000 7ff9ad3f0740 1 -- start start 2024-03-26T14:03:39.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.314+0000 7ff9ad3f0740 1 Processor -- start 2024-03-26T14:03:39.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.314+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 0x5562c5257200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9ad3f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5257740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 0x5562c5257200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 0x5562c5257200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:35430/0 (socket says 172.21.15.137:35430) 2024-03-26T14:03:39.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9a3e00640 1 -- 172.21.15.137:0/952026906 learned_addr learned my addr 172.21.15.137:0/952026906 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c525a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41aa830 con 0x5562c524fbb0 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41abb50 con 0x5562c524a050 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41aae90 con 0x5562c524aa40 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9ad3f0740 10 client.0 _ll_register_callbacks cb 0x7ff990005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.315+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c525a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a3e00640 1 -- 172.21.15.137:0/952026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 msgr2=0x5562c5257740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5257740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5257740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a3e00640 1 -- 172.21.15.137:0/952026906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 msgr2=0x5562c525a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c525a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a3e00640 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5562c525ad50 con 0x5562c524a050 2024-03-26T14:03:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5257740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:03:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff9a3e00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 0x5562c5257200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff99c02c630 tx=0x7ff99c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5730a2bb178f091 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:03:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.316+0000 7ff98d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:03:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.317+0000 7ff98d600640 1 -- 172.21.15.137:0/952026906 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c061060 con 0x5562c524a050 2024-03-26T14:03:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.317+0000 7ff98d600640 1 -- 172.21.15.137:0/952026906 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff99c003f50 con 0x5562c524a050 2024-03-26T14:03:39.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.317+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff99001f020 con 0x5562c524a050 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff98d600640 1 -- 172.21.15.137:0/952026906 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff99c043070 con 0x5562c524a050 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 10 client.35624 fetch_fsmap learned FSMap version 24 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5562c52c3f10 con 0x5562c524a050 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff98d600640 1 -- 172.21.15.137:0/952026906 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff99001f020 con 0x5562c524a050 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff98d600640 1 -- 172.21.15.137:0/952026906 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff99c034740 con 0x5562c524a050 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 10 client.35624 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:03:39.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 10 client.35624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:03:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 20 client.35624 populate_metadata read hostname 'smithi137' 2024-03-26T14:03:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c52c47e0 0x5562c52c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:03:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 4 client.35624 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:03:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.318+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5562c51b6ae0 con 0x5562c52c47e0 2024-03-26T14:03:39.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.319+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c52c47e0 0x5562c52c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:03:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.320+0000 7ff9a2a00640 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c52c47e0 0x5562c52c6c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a27ca93c42a55334 server_cookie=f7a24e8e232170d7 in_seq=0 out_seq=0 2024-03-26T14:03:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.320+0000 7ff98d600640 10 client.35624 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff98d600640 1 -- 172.21.15.137:0/952026906 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5562c51b6ae0 con 0x5562c52c47e0 2024-03-26T14:03:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff98d600640 10 client.35624 handle_command_reply: tid=0 2024-03-26T14:03:39.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 client.35624 shutdown 2024-03-26T14:03:39.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 msgr2=0x5562c5257200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 0x5562c5257200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff99c02c630 tx=0x7ff99c000f40 comp rx=0 tx=0).stop 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 shutdown_connections 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5562c52c47e0 0x5562c52c6c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5562c524fbb0 0x5562c525a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5562c524aa40 0x5562c5257740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 --2- 172.21.15.137:0/952026906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562c524a050 0x5562c5257200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 >> 172.21.15.137:0/952026906 conn(0x5562c5212200 msgr2=0x5562c5249490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.321+0000 7ff98d600640 0 client.35624 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.322+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 shutdown_connections 2024-03-26T14:03:39.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.322+0000 7ff9ad3f0740 1 -- 172.21.15.137:0/952026906 wait complete. 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.322+0000 7ff9ad3f0740 20 client.35624 trim_cache size 0 max 16384 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: "9ec57acc-473b-46e6-ab89-289d674661a3": { 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "9ec57acc-473b-46e6-ab89-289d674661a3", 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:03:39.321 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:03:39.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9880a2800 msgr2=0x7ff9880a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9880a2800 0x7ff9880a4cc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff994009610 tx=0x7ff994009f30 comp rx=0 tx=0).stop 2024-03-26T14:03:39.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 msgr2=0x7ff9a4199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:03:39.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4199940 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d770 tx=0x7ff99400dc30 comp rx=0 tx=0).stop 2024-03-26T14:03:39.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 shutdown_connections 2024-03-26T14:03:39.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9880a2800 0x7ff9880a4cc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9a4100750 0x7ff9a4199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9a41010f0 0x7ff9a41a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:03:39.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 --2- 172.21.15.137:0/1600315510 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9a40ffdf0 0x7ff9a4199940 secure :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff99400d770 tx=0x7ff99400dc30 comp rx=0 tx=0).stop 2024-03-26T14:03:39.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.323+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 >> 172.21.15.137:0/1600315510 conn(0x7ff9a40fba10 msgr2=0x7ff9a40fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:03:39.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.324+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 shutdown_connections 2024-03-26T14:03:39.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:03:39.324+0000 7ff9a9200640 1 -- 172.21.15.137:0/1600315510 wait complete. 2024-03-26T14:03:39.338 INFO:tasks.cephfs.filesystem:scrub status for tag:9ec57acc-473b-46e6-ab89-289d674661a3 - {'path': '/', 'tag': '9ec57acc-473b-46e6-ab89-289d674661a3', 'options': 'recursive,force'} 2024-03-26T14:03:39.532 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:39 smithi137 ceph-mon[31539]: pgmap v497: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-26T14:03:39.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:39 smithi179 ceph-mon[35472]: pgmap v497: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-26T14:03:39.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:39 smithi159 ceph-mon[35501]: pgmap v497: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-03-26T14:03:40.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:40 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:03:40.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:40 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:40.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:40 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:03:40.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:40 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:40.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:40 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:03:40.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:40 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:40.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:40 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:03:40.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:40 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:40.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:40 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:03:40.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:40 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:40.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:40 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:03:40.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:40 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:03:41.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:41 smithi179 ceph-mon[35472]: pgmap v498: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:03:41.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:41 smithi159 ceph-mon[35501]: pgmap v498: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:03:41.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:41 smithi137 ceph-mon[31539]: pgmap v498: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:03:42.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:42 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:03:42.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:42 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:03:42.782 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:42 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:03:43.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:43 smithi179 ceph-mon[35472]: pgmap v499: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:03:43.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:43 smithi159 ceph-mon[35501]: pgmap v499: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:03:43.782 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:43 smithi137 ceph-mon[31539]: pgmap v499: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:03:46.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:46 smithi137 ceph-mon[31539]: pgmap v500: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:46.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:46 smithi179 ceph-mon[35472]: pgmap v500: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:46.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:46 smithi159 ceph-mon[35501]: pgmap v500: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:48.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:48 smithi137 ceph-mon[31539]: pgmap v501: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:03:48.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:48 smithi179 ceph-mon[35472]: pgmap v501: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:03:48.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:48 smithi159 ceph-mon[35501]: pgmap v501: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:03:50.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:50 smithi137 ceph-mon[31539]: pgmap v502: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:50.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:50 smithi179 ceph-mon[35472]: pgmap v502: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:50.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:50 smithi159 ceph-mon[35501]: pgmap v502: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:52.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:52 smithi137 ceph-mon[31539]: pgmap v503: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:52.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:52 smithi179 ceph-mon[35472]: pgmap v503: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:52.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:52 smithi159 ceph-mon[35501]: pgmap v503: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:03:54.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:54 smithi179 ceph-mon[35472]: pgmap v504: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:03:54.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:54 smithi159 ceph-mon[35501]: pgmap v504: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:03:54.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:54 smithi137 ceph-mon[31539]: pgmap v504: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:03:56.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:56 smithi179 ceph-mon[35472]: pgmap v505: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:56.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:56 smithi159 ceph-mon[35501]: pgmap v505: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:56.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:56 smithi137 ceph-mon[31539]: pgmap v505: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:58.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:58 smithi179 ceph-mon[35472]: pgmap v506: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:03:58.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:58 smithi159 ceph-mon[35501]: pgmap v506: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:03:58.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:58 smithi137 ceph-mon[31539]: pgmap v506: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:03:59.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:03:59 smithi179 ceph-mon[35472]: pgmap v507: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:59.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:03:59 smithi159 ceph-mon[35501]: pgmap v507: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:03:59.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:03:59 smithi137 ceph-mon[31539]: pgmap v507: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:04:00.724 INFO:tasks.workunit.client.0.smithi137.stderr:Updating files: 63% (7774/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-26T14:04:01.453 INFO:tasks.workunit.client.0.smithi137.stderr:Note: switching to '3be9a5c9bc793e11e2800a8c0c696e8b46742033'. 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr:state without impacting any branches by switching back to a branch. 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: git switch -c 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr:Or undo this operation with: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.454 INFO:tasks.workunit.client.0.smithi137.stderr: git switch - 2024-03-26T14:04:01.455 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.455 INFO:tasks.workunit.client.0.smithi137.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-26T14:04:01.455 INFO:tasks.workunit.client.0.smithi137.stderr: 2024-03-26T14:04:01.455 INFO:tasks.workunit.client.0.smithi137.stderr:HEAD is now at 3be9a5c9bc7 dummy commit #1 2024-03-26T14:04:01.463 DEBUG:teuthology.orchestra.run.smithi137:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-26T14:04:01.517 INFO:tasks.workunit.client.0.smithi137.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-26T14:04:01.523 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T14:04:01.523 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-26T14:04:02.466 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-26T14:04:02.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:02 smithi137 ceph-mon[31539]: pgmap v508: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:04:02.591 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-26T14:04:02.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:02 smithi179 ceph-mon[35472]: pgmap v508: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:04:02.674 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-26T14:04:02.680 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T14:04:02.681 INFO:tasks.workunit.client.0.smithi137.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-26T14:04:02.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:02 smithi159 ceph-mon[35501]: pgmap v508: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:04:02.763 INFO:tasks.workunit.client.0.smithi137.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-26T14:04:02.773 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T14:04:02.774 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-26T14:04:02.824 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-03-26T14:04:02.825 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-03-26T14:04:02.825 DEBUG:teuthology.orchestra.run.smithi137:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-03-26T14:04:02.905 INFO:tasks.workunit.client.0.smithi137.stdout:test large (16MB) dio write 2024-03-26T14:04:02.905 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo 'test large (16MB) dio write' 2024-03-26T14:04:02.905 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-03-26T14:04:03.542 INFO:tasks.workunit.client.0.smithi137.stderr:1+0 records in 2024-03-26T14:04:03.543 INFO:tasks.workunit.client.0.smithi137.stderr:1+0 records out 2024-03-26T14:04:03.543 INFO:tasks.workunit.client.0.smithi137.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.630214 s, 26.6 MB/s 2024-03-26T14:04:03.545 INFO:tasks.workunit.client.0.smithi137.stdout:OK 2024-03-26T14:04:03.545 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo OK 2024-03-26T14:04:03.546 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T14:04:03.546 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T14:04:03.657 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-03-26T14:04:03.657 DEBUG:teuthology.orchestra.run.smithi137:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-03-26T14:04:04.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:04 smithi137 ceph-mon[31539]: pgmap v509: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:04:04.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:04 smithi179 ceph-mon[35472]: pgmap v509: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:04:04.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:04 smithi159 ceph-mon[35501]: pgmap v509: 129 pgs: 129 active+clean; 667 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:04:06.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:06 smithi179 ceph-mon[35472]: pgmap v510: 129 pgs: 129 active+clean; 8.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 682 KiB/s wr, 10 op/s 2024-03-26T14:04:06.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:06 smithi159 ceph-mon[35501]: pgmap v510: 129 pgs: 129 active+clean; 8.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 682 KiB/s wr, 10 op/s 2024-03-26T14:04:06.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:06 smithi137 ceph-mon[31539]: pgmap v510: 129 pgs: 129 active+clean; 8.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 682 KiB/s wr, 10 op/s 2024-03-26T14:04:08.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:08 smithi179 ceph-mon[35472]: pgmap v511: 129 pgs: 129 active+clean; 8.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.0 MiB/s wr, 15 op/s 2024-03-26T14:04:08.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:08 smithi159 ceph-mon[35501]: pgmap v511: 129 pgs: 129 active+clean; 8.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.0 MiB/s wr, 15 op/s 2024-03-26T14:04:08.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:08 smithi137 ceph-mon[31539]: pgmap v511: 129 pgs: 129 active+clean; 8.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.0 MiB/s wr, 15 op/s 2024-03-26T14:04:09.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:04:09.339 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.564+0000 7f811c600640 1 -- 172.21.15.137:0/2134823312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8114104ad0 msgr2=0x7f8114104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.564+0000 7f811c600640 1 --2- 172.21.15.137:0/2134823312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8114104ad0 0x7f8114104f50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8100009900 tx=0x7f810002bc30 comp rx=0 tx=0).stop 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 -- 172.21.15.137:0/2134823312 shutdown_connections 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 --2- 172.21.15.137:0/2134823312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8114105490 0x7f811410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 --2- 172.21.15.137:0/2134823312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8114104ad0 0x7f8114104f50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 --2- 172.21.15.137:0/2134823312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81141038e0 0x7f8114103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 -- 172.21.15.137:0/2134823312 >> 172.21.15.137:0/2134823312 conn(0x7f81140fee50 msgr2=0x7f8114101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 -- 172.21.15.137:0/2134823312 shutdown_connections 2024-03-26T14:04:09.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.565+0000 7f811c600640 1 -- 172.21.15.137:0/2134823312 wait complete. 2024-03-26T14:04:09.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 Processor -- start 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 -- start start 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 0x7f8114199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8114104ad0 0x7f811419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8114105490 0x7f81141a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811410f440 con 0x7f81141038e0 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811410f2c0 con 0x7f8114105490 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f811c600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811410f5c0 con 0x7f8114104ad0 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f8119c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 0x7f8114199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f8119c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 0x7f8114199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60024/0 (socket says 172.21.15.137:60024) 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.566+0000 7f8119c00640 1 -- 172.21.15.137:0/2585550330 learned_addr learned my addr 172.21.15.137:0/2585550330 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f811a600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8114105490 0x7f81141a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8119c00640 1 -- 172.21.15.137:0/2585550330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8114104ad0 msgr2=0x7f811419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8119c00640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8114104ad0 0x7f811419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8119c00640 1 -- 172.21.15.137:0/2585550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8114105490 msgr2=0x7f81141a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8119c00640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8114105490 0x7f81141a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8119c00640 1 -- 172.21.15.137:0/2585550330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81141a1710 con 0x7f81141038e0 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8119c00640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 0x7f8114199af0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8112000640 1 -- 172.21.15.137:0/2585550330 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108017020 con 0x7f81141038e0 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81141a1a00 con 0x7f81141038e0 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8112000640 1 -- 172.21.15.137:0/2585550330 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8108009c30 con 0x7f81141038e0 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f8112000640 1 -- 172.21.15.137:0/2585550330 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800f620 con 0x7f81141038e0 2024-03-26T14:04:09.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.567+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81141a1f50 con 0x7f81141038e0 2024-03-26T14:04:09.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f8112000640 1 -- 172.21.15.137:0/2585550330 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f810801a020 con 0x7f81141038e0 2024-03-26T14:04:09.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 Processor -- start 2024-03-26T14:04:09.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 -- start start 2024-03-26T14:04:09.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6eff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f06e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b6e64250 con 0x5597b6eff870 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b6e0ca30 con 0x5597b6efee80 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.571+0000 7f811f277740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b6e602f0 con 0x5597b6f049e0 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f811a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6eff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f04370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60040/0 (socket says 172.21.15.137:60040) 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f811a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6eff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:36636/0 (socket says 172.21.15.137:36636) 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 -- 172.21.15.137:0/1129205360 learned_addr learned my addr 172.21.15.137:0/1129205360 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119200640 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f06e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8112000640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80fc0a2800 0x7f80fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 -- 172.21.15.137:0/1129205360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 msgr2=0x5597b6f06e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f06e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 -- 172.21.15.137:0/1129205360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 msgr2=0x5597b6eff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6eff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f8119c00640 1 -- 172.21.15.137:0/1129205360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597b6df7690 con 0x5597b6eff870 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.572+0000 7f811a600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80fc0a2800 0x7f80fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.573+0000 7f8112000640 1 -- 172.21.15.137:0/2585550330 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f81080cee90 con 0x7f81141038e0 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.573+0000 7f8119c00640 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f04370 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f81080849b0 tx=0x7f8108090900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c244beeb58b1cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.573+0000 7f8106a00640 1 -- 172.21.15.137:0/1129205360 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810809af60 con 0x5597b6eff870 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.573+0000 7f8106a00640 1 -- 172.21.15.137:0/1129205360 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81080849e0 con 0x5597b6eff870 2024-03-26T14:04:09.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.573+0000 7f8106a00640 1 -- 172.21.15.137:0/1129205360 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080d29c0 con 0x5597b6eff870 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 -- 172.21.15.137:0/1129205360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 msgr2=0x5597b6f04370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f04370 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f81080849b0 tx=0x7f8108090900 comp rx=0 tx=0).stop 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811a600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80fc0a2800 0x7f80fc0a4cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5597b6f000d0 tx=0x7f810c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 -- 172.21.15.137:0/1129205360 shutdown_connections 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f06e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f04370 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 --2- 172.21.15.137:0/1129205360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6eff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 -- 172.21.15.137:0/1129205360 >> 172.21.15.137:0/1129205360 conn(0x5597b6ec7200 msgr2=0x5597b6efa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 -- 172.21.15.137:0/1129205360 shutdown_connections 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 -- 172.21.15.137:0/1129205360 wait complete. 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.574+0000 7f811f277740 1 -- start start 2024-03-26T14:04:09.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 Processor -- start 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b6f078d0 con 0x5597b6eff870 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b6e1fe80 con 0x5597b6efee80 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597b6f07a50 con 0x5597b6f049e0 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f811f277740 10 client.0 _ll_register_callbacks cb 0x5597b6e32130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f8119200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f8119200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60048/0 (socket says 172.21.15.137:60048) 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.575+0000 7f8119200640 1 -- 172.21.15.137:0/2290743546 learned_addr learned my addr 172.21.15.137:0/2290743546 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119c00640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f811a600640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119200640 1 -- 172.21.15.137:0/2290743546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 msgr2=0x5597b6f102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119200640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f102c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119200640 1 -- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 msgr2=0x5597b6f08db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119200640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f08db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119200640 1 -- 172.21.15.137:0/2290743546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597b6c427d0 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119c00640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f08db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f8119200640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f092f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f810002c240 tx=0x7f8100004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96077f91347643f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.576+0000 7f80fb400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f80fb400640 1 -- 172.21.15.137:0/2290743546 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100042030 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f80fb400640 1 -- 172.21.15.137:0/2290743546 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8100005040 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597b6e71790 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f80fb400640 1 -- 172.21.15.137:0/2290743546 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810004f050 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f811f277740 10 client.15942 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5597b6b783f0 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f80fb400640 1 -- 172.21.15.137:0/2290743546 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5597b6e71790 con 0x5597b6eff870 2024-03-26T14:04:09.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.577+0000 7f80fb400640 1 -- 172.21.15.137:0/2290743546 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8100004ca0 con 0x5597b6eff870 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f811f277740 10 client.15942 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f811f277740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f811f277740 20 client.15942 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f811f277740 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f14c40 0x5597b6f17050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f811f277740 4 client.15942 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597b6e6bae0 con 0x5597b6f14c40 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.578+0000 7f8119c00640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f14c40 0x5597b6f17050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.579+0000 7f8119c00640 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f14c40 0x5597b6f17050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c274a27064d99c53 server_cookie=24da1c0e4367ef25 in_seq=0 out_seq=0 2024-03-26T14:04:09.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.579+0000 7f80fb400640 10 client.15942 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:09.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.580+0000 7f80fb400640 1 -- 172.21.15.137:0/2290743546 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5597b6e6bae0 con 0x5597b6f14c40 2024-03-26T14:04:09.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.580+0000 7f80fb400640 10 client.15942 handle_command_reply: tid=0 2024-03-26T14:04:09.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.580+0000 7f811f277740 1 client.15942 shutdown 2024-03-26T14:04:09.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.580+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 msgr2=0x5597b6f092f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.580+0000 7f811f277740 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f092f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f810002c240 tx=0x7f8100004270 comp rx=0 tx=0).stop 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 shutdown_connections 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f14c40 0x5597b6f17050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6eff870 0x5597b6f092f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f80fb400640 0 client.15942 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 --2- 172.21.15.137:0/2290743546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f08db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 >> 172.21.15.137:0/2290743546 conn(0x5597b6ec7200 msgr2=0x5597b6ef9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 shutdown_connections 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 1 -- 172.21.15.137:0/2290743546 wait complete. 2024-03-26T14:04:09.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.581+0000 7f811f277740 20 client.15942 trim_cache size 0 max 16384 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.598+0000 7f811f277740 1 Processor -- start 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.598+0000 7f811f277740 1 -- start start 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811f277740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 0x5597b6efda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6f049e0 0x5597b6f10280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811f277740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f09420 0x5597b6efdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811f277740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141ae9c0 con 0x5597b6efee80 2024-03-26T14:04:09.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141ac4c0 con 0x5597b6f049e0 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811f277740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141aa540 con 0x5597b6f09420 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 0x5597b6efda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 0x5597b6efda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60056/0 (socket says 172.21.15.137:60056) 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 -- 172.21.15.137:0/1917285690 learned_addr learned my addr 172.21.15.137:0/1917285690 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119200640 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6f049e0 0x5597b6f10280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f811a600640 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f09420 0x5597b6efdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 -- 172.21.15.137:0/1917285690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f09420 msgr2=0x5597b6efdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f09420 0x5597b6efdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 -- 172.21.15.137:0/1917285690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6f049e0 msgr2=0x5597b6f10280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6f049e0 0x5597b6f10280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119c00640 1 -- 172.21.15.137:0/1917285690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597b6b380f0 con 0x5597b6efee80 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.599+0000 7f8119200640 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6f049e0 0x5597b6f10280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:09.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.600+0000 7f8119c00640 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 0x5597b6efda30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8108009610 tx=0x7f8108095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b544c30495c9e45 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.600+0000 7f8106000640 1 -- 172.21.15.137:0/1917285690 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080646f0 con 0x5597b6efee80 2024-03-26T14:04:09.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.600+0000 7f8106000640 1 -- 172.21.15.137:0/1917285690 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81080a3bc0 con 0x5597b6efee80 2024-03-26T14:04:09.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.600+0000 7f8106000640 1 -- 172.21.15.137:0/1917285690 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810809ad80 con 0x5597b6efee80 2024-03-26T14:04:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 -- 172.21.15.137:0/1917285690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 msgr2=0x5597b6efda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 0x5597b6efda30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8108009610 tx=0x7f8108095f90 comp rx=0 tx=0).stop 2024-03-26T14:04:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 -- 172.21.15.137:0/1917285690 shutdown_connections 2024-03-26T14:04:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6f049e0 0x5597b6f10280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f09420 0x5597b6efdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 --2- 172.21.15.137:0/1917285690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6efee80 0x5597b6efda30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 -- 172.21.15.137:0/1917285690 >> 172.21.15.137:0/1917285690 conn(0x5597b6ec7200 msgr2=0x5597b6f00ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 -- 172.21.15.137:0/1917285690 shutdown_connections 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 -- 172.21.15.137:0/1917285690 wait complete. 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.601+0000 7f811f277740 1 -- start start 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 Processor -- start 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f0e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 0x5597b6f0ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141aab40 con 0x5597b6f09420 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141abe60 con 0x5597b6efee80 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141ab1a0 con 0x5597b6f049e0 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811f277740 10 client.0 _ll_register_callbacks cb 0x7f8100009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:09.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811a600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 0x5597b6f0ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811a600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 0x5597b6f0ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60058/0 (socket says 172.21.15.137:60058) 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f811a600640 1 -- 172.21.15.137:0/2192937195 learned_addr learned my addr 172.21.15.137:0/2192937195 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f8119c00640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.602+0000 7f8119200640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f0e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811a600640 1 -- 172.21.15.137:0/2192937195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 msgr2=0x5597b6f0e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811a600640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f0e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811a600640 1 -- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 msgr2=0x5597b6f10060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811a600640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f10060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811a600640 1 -- 172.21.15.137:0/2192937195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597b6f0f2a0 con 0x5597b6f09420 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811a600640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 0x5597b6f0ebb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f810c013430 tx=0x7f810c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df52f9ed83e47e5d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f80fb400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f80fb400640 1 -- 172.21.15.137:0/2192937195 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c01f7f0 con 0x5597b6f09420 2024-03-26T14:04:09.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f80fb400640 1 -- 172.21.15.137:0/2192937195 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f810c01f990 con 0x5597b6f09420 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.603+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81080a3bc0 con 0x5597b6f09420 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f80fb400640 1 -- 172.21.15.137:0/2192937195 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f810c01fb90 con 0x5597b6f09420 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 10 client.15948 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5597b6f714e0 con 0x5597b6f09420 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f80fb400640 1 -- 172.21.15.137:0/2192937195 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81080a3bc0 con 0x5597b6f09420 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f80fb400640 1 -- 172.21.15.137:0/2192937195 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f810c026070 con 0x5597b6f09420 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 10 client.15948 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 20 client.15948 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f71ef0 0x5597b6f74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 4 client.15948 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.604+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5597b6f08ba0 con 0x5597b6f71ef0 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.605+0000 7f8119c00640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f71ef0 0x5597b6f74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.605+0000 7f8119c00640 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f71ef0 0x5597b6f74310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af201422099c204 server_cookie=124d53595c0138cc in_seq=0 out_seq=0 2024-03-26T14:04:09.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.605+0000 7f80fb400640 10 client.15948 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.605+0000 7f80fb400640 1 -- 172.21.15.137:0/2192937195 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5597b6f08ba0 con 0x5597b6f71ef0 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.605+0000 7f80fb400640 10 client.15948 handle_command_reply: tid=0 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.605+0000 7f811f277740 1 client.15948 shutdown 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 msgr2=0x5597b6f0ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 0x5597b6f0ebb0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f810c013430 tx=0x7f810c004ab0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 shutdown_connections 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5597b6f71ef0 0x5597b6f74310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5597b6f09420 0x5597b6f0ebb0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597b6f049e0 0x5597b6f0e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f80fb400640 0 client.15948 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 --2- 172.21.15.137:0/2192937195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597b6efee80 0x5597b6f10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 >> 172.21.15.137:0/2192937195 conn(0x5597b6ec7200 msgr2=0x5597b6f00ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 shutdown_connections 2024-03-26T14:04:09.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 1 -- 172.21.15.137:0/2192937195 wait complete. 2024-03-26T14:04:09.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811f277740 20 client.15948 trim_cache size 0 max 16384 2024-03-26T14:04:09.609 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:04:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:04:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:04:09.609 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80fc0a2800 msgr2=0x7f80fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.606+0000 7f811c600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80fc0a2800 0x7f80fc0a4cc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x5597b6f000d0 tx=0x7f810c004a40 comp rx=0 tx=0).stop 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 msgr2=0x7f8114199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 0x7f8114199af0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f810800d950 tx=0x7f810800de10 comp rx=0 tx=0).stop 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 shutdown_connections 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80fc0a2800 0x7f80fc0a4cc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8114105490 0x7f81141a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8114104ad0 0x7f811419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 --2- 172.21.15.137:0/2585550330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f81141038e0 0x7f8114199af0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 >> 172.21.15.137:0/2585550330 conn(0x7f81140fee50 msgr2=0x7f8114107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 shutdown_connections 2024-03-26T14:04:09.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.607+0000 7f811c600640 1 -- 172.21.15.137:0/2585550330 wait complete. 2024-03-26T14:04:09.620 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:09 smithi137 ceph-mon[31539]: pgmap v512: 129 pgs: 129 active+clean; 691 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-03-26T14:04:09.624 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:04:09.624 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:04:09.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:09 smithi179 ceph-mon[35472]: pgmap v512: 129 pgs: 129 active+clean; 691 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-03-26T14:04:09.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:09 smithi159 ceph-mon[35501]: pgmap v512: 129 pgs: 129 active+clean; 691 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 18 op/s 2024-03-26T14:04:09.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 -- 172.21.15.137:0/1127922663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030104ab0 msgr2=0x7fa030104f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 --2- 172.21.15.137:0/1127922663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030104ab0 0x7fa030104f30 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009900 tx=0x7fa01c02bc50 comp rx=0 tx=0).stop 2024-03-26T14:04:09.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 -- 172.21.15.137:0/1127922663 shutdown_connections 2024-03-26T14:04:09.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 --2- 172.21.15.137:0/1127922663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa030105470 0x7fa03010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 --2- 172.21.15.137:0/1127922663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030104ab0 0x7fa030104f30 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 --2- 172.21.15.137:0/1127922663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0301038c0 0x7fa030103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 -- 172.21.15.137:0/1127922663 >> 172.21.15.137:0/1127922663 conn(0x7fa0300fee50 msgr2=0x7fa030101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:09.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 -- 172.21.15.137:0/1127922663 shutdown_connections 2024-03-26T14:04:09.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.835+0000 7fa034c00640 1 -- 172.21.15.137:0/1127922663 wait complete. 2024-03-26T14:04:09.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 Processor -- start 2024-03-26T14:04:09.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 -- start start 2024-03-26T14:04:09.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0301038c0 0x7fa030199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa030104ab0 0x7fa03019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 0x7fa0301a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03010f440 con 0x7fa030105470 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03010f2c0 con 0x7fa0301038c0 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.836+0000 7fa034c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03010f5c0 con 0x7fa030104ab0 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 0x7fa0301a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 0x7fa0301a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60068/0 (socket says 172.21.15.137:60068) 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 -- 172.21.15.137:0/1451665112 learned_addr learned my addr 172.21.15.137:0/1451665112 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02de00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0301038c0 0x7fa030199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02d400640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa030104ab0 0x7fa03019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 -- 172.21.15.137:0/1451665112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa030104ab0 msgr2=0x7fa03019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa030104ab0 0x7fa03019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 -- 172.21.15.137:0/1451665112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0301038c0 msgr2=0x7fa030199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0301038c0 0x7fa030199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 -- 172.21.15.137:0/1451665112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0301a1740 con 0x7fa030105470 2024-03-26T14:04:09.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.837+0000 7fa02e800640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 0x7fa0301a10d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa02800eaa0 tx=0x7fa02800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.838+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800cdb0 con 0x7fa030105470 2024-03-26T14:04:09.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.838+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa028005020 con 0x7fa030105470 2024-03-26T14:04:09.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.838+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301a1a30 con 0x7fa030105470 2024-03-26T14:04:09.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.838+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0280116d0 con 0x7fa030105470 2024-03-26T14:04:09.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.838+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0301a1f00 con 0x7fa030105470 2024-03-26T14:04:09.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.839+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03010ad00 con 0x7fa030105470 2024-03-26T14:04:09.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.841+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa0280051d0 con 0x7fa030105470 2024-03-26T14:04:09.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.843+0000 7fa026a00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0140a2800 0x7fa0140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:09.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.843+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa0280cf0f0 con 0x7fa030105470 2024-03-26T14:04:09.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.843+0000 7fa02de00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0140a2800 0x7fa0140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:09.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.844+0000 7fa02de00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0140a2800 0x7fa0140a4cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa020005fd0 tx=0x7fa020005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:09.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:09.848+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa028016030 con 0x7fa030105470 2024-03-26T14:04:10.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.086+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa03002d150 con 0x7fa030105470 2024-03-26T14:04:10.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.087+0000 7fa026a00640 1 -- 172.21.15.137:0/1451665112 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fa02809bdc0 con 0x7fa030105470 2024-03-26T14:04:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:04:10.085 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:04:10.086 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:04:10.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0140a2800 msgr2=0x7fa0140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0140a2800 0x7fa0140a4cc0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa020005fd0 tx=0x7fa020005850 comp rx=0 tx=0).stop 2024-03-26T14:04:10.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 msgr2=0x7fa0301a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 0x7fa0301a10d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa02800eaa0 tx=0x7fa02800ef60 comp rx=0 tx=0).stop 2024-03-26T14:04:10.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 shutdown_connections 2024-03-26T14:04:10.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa0140a2800 0x7fa0140a4cc0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa030105470 0x7fa0301a10d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa030104ab0 0x7fa03019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 --2- 172.21.15.137:0/1451665112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0301038c0 0x7fa030199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 >> 172.21.15.137:0/1451665112 conn(0x7fa0300fee50 msgr2=0x7fa030107ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 shutdown_connections 2024-03-26T14:04:10.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.090+0000 7fa034c00640 1 -- 172.21.15.137:0/1451665112 wait complete. 2024-03-26T14:04:10.108 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:04:10.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 -- 172.21.15.137:0/1966468333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8108db0 msgr2=0x7f94f8109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 --2- 172.21.15.137:0/1966468333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8108db0 0x7f94f8109230 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009900 tx=0x7f94e002bc30 comp rx=0 tx=0).stop 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 -- 172.21.15.137:0/1966468333 shutdown_connections 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 --2- 172.21.15.137:0/1966468333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8109770 0x7f94f8110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 --2- 172.21.15.137:0/1966468333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8108db0 0x7f94f8109230 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 --2- 172.21.15.137:0/1966468333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94f8107bc0 0x7f94f8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 -- 172.21.15.137:0/1966468333 >> 172.21.15.137:0/1966468333 conn(0x7f94f8076b70 msgr2=0x7f94f8078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 -- 172.21.15.137:0/1966468333 shutdown_connections 2024-03-26T14:04:10.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.335+0000 7f94fda00640 1 -- 172.21.15.137:0/1966468333 wait complete. 2024-03-26T14:04:10.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.336+0000 7f94fda00640 1 Processor -- start 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 -- start start 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 0x7f94f819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8108db0 0x7f94f819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94f8109770 0x7f94f81a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f8113740 con 0x7f94f8107bc0 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81135c0 con 0x7f94f8108db0 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fda00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81138c0 con 0x7f94f8109770 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 0x7f94f819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 0x7f94f819dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60078/0 (socket says 172.21.15.137:60078) 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94f7e00640 1 -- 172.21.15.137:0/1077076835 learned_addr learned my addr 172.21.15.137:0/1077076835 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94ef400640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8108db0 0x7f94f819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.337+0000 7f94fd000640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94f8109770 0x7f94f81a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f7e00640 1 -- 172.21.15.137:0/1077076835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94f8109770 msgr2=0x7f94f81a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f7e00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94f8109770 0x7f94f81a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f7e00640 1 -- 172.21.15.137:0/1077076835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8108db0 msgr2=0x7f94f819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f7e00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8108db0 0x7f94f819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f7e00640 1 -- 172.21.15.137:0/1077076835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f81a5a00 con 0x7f94f8107bc0 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94ef400640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8108db0 0x7f94f819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f7e00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 0x7f94f819dde0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f94e800d950 tx=0x7f94e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f5600640 1 -- 172.21.15.137:0/1077076835 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8017070 con 0x7f94f8107bc0 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f5600640 1 -- 172.21.15.137:0/1077076835 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f94e8009c30 con 0x7f94f8107bc0 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94f5600640 1 -- 172.21.15.137:0/1077076835 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800f620 con 0x7f94f8107bc0 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.338+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f81a5cf0 con 0x7f94f8107bc0 2024-03-26T14:04:10.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.339+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f81a61c0 con 0x7f94f8107bc0 2024-03-26T14:04:10.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 Processor -- start 2024-03-26T14:04:10.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f94f5600640 1 -- 172.21.15.137:0/1077076835 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f94e801a070 con 0x7f94f8107bc0 2024-03-26T14:04:10.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 -- start start 2024-03-26T14:04:10.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d23be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d23d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d243e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838d1a02c0 con 0x55838d23cbc0 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838d148c90 con 0x55838d23d580 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.342+0000 7f9501c5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838d19c360 con 0x55838d23b9f0 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d23d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d23d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60082/0 (socket says 172.21.15.137:60082) 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 -- 172.21.15.137:0/126213711 learned_addr learned my addr 172.21.15.137:0/126213711 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94ef400640 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d243e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94fd000640 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d23be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 -- 172.21.15.137:0/126213711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 msgr2=0x55838d23be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d23be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 -- 172.21.15.137:0/126213711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 msgr2=0x55838d243e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d243e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.343+0000 7f94f7e00640 1 -- 172.21.15.137:0/126213711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55838d16d080 con 0x55838d23cbc0 2024-03-26T14:04:10.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94f7e00640 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d23d040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f94e80d4c20 tx=0x7f94e80d6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d158cf30f5b55e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94f5600640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94dc0a2880 0x7f94dc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94fd000640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94dc0a2880 0x7f94dc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94f5600640 1 -- 172.21.15.137:0/1077076835 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f94e80ce890 con 0x7f94f8107bc0 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94ed600640 1 -- 172.21.15.137:0/126213711 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80dd070 con 0x55838d23cbc0 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94ed600640 1 -- 172.21.15.137:0/126213711 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f94e8051e20 con 0x55838d23cbc0 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.344+0000 7f94ed600640 1 -- 172.21.15.137:0/126213711 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e809ac30 con 0x55838d23cbc0 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 -- 172.21.15.137:0/126213711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 msgr2=0x55838d23d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f94fd000640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94dc0a2880 0x7f94dc0a4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55838d23cac0 tx=0x7f94f0009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d23d040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f94e80d4c20 tx=0x7f94e80d6670 comp rx=0 tx=0).stop 2024-03-26T14:04:10.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 -- 172.21.15.137:0/126213711 shutdown_connections 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d243e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d23d040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 --2- 172.21.15.137:0/126213711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d23be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 -- 172.21.15.137:0/126213711 >> 172.21.15.137:0/126213711 conn(0x55838d236de0 msgr2=0x55838d2371e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 -- 172.21.15.137:0/126213711 shutdown_connections 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.345+0000 7f9501c5c740 1 -- 172.21.15.137:0/126213711 wait complete. 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 -- start start 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 Processor -- start 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d245ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d2461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d24d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838d244590 con 0x55838d23cbc0 2024-03-26T14:04:10.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838d15c0d0 con 0x55838d23d580 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55838d244710 con 0x55838d23b9f0 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.346+0000 7f9501c5c740 10 client.0 _ll_register_callbacks cb 0x55838d160e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d2461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d2461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60090/0 (socket says 172.21.15.137:60090) 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 -- 172.21.15.137:0/2693750164 learned_addr learned my addr 172.21.15.137:0/2693750164 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94f7e00640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d245ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94fd000640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d24d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 -- 172.21.15.137:0/2693750164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 msgr2=0x55838d245ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d245ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 -- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 msgr2=0x55838d24d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d24d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 -- 172.21.15.137:0/2693750164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55838ceb43f0 con 0x55838d23cbc0 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94fd000640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d24d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:10.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94ef400640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d2461e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005c40 tx=0x7f94e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792f04a396f8c64f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.347+0000 7f94e6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f94e6000640 1 -- 172.21.15.137:0/2693750164 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0005450 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f94e6000640 1 -- 172.21.15.137:0/2693750164 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f94e00055f0 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94e8051e20 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f94e6000640 1 -- 172.21.15.137:0/2693750164 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94e00044a0 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f9501c5c740 10 client.15966 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55838cf7e7d0 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f94e6000640 1 -- 172.21.15.137:0/2693750164 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f94e8051e20 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.348+0000 7f94e6000640 1 -- 172.21.15.137:0/2693750164 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f94e0003970 con 0x55838d23cbc0 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f9501c5c740 10 client.15966 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f9501c5c740 10 client.15966 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f9501c5c740 20 client.15966 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2517a0 0x55838d253c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f9501c5c740 4 client.15966 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:10.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55838d1ad800 con 0x55838d2517a0 2024-03-26T14:04:10.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.349+0000 7f94f7e00640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2517a0 0x55838d253c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.350+0000 7f94f7e00640 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2517a0 0x55838d253c00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d48efd549d20118 server_cookie=1ccdf82bb9889feb in_seq=0 out_seq=0 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.350+0000 7f94e6000640 10 client.15966 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f94e6000640 1 -- 172.21.15.137:0/2693750164 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55838d1ad800 con 0x55838d2517a0 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f94e6000640 10 client.15966 handle_command_reply: tid=0 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 client.15966 shutdown 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 msgr2=0x55838d2461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d2461e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f94e0005c40 tx=0x7f94e0004270 comp rx=0 tx=0).stop 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 shutdown_connections 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2517a0 0x55838d253c00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23d580 0x55838d24d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23cbc0 0x55838d2461e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f94e6000640 0 client.15966 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:10.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2693750164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23b9f0 0x55838d245ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 >> 172.21.15.137:0/2693750164 conn(0x55838d236de0 msgr2=0x55838d236160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 shutdown_connections 2024-03-26T14:04:10.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.352+0000 7f9501c5c740 1 -- 172.21.15.137:0/2693750164 wait complete. 2024-03-26T14:04:10.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.353+0000 7f9501c5c740 20 client.15966 trim_cache size 0 max 16384 2024-03-26T14:04:10.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 Processor -- start 2024-03-26T14:04:10.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 -- start start 2024-03-26T14:04:10.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 0x55838d235fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d24d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d2584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81b2b30 con 0x55838d23b9f0 2024-03-26T14:04:10.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81b0630 con 0x55838d23cbc0 2024-03-26T14:04:10.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.367+0000 7f9501c5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81ae6b0 con 0x55838d23d580 2024-03-26T14:04:10.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 0x55838d235fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 0x55838d235fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60096/0 (socket says 172.21.15.137:60096) 2024-03-26T14:04:10.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 -- 172.21.15.137:0/2203512308 learned_addr learned my addr 172.21.15.137:0/2203512308 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94ef400640 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d24d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94fd000640 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d2584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 -- 172.21.15.137:0/2203512308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 msgr2=0x55838d2584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d2584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 -- 172.21.15.137:0/2203512308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 msgr2=0x55838d24d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d24d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.368+0000 7f94f7e00640 1 -- 172.21.15.137:0/2203512308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55838d251560 con 0x55838d23b9f0 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.369+0000 7f94f7e00640 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 0x55838d235fe0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f94e80972c0 tx=0x7f94e8010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988e6bc6fa54648d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.369+0000 7f94ecc00640 1 -- 172.21.15.137:0/2203512308 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8021070 con 0x55838d23b9f0 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.369+0000 7f94ecc00640 1 -- 172.21.15.137:0/2203512308 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f94e8009de0 con 0x55838d23b9f0 2024-03-26T14:04:10.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.369+0000 7f94ecc00640 1 -- 172.21.15.137:0/2203512308 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80dd040 con 0x55838d23b9f0 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 -- 172.21.15.137:0/2203512308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 msgr2=0x55838d235fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 0x55838d235fe0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f94e80972c0 tx=0x7f94e8010f10 comp rx=0 tx=0).stop 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 -- 172.21.15.137:0/2203512308 shutdown_connections 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d24d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d2584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 --2- 172.21.15.137:0/2203512308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d23b9f0 0x55838d235fe0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 -- 172.21.15.137:0/2203512308 >> 172.21.15.137:0/2203512308 conn(0x55838d236de0 msgr2=0x55838d2358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 -- 172.21.15.137:0/2203512308 shutdown_connections 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 -- 172.21.15.137:0/2203512308 wait complete. 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.370+0000 7f9501c5c740 1 -- start start 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 Processor -- start 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d249e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d24a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 0x55838d2ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81aecb0 con 0x55838d24d310 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81affd0 con 0x55838d23cbc0 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81af310 con 0x55838d23d580 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f9501c5c740 10 client.0 _ll_register_callbacks cb 0x7f94e0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f94fd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 0x55838d2ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f94fd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 0x55838d2ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60100/0 (socket says 172.21.15.137:60100) 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f94fd000640 1 -- 172.21.15.137:0/1913425264 learned_addr learned my addr 172.21.15.137:0/1913425264 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.371+0000 7f94f7e00640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d249e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94ef400640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d24a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94fd000640 1 -- 172.21.15.137:0/1913425264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 msgr2=0x55838d24a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94fd000640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d24a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94fd000640 1 -- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 msgr2=0x55838d249e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94fd000640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d249e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94fd000640 1 -- 172.21.15.137:0/1913425264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55838d2ad650 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94fd000640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 0x55838d2ad090 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f94f0012980 tx=0x7f94f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f57e2bc4890df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.372+0000 7f94e6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f94e6000640 1 -- 172.21.15.137:0/1913425264 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f0021dd0 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94e8009de0 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f94e6000640 1 -- 172.21.15.137:0/1913425264 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f94f001f450 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f94e6000640 1 -- 172.21.15.137:0/1913425264 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94f001f650 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 10 client.15972 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55838d2aea70 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f94e6000640 1 -- 172.21.15.137:0/1913425264 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f94e8009de0 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f94e6000640 1 -- 172.21.15.137:0/1913425264 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f94f0028070 con 0x55838d24d310 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 10 client.15972 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:04:10.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 20 client.15972 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.373+0000 7f9501c5c740 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2af400 0x55838d2b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.374+0000 7f9501c5c740 4 client.15972 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:04:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.374+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55838d1a7b50 con 0x55838d2af400 2024-03-26T14:04:10.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.374+0000 7f94f7e00640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2af400 0x55838d2b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f94f7e00640 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2af400 0x55838d2b1820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79ce3c89be99e314 server_cookie=3ae893370a90729f in_seq=0 out_seq=0 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f94e6000640 10 client.15972 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f94e6000640 1 -- 172.21.15.137:0/1913425264 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55838d1a7b50 con 0x55838d2af400 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f94e6000640 10 client.15972 handle_command_reply: tid=0 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 client.15972 shutdown 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 msgr2=0x55838d2ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 0x55838d2ad090 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f94f0012980 tx=0x7f94f0004a40 comp rx=0 tx=0).stop 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 shutdown_connections 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55838d2af400 0x55838d2b1820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55838d24d310 0x55838d2ad090 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55838d23d580 0x55838d24a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f94e6000640 0 client.15972 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 --2- 172.21.15.137:0/1913425264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55838d23cbc0 0x55838d249e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 >> 172.21.15.137:0/1913425264 conn(0x55838d236de0 msgr2=0x55838d236160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 shutdown_connections 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.375+0000 7f9501c5c740 1 -- 172.21.15.137:0/1913425264 wait complete. 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f9501c5c740 20 client.15972 trim_cache size 0 max 16384 2024-03-26T14:04:10.377 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94dc0a2880 msgr2=0x7f94dc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94dc0a2880 0x7f94dc0a4d40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55838d23cac0 tx=0x7f94f0009580 comp rx=0 tx=0).stop 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 msgr2=0x7f94f819dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 0x7f94f819dde0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f94e800d950 tx=0x7f94e800de10 comp rx=0 tx=0).stop 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 shutdown_connections 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94dc0a2880 0x7f94dc0a4d40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94f8108db0 0x7f94f819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94f8109770 0x7f94f81a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 --2- 172.21.15.137:0/1077076835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f94f8107bc0 0x7f94f819dde0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 >> 172.21.15.137:0/1077076835 conn(0x7f94f8076b70 msgr2=0x7f94f810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 shutdown_connections 2024-03-26T14:04:10.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.376+0000 7f94fda00640 1 -- 172.21.15.137:0/1077076835 wait complete. 2024-03-26T14:04:10.389 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1451665112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:04:10.392 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:04:10.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:04:10.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 -- 172.21.15.137:0/30080469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 msgr2=0x7f6c7c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 --2- 172.21.15.137:0/30080469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c104f50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009900 tx=0x7f6c6802bc30 comp rx=0 tx=0).stop 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 -- 172.21.15.137:0/30080469 shutdown_connections 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 --2- 172.21.15.137:0/30080469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 0x7f6c7c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 --2- 172.21.15.137:0/30080469 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c104f50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 --2- 172.21.15.137:0/30080469 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c7c1038e0 0x7f6c7c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 -- 172.21.15.137:0/30080469 >> 172.21.15.137:0/30080469 conn(0x7f6c7c0fee50 msgr2=0x7f6c7c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 -- 172.21.15.137:0/30080469 shutdown_connections 2024-03-26T14:04:10.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.606+0000 7f6c80a00640 1 -- 172.21.15.137:0/30080469 wait complete. 2024-03-26T14:04:10.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.607+0000 7f6c80a00640 1 Processor -- start 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.607+0000 7f6c80a00640 1 -- start start 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c7c1038e0 0x7f6c7c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c80a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 0x7f6c7c1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c80a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c10f460 con 0x7f6c7c104ad0 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c80a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c10f2e0 con 0x7f6c7c105490 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c80a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c10f5e0 con 0x7f6c7c1038e0 2024-03-26T14:04:10.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c19a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60104/0 (socket says 172.21.15.137:60104) 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 -- 172.21.15.137:0/2011957635 learned_addr learned my addr 172.21.15.137:0/2011957635 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c7a800640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 0x7f6c7c1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79e00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c7c1038e0 0x7f6c7c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 -- 172.21.15.137:0/2011957635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c7c1038e0 msgr2=0x7f6c7c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c7c1038e0 0x7f6c7c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 -- 172.21.15.137:0/2011957635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 msgr2=0x7f6c7c1a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 0x7f6c7c1a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c79400640 1 -- 172.21.15.137:0/2011957635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c1a1770 con 0x7f6c7c104ad0 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.608+0000 7f6c7a800640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 0x7f6c7c1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:10.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.609+0000 7f6c79400640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c19a090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6c68000c00 tx=0x7f6c680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.609+0000 7f6c72a00640 1 -- 172.21.15.137:0/2011957635 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68043020 con 0x7f6c7c104ad0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.609+0000 7f6c72a00640 1 -- 172.21.15.137:0/2011957635 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c68004aa0 con 0x7f6c7c104ad0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.609+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c1a1a00 con 0x7f6c7c104ad0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.609+0000 7f6c72a00640 1 -- 172.21.15.137:0/2011957635 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68003d00 con 0x7f6c7c104ad0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.609+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c1a1fa0 con 0x7f6c7c104ad0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.612+0000 7f6c72a00640 1 -- 172.21.15.137:0/2011957635 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6c68046020 con 0x7f6c7c104ad0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 Processor -- start 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 -- start start 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb367c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb368e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3693a0 0x5621cb36fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cb2cc2c0 con 0x5621cb367810 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cb274c90 con 0x5621cb3693a0 2024-03-26T14:04:10.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c837ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cb2c8360 con 0x5621cb3689e0 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c7a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb367c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c7a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb367c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60106/0 (socket says 172.21.15.137:60106) 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c7a800640 1 -- 172.21.15.137:0/2957311088 learned_addr learned my addr 172.21.15.137:0/2957311088 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.613+0000 7f6c79400640 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3693a0 0x5621cb36fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c79e00640 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb368e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c7a800640 1 -- 172.21.15.137:0/2957311088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 msgr2=0x5621cb368e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c7a800640 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb368e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c7a800640 1 -- 172.21.15.137:0/2957311088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3693a0 msgr2=0x5621cb36fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c7a800640 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3693a0 0x5621cb36fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c7a800640 1 -- 172.21.15.137:0/2957311088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621cb299080 con 0x5621cb367810 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c72a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c600a2880 0x7f6c600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c7a800640 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb367c30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009fd0 tx=0x7f6c740108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28019df8644e7fab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.614+0000 7f6c72a00640 1 -- 172.21.15.137:0/2011957635 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6c680f11f0 con 0x7f6c7c104ad0 2024-03-26T14:04:10.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c79e00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c600a2880 0x7f6c600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c67000640 1 -- 172.21.15.137:0/2957311088 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74011760 con 0x5621cb367810 2024-03-26T14:04:10.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c67000640 1 -- 172.21.15.137:0/2957311088 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c74011db0 con 0x5621cb367810 2024-03-26T14:04:10.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c67000640 1 -- 172.21.15.137:0/2957311088 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400e3c0 con 0x5621cb367810 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c837ae740 1 -- 172.21.15.137:0/2957311088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 msgr2=0x5621cb367c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c79e00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c600a2880 0x7f6c600a4d40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.615+0000 7f6c837ae740 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb367c30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009fd0 tx=0x7f6c740108e0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 -- 172.21.15.137:0/2957311088 shutdown_connections 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3693a0 0x5621cb36fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb368e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 --2- 172.21.15.137:0/2957311088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb367c30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 -- 172.21.15.137:0/2957311088 >> 172.21.15.137:0/2957311088 conn(0x5621cb362c00 msgr2=0x5621cb363000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 -- 172.21.15.137:0/2957311088 shutdown_connections 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 -- 172.21.15.137:0/2957311088 wait complete. 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 -- start start 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.616+0000 7f6c837ae740 1 Processor -- start 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb371ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb371ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 0x5621cb379030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cb3703b0 con 0x5621cb3693a0 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cb2880d0 con 0x5621cb367810 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cb370530 con 0x5621cb3689e0 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c837ae740 10 client.0 _ll_register_callbacks cb 0x5621cb28ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c7a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 0x5621cb379030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c7a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 0x5621cb379030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60108/0 (socket says 172.21.15.137:60108) 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c7a800640 1 -- 172.21.15.137:0/884592322 learned_addr learned my addr 172.21.15.137:0/884592322 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c79e00640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb371ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.617+0000 7f6c79400640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb371ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c7a800640 1 -- 172.21.15.137:0/884592322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 msgr2=0x5621cb371ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c7a800640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb371ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c7a800640 1 -- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 msgr2=0x5621cb371ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c7a800640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb371ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c7a800640 1 -- 172.21.15.137:0/884592322 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621cafe03f0 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c7a800640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 0x5621cb379030 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000f80 tx=0x7f6c7401b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686560ec0de87b8c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c5b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c5b400640 1 -- 172.21.15.137:0/884592322 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74004e50 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c5b400640 1 -- 172.21.15.137:0/884592322 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6c74011560 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c74011db0 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c5b400640 1 -- 172.21.15.137:0/884592322 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c74033050 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c837ae740 10 client.15984 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.618+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5621cb0aa7d0 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c5b400640 1 -- 172.21.15.137:0/884592322 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c74011db0 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c5b400640 1 -- 172.21.15.137:0/884592322 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6c74005540 con 0x5621cb3693a0 2024-03-26T14:04:10.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c837ae740 10 client.15984 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:10.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c837ae740 10 client.15984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:04:10.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c837ae740 20 client.15984 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:10.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c837ae740 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb37d240 0x5621cb37f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c837ae740 4 client.15984 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:10.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.619+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621cb2d9800 con 0x5621cb37d240 2024-03-26T14:04:10.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.620+0000 7f6c79e00640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb37d240 0x5621cb37f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.621+0000 7f6c79e00640 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb37d240 0x5621cb37f630 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65aaae13ba6bfcaa server_cookie=2dc9d183828f586 in_seq=0 out_seq=0 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.621+0000 7f6c5b400640 10 client.15984 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c5b400640 1 -- 172.21.15.137:0/884592322 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5621cb2d9800 con 0x5621cb37d240 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c5b400640 10 client.15984 handle_command_reply: tid=0 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 client.15984 shutdown 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 msgr2=0x5621cb379030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 0x5621cb379030 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000f80 tx=0x7f6c7401b660 comp rx=0 tx=0).stop 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 shutdown_connections 2024-03-26T14:04:10.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb37d240 0x5621cb37f630 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3693a0 0x5621cb379030 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3689e0 0x5621cb371ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 --2- 172.21.15.137:0/884592322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb371ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c5b400640 0 client.15984 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.622+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 >> 172.21.15.137:0/884592322 conn(0x5621cb362c00 msgr2=0x5621cb361eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.623+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 shutdown_connections 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.623+0000 7f6c837ae740 1 -- 172.21.15.137:0/884592322 wait complete. 2024-03-26T14:04:10.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.623+0000 7f6c837ae740 20 client.15984 trim_cache size 0 max 16384 2024-03-26T14:04:10.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.637+0000 7f6c837ae740 1 Processor -- start 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.637+0000 7f6c837ae740 1 -- start start 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb361d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 0x5621cb378ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb383e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c837ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1ae950 con 0x5621cb3689e0 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c837ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1ac450 con 0x5621cb367810 2024-03-26T14:04:10.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c837ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1aa4d0 con 0x5621cb3693a0 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 0x5621cb378ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 0x5621cb378ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60124/0 (socket says 172.21.15.137:60124) 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 -- 172.21.15.137:0/4123188708 learned_addr learned my addr 172.21.15.137:0/4123188708 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79e00640 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb361d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c7a800640 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb383e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 -- 172.21.15.137:0/4123188708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 msgr2=0x5621cb383e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb383e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 -- 172.21.15.137:0/4123188708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 msgr2=0x5621cb361d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb361d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.638+0000 7f6c79400640 1 -- 172.21.15.137:0/4123188708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621cb363970 con 0x5621cb3689e0 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.639+0000 7f6c79400640 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 0x5621cb378ff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ba270 tx=0x7f6c680f4c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73e067bf542c11e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.639+0000 7f6c66600640 1 -- 172.21.15.137:0/4123188708 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680bd670 con 0x5621cb3689e0 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.639+0000 7f6c66600640 1 -- 172.21.15.137:0/4123188708 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c68004c50 con 0x5621cb3689e0 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.639+0000 7f6c66600640 1 -- 172.21.15.137:0/4123188708 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680044e0 con 0x5621cb3689e0 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 -- 172.21.15.137:0/4123188708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 msgr2=0x5621cb378ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 0x5621cb378ff0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6c680ba270 tx=0x7f6c680f4c30 comp rx=0 tx=0).stop 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 -- 172.21.15.137:0/4123188708 shutdown_connections 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb383e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb3689e0 0x5621cb378ff0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 --2- 172.21.15.137:0/4123188708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb367810 0x5621cb361d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 -- 172.21.15.137:0/4123188708 >> 172.21.15.137:0/4123188708 conn(0x5621cb362c00 msgr2=0x5621cb361620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 -- 172.21.15.137:0/4123188708 shutdown_connections 2024-03-26T14:04:10.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 -- 172.21.15.137:0/4123188708 wait complete. 2024-03-26T14:04:10.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.640+0000 7f6c837ae740 1 -- start start 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 Processor -- start 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb378970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3689e0 0x5621cb378eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb375410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1aaad0 con 0x5621cb367810 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1abdf0 con 0x5621cb3689e0 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1ab130 con 0x5621cb3693a0 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c837ae740 10 client.0 _ll_register_callbacks cb 0x7f6c680f4b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c79e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb378970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c79e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb378970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60140/0 (socket says 172.21.15.137:60140) 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c79e00640 1 -- 172.21.15.137:0/1863165915 learned_addr learned my addr 172.21.15.137:0/1863165915 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.641+0000 7f6c79400640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3689e0 0x5621cb378eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c7a800640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb375410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79e00640 1 -- 172.21.15.137:0/1863165915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 msgr2=0x5621cb375410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79e00640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb375410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79e00640 1 -- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3689e0 msgr2=0x5621cb378eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79e00640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3689e0 0x5621cb378eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79e00640 1 -- 172.21.15.137:0/1863165915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621cb375bf0 con 0x5621cb367810 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79400640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3689e0 0x5621cb378eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c79e00640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb378970 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5621cb363b70 tx=0x7f6c6c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19936e48d60d97fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c5b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c5b400640 1 -- 172.21.15.137:0/1863165915 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004240 con 0x5621cb367810 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c5b400640 1 -- 172.21.15.137:0/1863165915 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6c6c0043e0 con 0x5621cb367810 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c68004c50 con 0x5621cb367810 2024-03-26T14:04:10.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.642+0000 7f6c5b400640 1 -- 172.21.15.137:0/1863165915 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c6c037050 con 0x5621cb367810 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 10 client.15990 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5621cb3d97d0 con 0x5621cb367810 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c5b400640 1 -- 172.21.15.137:0/1863165915 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c68004c50 con 0x5621cb367810 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c5b400640 1 -- 172.21.15.137:0/1863165915 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6c6c0286b0 con 0x5621cb367810 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 10 client.15990 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 10 client.15990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 20 client.15990 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb3da160 0x5621cb3dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 4 client.15990 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.643+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621cb3718a0 con 0x5621cb3da160 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.644+0000 7f6c79400640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb3da160 0x5621cb3dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.644+0000 7f6c79400640 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb3da160 0x5621cb3dc580 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8cc9190399f0e4d server_cookie=253ab0dbcb781b6f in_seq=0 out_seq=0 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.644+0000 7f6c5b400640 10 client.15990 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:10.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c5b400640 1 -- 172.21.15.137:0/1863165915 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621cb3718a0 con 0x5621cb3da160 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c5b400640 10 client.15990 handle_command_reply: tid=0 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 client.15990 shutdown 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 msgr2=0x5621cb378970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb378970 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x5621cb363b70 tx=0x7f6c6c014000 comp rx=0 tx=0).stop 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 shutdown_connections 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5621cb3da160 0x5621cb3dc580 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5621cb3689e0 0x5621cb378eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5621cb3693a0 0x5621cb375410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 --2- 172.21.15.137:0/1863165915 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5621cb367810 0x5621cb378970 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c5b400640 0 client.15990 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 >> 172.21.15.137:0/1863165915 conn(0x5621cb362c00 msgr2=0x5621cb361eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 shutdown_connections 2024-03-26T14:04:10.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.645+0000 7f6c837ae740 1 -- 172.21.15.137:0/1863165915 wait complete. 2024-03-26T14:04:10.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c837ae740 20 client.15990 trim_cache size 0 max 16384 2024-03-26T14:04:10.648 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c600a2880 msgr2=0x7f6c600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c600a2880 0x7f6c600a4d40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c005fd0 tx=0x7f6c6c005c50 comp rx=0 tx=0).stop 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 msgr2=0x7f6c7c19a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c19a090 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6c68000c00 tx=0x7f6c680042f0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 shutdown_connections 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c600a2880 0x7f6c600a4d40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c7c105490 0x7f6c7c1a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c7c104ad0 0x7f6c7c19a090 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 --2- 172.21.15.137:0/2011957635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c7c1038e0 0x7f6c7c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.646+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 >> 172.21.15.137:0/2011957635 conn(0x7f6c7c0fee50 msgr2=0x7f6c7c107ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.647+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 shutdown_connections 2024-03-26T14:04:10.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.647+0000 7f6c80a00640 1 -- 172.21.15.137:0/2011957635 wait complete. 2024-03-26T14:04:10.664 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:04:10.665 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:04:10.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:10 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1451665112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:04:10.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:10 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1451665112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:04:10.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 -- 172.21.15.137:0/431096792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 msgr2=0x7f6f44103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 --2- 172.21.15.137:0/431096792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44103d00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009900 tx=0x7f6f3402bc40 comp rx=0 tx=0).stop 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 -- 172.21.15.137:0/431096792 shutdown_connections 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 --2- 172.21.15.137:0/431096792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f44105490 0x7f6f4410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 --2- 172.21.15.137:0/431096792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f44104ad0 0x7f6f44104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 --2- 172.21.15.137:0/431096792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44103d00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 -- 172.21.15.137:0/431096792 >> 172.21.15.137:0/431096792 conn(0x7f6f440fee50 msgr2=0x7f6f44101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 -- 172.21.15.137:0/431096792 shutdown_connections 2024-03-26T14:04:10.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.885+0000 7f6f4b000640 1 -- 172.21.15.137:0/431096792 wait complete. 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.886+0000 7f6f4b000640 1 Processor -- start 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.886+0000 7f6f4b000640 1 -- start start 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.886+0000 7f6f4b000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44079450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.886+0000 7f6f4b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f44104ad0 0x7f6f44079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f4b000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f44105490 0x7f6f44075fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f4b000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4410f3e0 con 0x7f6f441038e0 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f4b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4410f260 con 0x7f6f44104ad0 2024-03-26T14:04:10.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f4b000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4410f560 con 0x7f6f44105490 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44079450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44079450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60152/0 (socket says 172.21.15.137:60152) 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 -- 172.21.15.137:0/3995075502 learned_addr learned my addr 172.21.15.137:0/3995075502 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43400640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f44104ad0 0x7f6f44079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 -- 172.21.15.137:0/3995075502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f44105490 msgr2=0x7f6f44075fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f49000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f44105490 0x7f6f44075fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f44105490 0x7f6f44075fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 -- 172.21.15.137:0/3995075502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f44104ad0 msgr2=0x7f6f44079990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f44104ad0 0x7f6f44079990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.887+0000 7f6f43e00640 1 -- 172.21.15.137:0/3995075502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f44076880 con 0x7f6f441038e0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.888+0000 7f6f43e00640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44079450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6f34000c00 tx=0x7f6f34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.888+0000 7f6f40c00640 1 -- 172.21.15.137:0/3995075502 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34043020 con 0x7f6f441038e0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.888+0000 7f6f40c00640 1 -- 172.21.15.137:0/3995075502 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f34004aa0 con 0x7f6f441038e0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.888+0000 7f6f40c00640 1 -- 172.21.15.137:0/3995075502 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34003d00 con 0x7f6f441038e0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.888+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f441a5f60 con 0x7f6f441038e0 2024-03-26T14:04:10.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.888+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f441a6310 con 0x7f6f441038e0 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.892+0000 7f6f4dd97740 1 Processor -- start 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.892+0000 7f6f4dd97740 1 -- start start 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f40c00640 1 -- 172.21.15.137:0/3995075502 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6f34046020 con 0x7f6f441038e0 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d054e2c0 con 0x5577d05e9810 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d04f6c90 con 0x5577d05ea9e0 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d054a360 con 0x5577d05eb3a0 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60160/0 (socket says 172.21.15.137:60160) 2024-03-26T14:04:10.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f49000640 1 -- 172.21.15.137:0/3866979539 learned_addr learned my addr 172.21.15.137:0/3866979539 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f43e00640 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.893+0000 7f6f43400640 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f49000640 1 -- 172.21.15.137:0/3866979539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 msgr2=0x5577d05f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f49000640 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f49000640 1 -- 172.21.15.137:0/3866979539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 msgr2=0x5577d05eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f49000640 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f49000640 1 -- 172.21.15.137:0/3866979539 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d051b080 con 0x5577d05e9810 2024-03-26T14:04:10.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f49000640 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e9c30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009e70 tx=0x7f6f3c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6096f66d2c130d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f40c00640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f280a2800 0x7f6f280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f40c00640 1 -- 172.21.15.137:0/3995075502 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6f340f1500 con 0x7f6f441038e0 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f39600640 1 -- 172.21.15.137:0/3866979539 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c016070 con 0x5577d05e9810 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f43e00640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f280a2800 0x7f6f280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f39600640 1 -- 172.21.15.137:0/3866979539 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f3c00ee90 con 0x5577d05e9810 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.894+0000 7f6f39600640 1 -- 172.21.15.137:0/3866979539 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0116d0 con 0x5577d05e9810 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3866979539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 msgr2=0x5577d05e9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e9c30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009e70 tx=0x7f6f3c00e6c0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f43e00640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f280a2800 0x7f6f280a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f340ba4a0 tx=0x7f6f34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3866979539 shutdown_connections 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3866979539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e9c30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3866979539 >> 172.21.15.137:0/3866979539 conn(0x5577d05e4c00 msgr2=0x5577d05e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3866979539 shutdown_connections 2024-03-26T14:04:10.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.895+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3866979539 wait complete. 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.896+0000 7f6f4dd97740 1 -- start start 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.896+0000 7f6f4dd97740 1 Processor -- start 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05e9810 0x5577d05f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 0x5577d05fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d05f23b0 con 0x5577d05eb3a0 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d050a0d0 con 0x5577d05ea9e0 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577d05f2530 con 0x5577d05e9810 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f4dd97740 10 client.0 _ll_register_callbacks cb 0x5577d050ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 0x5577d05fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 0x5577d05fb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60172/0 (socket says 172.21.15.137:60172) 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f49000640 1 -- 172.21.15.137:0/426187036 learned_addr learned my addr 172.21.15.137:0/426187036 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.898 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f43400640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.897+0000 7f6f49000640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05e9810 0x5577d05f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f49000640 1 -- 172.21.15.137:0/426187036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05e9810 msgr2=0x5577d05f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f49000640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05e9810 0x5577d05f3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f49000640 1 -- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 msgr2=0x5577d05f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f49000640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f49000640 1 -- 172.21.15.137:0/426187036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577d02623f0 con 0x5577d05eb3a0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f49000640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 0x5577d05fb090 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c004fa0 tx=0x7f6f3c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c13f63e74c8e0ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f2e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f2e000640 1 -- 172.21.15.137:0/426187036 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c016040 con 0x5577d05eb3a0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f2e000640 1 -- 172.21.15.137:0/426187036 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6f3c011470 con 0x5577d05eb3a0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.898+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f3c00ee90 con 0x5577d05eb3a0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f2e000640 1 -- 172.21.15.137:0/426187036 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f3c031050 con 0x5577d05eb3a0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 10 client.16002 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5577d032c7d0 con 0x5577d05eb3a0 2024-03-26T14:04:10.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f2e000640 1 -- 172.21.15.137:0/426187036 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f3c00ee90 con 0x5577d05eb3a0 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f2e000640 1 -- 172.21.15.137:0/426187036 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6f3c026020 con 0x5577d05eb3a0 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 10 client.16002 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 20 client.16002 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d05ff300 0x5577d06016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 4 client.16002 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.899+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577d055b800 con 0x5577d05ff300 2024-03-26T14:04:10.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.900+0000 7f6f43400640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d05ff300 0x5577d06016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.901+0000 7f6f43400640 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d05ff300 0x5577d06016f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9512e511c2ee9ef server_cookie=5ab6610f0c641efb in_seq=0 out_seq=0 2024-03-26T14:04:10.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.901+0000 7f6f2e000640 10 client.16002 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:10.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f2e000640 1 -- 172.21.15.137:0/426187036 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5577d055b800 con 0x5577d05ff300 2024-03-26T14:04:10.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f2e000640 10 client.16002 handle_command_reply: tid=0 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 client.16002 shutdown 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 msgr2=0x5577d05fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 0x5577d05fb090 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c004fa0 tx=0x7f6f3c009440 comp rx=0 tx=0).stop 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 shutdown_connections 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d05ff300 0x5577d06016f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05eb3a0 0x5577d05fb090 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/426187036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05e9810 0x5577d05f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f2e000640 0 client.16002 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 >> 172.21.15.137:0/426187036 conn(0x5577d05e4c00 msgr2=0x5577d05e3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 shutdown_connections 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 1 -- 172.21.15.137:0/426187036 wait complete. 2024-03-26T14:04:10.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.903+0000 7f6f4dd97740 20 client.16002 trim_cache size 0 max 16384 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 Processor -- start 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 -- start start 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d0605f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441b2d30 con 0x5577d05e9810 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441b0830 con 0x5577d05ea9e0 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.918+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441ae8b0 con 0x5577d05eb3a0 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60180/0 (socket says 172.21.15.137:60180) 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 -- 172.21.15.137:0/4024598285 learned_addr learned my addr 172.21.15.137:0/4024598285 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f43400640 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 -- 172.21.15.137:0/4024598285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 msgr2=0x5577d0605f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d0605f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 -- 172.21.15.137:0/4024598285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 msgr2=0x5577d05fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 -- 172.21.15.137:0/4024598285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577d05e5970 con 0x5577d05e9810 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.919+0000 7f6f49000640 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e8570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009c00 tx=0x7f6f3c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fa6b13c9309631 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f38c00640 1 -- 172.21.15.137:0/4024598285 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03a070 con 0x5577d05e9810 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f38c00640 1 -- 172.21.15.137:0/4024598285 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f3c010eb0 con 0x5577d05e9810 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f38c00640 1 -- 172.21.15.137:0/4024598285 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c016040 con 0x5577d05e9810 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 -- 172.21.15.137:0/4024598285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 msgr2=0x5577d05e8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e8570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009c00 tx=0x7f6f3c004dd0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 -- 172.21.15.137:0/4024598285 shutdown_connections 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d0605f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/4024598285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05e8570 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 -- 172.21.15.137:0/4024598285 >> 172.21.15.137:0/4024598285 conn(0x5577d05e4c00 msgr2=0x5577d05e7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.920+0000 7f6f4dd97740 1 -- 172.21.15.137:0/4024598285 shutdown_connections 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 -- 172.21.15.137:0/4024598285 wait complete. 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 -- start start 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 Processor -- start 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441aeeb0 con 0x5577d05e9810 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441b01d0 con 0x5577d05ea9e0 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f441af510 con 0x5577d05eb3a0 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.921+0000 7f6f4dd97740 10 client.0 _ll_register_callbacks cb 0x7f6f3c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60186/0 (socket says 172.21.15.137:60186) 2024-03-26T14:04:10.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 -- 172.21.15.137:0/3288537531 learned_addr learned my addr 172.21.15.137:0/3288537531 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f43400640 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 -- 172.21.15.137:0/3288537531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 msgr2=0x5577d05f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 -- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 msgr2=0x5577d05f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 -- 172.21.15.137:0/3288537531 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577d05f9ea0 con 0x5577d05e9810 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f49000640 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05f63a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c038040 tx=0x7f6f3c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aad986e4b6b5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.922+0000 7f6f2e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f2e000640 1 -- 172.21.15.137:0/3288537531 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c03a030 con 0x5577d05e9810 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f2e000640 1 -- 172.21.15.137:0/3288537531 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6f3c0053f0 con 0x5577d05e9810 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577d0555b50 con 0x5577d05e9810 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f2e000640 1 -- 172.21.15.137:0/3288537531 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f3c047050 con 0x5577d05e9810 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f4dd97740 10 client.16008 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:10.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5577d05fb1a0 con 0x5577d05e9810 2024-03-26T14:04:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f2e000640 1 -- 172.21.15.137:0/3288537531 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5577d0555b50 con 0x5577d05e9810 2024-03-26T14:04:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.923+0000 7f6f2e000640 1 -- 172.21.15.137:0/3288537531 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6f3c026050 con 0x5577d05e9810 2024-03-26T14:04:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f4dd97740 10 client.16008 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f4dd97740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:10.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f4dd97740 20 client.16008 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d065afa0 0x5577d065d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f4dd97740 4 client.16008 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5577d04d3500 con 0x5577d065afa0 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.924+0000 7f6f43400640 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d065afa0 0x5577d065d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.925+0000 7f6f43400640 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d065afa0 0x5577d065d390 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd866136cc9ab29e server_cookie=7895910de477dbfb in_seq=0 out_seq=0 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.925+0000 7f6f2e000640 10 client.16008 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.925+0000 7f6f2e000640 1 -- 172.21.15.137:0/3288537531 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5577d04d3500 con 0x5577d065afa0 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.925+0000 7f6f2e000640 10 client.16008 handle_command_reply: tid=0 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.925+0000 7f6f4dd97740 1 client.16008 shutdown 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 msgr2=0x5577d05f63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05f63a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c038040 tx=0x7f6f3c005ce0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 shutdown_connections 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5577d065afa0 0x5577d065d390 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5577d05ea9e0 0x5577d05f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577d05eb3a0 0x5577d05f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 --2- 172.21.15.137:0/3288537531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5577d05e9810 0x5577d05f63a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 >> 172.21.15.137:0/3288537531 conn(0x5577d05e4c00 msgr2=0x5577d05e86f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f2e000640 0 client.16008 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 shutdown_connections 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 1 -- 172.21.15.137:0/3288537531 wait complete. 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.926+0000 7f6f4dd97740 20 client.16008 trim_cache size 0 max 16384 2024-03-26T14:04:10.927 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f280a2800 msgr2=0x7f6f280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f280a2800 0x7f6f280a4cc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f340ba4a0 tx=0x7f6f34009260 comp rx=0 tx=0).stop 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 msgr2=0x7f6f44079450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44079450 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6f34000c00 tx=0x7f6f34004270 comp rx=0 tx=0).stop 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 shutdown_connections 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f280a2800 0x7f6f280a4cc0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f44104ad0 0x7f6f44079990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f44105490 0x7f6f44075fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 --2- 172.21.15.137:0/3995075502 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f441038e0 0x7f6f44079450 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:10.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 >> 172.21.15.137:0/3995075502 conn(0x7f6f440fee50 msgr2=0x7f6f44107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:10.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 shutdown_connections 2024-03-26T14:04:10.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:10.927+0000 7f6f4b000640 1 -- 172.21.15.137:0/3995075502 wait complete. 2024-03-26T14:04:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:11 smithi179 ceph-mon[35472]: pgmap v513: 129 pgs: 129 active+clean; 691 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-03-26T14:04:11.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:11 smithi159 ceph-mon[35501]: pgmap v513: 129 pgs: 129 active+clean; 691 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-03-26T14:04:11.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:11 smithi137 ceph-mon[31539]: pgmap v513: 129 pgs: 129 active+clean; 691 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-03-26T14:04:11.944 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:04:11.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:04:11.944 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:04:12.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 -- 172.21.15.137:0/1802205328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 msgr2=0x7f62b4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 --2- 172.21.15.137:0/1802205328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4103be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009900 tx=0x7f62a802bc30 comp rx=0 tx=0).stop 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 -- 172.21.15.137:0/1802205328 shutdown_connections 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 --2- 172.21.15.137:0/1802205328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b4105370 0x7f62b410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 --2- 172.21.15.137:0/1802205328 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 0x7f62b4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 --2- 172.21.15.137:0/1802205328 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4103be0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 -- 172.21.15.137:0/1802205328 >> 172.21.15.137:0/1802205328 conn(0x7f62b40fed50 msgr2=0x7f62b4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 -- 172.21.15.137:0/1802205328 shutdown_connections 2024-03-26T14:04:12.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.173+0000 7f62bb000640 1 -- 172.21.15.137:0/1802205328 wait complete. 2024-03-26T14:04:12.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.174+0000 7f62bb000640 1 Processor -- start 2024-03-26T14:04:12.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.174+0000 7f62bb000640 1 -- start start 2024-03-26T14:04:12.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62bb000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62bb000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 0x7f62b4199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62bb000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b4105370 0x7f62b41a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62bb000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b410f330 con 0x7f62b41037c0 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62bb000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b410f1b0 con 0x7f62b4105370 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62bb000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b410f4b0 con 0x7f62b41049b0 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60210/0 (socket says 172.21.15.137:60210) 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 -- 172.21.15.137:0/4083167124 learned_addr learned my addr 172.21.15.137:0/4083167124 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62ba600640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b4105370 0x7f62b41a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9200640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 0x7f62b4199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 -- 172.21.15.137:0/4083167124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 msgr2=0x7f62b4199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 0x7f62b4199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 -- 172.21.15.137:0/4083167124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b4105370 msgr2=0x7f62b41a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b4105370 0x7f62b41a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.175+0000 7f62b9c00640 1 -- 172.21.15.137:0/4083167124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b41a1650 con 0x7f62b41037c0 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62b9200640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 0x7f62b4199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62b9c00640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4199a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005ab0 tx=0x7f62a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62b2000640 1 -- 172.21.15.137:0/4083167124 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8043020 con 0x7f62b41037c0 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b41a18e0 con 0x7f62b41037c0 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62b2000640 1 -- 172.21.15.137:0/4083167124 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62a80054c0 con 0x7f62b41037c0 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62b2000640 1 -- 172.21.15.137:0/4083167124 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a8003d10 con 0x7f62b41037c0 2024-03-26T14:04:12.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.176+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b41a1e30 con 0x7f62b41037c0 2024-03-26T14:04:12.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.179+0000 7f62bf203740 1 Processor -- start 2024-03-26T14:04:12.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.179+0000 7f62b2000640 1 -- 172.21.15.137:0/4083167124 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f62a8046020 con 0x7f62b41037c0 2024-03-26T14:04:12.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 -- start start 2024-03-26T14:04:12.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13acd2b0 0x556a13acd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13ad2e10 0x556a13ad52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a13a32eb0 con 0x556a13acdca0 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a139dba60 con 0x556a13acd2b0 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62bf203740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a13a2ef50 con 0x556a13ad2e10 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62b9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62ba600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13acd2b0 0x556a13acd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62b9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60226/0 (socket says 172.21.15.137:60226) 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.180+0000 7f62b9c00640 1 -- 172.21.15.137:0/3855194989 learned_addr learned my addr 172.21.15.137:0/3855194989 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9200640 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13ad2e10 0x556a13ad52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9c00640 1 -- 172.21.15.137:0/3855194989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13ad2e10 msgr2=0x556a13ad52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13ad2e10 0x556a13ad52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9c00640 1 -- 172.21.15.137:0/3855194989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13acd2b0 msgr2=0x556a13acd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13acd2b0 0x556a13acd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9c00640 1 -- 172.21.15.137:0/3855194989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a1380f7e0 con 0x556a13acdca0 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9200640 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13ad2e10 0x556a13ad52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62ba600640 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13acd2b0 0x556a13acd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b2000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f629c0a2800 0x7f629c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad27a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f62a80f9290 tx=0x7f62a80f9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dfe53c85082b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62ba600640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f629c0a2800 0x7f629c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.181+0000 7f62b2000640 1 -- 172.21.15.137:0/4083167124 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f62a80f12a0 con 0x7f62b41037c0 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62a6a00640 1 -- 172.21.15.137:0/3855194989 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80b9bf0 con 0x556a13acdca0 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62a6a00640 1 -- 172.21.15.137:0/3855194989 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62a8004c50 con 0x556a13acdca0 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62a6a00640 1 -- 172.21.15.137:0/3855194989 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a80f4dd0 con 0x556a13acdca0 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62bf203740 1 -- 172.21.15.137:0/3855194989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 msgr2=0x556a13ad27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62ba600640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f629c0a2800 0x7f629c0a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x556a13ace500 tx=0x7f62ac0096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62bf203740 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad27a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f62a80f9290 tx=0x7f62a80f9380 comp rx=0 tx=0).stop 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62bf203740 1 -- 172.21.15.137:0/3855194989 shutdown_connections 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.182+0000 7f62bf203740 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13ad2e10 0x556a13ad52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad27a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 --2- 172.21.15.137:0/3855194989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13acd2b0 0x556a13acd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 -- 172.21.15.137:0/3855194989 >> 172.21.15.137:0/3855194989 conn(0x556a13ac8820 msgr2=0x556a13ac8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 -- 172.21.15.137:0/3855194989 shutdown_connections 2024-03-26T14:04:12.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 -- 172.21.15.137:0/3855194989 wait complete. 2024-03-26T14:04:12.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 -- start start 2024-03-26T14:04:12.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.183+0000 7f62bf203740 1 Processor -- start 2024-03-26T14:04:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ad7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13ade6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a13a32eb0 con 0x556a13acdca0 2024-03-26T14:04:12.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a139eeeb0 con 0x556a13ad2e10 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a13ad5b80 con 0x556a13acd2b0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62bf203740 10 client.0 _ll_register_callbacks cb 0x556a139e3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60232/0 (socket says 172.21.15.137:60232) 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 -- 172.21.15.137:0/3705442987 learned_addr learned my addr 172.21.15.137:0/3705442987 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62ba600640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13ade6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ad7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 -- 172.21.15.137:0/3705442987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 msgr2=0x556a13ad7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ad7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 -- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 msgr2=0x556a13ade6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.184+0000 7f62b9200640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13ade6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f62b9200640 1 -- 172.21.15.137:0/3705442987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a1386ff70 con 0x556a13acdca0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f62b9200640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad76a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f62a0005b30 tx=0x7f62a000fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a401c92c5288c9b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f629b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:12.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f629b400640 1 -- 172.21.15.137:0/3705442987 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0016070 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f629b400640 1 -- 172.21.15.137:0/3705442987 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f62a000baa0 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62a8004c50 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.185+0000 7f629b400640 1 -- 172.21.15.137:0/3705442987 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a0025050 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 10 client.16020 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a13707170 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f629b400640 1 -- 172.21.15.137:0/3705442987 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62a8004c50 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f629b400640 1 -- 172.21.15.137:0/3705442987 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f62a001a400 con 0x556a13acdca0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 10 client.16020 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 20 client.16020 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13ae3110 0x556a13ae5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 4 client.16020 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.186+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a13a403f0 con 0x556a13ae3110 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.187+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13ae3110 0x556a13ae5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.187+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13ae3110 0x556a13ae5530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b862e9816a178e42 server_cookie=3227c272d7ffc4fd in_seq=0 out_seq=0 2024-03-26T14:04:12.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.187+0000 7f629b400640 10 client.16020 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f629b400640 1 -- 172.21.15.137:0/3705442987 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556a13a403f0 con 0x556a13ae3110 2024-03-26T14:04:12.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f629b400640 10 client.16020 handle_command_reply: tid=0 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 client.16020 shutdown 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 msgr2=0x556a13ad76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad76a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f62a0005b30 tx=0x7f62a000fbc0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 shutdown_connections 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13ae3110 0x556a13ae5530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13ade6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ad76a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 --2- 172.21.15.137:0/3705442987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ad7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f629b400640 0 client.16020 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.189+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 >> 172.21.15.137:0/3705442987 conn(0x556a13ac8820 msgr2=0x556a13ac7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.190+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 shutdown_connections 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.190+0000 7f62bf203740 1 -- 172.21.15.137:0/3705442987 wait complete. 2024-03-26T14:04:12.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.190+0000 7f62bf203740 20 client.16020 trim_cache size 0 max 16384 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 Processor -- start 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 -- start start 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ace740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ade6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13acec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41ae7e0 con 0x556a13acdca0 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41ac2e0 con 0x556a13ad2e10 2024-03-26T14:04:12.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62bf203740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41aa360 con 0x556a13acd2b0 2024-03-26T14:04:12.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62b9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ade6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62b9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ade6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60246/0 (socket says 172.21.15.137:60246) 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62b9200640 1 -- 172.21.15.137:0/48573377 learned_addr learned my addr 172.21.15.137:0/48573377 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.208+0000 7f62ba600640 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13acec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9c00640 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ace740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9200640 1 -- 172.21.15.137:0/48573377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 msgr2=0x556a13ace740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9200640 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ace740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9200640 1 -- 172.21.15.137:0/48573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 msgr2=0x556a13acec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9200640 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13acec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9200640 1 -- 172.21.15.137:0/48573377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a13ace530 con 0x556a13acdca0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9c00640 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ace740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62b9200640 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ade6a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f62a001a8f0 tx=0x7f62a000dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9429c92a3607494f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.209+0000 7f62a6000640 1 -- 172.21.15.137:0/48573377 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a002f020 con 0x556a13acdca0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62a6000640 1 -- 172.21.15.137:0/48573377 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62a001a400 con 0x556a13acdca0 2024-03-26T14:04:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62a6000640 1 -- 172.21.15.137:0/48573377 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0016040 con 0x556a13acdca0 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 -- 172.21.15.137:0/48573377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 msgr2=0x556a13ade6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ade6a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f62a001a8f0 tx=0x7f62a000dba0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 -- 172.21.15.137:0/48573377 shutdown_connections 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13acec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13ade6a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 --2- 172.21.15.137:0/48573377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ace740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.210+0000 7f62bf203740 1 -- 172.21.15.137:0/48573377 >> 172.21.15.137:0/48573377 conn(0x556a13ac8820 msgr2=0x556a13acc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 -- 172.21.15.137:0/48573377 shutdown_connections 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 -- 172.21.15.137:0/48573377 wait complete. 2024-03-26T14:04:12.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 -- start start 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 Processor -- start 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ade240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13adc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13adce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41aa960 con 0x556a13acdca0 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41abc80 con 0x556a13ad2e10 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b41aafc0 con 0x556a13acd2b0 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.211+0000 7f62bf203740 10 client.0 _ll_register_callbacks cb 0x7f62a000d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13adc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13adc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60260/0 (socket says 172.21.15.137:60260) 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 -- 172.21.15.137:0/3833510246 learned_addr learned my addr 172.21.15.137:0/3833510246 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62ba600640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13adce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ade240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 -- 172.21.15.137:0/3833510246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 msgr2=0x556a13ade240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ade240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 -- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 msgr2=0x556a13adce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13adce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 -- 172.21.15.137:0/3833510246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a13add390 con 0x556a13acdca0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f62b9200640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13adc910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f62a0005940 tx=0x7f62a000d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57b6b211d81a1e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.212+0000 7f629b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f629b400640 1 -- 172.21.15.137:0/3833510246 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a002f070 con 0x556a13acdca0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f629b400640 1 -- 172.21.15.137:0/3833510246 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f62a000b8f0 con 0x556a13acdca0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62a001a400 con 0x556a13acdca0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f629b400640 1 -- 172.21.15.137:0/3833510246 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a0044050 con 0x556a13acdca0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f62bf203740 10 client.16026 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556a13b3f6d0 con 0x556a13acdca0 2024-03-26T14:04:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f629b400640 1 -- 172.21.15.137:0/3833510246 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62a001a400 con 0x556a13acdca0 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f629b400640 1 -- 172.21.15.137:0/3833510246 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f62a0034450 con 0x556a13acdca0 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f62bf203740 10 client.16026 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f62bf203740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.213+0000 7f62bf203740 20 client.16026 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.214+0000 7f62bf203740 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13b40080 0x556a13b424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.214+0000 7f62bf203740 4 client.16026 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:04:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.214+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x556a13a3a740 con 0x556a13b40080 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.214+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13b40080 0x556a13b424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.215+0000 7f62b9c00640 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13b40080 0x556a13b424a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a7ee9b17a47bb6a server_cookie=612b8ebae0a25d53 in_seq=0 out_seq=0 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.215+0000 7f629b400640 10 client.16026 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f629b400640 1 -- 172.21.15.137:0/3833510246 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f62a80eee50 con 0x556a13b40080 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f629b400640 10 client.16026 handle_command_reply: tid=0 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 client.16026 shutdown 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 msgr2=0x556a13adc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13adc910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f62a0005940 tx=0x7f62a000d650 comp rx=0 tx=0).stop 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 shutdown_connections 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556a13b40080 0x556a13b424a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a13ad2e10 0x556a13adce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556a13acdca0 0x556a13adc910 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 --2- 172.21.15.137:0/3833510246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a13acd2b0 0x556a13ade240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f629b400640 0 client.16026 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 >> 172.21.15.137:0/3833510246 conn(0x556a13ac8820 msgr2=0x556a13ac71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.216+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 shutdown_connections 2024-03-26T14:04:12.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.217+0000 7f62bf203740 1 -- 172.21.15.137:0/3833510246 wait complete. 2024-03-26T14:04:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.217+0000 7f62bf203740 20 client.16026 trim_cache size 0 max 16384 2024-03-26T14:04:12.217 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:04:12.217 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:04:12.217 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "5a36617a-c3b7-4851-a0c8-7c5cbe59f42e", 2024-03-26T14:04:12.217 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:04:12.217 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.217+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f629c0a2800 msgr2=0x7f629c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.217+0000 7f62bb000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f629c0a2800 0x7f629c0a4cc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x556a13ace500 tx=0x7f62ac0096a0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.217+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 msgr2=0x7f62b4199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.217+0000 7f62bb000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4199a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005ab0 tx=0x7f62a80042f0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 shutdown_connections 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f629c0a2800 0x7f629c0a4cc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62b4105370 0x7f62b41a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f62b41049b0 0x7f62b4199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 --2- 172.21.15.137:0/4083167124 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f62b41037c0 0x7f62b4199a30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 >> 172.21.15.137:0/4083167124 conn(0x7f62b40fed50 msgr2=0x7f62b4107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 shutdown_connections 2024-03-26T14:04:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.218+0000 7f62bb000640 1 -- 172.21.15.137:0/4083167124 wait complete. 2024-03-26T14:04:12.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:04:12.232 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:04:12.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 -- 172.21.15.137:0/2660743764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340077c40 msgr2=0x7f13400784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 --2- 172.21.15.137:0/2660743764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340077c40 0x7f13400784e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1338009be0 tx=0x7f133802bef0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 -- 172.21.15.137:0/2660743764 shutdown_connections 2024-03-26T14:04:12.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 --2- 172.21.15.137:0/2660743764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13400772a0 0x7f1340077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 --2- 172.21.15.137:0/2660743764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340077c40 0x7f13400784e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 --2- 172.21.15.137:0/2660743764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1340076060 0x7f1340076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 -- 172.21.15.137:0/2660743764 >> 172.21.15.137:0/2660743764 conn(0x7f13400fee50 msgr2=0x7f1340101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 -- 172.21.15.137:0/2660743764 shutdown_connections 2024-03-26T14:04:12.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.447+0000 7f1345800640 1 -- 172.21.15.137:0/2660743764 wait complete. 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 Processor -- start 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 -- start start 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 0x7f134019ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13400772a0 0x7f134019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1340077c40 0x7f13401a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340113810 con 0x7f1340076060 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340113690 con 0x7f13400772a0 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.448+0000 7f1345800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1340113990 con 0x7f1340077c40 2024-03-26T14:04:12.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 0x7f134019ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 0x7f134019ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60274/0 (socket says 172.21.15.137:60274) 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 -- 172.21.15.137:0/2165482965 learned_addr learned my addr 172.21.15.137:0/2165482965 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133e000640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13400772a0 0x7f134019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133f400640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1340077c40 0x7f13401a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 -- 172.21.15.137:0/2165482965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1340077c40 msgr2=0x7f13401a5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1340077c40 0x7f13401a5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 -- 172.21.15.137:0/2165482965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13400772a0 msgr2=0x7f134019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13400772a0 0x7f134019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.449+0000 7f133ea00640 1 -- 172.21.15.137:0/2165482965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13401a59d0 con 0x7f1340076060 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.450+0000 7f133ea00640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 0x7f134019ddb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f133000c920 tx=0x7f133000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.450+0000 7f1337400640 1 -- 172.21.15.137:0/2165482965 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330007c70 con 0x7f1340076060 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.450+0000 7f1337400640 1 -- 172.21.15.137:0/2165482965 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1330007e10 con 0x7f1340076060 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.450+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13401a5cc0 con 0x7f1340076060 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.450+0000 7f1337400640 1 -- 172.21.15.137:0/2165482965 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330018820 con 0x7f1340076060 2024-03-26T14:04:12.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.450+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13401a61b0 con 0x7f1340076060 2024-03-26T14:04:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.454+0000 7f1348618740 1 Processor -- start 2024-03-26T14:04:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.454+0000 7f1348618740 1 -- start start 2024-03-26T14:04:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1337400640 1 -- 172.21.15.137:0/2165482965 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f13300189c0 con 0x7f1340076060 2024-03-26T14:04:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1348618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdbd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1348618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1348618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdc4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1348618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ffd22250 con 0x5613ffdbce80 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1348618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ffccaa30 con 0x5613ffdc29e0 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f1348618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ffd1e2f0 con 0x5613ffdbd870 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdbd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdbd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60276/0 (socket says 172.21.15.137:60276) 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f133f400640 1 -- 172.21.15.137:0/2973559870 learned_addr learned my addr 172.21.15.137:0/2973559870 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f133e000640 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdc4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.455+0000 7f133ea00640 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f133f400640 1 -- 172.21.15.137:0/2973559870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 msgr2=0x5613ffdc2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f133f400640 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f133f400640 1 -- 172.21.15.137:0/2973559870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 msgr2=0x5613ffdc4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f133f400640 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdc4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f133f400640 1 -- 172.21.15.137:0/2973559870 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613ffcb5690 con 0x5613ffdbce80 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f133f400640 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdbd2a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1338009be0 tx=0x7f1338009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8f740513d9de42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f132be00640 1 -- 172.21.15.137:0/2973559870 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338043070 con 0x5613ffdbce80 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.456+0000 7f1337400640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13240a2880 0x7f13240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f132be00640 1 -- 172.21.15.137:0/2973559870 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f13380334d0 con 0x5613ffdbce80 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f132be00640 1 -- 172.21.15.137:0/2973559870 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133802caf0 con 0x5613ffdbce80 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f1337400640 1 -- 172.21.15.137:0/2165482965 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f13300cef00 con 0x7f1340076060 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f133ea00640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13240a2880 0x7f13240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f1348618740 1 -- 172.21.15.137:0/2973559870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 msgr2=0x5613ffdbd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f1348618740 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdbd2a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1338009be0 tx=0x7f1338009840 comp rx=0 tx=0).stop 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.457+0000 7f133ea00640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13240a2880 0x7f13240a4d40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f133000c8f0 tx=0x7f1330097040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 -- 172.21.15.137:0/2973559870 shutdown_connections 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdc4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 --2- 172.21.15.137:0/2973559870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdbd2a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 -- 172.21.15.137:0/2973559870 >> 172.21.15.137:0/2973559870 conn(0x5613ffd85200 msgr2=0x5613ffdb87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 -- 172.21.15.137:0/2973559870 shutdown_connections 2024-03-26T14:04:12.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 -- 172.21.15.137:0/2973559870 wait complete. 2024-03-26T14:04:12.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 -- start start 2024-03-26T14:04:12.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.458+0000 7f1348618740 1 Processor -- start 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdc6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ffdc5930 con 0x5613ffdbce80 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ffcdde80 con 0x5613ffdc29e0 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ffdc5ab0 con 0x5613ffdbd870 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f1348618740 10 client.0 _ll_register_callbacks cb 0x5613ffcf0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdc6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdc6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60282/0 (socket says 172.21.15.137:60282) 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f133f400640 1 -- 172.21.15.137:0/1272769671 learned_addr learned my addr 172.21.15.137:0/1272769671 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f133e000640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.459+0000 7f133f400640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f133f400640 1 -- 172.21.15.137:0/1272769671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 msgr2=0x5613ffdc7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f133f400640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f133f400640 1 -- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 msgr2=0x5613ffdce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f133f400640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f133f400640 1 -- 172.21.15.137:0/1272769671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613ffb007d0 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f133f400640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdc6dc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f133802c430 tx=0x7f1338009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d326b354d6f7bea6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f1328c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f1328c00640 1 -- 172.21.15.137:0/1272769671 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338054060 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f1328c00640 1 -- 172.21.15.137:0/1272769671 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f133802cca0 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.460+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13380334d0 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1328c00640 1 -- 172.21.15.137:0/1272769671 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f133805b050 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 10 client.16038 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5613ffa363f0 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1328c00640 1 -- 172.21.15.137:0/1272769671 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f13380334d0 con 0x5613ffdbce80 2024-03-26T14:04:12.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1328c00640 1 -- 172.21.15.137:0/1272769671 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1338003ea0 con 0x5613ffdbce80 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 10 client.16038 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 20 client.16038 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffdd2830 0x5613ffdd4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 4 client.16038 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.461+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613ffd2f790 con 0x5613ffdd2830 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.462+0000 7f133e000640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffdd2830 0x5613ffdd4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.462+0000 7f133e000640 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffdd2830 0x5613ffdd4c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb244412477bc1f server_cookie=145a14bffaa89607 in_seq=0 out_seq=0 2024-03-26T14:04:12.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.462+0000 7f1328c00640 10 client.16038 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.463+0000 7f1328c00640 1 -- 172.21.15.137:0/1272769671 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5613ffd2f790 con 0x5613ffdd2830 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.463+0000 7f1328c00640 10 client.16038 handle_command_reply: tid=0 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 client.16038 shutdown 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 msgr2=0x5613ffdc6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdc6dc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f133802c430 tx=0x7f1338009500 comp rx=0 tx=0).stop 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 shutdown_connections 2024-03-26T14:04:12.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffdd2830 0x5613ffdd4c20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdc7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 --2- 172.21.15.137:0/1272769671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdc6dc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1328c00640 0 client.16038 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 >> 172.21.15.137:0/1272769671 conn(0x5613ffd85200 msgr2=0x5613ffdb7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 shutdown_connections 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 1 -- 172.21.15.137:0/1272769671 wait complete. 2024-03-26T14:04:12.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.464+0000 7f1348618740 20 client.16038 trim_cache size 0 max 16384 2024-03-26T14:04:12.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 Processor -- start 2024-03-26T14:04:12.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 -- start start 2024-03-26T14:04:12.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdd9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdd98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401b2c10 con 0x5613ffdbce80 2024-03-26T14:04:12.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401b0710 con 0x5613ffdc29e0 2024-03-26T14:04:12.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.482+0000 7f1348618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401ae790 con 0x5613ffdbd870 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdd9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdd9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60284/0 (socket says 172.21.15.137:60284) 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 -- 172.21.15.137:0/2744980671 learned_addr learned my addr 172.21.15.137:0/2744980671 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133e000640 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdd98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 -- 172.21.15.137:0/2744980671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 msgr2=0x5613ffdce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 -- 172.21.15.137:0/2744980671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 msgr2=0x5613ffdd98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdd98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 -- 172.21.15.137:0/2744980671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613ff9f60f0 con 0x5613ffdbce80 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.483+0000 7f133f400640 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdd9380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1338006010 tx=0x7f1338009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178df185719f7ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f132b400640 1 -- 172.21.15.137:0/2744980671 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133804c070 con 0x5613ffdbce80 2024-03-26T14:04:12.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f132b400640 1 -- 172.21.15.137:0/2744980671 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f13380337c0 con 0x5613ffdbce80 2024-03-26T14:04:12.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f132b400640 1 -- 172.21.15.137:0/2744980671 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338054070 con 0x5613ffdbce80 2024-03-26T14:04:12.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 -- 172.21.15.137:0/2744980671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 msgr2=0x5613ffdd9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdd9380 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1338006010 tx=0x7f1338009310 comp rx=0 tx=0).stop 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 -- 172.21.15.137:0/2744980671 shutdown_connections 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdc29e0 0x5613ffdd98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 --2- 172.21.15.137:0/2744980671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdbce80 0x5613ffdd9380 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 -- 172.21.15.137:0/2744980671 >> 172.21.15.137:0/2744980671 conn(0x5613ffd85200 msgr2=0x5613ffdbeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.484+0000 7f1348618740 1 -- 172.21.15.137:0/2744980671 shutdown_connections 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 -- 172.21.15.137:0/2744980671 wait complete. 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 -- start start 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 Processor -- start 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdbce80 0x5613ffdcbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdcc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 0x5613ffdcd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401aed90 con 0x5613ffdc29e0 2024-03-26T14:04:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401b00b0 con 0x5613ffdbce80 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13401af3f0 con 0x5613ffdbd870 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.485+0000 7f1348618740 10 client.0 _ll_register_callbacks cb 0x7f132c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 0x5613ffdcd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 0x5613ffdcd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60288/0 (socket says 172.21.15.137:60288) 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 -- 172.21.15.137:0/2663756645 learned_addr learned my addr 172.21.15.137:0/2663756645 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdbce80 0x5613ffdcbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133e000640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdcc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 -- 172.21.15.137:0/2663756645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 msgr2=0x5613ffdcc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdcc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 -- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdbce80 msgr2=0x5613ffdcbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdbce80 0x5613ffdcbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133f400640 1 -- 172.21.15.137:0/2663756645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613ffdcd9e0 con 0x5613ffdc29e0 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.486+0000 7f133e000640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdcc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f133f400640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 0x5613ffdcd3f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1338009be0 tx=0x7f1338034f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43b767806d37358 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1328c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1328c00640 1 -- 172.21.15.137:0/2663756645 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338054070 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1328c00640 1 -- 172.21.15.137:0/2663756645 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1338004d50 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613ffd29ae0 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1328c00640 1 -- 172.21.15.137:0/2663756645 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f133804c040 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1348618740 10 client.16044 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5613ffe2e310 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.487+0000 7f1328c00640 1 -- 172.21.15.137:0/2663756645 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5613ffd29ae0 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1328c00640 1 -- 172.21.15.137:0/2663756645 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f13380345a0 con 0x5613ffdc29e0 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1348618740 10 client.16044 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1348618740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1348618740 20 client.16044 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1348618740 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffe2eaf0 0x5613ffe30ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1348618740 4 client.16044 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:04:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5613ffca73a0 con 0x5613ffe2eaf0 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.488+0000 7f133e000640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffe2eaf0 0x5613ffe30ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f133e000640 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffe2eaf0 0x5613ffe30ee0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cabdbce9b5fe7f server_cookie=a04467ebd4f63e28 in_seq=0 out_seq=0 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1328c00640 10 client.16044 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1328c00640 1 -- 172.21.15.137:0/2663756645 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5613ffca73a0 con 0x5613ffe2eaf0 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1328c00640 10 client.16044 handle_command_reply: tid=0 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 client.16044 shutdown 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 msgr2=0x5613ffdcd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 0x5613ffdcd3f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1338009be0 tx=0x7f1338034f90 comp rx=0 tx=0).stop 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 shutdown_connections 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5613ffe2eaf0 0x5613ffe30ee0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5613ffdc29e0 0x5613ffdcd3f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5613ffdbd870 0x5613ffdcc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 --2- 172.21.15.137:0/2663756645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613ffdbce80 0x5613ffdcbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 >> 172.21.15.137:0/2663756645 conn(0x5613ffd85200 msgr2=0x5613ffdbba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1328c00640 0 client.16044 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 shutdown_connections 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.489+0000 7f1348618740 1 -- 172.21.15.137:0/2663756645 wait complete. 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1348618740 20 client.16044 trim_cache size 0 max 16384 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "5a36617a-c3b7-4851-a0c8-7c5cbe59f42e": { 2024-03-26T14:04:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "5a36617a-c3b7-4851-a0c8-7c5cbe59f42e", 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13240a2880 msgr2=0x7f13240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13240a2880 0x7f13240a4d40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f133000c8f0 tx=0x7f1330097040 comp rx=0 tx=0).stop 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 msgr2=0x7f134019ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 0x7f134019ddb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f133000c920 tx=0x7f133000cde0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 shutdown_connections 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f13240a2880 0x7f13240a4d40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13400772a0 0x7f134019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1340077c40 0x7f13401a5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 --2- 172.21.15.137:0/2165482965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1340076060 0x7f134019ddb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 >> 172.21.15.137:0/2165482965 conn(0x7f13400fee50 msgr2=0x7f13401004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 shutdown_connections 2024-03-26T14:04:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:12.490+0000 7f1345800640 1 -- 172.21.15.137:0/2165482965 wait complete. 2024-03-26T14:04:12.508 INFO:tasks.cephfs.filesystem:scrub status for tag:5a36617a-c3b7-4851-a0c8-7c5cbe59f42e - {'path': '/', 'tag': '5a36617a-c3b7-4851-a0c8-7c5cbe59f42e', 'options': 'recursive,force'} 2024-03-26T14:04:13.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:13 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:04:13.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:13 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:13.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:13 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:04:13.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:13 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:13.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:13 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:04:13.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:13 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:13.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:13 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:04:13.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:13 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:13.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:13 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:04:13.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:13 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:13.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:13 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:04:13.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:13 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:14.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:14 smithi179 ceph-mon[35472]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-03-26T14:04:14.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:14 smithi159 ceph-mon[35501]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-03-26T14:04:14.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:14 smithi137 ceph-mon[31539]: pgmap v514: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.4 MiB/s wr, 132 op/s 2024-03-26T14:04:16.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:16 smithi137 ceph-mon[31539]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-03-26T14:04:16.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:16 smithi179 ceph-mon[35472]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-03-26T14:04:16.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:16 smithi159 ceph-mon[35501]: pgmap v515: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-03-26T14:04:17.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:17 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:04:17.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:17 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:04:17.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:17 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:04:17.997 INFO:tasks.workunit.client.0.smithi137.stdout:.......... 2024-03-26T14:04:17.998 INFO:tasks.workunit.client.0.smithi137.stdout:do_o_direct_reads: SUCCESS 2024-03-26T14:04:18.004 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T14:04:18.005 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T14:04:18.093 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-03-26T14:04:18.094 DEBUG:teuthology.orchestra.run.smithi137:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-03-26T14:04:18.164 INFO:tasks.workunit.client.0.smithi137.stdout:test read from hole 2024-03-26T14:04:18.164 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo 'test read from hole' 2024-03-26T14:04:18.164 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-03-26T14:04:18.171 INFO:tasks.workunit.client.0.smithi137.stderr:0+0 records in 2024-03-26T14:04:18.171 INFO:tasks.workunit.client.0.smithi137.stderr:0+0 records out 2024-03-26T14:04:18.171 INFO:tasks.workunit.client.0.smithi137.stderr:0 bytes copied, 0.00015543 s, 0.0 kB/s 2024-03-26T14:04:18.171 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-03-26T14:04:18.178 INFO:tasks.workunit.client.0.smithi137.stderr:2+0 records in 2024-03-26T14:04:18.178 INFO:tasks.workunit.client.0.smithi137.stderr:2+0 records out 2024-03-26T14:04:18.179 INFO:tasks.workunit.client.0.smithi137.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00325963 s, 314 kB/s 2024-03-26T14:04:18.179 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-03-26T14:04:18.181 INFO:tasks.workunit.client.0.smithi137.stderr:2+0 records in 2024-03-26T14:04:18.181 INFO:tasks.workunit.client.0.smithi137.stderr:2+0 records out 2024-03-26T14:04:18.181 INFO:tasks.workunit.client.0.smithi137.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 9.8338e-05 s, 10.4 MB/s 2024-03-26T14:04:18.182 INFO:tasks.workunit.client.0.smithi137.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-03-26T14:04:18.184 INFO:tasks.workunit.client.0.smithi137.stdout:other thing 2024-03-26T14:04:18.184 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo 'other thing' 2024-03-26T14:04:18.184 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-03-26T14:04:18.186 INFO:tasks.workunit.client.0.smithi137.stderr:1+0 records in 2024-03-26T14:04:18.186 INFO:tasks.workunit.client.0.smithi137.stderr:1+0 records out 2024-03-26T14:04:18.186 INFO:tasks.workunit.client.0.smithi137.stderr:500 bytes copied, 9.9997e-05 s, 5.0 MB/s 2024-03-26T14:04:18.187 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-03-26T14:04:18.205 INFO:tasks.workunit.client.0.smithi137.stderr:0+1 records in 2024-03-26T14:04:18.205 INFO:tasks.workunit.client.0.smithi137.stderr:0+1 records out 2024-03-26T14:04:18.206 INFO:tasks.workunit.client.0.smithi137.stderr:500 bytes copied, 0.00643066 s, 77.8 kB/s 2024-03-26T14:04:18.206 INFO:tasks.workunit.client.0.smithi137.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-03-26T14:04:18.216 INFO:tasks.workunit.client.0.smithi137.stderr:0+1 records in 2024-03-26T14:04:18.216 INFO:tasks.workunit.client.0.smithi137.stderr:0+1 records out 2024-03-26T14:04:18.216 INFO:tasks.workunit.client.0.smithi137.stderr:500 bytes copied, 0.00461598 s, 108 kB/s 2024-03-26T14:04:18.216 INFO:tasks.workunit.client.0.smithi137.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-03-26T14:04:18.219 INFO:tasks.workunit.client.0.smithi137.stdout:OK 2024-03-26T14:04:18.220 INFO:tasks.workunit.client.0.smithi137.stderr:+ echo OK 2024-03-26T14:04:18.221 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T14:04:18.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T14:04:18.343 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:18 smithi137 ceph-mon[31539]: pgmap v516: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 720 KiB/s wr, 127 op/s 2024-03-26T14:04:18.343 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:18 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:04:18.346 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-03-26T14:04:18.346 DEBUG:teuthology.orchestra.run.smithi137:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-03-26T14:04:18.443 INFO:tasks.workunit.client.0.smithi137.stdout:writing first 3 bytes of 10k file 2024-03-26T14:04:18.443 INFO:tasks.workunit.client.0.smithi137.stdout:reading O_DIRECT 2024-03-26T14:04:18.443 INFO:tasks.workunit.client.0.smithi137.stdout:got 10000 2024-03-26T14:04:18.444 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T14:04:18.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T14:04:18.559 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-03-26T14:04:18.559 DEBUG:teuthology.orchestra.run.smithi137:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3be9a5c9bc793e11e2800a8c0c696e8b46742033 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-03-26T14:04:18.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:18 smithi179 ceph-mon[35472]: pgmap v516: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 720 KiB/s wr, 127 op/s 2024-03-26T14:04:18.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:18 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:04:18.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:18 smithi159 ceph-mon[35501]: pgmap v516: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 720 KiB/s wr, 127 op/s 2024-03-26T14:04:18.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:18 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:04:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:19 smithi179 ceph-mon[35472]: pgmap v517: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 377 KiB/s wr, 270 op/s 2024-03-26T14:04:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:04:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:04:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:19 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:04:19.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:19 smithi159 ceph-mon[35501]: pgmap v517: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 377 KiB/s wr, 270 op/s 2024-03-26T14:04:19.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:04:19.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:04:19.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:19 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:04:19.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:19 smithi137 ceph-mon[31539]: pgmap v517: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 377 KiB/s wr, 270 op/s 2024-03-26T14:04:19.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:04:19.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:04:19.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:19 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:04:22.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:22 smithi179 ceph-mon[35472]: pgmap v518: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 35 KiB/s wr, 267 op/s 2024-03-26T14:04:22.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:22 smithi159 ceph-mon[35501]: pgmap v518: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 35 KiB/s wr, 267 op/s 2024-03-26T14:04:22.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:22 smithi137 ceph-mon[31539]: pgmap v518: 129 pgs: 129 active+clean; 1.1 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 35 KiB/s wr, 267 op/s 2024-03-26T14:04:24.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:24 smithi179 ceph-mon[35472]: pgmap v519: 129 pgs: 129 active+clean; 761 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 41 KiB/s wr, 294 op/s 2024-03-26T14:04:24.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:24 smithi159 ceph-mon[35501]: pgmap v519: 129 pgs: 129 active+clean; 761 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 41 KiB/s wr, 294 op/s 2024-03-26T14:04:24.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:24 smithi137 ceph-mon[31539]: pgmap v519: 129 pgs: 129 active+clean; 761 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 41 KiB/s wr, 294 op/s 2024-03-26T14:04:26.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:26 smithi179 ceph-mon[35472]: pgmap v520: 129 pgs: 129 active+clean; 761 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 7.1 KiB/s wr, 179 op/s 2024-03-26T14:04:26.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:26 smithi159 ceph-mon[35501]: pgmap v520: 129 pgs: 129 active+clean; 761 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 7.1 KiB/s wr, 179 op/s 2024-03-26T14:04:26.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:26 smithi137 ceph-mon[31539]: pgmap v520: 129 pgs: 129 active+clean; 761 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 7.1 KiB/s wr, 179 op/s 2024-03-26T14:04:27.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:27 smithi179 ceph-mon[35472]: pgmap v521: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 8.0 KiB/s wr, 181 op/s 2024-03-26T14:04:27.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:27 smithi159 ceph-mon[35501]: pgmap v521: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 8.0 KiB/s wr, 181 op/s 2024-03-26T14:04:27.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:27 smithi137 ceph-mon[31539]: pgmap v521: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 8.0 KiB/s wr, 181 op/s 2024-03-26T14:04:30.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:30 smithi179 ceph-mon[35472]: pgmap v522: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 6.6 KiB/s wr, 180 op/s 2024-03-26T14:04:30.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:30 smithi159 ceph-mon[35501]: pgmap v522: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 6.6 KiB/s wr, 180 op/s 2024-03-26T14:04:30.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:30 smithi137 ceph-mon[31539]: pgmap v522: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 6.6 KiB/s wr, 180 op/s 2024-03-26T14:04:32.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:32 smithi179 ceph-mon[35472]: pgmap v523: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 KiB/s wr, 31 op/s 2024-03-26T14:04:32.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:32 smithi159 ceph-mon[35501]: pgmap v523: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 KiB/s wr, 31 op/s 2024-03-26T14:04:32.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:32 smithi137 ceph-mon[31539]: pgmap v523: 129 pgs: 129 active+clean; 769 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.6 KiB/s wr, 31 op/s 2024-03-26T14:04:34.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:34 smithi179 ceph-mon[35472]: pgmap v524: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.2 KiB/s wr, 33 op/s 2024-03-26T14:04:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:34 smithi159 ceph-mon[35501]: pgmap v524: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.2 KiB/s wr, 33 op/s 2024-03-26T14:04:34.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:34 smithi137 ceph-mon[31539]: pgmap v524: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 7.2 KiB/s wr, 33 op/s 2024-03-26T14:04:35.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:35 smithi179 ceph-mon[35472]: pgmap v525: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-26T14:04:35.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:35 smithi159 ceph-mon[35501]: pgmap v525: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-26T14:04:35.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:35 smithi137 ceph-mon[31539]: pgmap v525: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-26T14:04:38.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:38 smithi179 ceph-mon[35472]: pgmap v526: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-26T14:04:38.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:38 smithi159 ceph-mon[35501]: pgmap v526: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-26T14:04:38.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:38 smithi137 ceph-mon[31539]: pgmap v526: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-03-26T14:04:40.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:40 smithi179 ceph-mon[35472]: pgmap v527: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-26T14:04:40.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:40 smithi159 ceph-mon[35501]: pgmap v527: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-26T14:04:40.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:40 smithi137 ceph-mon[31539]: pgmap v527: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 6 op/s 2024-03-26T14:04:41.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:41 smithi179 ceph-mon[35472]: pgmap v528: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:04:41.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:41 smithi159 ceph-mon[35501]: pgmap v528: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:04:41.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:41 smithi137 ceph-mon[31539]: pgmap v528: 129 pgs: 129 active+clean; 776 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:04:42.509 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:04:42.509 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:04:42.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 -- 172.21.15.137:0/3456782606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c105350 msgr2=0x7eff7c10bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 --2- 172.21.15.137:0/3456782606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c105350 0x7eff7c10bbe0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7eff74009ca0 tx=0x7eff7402c100 comp rx=0 tx=0).stop 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 -- 172.21.15.137:0/3456782606 shutdown_connections 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 --2- 172.21.15.137:0/3456782606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff7c104990 0x7eff7c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 --2- 172.21.15.137:0/3456782606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c105350 0x7eff7c10bbe0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 --2- 172.21.15.137:0/3456782606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff7c1037a0 0x7eff7c103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 -- 172.21.15.137:0/3456782606 >> 172.21.15.137:0/3456782606 conn(0x7eff7c0fed10 msgr2=0x7eff7c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 -- 172.21.15.137:0/3456782606 shutdown_connections 2024-03-26T14:04:42.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.732+0000 7eff81c00640 1 -- 172.21.15.137:0/3456782606 wait complete. 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.733+0000 7eff81c00640 1 Processor -- start 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.733+0000 7eff81c00640 1 -- start start 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.733+0000 7eff81c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff7c1037a0 0x7eff7c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff81c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 0x7eff7c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff81c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff7c105350 0x7eff7c1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff81c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c10f2d0 con 0x7eff7c104990 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff81c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c10f150 con 0x7eff7c105350 2024-03-26T14:04:42.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff81c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c10f450 con 0x7eff7c1037a0 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 0x7eff7c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 0x7eff7c199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48256/0 (socket says 172.21.15.137:48256) 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 -- 172.21.15.137:0/3072483816 learned_addr learned my addr 172.21.15.137:0/3072483816 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff81200640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff7c105350 0x7eff7c1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7be00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff7c1037a0 0x7eff7c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 -- 172.21.15.137:0/3072483816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff7c1037a0 msgr2=0x7eff7c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff7c1037a0 0x7eff7c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 -- 172.21.15.137:0/3072483816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff7c105350 msgr2=0x7eff7c1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff7c105350 0x7eff7c1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.734+0000 7eff7b400640 1 -- 172.21.15.137:0/3072483816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c1a1560 con 0x7eff7c104990 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.735+0000 7eff7b400640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 0x7eff7c199e80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e9f0 tx=0x7eff6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:42.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.735+0000 7eff78c00640 1 -- 172.21.15.137:0/3072483816 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800cd80 con 0x7eff7c104990 2024-03-26T14:04:42.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.735+0000 7eff78c00640 1 -- 172.21.15.137:0/3072483816 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff680108e0 con 0x7eff7c104990 2024-03-26T14:04:42.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.735+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c1a1850 con 0x7eff7c104990 2024-03-26T14:04:42.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.735+0000 7eff78c00640 1 -- 172.21.15.137:0/3072483816 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68018920 con 0x7eff7c104990 2024-03-26T14:04:42.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.735+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c1a1da0 con 0x7eff7c104990 2024-03-26T14:04:42.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 Processor -- start 2024-03-26T14:04:42.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 -- start start 2024-03-26T14:04:42.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0e050 0x560ecea0e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea13540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea13bb0 0x560ecea16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ece973250 con 0x560ecea13bb0 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ece91ba30 con 0x560ecea0e050 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.739+0000 7eff8600b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ece96f2f0 con 0x560ecea0ea40 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.740+0000 7eff78c00640 1 -- 172.21.15.137:0/3072483816 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7eff68010a90 con 0x7eff7c104990 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.740+0000 7eff81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0e050 0x560ecea0e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.740+0000 7eff81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0e050 0x560ecea0e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:60596/0 (socket says 172.21.15.137:60596) 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.740+0000 7eff81200640 1 -- 172.21.15.137:0/1667353612 learned_addr learned my addr 172.21.15.137:0/1667353612 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.740+0000 7eff7b400640 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea13bb0 0x560ecea16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.740+0000 7eff7be00640 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea13540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff7b400640 1 -- 172.21.15.137:0/1667353612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 msgr2=0x560ecea13540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff7b400640 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea13540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff7b400640 1 -- 172.21.15.137:0/1667353612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0e050 msgr2=0x560ecea0e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff7b400640 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0e050 0x560ecea0e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff7b400640 1 -- 172.21.15.137:0/1667353612 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ece906690 con 0x560ecea13bb0 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff7b400640 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea13bb0 0x560ecea16040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7eff680cc8a0 tx=0x7eff680d7910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96a7a0ff5403398 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff71600640 1 -- 172.21.15.137:0/1667353612 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680e2070 con 0x560ecea13bb0 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff71600640 1 -- 172.21.15.137:0/1667353612 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff680cfab0 con 0x560ecea13bb0 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.741+0000 7eff71600640 1 -- 172.21.15.137:0/1667353612 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680dd9e0 con 0x560ecea13bb0 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff78c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff600a2800 0x7eff600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff78c00640 1 -- 172.21.15.137:0/3072483816 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff680cf1d0 con 0x7eff7c104990 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff81200640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff600a2800 0x7eff600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 -- 172.21.15.137:0/1667353612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea13bb0 msgr2=0x560ecea16040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea13bb0 0x560ecea16040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7eff680cc8a0 tx=0x7eff680d7910 comp rx=0 tx=0).stop 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 -- 172.21.15.137:0/1667353612 shutdown_connections 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea13bb0 0x560ecea16040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea13540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 --2- 172.21.15.137:0/1667353612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0e050 0x560ecea0e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.742+0000 7eff8600b740 1 -- 172.21.15.137:0/1667353612 >> 172.21.15.137:0/1667353612 conn(0x560ece9d6200 msgr2=0x560ecea099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.743+0000 7eff8600b740 1 -- 172.21.15.137:0/1667353612 shutdown_connections 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.743+0000 7eff8600b740 1 -- 172.21.15.137:0/1667353612 wait complete. 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.743+0000 7eff81200640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff600a2800 0x7eff600a4cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x560ecea0f2a0 tx=0x7eff740324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:42.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.743+0000 7eff8600b740 1 -- start start 2024-03-26T14:04:42.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.743+0000 7eff8600b740 1 Processor -- start 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea18620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea1f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ecea16b60 con 0x560ecea0e050 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ece92ee80 con 0x560ecea13bb0 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ecea16ce0 con 0x560ecea0ea40 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff8600b740 10 client.0 _ll_register_callbacks cb 0x560ece941130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff7be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff7be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea180e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48272/0 (socket says 172.21.15.137:48272) 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.744+0000 7eff7be00640 1 -- 172.21.15.137:0/4169331898 learned_addr learned my addr 172.21.15.137:0/4169331898 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:42.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff81200640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea1f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7b400640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea18620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7be00640 1 -- 172.21.15.137:0/4169331898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 msgr2=0x560ecea18620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7be00640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea18620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7be00640 1 -- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 msgr2=0x560ecea1f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7be00640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea1f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7be00640 1 -- 172.21.15.137:0/4169331898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ece7517d0 con 0x560ecea0e050 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff7be00640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea180e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00bec0 tx=0x7eff6c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df5d5ec558a3906 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff66000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.745+0000 7eff66000640 1 -- 172.21.15.137:0/4169331898 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0035f0 con 0x560ecea0e050 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff66000640 1 -- 172.21.15.137:0/4169331898 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff6c003790 con 0x560ecea0e050 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff680cfab0 con 0x560ecea0e050 2024-03-26T14:04:42.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff66000640 1 -- 172.21.15.137:0/4169331898 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff6c003990 con 0x560ecea0e050 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff8600b740 10 client.16056 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560ece6873f0 con 0x560ecea0e050 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff66000640 1 -- 172.21.15.137:0/4169331898 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff680cfab0 con 0x560ecea0e050 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.746+0000 7eff66000640 1 -- 172.21.15.137:0/4169331898 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7eff6c019070 con 0x560ecea0e050 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff8600b740 10 client.16056 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff8600b740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff8600b740 20 client.16056 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff8600b740 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea23f40 0x560ecea263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff8600b740 4 client.16056 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ece980790 con 0x560ecea23f40 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.747+0000 7eff7b400640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea23f40 0x560ecea263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.748+0000 7eff66000640 10 client.16056 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:42.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.748+0000 7eff7b400640 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea23f40 0x560ecea263c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6ca8a13bb6e736 server_cookie=651ecc2cfd63bbfa in_seq=0 out_seq=0 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff66000640 1 -- 172.21.15.137:0/4169331898 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560ece980790 con 0x560ecea23f40 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff66000640 10 client.16056 handle_command_reply: tid=0 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 client.16056 shutdown 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 msgr2=0x560ecea180e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea180e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00bec0 tx=0x7eff6c012820 comp rx=0 tx=0).stop 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 shutdown_connections 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea23f40 0x560ecea263c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea1f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0ea40 0x560ecea18620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 --2- 172.21.15.137:0/4169331898 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea180e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 >> 172.21.15.137:0/4169331898 conn(0x560ece9d6200 msgr2=0x560ecea08bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:42.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.749+0000 7eff66000640 0 client.16056 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:42.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.750+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 shutdown_connections 2024-03-26T14:04:42.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.750+0000 7eff8600b740 1 -- 172.21.15.137:0/4169331898 wait complete. 2024-03-26T14:04:42.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.750+0000 7eff8600b740 20 client.16056 trim_cache size 0 max 16384 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.772+0000 7eff8600b740 1 Processor -- start 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 -- start start 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0e050 0x560ecea0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 0x560ecea1f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea0fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1ae780 con 0x560ecea0ea40 2024-03-26T14:04:42.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1ac280 con 0x560ecea13bb0 2024-03-26T14:04:42.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff8600b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1aa300 con 0x560ecea0e050 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff7b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 0x560ecea1f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff7b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 0x560ecea1f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48284/0 (socket says 172.21.15.137:48284) 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff7b400640 1 -- 172.21.15.137:0/3323971627 learned_addr learned my addr 172.21.15.137:0/3323971627 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.773+0000 7eff81200640 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea0fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7be00640 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0e050 0x560ecea0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7b400640 1 -- 172.21.15.137:0/3323971627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0e050 msgr2=0x560ecea0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7b400640 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0e050 0x560ecea0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7b400640 1 -- 172.21.15.137:0/3323971627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 msgr2=0x560ecea0fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7b400640 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea0fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7b400640 1 -- 172.21.15.137:0/3323971627 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ece6470f0 con 0x560ecea0ea40 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7be00640 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0e050 0x560ecea0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff7b400640 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 0x560ecea1f5b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7eff680a3a10 tx=0x7eff68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50db87f0360674e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff70c00640 1 -- 172.21.15.137:0/3323971627 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6809b570 con 0x560ecea0ea40 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.774+0000 7eff70c00640 1 -- 172.21.15.137:0/3323971627 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff68010de0 con 0x560ecea0ea40 2024-03-26T14:04:42.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff70c00640 1 -- 172.21.15.137:0/3323971627 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6809b100 con 0x560ecea0ea40 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 -- 172.21.15.137:0/3323971627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 msgr2=0x560ecea1f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 0x560ecea1f5b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7eff680a3a10 tx=0x7eff68000bc0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 -- 172.21.15.137:0/3323971627 shutdown_connections 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea13bb0 0x560ecea0fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0ea40 0x560ecea1f5b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 --2- 172.21.15.137:0/3323971627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea0e050 0x560ecea0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.775+0000 7eff8600b740 1 -- 172.21.15.137:0/3323971627 >> 172.21.15.137:0/3323971627 conn(0x560ece9d6200 msgr2=0x560ecea0f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.776+0000 7eff8600b740 1 -- 172.21.15.137:0/3323971627 shutdown_connections 2024-03-26T14:04:42.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.776+0000 7eff8600b740 1 -- 172.21.15.137:0/3323971627 wait complete. 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.776+0000 7eff8600b740 1 -- start start 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.776+0000 7eff8600b740 1 Processor -- start 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea1c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0ea40 0x560ecea1ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea13bb0 0x560ecea1dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1aa900 con 0x560ecea0e050 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1abc20 con 0x560ecea0ea40 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c1aaf60 con 0x560ecea13bb0 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff8600b740 10 client.0 _ll_register_callbacks cb 0x7eff6c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea1c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea1c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48294/0 (socket says 172.21.15.137:48294) 2024-03-26T14:04:42.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 -- 172.21.15.137:0/1830523602 learned_addr learned my addr 172.21.15.137:0/1830523602 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7b400640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0ea40 0x560ecea1ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff81200640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea13bb0 0x560ecea1dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 -- 172.21.15.137:0/1830523602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea13bb0 msgr2=0x560ecea1dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea13bb0 0x560ecea1dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 -- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0ea40 msgr2=0x560ecea1ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0ea40 0x560ecea1ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.777+0000 7eff7be00640 1 -- 172.21.15.137:0/1830523602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ecea1e2c0 con 0x560ecea0e050 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff7be00640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea1c780 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7eff6c01bdb0 tx=0x7eff6c00ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f2e474651f1345 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff66000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff66000640 1 -- 172.21.15.137:0/1830523602 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c003d30 con 0x560ecea0e050 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff66000640 1 -- 172.21.15.137:0/1830523602 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff6c005040 con 0x560ecea0e050 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ece97aae0 con 0x560ecea0e050 2024-03-26T14:04:42.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff66000640 1 -- 172.21.15.137:0/1830523602 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff6c03a050 con 0x560ecea0e050 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff8600b740 10 client.16062 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.778+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560ecea1f640 con 0x560ecea0e050 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff66000640 1 -- 172.21.15.137:0/1830523602 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x560ece97aae0 con 0x560ecea0e050 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff66000640 1 -- 172.21.15.137:0/1830523602 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7eff6c019070 con 0x560ecea0e050 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff8600b740 10 client.16062 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff8600b740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff8600b740 20 client.16062 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff8600b740 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea7fce0 0x560ecea82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff8600b740 4 client.16062 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.779+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ece8f83a0 con 0x560ecea7fce0 2024-03-26T14:04:42.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.780+0000 7eff7b400640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea7fce0 0x560ecea82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:42.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.780+0000 7eff7b400640 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea7fce0 0x560ecea82100 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff692814a3ff68e server_cookie=baa7b1f6c48da30d in_seq=0 out_seq=0 2024-03-26T14:04:42.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.780+0000 7eff66000640 10 client.16062 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:42.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff66000640 1 -- 172.21.15.137:0/1830523602 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560ece8f83a0 con 0x560ecea7fce0 2024-03-26T14:04:42.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff66000640 10 client.16062 handle_command_reply: tid=0 2024-03-26T14:04:42.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 client.16062 shutdown 2024-03-26T14:04:42.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 msgr2=0x560ecea1c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea1c780 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7eff6c01bdb0 tx=0x7eff6c00ba60 comp rx=0 tx=0).stop 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 shutdown_connections 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560ecea7fce0 0x560ecea82100 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ecea0ea40 0x560ecea1ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ecea13bb0 0x560ecea1dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 --2- 172.21.15.137:0/1830523602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560ecea0e050 0x560ecea1c780 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 >> 172.21.15.137:0/1830523602 conn(0x560ece9d6200 msgr2=0x560ecea08bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff66000640 0 client.16062 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 shutdown_connections 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 1 -- 172.21.15.137:0/1830523602 wait complete. 2024-03-26T14:04:42.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.781+0000 7eff8600b740 20 client.16062 trim_cache size 0 max 16384 2024-03-26T14:04:42.785 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:04:42.785 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:04:42.785 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:04:42.785 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:04:42.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff600a2800 msgr2=0x7eff600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff600a2800 0x7eff600a4cc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x560ecea0f2a0 tx=0x7eff740324c0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 msgr2=0x7eff7c199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 0x7eff7c199e80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e9f0 tx=0x7eff6800eeb0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 shutdown_connections 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff600a2800 0x7eff600a4cc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff7c105350 0x7eff7c1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff7c104990 0x7eff7c199e80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 --2- 172.21.15.137:0/3072483816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff7c1037a0 0x7eff7c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 >> 172.21.15.137:0/3072483816 conn(0x7eff7c0fed10 msgr2=0x7eff7c107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 shutdown_connections 2024-03-26T14:04:42.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:42.782+0000 7eff81c00640 1 -- 172.21.15.137:0/3072483816 wait complete. 2024-03-26T14:04:42.799 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:04:42.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 -- 172.21.15.137:0/1293263325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350107270 msgr2=0x7ff350109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 --2- 172.21.15.137:0/1293263325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350107270 0x7ff350109660 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff348009ca0 tx=0x7ff34802c100 comp rx=0 tx=0).stop 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 -- 172.21.15.137:0/1293263325 shutdown_connections 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 --2- 172.21.15.137:0/1293263325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff350104940 0x7ff350106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 --2- 172.21.15.137:0/1293263325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350107270 0x7ff350109660 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 --2- 172.21.15.137:0/1293263325 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff350102010 0x7ff350104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 -- 172.21.15.137:0/1293263325 >> 172.21.15.137:0/1293263325 conn(0x7ff3500fb980 msgr2=0x7ff3500fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 -- 172.21.15.137:0/1293263325 shutdown_connections 2024-03-26T14:04:43.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.017+0000 7ff356c00640 1 -- 172.21.15.137:0/1293263325 wait complete. 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.018+0000 7ff356c00640 1 Processor -- start 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.018+0000 7ff356c00640 1 -- start start 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 0x7ff350199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff350104940 0x7ff350199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff350107270 0x7ff3501a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35010d050 con 0x7ff350102010 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35010ced0 con 0x7ff350107270 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35010d1d0 con 0x7ff350104940 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff355800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 0x7ff350199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff355800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 0x7ff350199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48320/0 (socket says 172.21.15.137:48320) 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff355800640 1 -- 172.21.15.137:0/2223859204 learned_addr learned my addr 172.21.15.137:0/2223859204 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff354e00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff350104940 0x7ff350199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.019+0000 7ff356200640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff350107270 0x7ff3501a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff355800640 1 -- 172.21.15.137:0/2223859204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff350104940 msgr2=0x7ff350199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff355800640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff350104940 0x7ff350199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff355800640 1 -- 172.21.15.137:0/2223859204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff350107270 msgr2=0x7ff3501a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff355800640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff350107270 0x7ff3501a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff355800640 1 -- 172.21.15.137:0/2223859204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3501a1540 con 0x7ff350102010 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff355800640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 0x7ff350199920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff34400d770 tx=0x7ff34400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344017070 con 0x7ff350102010 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff3440054d0 con 0x7ff350102010 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3501a1830 con 0x7ff350102010 2024-03-26T14:04:43.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344004cf0 con 0x7ff350102010 2024-03-26T14:04:43.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.020+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3501a1d80 con 0x7ff350102010 2024-03-26T14:04:43.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.022+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff350108d60 con 0x7ff350102010 2024-03-26T14:04:43.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.024+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff344004e90 con 0x7ff350102010 2024-03-26T14:04:43.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.026+0000 7ff34e000640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3380a2880 0x7ff3380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.026+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff3440ce900 con 0x7ff350102010 2024-03-26T14:04:43.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.026+0000 7ff354e00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3380a2880 0x7ff3380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.027+0000 7ff354e00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3380a2880 0x7ff3380a4d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff35019ae30 tx=0x7ff33c00b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.031+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff34409b590 con 0x7ff350102010 2024-03-26T14:04:43.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.308+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff35002d070 con 0x7ff350102010 2024-03-26T14:04:43.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.310+0000 7ff34e000640 1 -- 172.21.15.137:0/2223859204 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7ff34409ae90 con 0x7ff350102010 2024-03-26T14:04:43.308 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:04:43.308 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:04:43.309 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:04:43.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3380a2880 msgr2=0x7ff3380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3380a2880 0x7ff3380a4d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff35019ae30 tx=0x7ff33c00b590 comp rx=0 tx=0).stop 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 msgr2=0x7ff350199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 0x7ff350199920 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff34400d770 tx=0x7ff34400dc30 comp rx=0 tx=0).stop 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 shutdown_connections 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3380a2880 0x7ff3380a4d40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff350107270 0x7ff3501a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff350104940 0x7ff350199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 --2- 172.21.15.137:0/2223859204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff350102010 0x7ff350199920 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 >> 172.21.15.137:0/2223859204 conn(0x7ff3500fb980 msgr2=0x7ff350101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 shutdown_connections 2024-03-26T14:04:43.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.313+0000 7ff356c00640 1 -- 172.21.15.137:0/2223859204 wait complete. 2024-03-26T14:04:43.332 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:04:43.333 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:04:43.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 -- 172.21.15.137:0/1854876752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54601038c0 msgr2=0x7f5460103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 --2- 172.21.15.137:0/1854876752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54601038c0 0x7f5460103ce0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5454009900 tx=0x7f545402bc30 comp rx=0 tx=0).stop 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 -- 172.21.15.137:0/1854876752 shutdown_connections 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 --2- 172.21.15.137:0/1854876752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5460105470 0x7f546010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 --2- 172.21.15.137:0/1854876752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460104ab0 0x7f5460104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 --2- 172.21.15.137:0/1854876752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54601038c0 0x7f5460103ce0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 -- 172.21.15.137:0/1854876752 >> 172.21.15.137:0/1854876752 conn(0x7f54600fee50 msgr2=0x7f5460101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 -- 172.21.15.137:0/1854876752 shutdown_connections 2024-03-26T14:04:43.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.542+0000 7f5467e00640 1 -- 172.21.15.137:0/1854876752 wait complete. 2024-03-26T14:04:43.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.543+0000 7f5467e00640 1 Processor -- start 2024-03-26T14:04:43.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.543+0000 7f5467e00640 1 -- start start 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5467e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54601038c0 0x7f5460199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5467e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5460104ab0 0x7f546019a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5467e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 0x7f54601a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5467e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546010f3f0 con 0x7f5460105470 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5467e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546010f270 con 0x7f5460104ab0 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5467e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546010f570 con 0x7f54601038c0 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5465e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 0x7f54601a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5465e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 0x7f54601a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48348/0 (socket says 172.21.15.137:48348) 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5465e00640 1 -- 172.21.15.137:0/2379824625 learned_addr learned my addr 172.21.15.137:0/2379824625 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5464a00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5460104ab0 0x7f546019a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.544+0000 7f5465400640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54601038c0 0x7f5460199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5465e00640 1 -- 172.21.15.137:0/2379824625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54601038c0 msgr2=0x7f5460199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5465e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54601038c0 0x7f5460199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5465e00640 1 -- 172.21.15.137:0/2379824625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5460104ab0 msgr2=0x7f546019a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5465e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5460104ab0 0x7f546019a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5465e00640 1 -- 172.21.15.137:0/2379824625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54601a1710 con 0x7f5460105470 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5464a00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5460104ab0 0x7f546019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5465e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 0x7f54601a10a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f545800eb10 tx=0x7f545800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f545e000640 1 -- 172.21.15.137:0/2379824625 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458004b20 con 0x7f5460105470 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f545e000640 1 -- 172.21.15.137:0/2379824625 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5458004cc0 con 0x7f5460105470 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54601a1a00 con 0x7f5460105470 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.545+0000 7f545e000640 1 -- 172.21.15.137:0/2379824625 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458010690 con 0x7f5460105470 2024-03-26T14:04:43.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.546+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54601a1fa0 con 0x7f5460105470 2024-03-26T14:04:43.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.549+0000 7f546ac30740 1 Processor -- start 2024-03-26T14:04:43.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.549+0000 7f545e000640 1 -- 172.21.15.137:0/2379824625 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5458005020 con 0x7f5460105470 2024-03-26T14:04:43.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 -- start start 2024-03-26T14:04:43.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 0x5610c9412c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94139e0 0x5610c9413e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c941ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c93772c0 con 0x5610c9412810 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c931fc90 con 0x5610c94139e0 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f546ac30740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c9373360 con 0x5610c94143a0 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f5464a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 0x5610c9412c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f5464a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 0x5610c9412c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48354/0 (socket says 172.21.15.137:48354) 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f5464a00640 1 -- 172.21.15.137:0/4094258737 learned_addr learned my addr 172.21.15.137:0/4094258737 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.550+0000 7f5465400640 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94139e0 0x5610c9413e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5464a00640 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c941ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465400640 1 -- 172.21.15.137:0/4094258737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 msgr2=0x5610c941ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465400640 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c941ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465400640 1 -- 172.21.15.137:0/4094258737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 msgr2=0x5610c9412c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465400640 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 0x5610c9412c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465400640 1 -- 172.21.15.137:0/4094258737 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610c9344080 con 0x5610c94139e0 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f545e000640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f54480a2880 0x7f54480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5464a00640 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 0x5610c9412c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:04:43.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f545e000640 1 -- 172.21.15.137:0/2379824625 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f54580cf520 con 0x7f5460105470 2024-03-26T14:04:43.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f54480a2880 0x7f54480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.551+0000 7f5465400640 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94139e0 0x5610c9413e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5454036040 tx=0x7f54540095e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9dc1734198202a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.552+0000 7f5452600640 1 -- 172.21.15.137:0/4094258737 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454044020 con 0x5610c94139e0 2024-03-26T14:04:43.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.552+0000 7f5452600640 1 -- 172.21.15.137:0/4094258737 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f545402cba0 con 0x5610c94139e0 2024-03-26T14:04:43.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.552+0000 7f5452600640 1 -- 172.21.15.137:0/4094258737 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54540046c0 con 0x5610c94139e0 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 -- 172.21.15.137:0/4094258737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94139e0 msgr2=0x5610c9413e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f5465e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f54480a2880 0x7f54480a4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54580a3d60 tx=0x7f5458009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94139e0 0x5610c9413e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5454036040 tx=0x7f54540095e0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 -- 172.21.15.137:0/4094258737 shutdown_connections 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94139e0 0x5610c9413e60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c941ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 --2- 172.21.15.137:0/4094258737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c9412810 0x5610c9412c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 -- 172.21.15.137:0/4094258737 >> 172.21.15.137:0/4094258737 conn(0x5610c940dc00 msgr2=0x5610c940e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 -- 172.21.15.137:0/4094258737 shutdown_connections 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 -- 172.21.15.137:0/4094258737 wait complete. 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.553+0000 7f546ac30740 1 -- start start 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 Processor -- start 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c9412810 0x5610c941c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c9423f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c941b350 con 0x5610c94139e0 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c93330d0 con 0x5610c9412810 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610c941b4d0 con 0x5610c94143a0 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f546ac30740 10 client.0 _ll_register_callbacks cb 0x5610c9337e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f5465400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f5465400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48364/0 (socket says 172.21.15.137:48364) 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f5465400640 1 -- 172.21.15.137:0/162934377 learned_addr learned my addr 172.21.15.137:0/162934377 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.554+0000 7f5464a00640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c9412810 0x5610c941c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5464a00640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c9423f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5465400640 1 -- 172.21.15.137:0/162934377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 msgr2=0x5610c9423f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5465400640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c9423f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5465400640 1 -- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c9412810 msgr2=0x5610c941c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5465400640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c9412810 0x5610c941c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5465400640 1 -- 172.21.15.137:0/162934377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610c908b3f0 con 0x5610c94139e0 2024-03-26T14:04:43.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5465400640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941cf10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5454005ab0 tx=0x7f545402c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c5ebac68d1d9ec server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.555+0000 7f5446a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:43.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f5446a00640 1 -- 172.21.15.137:0/162934377 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454050060 con 0x5610c94139e0 2024-03-26T14:04:43.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f5446a00640 1 -- 172.21.15.137:0/162934377 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f54540037f0 con 0x5610c94139e0 2024-03-26T14:04:43.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f545402cba0 con 0x5610c94139e0 2024-03-26T14:04:43.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f5446a00640 1 -- 172.21.15.137:0/162934377 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5454057050 con 0x5610c94139e0 2024-03-26T14:04:43.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f546ac30740 10 client.16080 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:43.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5610c91557d0 con 0x5610c94139e0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.556+0000 7f5446a00640 1 -- 172.21.15.137:0/162934377 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f545402cba0 con 0x5610c94139e0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f5446a00640 1 -- 172.21.15.137:0/162934377 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f545404b460 con 0x5610c94139e0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f546ac30740 10 client.16080 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f546ac30740 10 client.16080 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f546ac30740 20 client.16080 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f546ac30740 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c94280b0 0x5610c942a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f546ac30740 4 client.16080 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610c9384800 con 0x5610c94280b0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.557+0000 7f5464a00640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c94280b0 0x5610c942a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.558+0000 7f5464a00640 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c94280b0 0x5610c942a4a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f658c4e8a275db2 server_cookie=d8723a1c04c48e9a in_seq=0 out_seq=0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.558+0000 7f5446a00640 10 client.16080 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.559+0000 7f5446a00640 1 -- 172.21.15.137:0/162934377 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5610c9384800 con 0x5610c94280b0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.559+0000 7f5446a00640 10 client.16080 handle_command_reply: tid=0 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.559+0000 7f546ac30740 1 client.16080 shutdown 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.559+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 msgr2=0x5610c941cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.559+0000 7f546ac30740 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941cf10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5454005ab0 tx=0x7f545402c660 comp rx=0 tx=0).stop 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 shutdown_connections 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c94280b0 0x5610c942a4a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c94143a0 0x5610c9423f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941cf10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 --2- 172.21.15.137:0/162934377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c9412810 0x5610c941c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 >> 172.21.15.137:0/162934377 conn(0x5610c940dc00 msgr2=0x5610c940ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f5446a00640 0 client.16080 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 shutdown_connections 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 1 -- 172.21.15.137:0/162934377 wait complete. 2024-03-26T14:04:43.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.560+0000 7f546ac30740 20 client.16080 trim_cache size 0 max 16384 2024-03-26T14:04:43.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 Processor -- start 2024-03-26T14:04:43.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 -- start start 2024-03-26T14:04:43.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c9416a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c9423f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9416fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601ae910 con 0x5610c94139e0 2024-03-26T14:04:43.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601ac410 con 0x5610c94143a0 2024-03-26T14:04:43.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f546ac30740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601aa490 con 0x5610c9412810 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.574+0000 7f5465400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c9423f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c9423f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48370/0 (socket says 172.21.15.137:48370) 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 -- 172.21.15.137:0/4119993273 learned_addr learned my addr 172.21.15.137:0/4119993273 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5464a00640 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9416fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 -- 172.21.15.137:0/4119993273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 msgr2=0x5610c9416a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c9416a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 -- 172.21.15.137:0/4119993273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 msgr2=0x5610c9416fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9416fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 -- 172.21.15.137:0/4119993273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610c940e580 con 0x5610c94139e0 2024-03-26T14:04:43.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5464a00640 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9416fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:04:43.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.575+0000 7f5465400640 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c9423f10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f545402c170 tx=0x7f54540097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af881ff2e5b5567c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.576+0000 7f5451c00640 1 -- 172.21.15.137:0/4119993273 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454044030 con 0x5610c94139e0 2024-03-26T14:04:43.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.576+0000 7f5451c00640 1 -- 172.21.15.137:0/4119993273 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f545402c8e0 con 0x5610c94139e0 2024-03-26T14:04:43.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.576+0000 7f5451c00640 1 -- 172.21.15.137:0/4119993273 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454003c00 con 0x5610c94139e0 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 -- 172.21.15.137:0/4119993273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 msgr2=0x5610c9423f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c9423f10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f545402c170 tx=0x7f54540097c0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 -- 172.21.15.137:0/4119993273 shutdown_connections 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9416fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c9423f10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 --2- 172.21.15.137:0/4119993273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c9416a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 -- 172.21.15.137:0/4119993273 >> 172.21.15.137:0/4119993273 conn(0x5610c940dc00 msgr2=0x5610c94116f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 -- 172.21.15.137:0/4119993273 shutdown_connections 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 -- 172.21.15.137:0/4119993273 wait complete. 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.577+0000 7f546ac30740 1 -- start start 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 Processor -- start 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c941f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9422e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601aaa90 con 0x5610c94139e0 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601abdb0 con 0x5610c94143a0 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54601ab0f0 con 0x5610c9412810 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f546ac30740 10 client.0 _ll_register_callbacks cb 0x7f544c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f5465400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f5465400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48378/0 (socket says 172.21.15.137:48378) 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.578+0000 7f5465400640 1 -- 172.21.15.137:0/2523026895 learned_addr learned my addr 172.21.15.137:0/2523026895 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5464a00640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9422e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5464a00640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c941f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5465400640 1 -- 172.21.15.137:0/2523026895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 msgr2=0x5610c941f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5465400640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c941f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5465400640 1 -- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 msgr2=0x5610c9422e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5465400640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9422e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5465400640 1 -- 172.21.15.137:0/2523026895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610c9423400 con 0x5610c94139e0 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5464a00640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c941f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5465400640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941fdf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f545402bc70 tx=0x7f545402c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ca7c5c3e2c04c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5446a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:43.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5446a00640 1 -- 172.21.15.137:0/2523026895 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454050020 con 0x5610c94139e0 2024-03-26T14:04:43.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f5446a00640 1 -- 172.21.15.137:0/2523026895 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5454003c00 con 0x5610c94139e0 2024-03-26T14:04:43.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.579+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610c937eb50 con 0x5610c94139e0 2024-03-26T14:04:43.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f5446a00640 1 -- 172.21.15.137:0/2523026895 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f545406a050 con 0x5610c94139e0 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 10 client.16086 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5610c94839e0 con 0x5610c94139e0 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f5446a00640 1 -- 172.21.15.137:0/2523026895 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5610c937eb50 con 0x5610c94139e0 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f5446a00640 1 -- 172.21.15.137:0/2523026895 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5454040c00 con 0x5610c94139e0 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 10 client.16086 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 10 client.16086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 20 client.16086 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c9484310 0x5610c9486700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 4 client.16086 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.580+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610c92fc500 con 0x5610c9484310 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.581+0000 7f5464a00640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c9484310 0x5610c9486700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f5464a00640 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c9484310 0x5610c9486700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4be710312e15159e server_cookie=c904b52404112717 in_seq=0 out_seq=0 2024-03-26T14:04:43.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f5446a00640 10 client.16086 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f5446a00640 1 -- 172.21.15.137:0/2523026895 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610c92fc500 con 0x5610c9484310 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f5446a00640 10 client.16086 handle_command_reply: tid=0 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 client.16086 shutdown 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 msgr2=0x5610c941fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941fdf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f545402bc70 tx=0x7f545402c2b0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 shutdown_connections 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5610c9484310 0x5610c9486700 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610c94143a0 0x5610c9422e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5610c94139e0 0x5610c941fdf0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 --2- 172.21.15.137:0/2523026895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610c9412810 0x5610c941f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f5446a00640 0 client.16086 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.582+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 >> 172.21.15.137:0/2523026895 conn(0x5610c940dc00 msgr2=0x5610c940c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 shutdown_connections 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f546ac30740 1 -- 172.21.15.137:0/2523026895 wait complete. 2024-03-26T14:04:43.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f546ac30740 20 client.16086 trim_cache size 0 max 16384 2024-03-26T14:04:43.584 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:04:43.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f54480a2880 msgr2=0x7f54480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f5467e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f54480a2880 0x7f54480a4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54580a3d60 tx=0x7f5458009a70 comp rx=0 tx=0).stop 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 msgr2=0x7f54601a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.583+0000 7f5467e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 0x7f54601a10a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f545800eb10 tx=0x7f545800c2d0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 shutdown_connections 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f54480a2880 0x7f54480a4d40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5460104ab0 0x7f546019a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460105470 0x7f54601a10a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 --2- 172.21.15.137:0/2379824625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54601038c0 0x7f5460199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 >> 172.21.15.137:0/2379824625 conn(0x7f54600fee50 msgr2=0x7f5460107e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 shutdown_connections 2024-03-26T14:04:43.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.584+0000 7f5467e00640 1 -- 172.21.15.137:0/2379824625 wait complete. 2024-03-26T14:04:43.600 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:04:43.600 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 -- 172.21.15.137:0/2360272133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 msgr2=0x7f9c30073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/2360272133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30073ad0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009900 tx=0x7f9c2002bcd0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 -- 172.21.15.137:0/2360272133 shutdown_connections 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/2360272133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c300740a0 0x7f9c30074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/2360272133 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c30109430 0x7f9c3010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/2360272133 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30073ad0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 -- 172.21.15.137:0/2360272133 >> 172.21.15.137:0/2360272133 conn(0x7f9c300fcb10 msgr2=0x7f9c300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 -- 172.21.15.137:0/2360272133 shutdown_connections 2024-03-26T14:04:43.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.814+0000 7f9c2fe00640 1 -- 172.21.15.137:0/2360272133 wait complete. 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.815+0000 7f9c2fe00640 1 Processor -- start 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.815+0000 7f9c2fe00640 1 -- start start 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c300740a0 0x7f9c301999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c30109430 0x7f9c301a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3010f310 con 0x7f9c30075680 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3010f190 con 0x7f9c30109430 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3010f490 con 0x7f9c300740a0 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48404/0 (socket says 172.21.15.137:48404) 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2e000640 1 -- 172.21.15.137:0/4249652039 learned_addr learned my addr 172.21.15.137:0/4249652039 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2f400640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c30109430 0x7f9c301a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.816+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c300740a0 0x7f9c301999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2e000640 1 -- 172.21.15.137:0/4249652039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c300740a0 msgr2=0x7f9c301999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2e000640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c300740a0 0x7f9c301999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2e000640 1 -- 172.21.15.137:0/4249652039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c30109430 msgr2=0x7f9c301a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2e000640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c30109430 0x7f9c301a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2e000640 1 -- 172.21.15.137:0/4249652039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c301a15d0 con 0x7f9c30075680 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2f400640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c30109430 0x7f9c301a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:43.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2e000640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30199ef0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00e9f0 tx=0x7f9c1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c27400640 1 -- 172.21.15.137:0/4249652039 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00cd80 con 0x7f9c30075680 2024-03-26T14:04:43.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c301a18c0 con 0x7f9c30075680 2024-03-26T14:04:43.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c27400640 1 -- 172.21.15.137:0/4249652039 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c1c0108e0 con 0x7f9c30075680 2024-03-26T14:04:43.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c27400640 1 -- 172.21.15.137:0/4249652039 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c018920 con 0x7f9c30075680 2024-03-26T14:04:43.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.817+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c301a1e10 con 0x7f9c30075680 2024-03-26T14:04:43.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.821+0000 7f9c389a8740 1 Processor -- start 2024-03-26T14:04:43.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 -- start start 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e487fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 0x55f6e4881040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4881580 0x55f6e4887e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e47e42c0 con 0x55f6e4880bc0 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e478cc90 con 0x55f6e487f9f0 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.822+0000 7f9c389a8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e47e0360 con 0x55f6e4881580 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 0x55f6e4881040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c27400640 1 -- 172.21.15.137:0/4249652039 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9c1c010a90 con 0x7f9c30075680 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 0x55f6e4881040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48416/0 (socket says 172.21.15.137:48416) 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 -- 172.21.15.137:0/2216005344 learned_addr learned my addr 172.21.15.137:0/2216005344 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2f400640 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e487fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 -- 172.21.15.137:0/2216005344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4881580 msgr2=0x55f6e4887e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4881580 0x55f6e4887e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 -- 172.21.15.137:0/2216005344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 msgr2=0x55f6e487fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e487fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 -- 172.21.15.137:0/2216005344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6e47b1080 con 0x55f6e4880bc0 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.823+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 0x55f6e4881040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9c2002c8e0 tx=0x7f9c20009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1063cab0cd917ca1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.824+0000 7f9c1be00640 1 -- 172.21.15.137:0/2216005344 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004240 con 0x55f6e4880bc0 2024-03-26T14:04:43.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.824+0000 7f9c1be00640 1 -- 172.21.15.137:0/2216005344 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c2002ce30 con 0x55f6e4880bc0 2024-03-26T14:04:43.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.824+0000 7f9c1be00640 1 -- 172.21.15.137:0/2216005344 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20004920 con 0x55f6e4880bc0 2024-03-26T14:04:43.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.824+0000 7f9c27400640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c140a2800 0x7f9c140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.824+0000 7f9c27400640 1 -- 172.21.15.137:0/4249652039 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9c1c0cf380 con 0x7f9c30075680 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.824+0000 7f9c2f400640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c140a2800 0x7f9c140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 -- 172.21.15.137:0/2216005344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 msgr2=0x55f6e4881040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 0x55f6e4881040 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9c2002c8e0 tx=0x7f9c20009670 comp rx=0 tx=0).stop 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 -- 172.21.15.137:0/2216005344 shutdown_connections 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4881580 0x55f6e4887e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4880bc0 0x55f6e4881040 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2216005344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e487fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 -- 172.21.15.137:0/2216005344 >> 172.21.15.137:0/2216005344 conn(0x55f6e487ade0 msgr2=0x55f6e487b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 -- 172.21.15.137:0/2216005344 shutdown_connections 2024-03-26T14:04:43.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c2f400640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c140a2800 0x7f9c140a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55f6e4880ac0 tx=0x7f9c28009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.825+0000 7f9c389a8740 1 -- 172.21.15.137:0/2216005344 wait complete. 2024-03-26T14:04:43.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.826+0000 7f9c389a8740 1 -- start start 2024-03-26T14:04:43.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.826+0000 7f9c389a8740 1 Processor -- start 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4889c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e488a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e48911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e4888530 con 0x55f6e4881580 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e47a00d0 con 0x55f6e4880bc0 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6e48886b0 con 0x55f6e487f9f0 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c389a8740 10 client.0 _ll_register_callbacks cb 0x55f6e47a4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e48911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e48911e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48432/0 (socket says 172.21.15.137:48432) 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c2f400640 1 -- 172.21.15.137:0/3066230413 learned_addr learned my addr 172.21.15.137:0/3066230413 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c2e000640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e488a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.827+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4889c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2f400640 1 -- 172.21.15.137:0/3066230413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 msgr2=0x55f6e4889c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2f400640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4889c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2f400640 1 -- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 msgr2=0x55f6e488a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2f400640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e488a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2f400640 1 -- 172.21.15.137:0/3066230413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6e44f83f0 con 0x55f6e4881580 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4889c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c2f400640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e48911e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9c28010a40 tx=0x7f9c28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab6ed3274442466 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c18c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c18c00640 1 -- 172.21.15.137:0/3066230413 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2802c060 con 0x55f6e4881580 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c2002ce30 con 0x55f6e4881580 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.828+0000 7f9c18c00640 1 -- 172.21.15.137:0/3066230413 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9c28017030 con 0x55f6e4881580 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c18c00640 1 -- 172.21.15.137:0/3066230413 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c28033050 con 0x55f6e4881580 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 10 client.16098 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:43.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6e45c27d0 con 0x55f6e4881580 2024-03-26T14:04:43.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c18c00640 1 -- 172.21.15.137:0/3066230413 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c2002ce30 con 0x55f6e4881580 2024-03-26T14:04:43.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c18c00640 1 -- 172.21.15.137:0/3066230413 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9c2801cc30 con 0x55f6e4881580 2024-03-26T14:04:43.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 10 client.16098 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:43.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 10 client.16098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:04:43.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 20 client.16098 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e4895710 0x55f6e4897b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 4 client.16098 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.829+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6e47f1800 con 0x55f6e4895710 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.830+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e4895710 0x55f6e4897b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.831+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e4895710 0x55f6e4897b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=720aef450f9b362b server_cookie=dc45933ec6793fa5 in_seq=0 out_seq=0 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.831+0000 7f9c18c00640 10 client.16098 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.832+0000 7f9c18c00640 1 -- 172.21.15.137:0/3066230413 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f6e47f1800 con 0x55f6e4895710 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.832+0000 7f9c18c00640 10 client.16098 handle_command_reply: tid=0 2024-03-26T14:04:43.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.832+0000 7f9c389a8740 1 client.16098 shutdown 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 msgr2=0x55f6e48911e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e48911e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9c28010a40 tx=0x7f9c28004ab0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 shutdown_connections 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e4895710 0x55f6e4897b70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e488a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e48911e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 --2- 172.21.15.137:0/3066230413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4889c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c18c00640 0 client.16098 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:43.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 >> 172.21.15.137:0/3066230413 conn(0x55f6e487ade0 msgr2=0x55f6e487a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 shutdown_connections 2024-03-26T14:04:43.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 1 -- 172.21.15.137:0/3066230413 wait complete. 2024-03-26T14:04:43.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.833+0000 7f9c389a8740 20 client.16098 trim_cache size 0 max 16384 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 Processor -- start 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 -- start start 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e4879fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4880bc0 0x55f6e48911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e489c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301ae860 con 0x55f6e4881580 2024-03-26T14:04:43.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301ac360 con 0x55f6e487f9f0 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.852+0000 7f9c389a8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301aa3e0 con 0x55f6e4880bc0 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e489c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e489c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48442/0 (socket says 172.21.15.137:48442) 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 -- 172.21.15.137:0/2309645097 learned_addr learned my addr 172.21.15.137:0/2309645097 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e4879fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2e000640 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4880bc0 0x55f6e48911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 -- 172.21.15.137:0/2309645097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4880bc0 msgr2=0x55f6e48911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4880bc0 0x55f6e48911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 -- 172.21.15.137:0/2309645097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 msgr2=0x55f6e4879fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e4879fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2f400640 1 -- 172.21.15.137:0/2309645097 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6e487bb50 con 0x55f6e4881580 2024-03-26T14:04:43.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.853+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e4879fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:43.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.854+0000 7f9c2f400640 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e489c430 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9c28018040 tx=0x7f9c28005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf91e3cf27c22c3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.854+0000 7f9c1b400640 1 -- 172.21.15.137:0/2309645097 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2801fdc0 con 0x55f6e4881580 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.854+0000 7f9c1b400640 1 -- 172.21.15.137:0/2309645097 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c2800ea80 con 0x55f6e4881580 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.854+0000 7f9c1b400640 1 -- 172.21.15.137:0/2309645097 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2801c8f0 con 0x55f6e4881580 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 -- 172.21.15.137:0/2309645097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 msgr2=0x55f6e489c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e489c430 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9c28018040 tx=0x7f9c28005cc0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 -- 172.21.15.137:0/2309645097 shutdown_connections 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e489c430 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e4880bc0 0x55f6e48911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 --2- 172.21.15.137:0/2309645097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e487f9f0 0x55f6e4879fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 -- 172.21.15.137:0/2309645097 >> 172.21.15.137:0/2309645097 conn(0x55f6e487ade0 msgr2=0x55f6e48798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 -- 172.21.15.137:0/2309645097 shutdown_connections 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 -- 172.21.15.137:0/2309645097 wait complete. 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 -- start start 2024-03-26T14:04:43.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.855+0000 7f9c389a8740 1 Processor -- start 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4890bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e4891130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e488d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301aa9e0 con 0x55f6e4881580 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301abd00 con 0x55f6e4880bc0 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301ab040 con 0x55f6e487f9f0 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c389a8740 10 client.0 _ll_register_callbacks cb 0x7f9c28005c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e488d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e488d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48456/0 (socket says 172.21.15.137:48456) 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c2f400640 1 -- 172.21.15.137:0/828126641 learned_addr learned my addr 172.21.15.137:0/828126641 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c2e000640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e4891130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.856+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4890bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2f400640 1 -- 172.21.15.137:0/828126641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 msgr2=0x55f6e4890bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2f400640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4890bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2f400640 1 -- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 msgr2=0x55f6e4891130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2f400640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e4891130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2f400640 1 -- 172.21.15.137:0/828126641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6e488df40 con 0x55f6e4881580 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2e000640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e4891130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c2f400640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e488d710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9c28021010 tx=0x7f9c2800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a678d151f212574 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c18c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c18c00640 1 -- 172.21.15.137:0/828126641 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2800e780 con 0x55f6e4881580 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c2800ea80 con 0x55f6e4881580 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.857+0000 7f9c18c00640 1 -- 172.21.15.137:0/828126641 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9c280049c0 con 0x55f6e4881580 2024-03-26T14:04:43.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c18c00640 1 -- 172.21.15.137:0/828126641 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c28051050 con 0x55f6e4881580 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 10 client.16104 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f6e48f1df0 con 0x55f6e4881580 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c18c00640 1 -- 172.21.15.137:0/828126641 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9c2800ea80 con 0x55f6e4881580 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c18c00640 1 -- 172.21.15.137:0/828126641 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9c28005040 con 0x55f6e4881580 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 10 client.16104 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 10 client.16104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 20 client.16104 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e48f26e0 0x55f6e48f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 4 client.16104 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.858+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6e47ebb50 con 0x55f6e48f26e0 2024-03-26T14:04:43.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.859+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e48f26e0 0x55f6e48f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c2ea00640 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e48f26e0 0x55f6e48f4ad0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b65136ae4277e225 server_cookie=61f7c8009cf9be61 in_seq=0 out_seq=0 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c18c00640 10 client.16104 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c18c00640 1 -- 172.21.15.137:0/828126641 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6e47ebb50 con 0x55f6e48f26e0 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c18c00640 10 client.16104 handle_command_reply: tid=0 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 client.16104 shutdown 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 msgr2=0x55f6e488d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e488d710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9c28021010 tx=0x7f9c2800ee60 comp rx=0 tx=0).stop 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 shutdown_connections 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f6e48f26e0 0x55f6e48f4ad0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6e4880bc0 0x55f6e4891130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f6e4881580 0x55f6e488d710 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 --2- 172.21.15.137:0/828126641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6e487f9f0 0x55f6e4890bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c18c00640 0 client.16104 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 >> 172.21.15.137:0/828126641 conn(0x55f6e487ade0 msgr2=0x55f6e487a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.860+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 shutdown_connections 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c389a8740 1 -- 172.21.15.137:0/828126641 wait complete. 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c389a8740 20 client.16104 trim_cache size 0 max 16384 2024-03-26T14:04:43.861 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c140a2800 msgr2=0x7f9c140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c140a2800 0x7f9c140a4cc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55f6e4880ac0 tx=0x7f9c28009500 comp rx=0 tx=0).stop 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 msgr2=0x7f9c30199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30199ef0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00e9f0 tx=0x7f9c1c00eeb0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 shutdown_connections 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9c140a2800 0x7f9c140a4cc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c30109430 0x7f9c301a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c30075680 0x7f9c30199ef0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 --2- 172.21.15.137:0/4249652039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9c300740a0 0x7f9c301999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:43.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.861+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 >> 172.21.15.137:0/4249652039 conn(0x7f9c300fcb10 msgr2=0x7f9c301053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:43.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.862+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 shutdown_connections 2024-03-26T14:04:43.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:43.862+0000 7f9c2fe00640 1 -- 172.21.15.137:0/4249652039 wait complete. 2024-03-26T14:04:43.876 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:04:43.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 -- 172.21.15.137:0/876250755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40241072c0 msgr2=0x7f40241096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 --2- 172.21.15.137:0/876250755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40241072c0 0x7f40241096b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f401c009ca0 tx=0x7f401c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 -- 172.21.15.137:0/876250755 shutdown_connections 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 --2- 172.21.15.137:0/876250755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4024104990 0x7f4024106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 --2- 172.21.15.137:0/876250755 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f40241072c0 0x7f40241096b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 --2- 172.21.15.137:0/876250755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4024102060 0x7f4024104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.096+0000 7f4028c00640 1 -- 172.21.15.137:0/876250755 >> 172.21.15.137:0/876250755 conn(0x7f40240fba10 msgr2=0x7f40240fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.097+0000 7f4028c00640 1 -- 172.21.15.137:0/876250755 shutdown_connections 2024-03-26T14:04:44.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.097+0000 7f4028c00640 1 -- 172.21.15.137:0/876250755 wait complete. 2024-03-26T14:04:44.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 Processor -- start 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 -- start start 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 0x7f4024199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4024104990 0x7f4024199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40241072c0 0x7f40241a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402410d0c0 con 0x7f4024102060 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402410cf40 con 0x7f4024104990 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.098+0000 7f4028c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402410d240 con 0x7f40241072c0 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 0x7f4024199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 0x7f4024199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48472/0 (socket says 172.21.15.137:48472) 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 -- 172.21.15.137:0/1168672366 learned_addr learned my addr 172.21.15.137:0/1168672366 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:44.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4022a00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4024104990 0x7f4024199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023e00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40241072c0 0x7f40241a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 -- 172.21.15.137:0/1168672366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40241072c0 msgr2=0x7f40241a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40241072c0 0x7f40241a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 -- 172.21.15.137:0/1168672366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4024104990 msgr2=0x7f4024199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4024104990 0x7f4024199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.099+0000 7f4023400640 1 -- 172.21.15.137:0/1168672366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40241a1550 con 0x7f4024102060 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.100+0000 7f4023400640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 0x7f4024199930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f401400d770 tx=0x7f401400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.100+0000 7f401be00640 1 -- 172.21.15.137:0/1168672366 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014017070 con 0x7f4024102060 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.100+0000 7f401be00640 1 -- 172.21.15.137:0/1168672366 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40140054d0 con 0x7f4024102060 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.100+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40241a1840 con 0x7f4024102060 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.100+0000 7f401be00640 1 -- 172.21.15.137:0/1168672366 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014004cf0 con 0x7f4024102060 2024-03-26T14:04:44.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.100+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40241a1d40 con 0x7f4024102060 2024-03-26T14:04:44.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 Processor -- start 2024-03-26T14:04:44.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 -- start start 2024-03-26T14:04:44.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece96040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece9ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555decdf92c0 con 0x555dece96580 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.104+0000 7f402ce1e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555decda1c90 con 0x555dece95bc0 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f402ce1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555decdf5360 con 0x555dece949f0 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f401be00640 1 -- 172.21.15.137:0/1168672366 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4014004e90 con 0x7f4024102060 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece9ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece9ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48474/0 (socket says 172.21.15.137:48474) 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 -- 172.21.15.137:0/580939662 learned_addr learned my addr 172.21.15.137:0/580939662 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4023e00640 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4023400640 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece96040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 -- 172.21.15.137:0/580939662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 msgr2=0x555dece94e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece94e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 -- 172.21.15.137:0/580939662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 msgr2=0x555dece96040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece96040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4022a00640 1 -- 172.21.15.137:0/580939662 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555decdc6080 con 0x555dece96580 2024-03-26T14:04:44.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.105+0000 7f4023400640 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece96040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.106+0000 7f4022a00640 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece9ce10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4010009c60 tx=0x7f4010011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbc585b2faac944 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.106+0000 7f400fe00640 1 -- 172.21.15.137:0/580939662 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004230 con 0x555dece96580 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.106+0000 7f400fe00640 1 -- 172.21.15.137:0/580939662 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40100043d0 con 0x555dece96580 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.106+0000 7f400fe00640 1 -- 172.21.15.137:0/580939662 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004d40 con 0x555dece96580 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.106+0000 7f401be00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40080a2800 0x7f40080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f401be00640 1 -- 172.21.15.137:0/1168672366 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f40140ce820 con 0x7f4024102060 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f4023e00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40080a2800 0x7f40080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 -- 172.21.15.137:0/580939662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 msgr2=0x555dece9ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece9ce10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4010009c60 tx=0x7f4010011810 comp rx=0 tx=0).stop 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 -- 172.21.15.137:0/580939662 shutdown_connections 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece96040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece9ce10 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 --2- 172.21.15.137:0/580939662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece94e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.107+0000 7f402ce1e740 1 -- 172.21.15.137:0/580939662 >> 172.21.15.137:0/580939662 conn(0x555dece8fde0 msgr2=0x555dece901e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.108+0000 7f402ce1e740 1 -- 172.21.15.137:0/580939662 shutdown_connections 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.108+0000 7f4023e00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40080a2800 0x7f40080a4cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f401c02c630 tx=0x7f401c02e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.108+0000 7f402ce1e740 1 -- 172.21.15.137:0/580939662 wait complete. 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.108+0000 7f402ce1e740 1 -- start start 2024-03-26T14:04:44.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 Processor -- start 2024-03-26T14:04:44.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece9ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece9f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dece9d590 con 0x555dece96580 2024-03-26T14:04:44.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555decdb50d0 con 0x555dece95bc0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dece9d710 con 0x555dece949f0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f402ce1e740 10 client.0 _ll_register_callbacks cb 0x555decdb9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.109+0000 7f4023e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48484/0 (socket says 172.21.15.137:48484) 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 -- 172.21.15.137:0/2083430776 learned_addr learned my addr 172.21.15.137:0/2083430776 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4022a00640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece9f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023400640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece9ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 -- 172.21.15.137:0/2083430776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 msgr2=0x555dece9ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece9ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 -- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 msgr2=0x555dece9f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece9f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 -- 172.21.15.137:0/2083430776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555decb0d3f0 con 0x555dece96580 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4022a00640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece9f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f4023e00640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea6190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f401c032550 tx=0x7f401c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04fb22b1f3939a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:44.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.110+0000 7f400cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.111+0000 7f400cc00640 1 -- 172.21.15.137:0/2083430776 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c039040 con 0x555dece96580 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.111+0000 7f400cc00640 1 -- 172.21.15.137:0/2083430776 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f401c04c660 con 0x555dece96580 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.111+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40100043d0 con 0x555dece96580 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.111+0000 7f400cc00640 1 -- 172.21.15.137:0/2083430776 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c05a050 con 0x555dece96580 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.111+0000 7f402ce1e740 10 client.16116 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.111+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555decbd77d0 con 0x555dece96580 2024-03-26T14:04:44.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f400cc00640 1 -- 172.21.15.137:0/2083430776 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f40100043d0 con 0x555dece96580 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f400cc00640 1 -- 172.21.15.137:0/2083430776 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f401c040b70 con 0x555dece96580 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f402ce1e740 10 client.16116 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f402ce1e740 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f402ce1e740 20 client.16116 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555deceaa6c0 0x555deceacae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f402ce1e740 4 client.16116 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.112+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dece06800 con 0x555deceaa6c0 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.113+0000 7f4023400640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555deceaa6c0 0x555deceacae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.113+0000 7f4023400640 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555deceaa6c0 0x555deceacae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93433c83f995c9f server_cookie=b311f0ddf17712e in_seq=0 out_seq=0 2024-03-26T14:04:44.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.113+0000 7f400cc00640 10 client.16116 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f400cc00640 1 -- 172.21.15.137:0/2083430776 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555dece06800 con 0x555deceaa6c0 2024-03-26T14:04:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f400cc00640 10 client.16116 handle_command_reply: tid=0 2024-03-26T14:04:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 client.16116 shutdown 2024-03-26T14:04:44.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 msgr2=0x555decea6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea6190 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f401c032550 tx=0x7f401c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 shutdown_connections 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555deceaa6c0 0x555deceacae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555dece9f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f400cc00640 0 client.16116 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea6190 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2083430776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece9ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.115+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 >> 172.21.15.137:0/2083430776 conn(0x555dece8fde0 msgr2=0x555dece8f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.116+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 shutdown_connections 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.116+0000 7f402ce1e740 1 -- 172.21.15.137:0/2083430776 wait complete. 2024-03-26T14:04:44.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.116+0000 7f402ce1e740 20 client.16116 trim_cache size 0 max 16384 2024-03-26T14:04:44.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 Processor -- start 2024-03-26T14:04:44.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 -- start start 2024-03-26T14:04:44.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece98df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece99330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241ae5c0 con 0x555dece96580 2024-03-26T14:04:44.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241ac0c0 con 0x555dece95bc0 2024-03-26T14:04:44.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.131+0000 7f402ce1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241aa140 con 0x555dece949f0 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece99330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece99330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48486/0 (socket says 172.21.15.137:48486) 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 -- 172.21.15.137:0/556880005 learned_addr learned my addr 172.21.15.137:0/556880005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4022a00640 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023400640 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece98df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 -- 172.21.15.137:0/556880005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 msgr2=0x555dece98df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece98df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 -- 172.21.15.137:0/556880005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 msgr2=0x555decea6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 -- 172.21.15.137:0/556880005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dece95640 con 0x555dece96580 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4022a00640 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.132+0000 7f4023e00640 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece99330 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f401c032e10 tx=0x7f401c02c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4840f4ff20269f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.133+0000 7f400f400640 1 -- 172.21.15.137:0/556880005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c039040 con 0x555dece96580 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.133+0000 7f400f400640 1 -- 172.21.15.137:0/556880005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f401c063070 con 0x555dece96580 2024-03-26T14:04:44.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.133+0000 7f400f400640 1 -- 172.21.15.137:0/556880005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c034b90 con 0x555dece96580 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.133+0000 7f402ce1e740 1 -- 172.21.15.137:0/556880005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 msgr2=0x555dece99330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.133+0000 7f402ce1e740 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece99330 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f401c032e10 tx=0x7f401c02c820 comp rx=0 tx=0).stop 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 -- 172.21.15.137:0/556880005 shutdown_connections 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555dece99330 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 --2- 172.21.15.137:0/556880005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555dece98df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 -- 172.21.15.137:0/556880005 >> 172.21.15.137:0/556880005 conn(0x555dece8fde0 msgr2=0x555dece938d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 -- 172.21.15.137:0/556880005 shutdown_connections 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 -- 172.21.15.137:0/556880005 wait complete. 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 -- start start 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.134+0000 7f402ce1e740 1 Processor -- start 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555decea5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241aa740 con 0x555dece96580 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241aba60 con 0x555dece95bc0 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241aada0 con 0x555dece949f0 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f402ce1e740 10 client.0 _ll_register_callbacks cb 0x7f4010005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f4023e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f4023e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48488/0 (socket says 172.21.15.137:48488) 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f4023e00640 1 -- 172.21.15.137:0/2246146552 learned_addr learned my addr 172.21.15.137:0/2246146552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:44.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f4022a00640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.135+0000 7f4023400640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555decea5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4023e00640 1 -- 172.21.15.137:0/2246146552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 msgr2=0x555decea5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4023e00640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555decea5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4023e00640 1 -- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 msgr2=0x555decea43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4023e00640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4023e00640 1 -- 172.21.15.137:0/2246146552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555decea4e90 con 0x555dece96580 2024-03-26T14:04:44.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4022a00640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:44.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f4023e00640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea4920 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f401c02c700 tx=0x7f401c040690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3a56e3ed578f61 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:44.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.136+0000 7f400cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:44.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f400cc00640 1 -- 172.21.15.137:0/2246146552 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c040ad0 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f400cc00640 1 -- 172.21.15.137:0/2246146552 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f401c040c70 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f401c063070 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f400cc00640 1 -- 172.21.15.137:0/2246146552 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f401c074050 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f402ce1e740 10 client.16122 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555decf06cf0 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.137+0000 7f400cc00640 1 -- 172.21.15.137:0/2246146552 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f401c063070 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f400cc00640 1 -- 172.21.15.137:0/2246146552 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f401c04cb00 con 0x555dece96580 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f402ce1e740 10 client.16122 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f402ce1e740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f402ce1e740 20 client.16122 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555decf075d0 0x555decf099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f402ce1e740 4 client.16122 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555dece00b50 con 0x555decf075d0 2024-03-26T14:04:44.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.138+0000 7f4023400640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555decf075d0 0x555decf099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f4023400640 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555decf075d0 0x555decf099f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=932db7e870558538 server_cookie=295377eba87eda43 in_seq=0 out_seq=0 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f400cc00640 10 client.16122 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f400cc00640 1 -- 172.21.15.137:0/2246146552 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555dece00b50 con 0x555decf075d0 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f400cc00640 10 client.16122 handle_command_reply: tid=0 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f402ce1e740 1 client.16122 shutdown 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 msgr2=0x555decea4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.139+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea4920 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f401c02c700 tx=0x7f401c040690 comp rx=0 tx=0).stop 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 shutdown_connections 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555decf075d0 0x555decf099f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555dece95bc0 0x555decea43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555dece96580 0x555decea4920 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 --2- 172.21.15.137:0/2246146552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dece949f0 0x555decea5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 >> 172.21.15.137:0/2246146552 conn(0x555dece8fde0 msgr2=0x555dece8e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f400cc00640 0 client.16122 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:44.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 shutdown_connections 2024-03-26T14:04:44.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 1 -- 172.21.15.137:0/2246146552 wait complete. 2024-03-26T14:04:44.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.140+0000 7f402ce1e740 20 client.16122 trim_cache size 0 max 16384 2024-03-26T14:04:44.141 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40080a2800 msgr2=0x7f40080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40080a2800 0x7f40080a4cc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f401c02c630 tx=0x7f401c02e040 comp rx=0 tx=0).stop 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 msgr2=0x7f4024199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 0x7f4024199930 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f401400d770 tx=0x7f401400dc30 comp rx=0 tx=0).stop 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 shutdown_connections 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f40080a2800 0x7f40080a4cc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4024104990 0x7f4024199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40241072c0 0x7f40241a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 --2- 172.21.15.137:0/1168672366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4024102060 0x7f4024199930 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 >> 172.21.15.137:0/1168672366 conn(0x7f40240fba10 msgr2=0x7f40241016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 shutdown_connections 2024-03-26T14:04:44.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:44.141+0000 7f4028c00640 1 -- 172.21.15.137:0/1168672366 wait complete. 2024-03-26T14:04:44.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:44 smithi137 ceph-mon[31539]: pgmap v529: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-26T14:04:44.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:44 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2223859204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:04:44.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:44 smithi179 ceph-mon[35472]: pgmap v529: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-26T14:04:44.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:44 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2223859204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:04:44.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:44 smithi159 ceph-mon[35501]: pgmap v529: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-03-26T14:04:44.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:44 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2223859204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:04:45.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:04:45.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:04:45.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 -- 172.21.15.137:0/3425219240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478102790 msgr2=0x7ff478102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 --2- 172.21.15.137:0/3425219240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478102790 0x7ff478102c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff464009900 tx=0x7ff46402bc30 comp rx=0 tx=0).stop 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 -- 172.21.15.137:0/3425219240 shutdown_connections 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 --2- 172.21.15.137:0/3425219240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff478103150 0x7ff4781099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 --2- 172.21.15.137:0/3425219240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478102790 0x7ff478102c10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 --2- 172.21.15.137:0/3425219240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4781015a0 0x7ff4781019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 -- 172.21.15.137:0/3425219240 >> 172.21.15.137:0/3425219240 conn(0x7ff4780fcb10 msgr2=0x7ff4780fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 -- 172.21.15.137:0/3425219240 shutdown_connections 2024-03-26T14:04:45.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.396+0000 7ff477200640 1 -- 172.21.15.137:0/3425219240 wait complete. 2024-03-26T14:04:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 Processor -- start 2024-03-26T14:04:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 -- start start 2024-03-26T14:04:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4781015a0 0x7ff478199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff478102790 0x7ff478199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 0x7ff4781a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47810d120 con 0x7ff478103150 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47810cfa0 con 0x7ff478102790 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff477200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47810d2a0 con 0x7ff4781015a0 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.398+0000 7ff476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 0x7ff4781a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 0x7ff4781a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48506/0 (socket says 172.21.15.137:48506) 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 -- 172.21.15.137:0/1874267087 learned_addr learned my addr 172.21.15.137:0/1874267087 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff475400640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff478102790 0x7ff478199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff475e00640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4781015a0 0x7ff478199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 -- 172.21.15.137:0/1874267087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4781015a0 msgr2=0x7ff478199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4781015a0 0x7ff478199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 -- 172.21.15.137:0/1874267087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff478102790 msgr2=0x7ff478199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff478102790 0x7ff478199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 -- 172.21.15.137:0/1874267087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4781a1630 con 0x7ff478103150 2024-03-26T14:04:45.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.399+0000 7ff476800640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 0x7ff4781a0fc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff47000eaa0 tx=0x7ff47000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.400+0000 7ff46ea00640 1 -- 172.21.15.137:0/1874267087 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47000cdb0 con 0x7ff478103150 2024-03-26T14:04:45.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.400+0000 7ff46ea00640 1 -- 172.21.15.137:0/1874267087 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff470005020 con 0x7ff478103150 2024-03-26T14:04:45.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.400+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4781a1920 con 0x7ff478103150 2024-03-26T14:04:45.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.400+0000 7ff46ea00640 1 -- 172.21.15.137:0/1874267087 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4700116d0 con 0x7ff478103150 2024-03-26T14:04:45.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.400+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4781a1e40 con 0x7ff478103150 2024-03-26T14:04:45.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.403+0000 7ff480005740 1 Processor -- start 2024-03-26T14:04:45.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.403+0000 7ff46ea00640 1 -- 172.21.15.137:0/1874267087 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff4700051d0 con 0x7ff478103150 2024-03-26T14:04:45.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 -- start start 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e7e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c91ddeeb0 con 0x555c91e792b0 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c91d87a60 con 0x555c91e79ca0 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff480005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c91ddaf50 con 0x555c91e7ee10 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.404+0000 7ff475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e796d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48522/0 (socket says 172.21.15.137:48522) 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 -- 172.21.15.137:0/3922688871 learned_addr learned my addr 172.21.15.137:0/3922688871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475e00640 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e7e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 -- 172.21.15.137:0/3922688871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 msgr2=0x555c91e812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e812a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 -- 172.21.15.137:0/3922688871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 msgr2=0x555c91e7e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e7e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 -- 172.21.15.137:0/3922688871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c91bbb7e0 con 0x555c91e792b0 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff46ea00640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff45c0a2880 0x7ff45c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.405+0000 7ff475400640 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e796d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff464005ab0 tx=0x7ff464005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319da635039fe083 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.406+0000 7ff46ea00640 1 -- 172.21.15.137:0/1874267087 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff4700cefa0 con 0x7ff478103150 2024-03-26T14:04:45.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.406+0000 7ff475e00640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff45c0a2880 0x7ff45c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.406+0000 7ff463400640 1 -- 172.21.15.137:0/3922688871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46402cb00 con 0x555c91e792b0 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.406+0000 7ff463400640 1 -- 172.21.15.137:0/3922688871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff464047070 con 0x555c91e792b0 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.406+0000 7ff463400640 1 -- 172.21.15.137:0/3922688871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4640046c0 con 0x555c91e792b0 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 -- 172.21.15.137:0/3922688871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 msgr2=0x555c91e796d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e796d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff464005ab0 tx=0x7ff464005c00 comp rx=0 tx=0).stop 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff475e00640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff45c0a2880 0x7ff45c0a4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x555c91e7a0a0 tx=0x7ff46800b4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 -- 172.21.15.137:0/3922688871 shutdown_connections 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e7e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 --2- 172.21.15.137:0/3922688871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e796d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 -- 172.21.15.137:0/3922688871 >> 172.21.15.137:0/3922688871 conn(0x555c91e74820 msgr2=0x555c91e74c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 -- 172.21.15.137:0/3922688871 shutdown_connections 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.407+0000 7ff480005740 1 -- 172.21.15.137:0/3922688871 wait complete. 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.408+0000 7ff480005740 1 -- start start 2024-03-26T14:04:45.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.408+0000 7ff480005740 1 Processor -- start 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.408+0000 7ff480005740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e792b0 0x555c91e834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.408+0000 7ff480005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e83a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.408+0000 7ff480005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e8aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.408+0000 7ff480005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c91e81d60 con 0x555c91e79ca0 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff480005740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c91d9aeb0 con 0x555c91e792b0 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff480005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c91e81ee0 con 0x555c91e7ee10 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff480005740 10 client.0 _ll_register_callbacks cb 0x555c91d8f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e83a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e83a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48536/0 (socket says 172.21.15.137:48536) 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 -- 172.21.15.137:0/3718615950 learned_addr learned my addr 172.21.15.137:0/3718615950 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475e00640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e792b0 0x555c91e834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff476800640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e8aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 -- 172.21.15.137:0/3718615950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 msgr2=0x555c91e8aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e8aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 -- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e792b0 msgr2=0x555c91e834c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e792b0 0x555c91e834c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.409+0000 7ff475400640 1 -- 172.21.15.137:0/3718615950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c91c1bf70 con 0x555c91e79ca0 2024-03-26T14:04:45.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff475400640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e83a00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff46402c170 tx=0x7ff464005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad4fab6bddf8edb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff457e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:45.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff457e00640 1 -- 172.21.15.137:0/3718615950 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464004240 con 0x555c91e79ca0 2024-03-26T14:04:45.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff457e00640 1 -- 172.21.15.137:0/3718615950 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff4640043e0 con 0x555c91e79ca0 2024-03-26T14:04:45.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff464047070 con 0x555c91e79ca0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff457e00640 1 -- 172.21.15.137:0/3718615950 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff46405d050 con 0x555c91e79ca0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff480005740 10 client.16134 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.410+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c91ab3170 con 0x555c91e79ca0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff457e00640 1 -- 172.21.15.137:0/3718615950 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff464047070 con 0x555c91e79ca0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff457e00640 1 -- 172.21.15.137:0/3718615950 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff464040040 con 0x555c91e79ca0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff480005740 10 client.16134 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff480005740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff480005740 20 client.16134 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff480005740 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91e8ef30 0x555c91e91320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff480005740 4 client.16134 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.411+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c91dec3f0 con 0x555c91e8ef30 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.412+0000 7ff475e00640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91e8ef30 0x555c91e91320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.413+0000 7ff475e00640 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91e8ef30 0x555c91e91320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0fd9fc2f250c27 server_cookie=a59cf488b0834006 in_seq=0 out_seq=0 2024-03-26T14:04:45.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.413+0000 7ff457e00640 10 client.16134 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.414+0000 7ff457e00640 1 -- 172.21.15.137:0/3718615950 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555c91dec3f0 con 0x555c91e8ef30 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.414+0000 7ff457e00640 10 client.16134 handle_command_reply: tid=0 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.414+0000 7ff480005740 1 client.16134 shutdown 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.414+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 msgr2=0x555c91e83a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.414+0000 7ff480005740 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e83a00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff46402c170 tx=0x7ff464005fd0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.414+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 shutdown_connections 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91e8ef30 0x555c91e91320 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e8aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e83a00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 --2- 172.21.15.137:0/3718615950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e792b0 0x555c91e834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 >> 172.21.15.137:0/3718615950 conn(0x555c91e74820 msgr2=0x555c91e73a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff457e00640 0 client.16134 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 shutdown_connections 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 1 -- 172.21.15.137:0/3718615950 wait complete. 2024-03-26T14:04:45.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.415+0000 7ff480005740 20 client.16134 trim_cache size 0 max 16384 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.432+0000 7ff480005740 1 Processor -- start 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.432+0000 7ff480005740 1 -- start start 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff480005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e792b0 0x555c91e78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff480005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e8aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff480005740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e7ee10 0x555c91e7a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff480005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781ae570 con 0x555c91e79ca0 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff480005740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781ac070 con 0x555c91e7ee10 2024-03-26T14:04:45.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff480005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781aa0f0 con 0x555c91e792b0 2024-03-26T14:04:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e8aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e8aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48544/0 (socket says 172.21.15.137:48544) 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff475400640 1 -- 172.21.15.137:0/2092716356 learned_addr learned my addr 172.21.15.137:0/2092716356 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff476800640 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e7ee10 0x555c91e7a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.433+0000 7ff475e00640 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e792b0 0x555c91e78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff475400640 1 -- 172.21.15.137:0/2092716356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e792b0 msgr2=0x555c91e78190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff475400640 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e792b0 0x555c91e78190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff475400640 1 -- 172.21.15.137:0/2092716356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e7ee10 msgr2=0x555c91e7a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff475400640 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e7ee10 0x555c91e7a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff475400640 1 -- 172.21.15.137:0/2092716356 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c91e75590 con 0x555c91e79ca0 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff475400640 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e8aa00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff464005ab0 tx=0x7ff464005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dc0d182e51c52e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff462a00640 1 -- 172.21.15.137:0/2092716356 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46404f020 con 0x555c91e79ca0 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff462a00640 1 -- 172.21.15.137:0/2092716356 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff464047070 con 0x555c91e79ca0 2024-03-26T14:04:45.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.434+0000 7ff462a00640 1 -- 172.21.15.137:0/2092716356 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464042480 con 0x555c91e79ca0 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 -- 172.21.15.137:0/2092716356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 msgr2=0x555c91e8aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e8aa00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff464005ab0 tx=0x7ff464005b50 comp rx=0 tx=0).stop 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 -- 172.21.15.137:0/2092716356 shutdown_connections 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e7ee10 0x555c91e7a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e79ca0 0x555c91e8aa00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 --2- 172.21.15.137:0/2092716356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e792b0 0x555c91e78190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 -- 172.21.15.137:0/2092716356 >> 172.21.15.137:0/2092716356 conn(0x555c91e74820 msgr2=0x555c91e77a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 -- 172.21.15.137:0/2092716356 shutdown_connections 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.435+0000 7ff480005740 1 -- 172.21.15.137:0/2092716356 wait complete. 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 -- start start 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 Processor -- start 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e88560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e89ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781aa6f0 con 0x555c91e792b0 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781aba10 con 0x555c91e79ca0 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4781aad50 con 0x555c91e7ee10 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.436+0000 7ff480005740 10 client.0 _ll_register_callbacks cb 0x7ff46800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e88560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e88560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48554/0 (socket says 172.21.15.137:48554) 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 -- 172.21.15.137:0/1990685595 learned_addr learned my addr 172.21.15.137:0/1990685595 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475400640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff476800640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e89ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 -- 172.21.15.137:0/1990685595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 msgr2=0x555c91e89ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e89ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 -- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 msgr2=0x555c91e88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e88aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 -- 172.21.15.137:0/1990685595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c91e8a0a0 con 0x555c91e792b0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff476800640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e89ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475400640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff475e00640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e88560 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b840 tx=0x7ff4680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0cef193819a8f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.437+0000 7ff457e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff457e00640 1 -- 172.21.15.137:0/1990685595 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468028310 con 0x555c91e792b0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff457e00640 1 -- 172.21.15.137:0/1990685595 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff468028520 con 0x555c91e792b0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff464047070 con 0x555c91e792b0 2024-03-26T14:04:45.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff457e00640 1 -- 172.21.15.137:0/1990685595 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff468038050 con 0x555c91e792b0 2024-03-26T14:04:45.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff480005740 10 client.16140 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:45.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c91eeac40 con 0x555c91e792b0 2024-03-26T14:04:45.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.438+0000 7ff457e00640 1 -- 172.21.15.137:0/1990685595 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff464047070 con 0x555c91e792b0 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff457e00640 1 -- 172.21.15.137:0/1990685595 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff46801f6b0 con 0x555c91e792b0 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff480005740 10 client.16140 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff480005740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff480005740 20 client.16140 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff480005740 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91eeb5d0 0x555c91eeda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff480005740 4 client.16140 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555c91de6740 con 0x555c91eeb5d0 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.439+0000 7ff475400640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91eeb5d0 0x555c91eeda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.440+0000 7ff475400640 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91eeb5d0 0x555c91eeda30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3dd4e219d28f44 server_cookie=13ec8e8b4f52ecea in_seq=0 out_seq=0 2024-03-26T14:04:45.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.440+0000 7ff457e00640 10 client.16140 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff457e00640 1 -- 172.21.15.137:0/1990685595 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555c91de6740 con 0x555c91eeb5d0 2024-03-26T14:04:45.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff457e00640 10 client.16140 handle_command_reply: tid=0 2024-03-26T14:04:45.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 client.16140 shutdown 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 msgr2=0x555c91e88560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e88560 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff46800b840 tx=0x7ff4680046c0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 shutdown_connections 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555c91eeb5d0 0x555c91eeda30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c91e79ca0 0x555c91e88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c91e7ee10 0x555c91e89ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 --2- 172.21.15.137:0/1990685595 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c91e792b0 0x555c91e88560 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 >> 172.21.15.137:0/1990685595 conn(0x555c91e74820 msgr2=0x555c91e78310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.442+0000 7ff457e00640 0 client.16140 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.443+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 shutdown_connections 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.443+0000 7ff480005740 1 -- 172.21.15.137:0/1990685595 wait complete. 2024-03-26T14:04:45.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.443+0000 7ff480005740 20 client.16140 trim_cache size 0 max 16384 2024-03-26T14:04:45.444 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:04:45.444 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:04:45.444 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "c1ac3c30-5316-4e9a-a1fe-1c65da075534", 2024-03-26T14:04:45.444 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:04:45.444 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:04:45.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.443+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff45c0a2880 msgr2=0x7ff45c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.443+0000 7ff477200640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff45c0a2880 0x7ff45c0a4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x555c91e7a0a0 tx=0x7ff46800b4d0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 msgr2=0x7ff4781a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 0x7ff4781a0fc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff47000eaa0 tx=0x7ff47000ef60 comp rx=0 tx=0).stop 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 shutdown_connections 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff45c0a2880 0x7ff45c0a4d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff478102790 0x7ff478199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff478103150 0x7ff4781a0fc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 --2- 172.21.15.137:0/1874267087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff4781015a0 0x7ff478199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 >> 172.21.15.137:0/1874267087 conn(0x7ff4780fcb10 msgr2=0x7ff4780738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 shutdown_connections 2024-03-26T14:04:45.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.444+0000 7ff477200640 1 -- 172.21.15.137:0/1874267087 wait complete. 2024-03-26T14:04:45.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:04:45.460 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3237253121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa4103150 msgr2=0x7f5fa41099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3237253121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa4103150 0x7f5fa41099e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009ca0 tx=0x7f5f9c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3237253121 shutdown_connections 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3237253121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa4102790 0x7f5fa4102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3237253121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa4103150 0x7f5fa41099e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3237253121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa41015a0 0x7f5fa41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3237253121 >> 172.21.15.137:0/3237253121 conn(0x7f5fa40fcb10 msgr2=0x7f5fa40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3237253121 shutdown_connections 2024-03-26T14:04:45.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.659+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3237253121 wait complete. 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.660+0000 7f5fa3e00640 1 Processor -- start 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.660+0000 7f5fa3e00640 1 -- start start 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 0x7f5fa41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa4102790 0x7f5fa4199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa4103150 0x7f5fa41a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa410d0e0 con 0x7f5fa41015a0 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa410cf60 con 0x7f5fa4102790 2024-03-26T14:04:45.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa410d260 con 0x7f5fa4103150 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 0x7f5fa41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 0x7f5fa41999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48564/0 (socket says 172.21.15.137:48564) 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 -- 172.21.15.137:0/3071753602 learned_addr learned my addr 172.21.15.137:0/3071753602 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa4103150 0x7f5fa41a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2000640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa4102790 0x7f5fa4199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 -- 172.21.15.137:0/3071753602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa4103150 msgr2=0x7f5fa41a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa4103150 0x7f5fa41a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 -- 172.21.15.137:0/3071753602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa4102790 msgr2=0x7f5fa4199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.661+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa4102790 0x7f5fa4199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5fa2a00640 1 -- 172.21.15.137:0/3071753602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa41a15f0 con 0x7f5fa41015a0 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa4103150 0x7f5fa41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:04:45.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 0x7f5fa41999d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400d950 tx=0x7f5f9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5f9b400640 1 -- 172.21.15.137:0/3071753602 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94017070 con 0x7f5fa41015a0 2024-03-26T14:04:45.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5f9b400640 1 -- 172.21.15.137:0/3071753602 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f94009c30 con 0x7f5fa41015a0 2024-03-26T14:04:45.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa41a18e0 con 0x7f5fa41015a0 2024-03-26T14:04:45.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5f9b400640 1 -- 172.21.15.137:0/3071753602 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9400f620 con 0x7f5fa41015a0 2024-03-26T14:04:45.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.662+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa41a1e30 con 0x7f5fa41015a0 2024-03-26T14:04:45.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 Processor -- start 2024-03-26T14:04:45.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 -- start start 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d74540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5f9b400640 1 -- 172.21.15.137:0/3071753602 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5f9401a070 con 0x7f5fa41015a0 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d77040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b3cd4250 con 0x55d9b3d6fa40 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b3c7ca30 con 0x55d9b3d6f050 2024-03-26T14:04:45.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.666+0000 7f5fac651740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b3cd02f0 con 0x55d9b3d74bb0 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d74540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:60894/0 (socket says 172.21.15.137:60894) 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa3400640 1 -- 172.21.15.137:0/748834273 learned_addr learned my addr 172.21.15.137:0/748834273 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d74540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48572/0 (socket says 172.21.15.137:48572) 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2000640 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d77040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2a00640 1 -- 172.21.15.137:0/748834273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d74bb0 msgr2=0x55d9b3d77040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d77040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2a00640 1 -- 172.21.15.137:0/748834273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 msgr2=0x55d9b3d6f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.667+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5fa2a00640 1 -- 172.21.15.137:0/748834273 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9b3c67690 con 0x55d9b3d6fa40 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d74540 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5f940cbf60 tx=0x7f5f940d6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844b67a953d9c518 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5f9b400640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5f880a2880 0x7f5f880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5f8fe00640 1 -- 172.21.15.137:0/748834273 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f940dd070 con 0x55d9b3d6fa40 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5f8fe00640 1 -- 172.21.15.137:0/748834273 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f9409b560 con 0x55d9b3d6fa40 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5f8fe00640 1 -- 172.21.15.137:0/748834273 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9409a9a0 con 0x55d9b3d6fa40 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.668+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5f880a2880 0x7f5f880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5f9b400640 1 -- 172.21.15.137:0/3071753602 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5f940ce890 con 0x7f5fa41015a0 2024-03-26T14:04:45.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 -- 172.21.15.137:0/748834273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 msgr2=0x55d9b3d74540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d74540 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5f940cbf60 tx=0x7f5f940d6670 comp rx=0 tx=0).stop 2024-03-26T14:04:45.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5f880a2880 0x7f5f880a4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d9b3d702a0 tx=0x7f5f9c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 -- 172.21.15.137:0/748834273 shutdown_connections 2024-03-26T14:04:45.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d77040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d74540 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 --2- 172.21.15.137:0/748834273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.669+0000 7f5fac651740 1 -- 172.21.15.137:0/748834273 >> 172.21.15.137:0/748834273 conn(0x55d9b3d37200 msgr2=0x55d9b3d6a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.670+0000 7f5fac651740 1 -- 172.21.15.137:0/748834273 shutdown_connections 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.670+0000 7f5fac651740 1 -- 172.21.15.137:0/748834273 wait complete. 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.670+0000 7f5fac651740 1 -- start start 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.670+0000 7f5fac651740 1 Processor -- start 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6f050 0x55d9b3d78fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d80520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b3d77b60 con 0x55d9b3d6fa40 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b3c8fe80 con 0x55d9b3d74bb0 2024-03-26T14:04:45.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9b3d77ce0 con 0x55d9b3d6f050 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fac651740 10 client.0 _ll_register_callbacks cb 0x55d9b3ca2130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fa2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fa2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d794e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48586/0 (socket says 172.21.15.137:48586) 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fa2000640 1 -- 172.21.15.137:0/3107449246 learned_addr learned my addr 172.21.15.137:0/3107449246 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6f050 0x55d9b3d78fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.671+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d80520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2000640 1 -- 172.21.15.137:0/3107449246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6f050 msgr2=0x55d9b3d78fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2000640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6f050 0x55d9b3d78fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2000640 1 -- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 msgr2=0x55d9b3d80520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2000640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d80520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2000640 1 -- 172.21.15.137:0/3107449246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9b3ab27d0 con 0x55d9b3d6fa40 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6f050 0x55d9b3d78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fa2000640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d794e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000da10 tx=0x7f5f9000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ec9f1f03d39278 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5f8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3107449246 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90011640 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3107449246 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5f900117e0 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.672+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f9409b560 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3107449246 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f90003e40 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 10 client.16152 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d9b39e83f0 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3107449246 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f9409b560 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3107449246 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5f90019030 con 0x55d9b3d6fa40 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 10 client.16152 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 20 client.16152 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3d84e70 0x55d9b3d87310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 4 client.16152 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:04:45.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.673+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9b3ce1790 con 0x55d9b3d84e70 2024-03-26T14:04:45.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.674+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3d84e70 0x55d9b3d87310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.675+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3d84e70 0x55d9b3d87310 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd8ab68c05b1747 server_cookie=86ce6b02987092b in_seq=0 out_seq=0 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.675+0000 7f5f8cc00640 10 client.16152 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3107449246 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d9b3ce1790 con 0x55d9b3d84e70 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5f8cc00640 10 client.16152 handle_command_reply: tid=0 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 client.16152 shutdown 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 msgr2=0x55d9b3d794e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d794e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000da10 tx=0x7f5f9000ded0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 shutdown_connections 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3d84e70 0x55d9b3d87310 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d794e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5f8cc00640 0 client.16152 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 --2- 172.21.15.137:0/3107449246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6f050 0x55d9b3d78fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 >> 172.21.15.137:0/3107449246 conn(0x55d9b3d37200 msgr2=0x55d9b3d69bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 shutdown_connections 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.677+0000 7f5fac651740 1 -- 172.21.15.137:0/3107449246 wait complete. 2024-03-26T14:04:45.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.678+0000 7f5fac651740 20 client.16152 trim_cache size 0 max 16384 2024-03-26T14:04:45.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 Processor -- start 2024-03-26T14:04:45.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 -- start start 2024-03-26T14:04:45.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d804e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41ae830 con 0x55d9b3d6f050 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41ac330 con 0x55d9b3d74bb0 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fac651740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41aa3b0 con 0x55d9b3d6fa40 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48598/0 (socket says 172.21.15.137:48598) 2024-03-26T14:04:45.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.701+0000 7f5fa2a00640 1 -- 172.21.15.137:0/1651195228 learned_addr learned my addr 172.21.15.137:0/1651195228 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2000640 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa3400640 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2a00640 1 -- 172.21.15.137:0/1651195228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 msgr2=0x55d9b3d804e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d804e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2a00640 1 -- 172.21.15.137:0/1651195228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 msgr2=0x55d9b3d8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2a00640 1 -- 172.21.15.137:0/1651195228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9b39a80f0 con 0x55d9b3d6f050 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6e310 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5f9409ada0 tx=0x7f5f940d4530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726332c553a97b2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5f8f400640 1 -- 172.21.15.137:0/1651195228 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f94091550 con 0x55d9b3d6f050 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5f8f400640 1 -- 172.21.15.137:0/1651195228 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f94009de0 con 0x55d9b3d6f050 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.702+0000 7f5f8f400640 1 -- 172.21.15.137:0/1651195228 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9408f690 con 0x55d9b3d6f050 2024-03-26T14:04:45.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 -- 172.21.15.137:0/1651195228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 msgr2=0x55d9b3d6e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6e310 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5f9409ada0 tx=0x7f5f940d4530 comp rx=0 tx=0).stop 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 -- 172.21.15.137:0/1651195228 shutdown_connections 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 --2- 172.21.15.137:0/1651195228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d6f050 0x55d9b3d6e310 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 -- 172.21.15.137:0/1651195228 >> 172.21.15.137:0/1651195228 conn(0x55d9b3d37200 msgr2=0x55d9b3d6dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 -- 172.21.15.137:0/1651195228 shutdown_connections 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.703+0000 7f5fac651740 1 -- 172.21.15.137:0/1651195228 wait complete. 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 -- start start 2024-03-26T14:04:45.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 Processor -- start 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d7ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d80450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d7e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41aa9b0 con 0x55d9b3d74bb0 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41abcd0 con 0x55d9b3d6f050 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa41ab010 con 0x55d9b3d6fa40 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.704+0000 7f5fac651740 10 client.0 _ll_register_callbacks cb 0x7f5f90004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d7e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d7e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48600/0 (socket says 172.21.15.137:48600) 2024-03-26T14:04:45.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 -- 172.21.15.137:0/3658857157 learned_addr learned my addr 172.21.15.137:0/3658857157 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d7ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa2000640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d80450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 -- 172.21.15.137:0/3658857157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 msgr2=0x55d9b3d80450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d80450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 -- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 msgr2=0x55d9b3d7ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d7ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 -- 172.21.15.137:0/3658857157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9b3d7ee50 con 0x55d9b3d74bb0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5fa3400640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d7e620 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c032420 tx=0x7f5f9c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b6e520bc48c99 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.705+0000 7f5f8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3658857157 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c052020 con 0x55d9b3d74bb0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f94009de0 con 0x55d9b3d74bb0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3658857157 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5f9c004240 con 0x55d9b3d74bb0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3658857157 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f9c039040 con 0x55d9b3d74bb0 2024-03-26T14:04:45.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5fac651740 10 client.16158 fetch_fsmap learned FSMap version 24 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d9b3de13f0 con 0x55d9b3d74bb0 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3658857157 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f94009de0 con 0x55d9b3d74bb0 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.706+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3658857157 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5f9c04ebf0 con 0x55d9b3d74bb0 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fac651740 10 client.16158 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fac651740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fac651740 20 client.16158 populate_metadata read hostname 'smithi137' 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fac651740 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3de1d80 0x55d9b3de41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fac651740 4 client.16158 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d9b3cdbae0 con 0x55d9b3de1d80 2024-03-26T14:04:45.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.707+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3de1d80 0x55d9b3de41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:04:45.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.708+0000 7f5fa2a00640 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3de1d80 0x55d9b3de41a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9af5e0aea10e5d server_cookie=6e7af16fd917d3f2 in_seq=0 out_seq=0 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.708+0000 7f5f8cc00640 10 client.16158 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.708+0000 7f5f8cc00640 1 -- 172.21.15.137:0/3658857157 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d9b3cdbae0 con 0x55d9b3de1d80 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.708+0000 7f5f8cc00640 10 client.16158 handle_command_reply: tid=0 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.708+0000 7f5fac651740 1 client.16158 shutdown 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 msgr2=0x55d9b3d7e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d7e620 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c032420 tx=0x7f5f9c0095c0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 shutdown_connections 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d9b3de1d80 0x55d9b3de41a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d9b3d74bb0 0x55d9b3d7e620 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9b3d6fa40 0x55d9b3d80450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 --2- 172.21.15.137:0/3658857157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9b3d6f050 0x55d9b3d7ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5f8cc00640 0 client.16158 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 >> 172.21.15.137:0/3658857157 conn(0x55d9b3d37200 msgr2=0x55d9b3d709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 shutdown_connections 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 1 -- 172.21.15.137:0/3658857157 wait complete. 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.709+0000 7f5fac651740 20 client.16158 trim_cache size 0 max 16384 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: "c1ac3c30-5316-4e9a-a1fe-1c65da075534": { 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "c1ac3c30-5316-4e9a-a1fe-1c65da075534", 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:04:45.712 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5f880a2880 msgr2=0x7f5f880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5f880a2880 0x7f5f880a4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d9b3d702a0 tx=0x7f5f9c004a40 comp rx=0 tx=0).stop 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 msgr2=0x7f5fa41999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 0x7f5fa41999d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5f9400d950 tx=0x7f5f9400de10 comp rx=0 tx=0).stop 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 shutdown_connections 2024-03-26T14:04:45.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5f880a2880 0x7f5f880a4d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fa4102790 0x7f5fa4199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fa4103150 0x7f5fa41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 --2- 172.21.15.137:0/3071753602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fa41015a0 0x7f5fa41999d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:04:45.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 >> 172.21.15.137:0/3071753602 conn(0x7f5fa40fcb10 msgr2=0x7f5fa40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:04:45.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 shutdown_connections 2024-03-26T14:04:45.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:04:45.710+0000 7f5fa3e00640 1 -- 172.21.15.137:0/3071753602 wait complete. 2024-03-26T14:04:45.729 INFO:tasks.cephfs.filesystem:scrub status for tag:c1ac3c30-5316-4e9a-a1fe-1c65da075534 - {'path': '/', 'tag': 'c1ac3c30-5316-4e9a-a1fe-1c65da075534', 'options': 'recursive,force'} 2024-03-26T14:04:46.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:46 smithi179 ceph-mon[35472]: pgmap v530: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-03-26T14:04:46.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:46 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:04:46.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:46 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:46.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:46 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:04:46.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:46 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:46.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:46 smithi159 ceph-mon[35501]: pgmap v530: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-03-26T14:04:46.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:46 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:04:46.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:46 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:46.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:46 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:04:46.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:46 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:46.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:46 smithi137 ceph-mon[31539]: pgmap v530: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 596 B/s wr, 5 op/s 2024-03-26T14:04:46.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:46 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:04:46.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:46 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:46.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:46 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:04:46.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:46 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:04:47.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:47 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:04:47.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:47 smithi179 ceph-mon[35472]: pgmap v531: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:04:47.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:47 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:04:47.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:47 smithi159 ceph-mon[35501]: pgmap v531: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:04:47.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:47 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:04:47.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:47 smithi137 ceph-mon[31539]: pgmap v531: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:04:50.172 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:49 smithi179 ceph-mon[35472]: pgmap v532: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 6 op/s 2024-03-26T14:04:50.201 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:49 smithi159 ceph-mon[35501]: pgmap v532: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 6 op/s 2024-03-26T14:04:50.283 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:49 smithi137 ceph-mon[31539]: pgmap v532: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 6 op/s 2024-03-26T14:04:52.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:52 smithi179 ceph-mon[35472]: pgmap v533: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 5 op/s 2024-03-26T14:04:52.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:52 smithi159 ceph-mon[35501]: pgmap v533: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 5 op/s 2024-03-26T14:04:52.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:52 smithi137 ceph-mon[31539]: pgmap v533: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 596 B/s wr, 5 op/s 2024-03-26T14:04:53.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:53 smithi179 ceph-mon[35472]: pgmap v534: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:04:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:53 smithi159 ceph-mon[35501]: pgmap v534: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:04:53.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:53 smithi137 ceph-mon[31539]: pgmap v534: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 7 op/s 2024-03-26T14:04:56.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:56 smithi179 ceph-mon[35472]: pgmap v535: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:04:56.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:56 smithi159 ceph-mon[35501]: pgmap v535: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:04:56.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:56 smithi137 ceph-mon[31539]: pgmap v535: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:04:57.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:57 smithi179 ceph-mon[35472]: pgmap v536: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:04:57.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:57 smithi159 ceph-mon[35501]: pgmap v536: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:04:57.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:57 smithi137 ceph-mon[31539]: pgmap v536: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:04:59.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:04:59 smithi179 ceph-mon[35472]: pgmap v537: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:04:59.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:04:59 smithi159 ceph-mon[35501]: pgmap v537: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:04:59.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:04:59 smithi137 ceph-mon[31539]: pgmap v537: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:01.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:01 smithi179 ceph-mon[35472]: pgmap v538: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:01.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:01 smithi159 ceph-mon[35501]: pgmap v538: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:01.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:01 smithi137 ceph-mon[31539]: pgmap v538: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:03.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:03 smithi179 ceph-mon[35472]: pgmap v539: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:03.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:03 smithi159 ceph-mon[35501]: pgmap v539: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:03.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:03 smithi137 ceph-mon[31539]: pgmap v539: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:05.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:05 smithi179 ceph-mon[35472]: pgmap v540: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:05.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:05 smithi159 ceph-mon[35501]: pgmap v540: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:05.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:05 smithi137 ceph-mon[31539]: pgmap v540: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:07.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:07 smithi179 ceph-mon[35472]: pgmap v541: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:07.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:07 smithi159 ceph-mon[35501]: pgmap v541: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:07.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:07 smithi137 ceph-mon[31539]: pgmap v541: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:09.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:09 smithi179 ceph-mon[35472]: pgmap v542: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:09.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:09 smithi159 ceph-mon[35501]: pgmap v542: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:09.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:09 smithi137 ceph-mon[31539]: pgmap v542: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:11.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:11 smithi179 ceph-mon[35472]: pgmap v543: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:11.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:11 smithi159 ceph-mon[35501]: pgmap v543: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:11.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:11 smithi137 ceph-mon[31539]: pgmap v543: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:13.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:13 smithi179 ceph-mon[35472]: pgmap v544: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:13.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:13 smithi159 ceph-mon[35501]: pgmap v544: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:13.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:13 smithi137 ceph-mon[31539]: pgmap v544: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:15.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:15 smithi179 ceph-mon[35472]: pgmap v545: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:15.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:15 smithi159 ceph-mon[35501]: pgmap v545: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:15.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:05:15.731 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:05:15.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:15 smithi137 ceph-mon[31539]: pgmap v545: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.950+0000 7f00b4200640 1 -- 172.21.15.137:0/3251811003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 msgr2=0x7f00ac1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.950+0000 7f00b4200640 1 --2- 172.21.15.137:0/3251811003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1097e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009ca0 tx=0x7f00a402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.950+0000 7f00b4200640 1 -- 172.21.15.137:0/3251811003 shutdown_connections 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.950+0000 7f00b4200640 1 --2- 172.21.15.137:0/3251811003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00ac104a60 0x7f00ac106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.950+0000 7f00b4200640 1 --2- 172.21.15.137:0/3251811003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1097e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.950+0000 7f00b4200640 1 --2- 172.21.15.137:0/3251811003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00ac102130 0x7f00ac104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.951+0000 7f00b4200640 1 -- 172.21.15.137:0/3251811003 >> 172.21.15.137:0/3251811003 conn(0x7f00ac0fbb40 msgr2=0x7f00ac0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:15.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.951+0000 7f00b4200640 1 -- 172.21.15.137:0/3251811003 shutdown_connections 2024-03-26T14:05:15.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.951+0000 7f00b4200640 1 -- 172.21.15.137:0/3251811003 wait complete. 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 Processor -- start 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 -- start start 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00ac102130 0x7f00ac199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00ac104a60 0x7f00ac19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac10d200 con 0x7f00ac107390 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac10d080 con 0x7f00ac104a60 2024-03-26T14:05:15.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.952+0000 7f00b4200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac10d380 con 0x7f00ac102130 2024-03-26T14:05:15.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b0e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00ac104a60 0x7f00ac19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1a1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32890/0 (socket says 172.21.15.137:32890) 2024-03-26T14:05:15.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 -- 172.21.15.137:0/751473056 learned_addr learned my addr 172.21.15.137:0/751473056 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b1800640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00ac102130 0x7f00ac199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 -- 172.21.15.137:0/751473056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00ac102130 msgr2=0x7f00ac199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00ac102130 0x7f00ac199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 -- 172.21.15.137:0/751473056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00ac104a60 msgr2=0x7f00ac19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00ac104a60 0x7f00ac19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 -- 172.21.15.137:0/751473056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac1a1770 con 0x7f00ac107390 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.953+0000 7f00b2200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1a1100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f00a402c700 tx=0x7f00a40328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.954+0000 7f00aa000640 1 -- 172.21.15.137:0/751473056 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4041020 con 0x7f00ac107390 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.954+0000 7f00aa000640 1 -- 172.21.15.137:0/751473056 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00a4003e20 con 0x7f00ac107390 2024-03-26T14:05:15.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.954+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac1a1a00 con 0x7f00ac107390 2024-03-26T14:05:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.954+0000 7f00aa000640 1 -- 172.21.15.137:0/751473056 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4039040 con 0x7f00ac107390 2024-03-26T14:05:15.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.954+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac1a1f40 con 0x7f00ac107390 2024-03-26T14:05:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.958+0000 7f00b7019740 1 Processor -- start 2024-03-26T14:05:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.958+0000 7f00aa000640 1 -- 172.21.15.137:0/751473056 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f00a4044020 con 0x7f00ac107390 2024-03-26T14:05:15.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.958+0000 7f00b7019740 1 -- start start 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b7019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b7019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091a870 0x559f0091f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b7019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091f9e0 0x559f00921e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b7019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0087f250 con 0x559f00919e80 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b7019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f00827a30 con 0x559f0091a870 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b7019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0087b2f0 con 0x559f0091f9e0 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32900/0 (socket says 172.21.15.137:32900) 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b0e00640 1 -- 172.21.15.137:0/1935183631 learned_addr learned my addr 172.21.15.137:0/1935183631 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b1800640 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091a870 0x559f0091f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b0e00640 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091f9e0 0x559f00921e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.959+0000 7f00b1800640 1 -- 172.21.15.137:0/1935183631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091f9e0 msgr2=0x559f00921e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b1800640 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091f9e0 0x559f00921e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b1800640 1 -- 172.21.15.137:0/1935183631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 msgr2=0x559f0091a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b1800640 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b1800640 1 -- 172.21.15.137:0/1935183631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f00812690 con 0x559f0091a870 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b0e00640 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091f9e0 0x559f00921e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00aa000640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00940a2880 0x7f00940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b1800640 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091a870 0x559f0091f370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00a000bbe0 tx=0x7f00a000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700fd88b8065b604 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00aa000640 1 -- 172.21.15.137:0/751473056 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f00a40f1100 con 0x7f00ac107390 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f00b0e00640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00940a2880 0x7f00940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f009ea00640 1 -- 172.21.15.137:0/1935183631 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0015070 con 0x559f0091a870 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.960+0000 7f009ea00640 1 -- 172.21.15.137:0/1935183631 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00a00054d0 con 0x559f0091a870 2024-03-26T14:05:15.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f009ea00640 1 -- 172.21.15.137:0/1935183631 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0011440 con 0x559f0091a870 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 -- 172.21.15.137:0/1935183631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091a870 msgr2=0x559f0091f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b0e00640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00940a2880 0x7f00940a4d40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559f0091b0d0 tx=0x7f00980099d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091a870 0x559f0091f370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00a000bbe0 tx=0x7f00a000f730 comp rx=0 tx=0).stop 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 -- 172.21.15.137:0/1935183631 shutdown_connections 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091a870 0x559f0091f370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091f9e0 0x559f00921e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 --2- 172.21.15.137:0/1935183631 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.961+0000 7f00b7019740 1 -- 172.21.15.137:0/1935183631 >> 172.21.15.137:0/1935183631 conn(0x559f008e2200 msgr2=0x559f009157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.962+0000 7f00b7019740 1 -- 172.21.15.137:0/1935183631 shutdown_connections 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.962+0000 7f00b7019740 1 -- 172.21.15.137:0/1935183631 wait complete. 2024-03-26T14:05:15.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.962+0000 7f00b7019740 1 -- start start 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.962+0000 7f00b7019740 1 Processor -- start 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f00919e80 0x559f00923e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 0x559f00924360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0092b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f00922990 con 0x559f0091a870 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f0083ae80 con 0x559f0091f9e0 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f00922b10 con 0x559f00919e80 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b7019740 10 client.0 _ll_register_callbacks cb 0x559f0084d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:15.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 0x559f00924360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 0x559f00924360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32908/0 (socket says 172.21.15.137:32908) 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 -- 172.21.15.137:0/4255479618 learned_addr learned my addr 172.21.15.137:0/4255479618 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b2200640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0092b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b1800640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f00919e80 0x559f00923e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 -- 172.21.15.137:0/4255479618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f00919e80 msgr2=0x559f00923e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f00919e80 0x559f00923e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 -- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 msgr2=0x559f0092b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0092b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.963+0000 7f00b0e00640 1 -- 172.21.15.137:0/4255479618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f0065d7d0 con 0x559f0091a870 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.964+0000 7f00b0e00640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 0x559f00924360 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0098004600 tx=0x7f0098004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f88ee434f8324aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.964+0000 7f0093400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.964+0000 7f0093400640 1 -- 172.21.15.137:0/4255479618 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098020630 con 0x559f0091a870 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.964+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00a00054d0 con 0x559f0091a870 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.964+0000 7f0093400640 1 -- 172.21.15.137:0/4255479618 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f00980207d0 con 0x559f0091a870 2024-03-26T14:05:15.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.964+0000 7f0093400640 1 -- 172.21.15.137:0/4255479618 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00980209d0 con 0x559f0091a870 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 10 client.16170 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559f005933f0 con 0x559f0091a870 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f0093400640 1 -- 172.21.15.137:0/4255479618 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00a00054d0 con 0x559f0091a870 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f0093400640 1 -- 172.21.15.137:0/4255479618 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0098028070 con 0x559f0091a870 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 10 client.16170 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 20 client.16170 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0092fe70 0x559f009322d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 4 client.16170 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.965+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f0088c790 con 0x559f0092fe70 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.966+0000 7f00b1800640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0092fe70 0x559f009322d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.966+0000 7f00b1800640 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0092fe70 0x559f009322d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ca8c3a4a519e8 server_cookie=dcc3d614072b90b8 in_seq=0 out_seq=0 2024-03-26T14:05:15.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.966+0000 7f0093400640 10 client.16170 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:15.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f0093400640 1 -- 172.21.15.137:0/4255479618 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f0088c790 con 0x559f0092fe70 2024-03-26T14:05:15.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f0093400640 10 client.16170 handle_command_reply: tid=0 2024-03-26T14:05:15.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 client.16170 shutdown 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 msgr2=0x559f00924360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 0x559f00924360 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f0098004600 tx=0x7f0098004730 comp rx=0 tx=0).stop 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 shutdown_connections 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0092fe70 0x559f009322d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0092b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f0091a870 0x559f00924360 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 --2- 172.21.15.137:0/4255479618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f00919e80 0x559f00923e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f0093400640 0 client.16170 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.968+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 >> 172.21.15.137:0/4255479618 conn(0x559f008e2200 msgr2=0x559f00914a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.969+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 shutdown_connections 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.969+0000 7f00b7019740 1 -- 172.21.15.137:0/4255479618 wait complete. 2024-03-26T14:05:15.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.969+0000 7f00b7019740 20 client.16170 trim_cache size 0 max 16384 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.987+0000 7f00b7019740 1 Processor -- start 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.987+0000 7f00b7019740 1 -- start start 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.987+0000 7f00b7019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.987+0000 7f00b7019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f0092b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b7019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0091b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b7019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1ae8f0 con 0x559f00919e80 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b7019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1ac3f0 con 0x559f0091f9e0 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b7019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1aa470 con 0x559f0091a870 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32910/0 (socket says 172.21.15.137:32910) 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 -- 172.21.15.137:0/1211287444 learned_addr learned my addr 172.21.15.137:0/1211287444 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:15.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b2200640 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0091b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 -- 172.21.15.137:0/1211287444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 msgr2=0x559f0092b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f0092b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 -- 172.21.15.137:0/1211287444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 msgr2=0x559f0091b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0091b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.988+0000 7f00b1800640 1 -- 172.21.15.137:0/1211287444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f005530f0 con 0x559f00919e80 2024-03-26T14:05:15.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.989+0000 7f00b1800640 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091b310 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f00a00057c0 tx=0x7f00a000b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72abd82b15747e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.989+0000 7f009e000640 1 -- 172.21.15.137:0/1211287444 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0015050 con 0x559f00919e80 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.989+0000 7f009e000640 1 -- 172.21.15.137:0/1211287444 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00a000bcd0 con 0x559f00919e80 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.989+0000 7f009e000640 1 -- 172.21.15.137:0/1211287444 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0026e30 con 0x559f00919e80 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 -- 172.21.15.137:0/1211287444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 msgr2=0x559f0091b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091b310 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f00a00057c0 tx=0x7f00a000b4d0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 -- 172.21.15.137:0/1211287444 shutdown_connections 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f0091b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f0092b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 --2- 172.21.15.137:0/1211287444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f0091b310 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 -- 172.21.15.137:0/1211287444 >> 172.21.15.137:0/1211287444 conn(0x559f008e2200 msgr2=0x559f0091edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:15.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 -- 172.21.15.137:0/1211287444 shutdown_connections 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 -- 172.21.15.137:0/1211287444 wait complete. 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.990+0000 7f00b7019740 1 -- start start 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 Processor -- start 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f00928450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f00928990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f009299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1aaa70 con 0x559f00919e80 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1abd90 con 0x559f0091f9e0 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac1ab0d0 con 0x559f0091a870 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b7019740 10 client.0 _ll_register_callbacks cb 0x7f0098009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:15.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b1800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f00928450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b1800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f00928450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32914/0 (socket says 172.21.15.137:32914) 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.991+0000 7f00b1800640 1 -- 172.21.15.137:0/2277059463 learned_addr learned my addr 172.21.15.137:0/2277059463 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b2200640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f009299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b0e00640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f00928990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b1800640 1 -- 172.21.15.137:0/2277059463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 msgr2=0x559f00928990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b1800640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f00928990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b1800640 1 -- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 msgr2=0x559f009299a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b1800640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f009299a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b1800640 1 -- 172.21.15.137:0/2277059463 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f00929f90 con 0x559f00919e80 2024-03-26T14:05:15.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b0e00640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f00928990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b2200640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f009299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f00b1800640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f00928450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005fa0 tx=0x7f00a000bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb0c03d93a55fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f0093400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.992+0000 7f0093400640 1 -- 172.21.15.137:0/2277059463 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0034060 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00a000bcd0 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f0093400640 1 -- 172.21.15.137:0/2277059463 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f00a0026db0 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f0093400640 1 -- 172.21.15.137:0/2277059463 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a0015020 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f00b7019740 10 client.16176 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559f0092b2a0 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f0093400640 1 -- 172.21.15.137:0/2277059463 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f00a000bcd0 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f0093400640 1 -- 172.21.15.137:0/2277059463 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f00a0030b40 con 0x559f00919e80 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f00b7019740 10 client.16176 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:15.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f00b7019740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:15.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.993+0000 7f00b7019740 20 client.16176 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:15.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.994+0000 7f00b7019740 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0098b870 0x559f0098dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:15.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.994+0000 7f00b7019740 4 client.16176 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:05:15.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.994+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559f00886ae0 con 0x559f0098b870 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.994+0000 7f00b0e00640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0098b870 0x559f0098dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f0093400640 10 client.16176 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f00b0e00640 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0098b870 0x559f0098dc90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b379c067aea959b4 server_cookie=4887f1dc0810ce97 in_seq=0 out_seq=0 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f0093400640 1 -- 172.21.15.137:0/2277059463 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559f00886ae0 con 0x559f0098b870 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f0093400640 10 client.16176 handle_command_reply: tid=0 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f00b7019740 1 client.16176 shutdown 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 msgr2=0x559f00928450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.995+0000 7f00b7019740 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f00928450 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005fa0 tx=0x7f00a000bf00 comp rx=0 tx=0).stop 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 shutdown_connections 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559f0098b870 0x559f0098dc90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f0091f9e0 0x559f009299a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f0091a870 0x559f00928990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 --2- 172.21.15.137:0/2277059463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f00919e80 0x559f00928450 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f0093400640 0 client.16176 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 >> 172.21.15.137:0/2277059463 conn(0x559f008e2200 msgr2=0x559f0091b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 shutdown_connections 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 1 -- 172.21.15.137:0/2277059463 wait complete. 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.996+0000 7f00b7019740 20 client.16176 trim_cache size 0 max 16384 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:05:15.998 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:05:15.999 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:05:15.999 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:05:15.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00940a2880 msgr2=0x7f00940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00940a2880 0x7f00940a4d40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559f0091b0d0 tx=0x7f00980099d0 comp rx=0 tx=0).stop 2024-03-26T14:05:15.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 msgr2=0x7f00ac1a1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:15.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1a1100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f00a402c700 tx=0x7f00a40328a0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 shutdown_connections 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f00940a2880 0x7f00940a4d40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00ac104a60 0x7f00ac19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00ac107390 0x7f00ac1a1100 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 --2- 172.21.15.137:0/751473056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f00ac102130 0x7f00ac199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 >> 172.21.15.137:0/751473056 conn(0x7f00ac0fbb40 msgr2=0x7f00ac101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 shutdown_connections 2024-03-26T14:05:16.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:15.997+0000 7f00b4200640 1 -- 172.21.15.137:0/751473056 wait complete. 2024-03-26T14:05:16.013 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:05:16.013 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 -- 172.21.15.137:0/45348216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c108cf0 msgr2=0x7ff30c109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 --2- 172.21.15.137:0/45348216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c108cf0 0x7ff30c109170 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009900 tx=0x7ff2f802bc50 comp rx=0 tx=0).stop 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 -- 172.21.15.137:0/45348216 shutdown_connections 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 --2- 172.21.15.137:0/45348216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff30c1096b0 0x7ff30c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 --2- 172.21.15.137:0/45348216 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c108cf0 0x7ff30c109170 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 --2- 172.21.15.137:0/45348216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 0x7ff30c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 -- 172.21.15.137:0/45348216 >> 172.21.15.137:0/45348216 conn(0x7ff30c076ab0 msgr2=0x7ff30c078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 -- 172.21.15.137:0/45348216 shutdown_connections 2024-03-26T14:05:16.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.225+0000 7ff312400640 1 -- 172.21.15.137:0/45348216 wait complete. 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.226+0000 7ff312400640 1 Processor -- start 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.226+0000 7ff312400640 1 -- start start 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff312400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 0x7ff30c19dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff312400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff30c108cf0 0x7ff30c19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff312400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 0x7ff30c1a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff312400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c113660 con 0x7ff30c1096b0 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff312400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1134e0 con 0x7ff30c107b00 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff312400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30c1137e0 con 0x7ff30c108cf0 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff311a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 0x7ff30c1a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff311a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 0x7ff30c1a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32922/0 (socket says 172.21.15.137:32922) 2024-03-26T14:05:16.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff311a00640 1 -- 172.21.15.137:0/2794196229 learned_addr learned my addr 172.21.15.137:0/2794196229 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff311000640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 0x7ff30c19dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.227+0000 7ff303e00640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff30c108cf0 0x7ff30c19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311a00640 1 -- 172.21.15.137:0/2794196229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff30c108cf0 msgr2=0x7ff30c19e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311a00640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff30c108cf0 0x7ff30c19e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311a00640 1 -- 172.21.15.137:0/2794196229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 msgr2=0x7ff30c19dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311a00640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 0x7ff30c19dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311a00640 1 -- 172.21.15.137:0/2794196229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c1a5900 con 0x7ff30c1096b0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311000640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 0x7ff30c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff311a00640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 0x7ff30c1a5260 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff30400eaa0 tx=0x7ff30400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400cdb0 con 0x7ff30c1096b0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff304005020 con 0x7ff30c1096b0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30c1a5bf0 con 0x7ff30c1096b0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.228+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040116d0 con 0x7ff30c1096b0 2024-03-26T14:05:16.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.229+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30c1a6140 con 0x7ff30c1096b0 2024-03-26T14:05:16.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.230+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30c10ee70 con 0x7ff30c1096b0 2024-03-26T14:05:16.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.232+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff3040051d0 con 0x7ff30c1096b0 2024-03-26T14:05:16.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.234+0000 7ff30a000640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2e80a2880 0x7ff2e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.235+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff3040cf150 con 0x7ff30c1096b0 2024-03-26T14:05:16.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.235+0000 7ff311000640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2e80a2880 0x7ff2e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.236+0000 7ff311000640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2e80a2880 0x7ff2e80a4d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff30c19f1c0 tx=0x7ff2f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.240+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff304016030 con 0x7ff30c1096b0 2024-03-26T14:05:16.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.516+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff30c02d070 con 0x7ff30c1096b0 2024-03-26T14:05:16.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.517+0000 7ff30a000640 1 -- 172.21.15.137:0/2794196229 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7ff30409bde0 con 0x7ff30c1096b0 2024-03-26T14:05:16.516 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:05:16.516 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:05:16.516 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.520+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2e80a2880 msgr2=0x7ff2e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.520+0000 7ff312400640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2e80a2880 0x7ff2e80a4d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff30c19f1c0 tx=0x7ff2f0005f90 comp rx=0 tx=0).stop 2024-03-26T14:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.520+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 msgr2=0x7ff30c1a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.520+0000 7ff312400640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 0x7ff30c1a5260 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff30400eaa0 tx=0x7ff30400ef60 comp rx=0 tx=0).stop 2024-03-26T14:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 shutdown_connections 2024-03-26T14:05:16.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2e80a2880 0x7ff2e80a4d40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff30c1096b0 0x7ff30c1a5260 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff30c108cf0 0x7ff30c19e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 --2- 172.21.15.137:0/2794196229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff30c107b00 0x7ff30c19dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 >> 172.21.15.137:0/2794196229 conn(0x7ff30c076ab0 msgr2=0x7ff30c10c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 shutdown_connections 2024-03-26T14:05:16.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.521+0000 7ff312400640 1 -- 172.21.15.137:0/2794196229 wait complete. 2024-03-26T14:05:16.540 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:05:16.540 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:05:16.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 -- 172.21.15.137:0/3833052242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 msgr2=0x7f2a1c1096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 --2- 172.21.15.137:0/3833052242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1096d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009900 tx=0x7f2a1402bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:16.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 -- 172.21.15.137:0/3833052242 shutdown_connections 2024-03-26T14:05:16.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 --2- 172.21.15.137:0/3833052242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a1c1049b0 0x7f2a1c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 --2- 172.21.15.137:0/3833052242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1096d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 --2- 172.21.15.137:0/3833052242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c102080 0x7f2a1c104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.773+0000 7f2a21a00640 1 -- 172.21.15.137:0/3833052242 >> 172.21.15.137:0/3833052242 conn(0x7f2a1c0fba10 msgr2=0x7f2a1c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.774+0000 7f2a21a00640 1 -- 172.21.15.137:0/3833052242 shutdown_connections 2024-03-26T14:05:16.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.774+0000 7f2a21a00640 1 -- 172.21.15.137:0/3833052242 wait complete. 2024-03-26T14:05:16.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 Processor -- start 2024-03-26T14:05:16.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 -- start start 2024-03-26T14:05:16.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a1c102080 0x7f2a1c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c1049b0 0x7f2a1c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c10d190 con 0x7f2a1c1072e0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c10d010 con 0x7f2a1c102080 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c10d310 con 0x7f2a1c1049b0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.775+0000 7f2a21000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1a0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32942/0 (socket says 172.21.15.137:32942) 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 -- 172.21.15.137:0/2710042547 learned_addr learned my addr 172.21.15.137:0/2710042547 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a1be00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a1c102080 0x7f2a1c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a13400640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c1049b0 0x7f2a1c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 -- 172.21.15.137:0/2710042547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c1049b0 msgr2=0x7f2a1c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c1049b0 0x7f2a1c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 -- 172.21.15.137:0/2710042547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a1c102080 msgr2=0x7f2a1c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a1c102080 0x7f2a1c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 -- 172.21.15.137:0/2710042547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c1a1640 con 0x7f2a1c1072e0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a13400640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c1049b0 0x7f2a1c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.776+0000 7f2a21000640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1a0fd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2a14036040 tx=0x7f2a14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.777+0000 7f2a19600640 1 -- 172.21.15.137:0/2710042547 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14043020 con 0x7f2a1c1072e0 2024-03-26T14:05:16.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.777+0000 7f2a19600640 1 -- 172.21.15.137:0/2710042547 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a14004490 con 0x7f2a1c1072e0 2024-03-26T14:05:16.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.777+0000 7f2a19600640 1 -- 172.21.15.137:0/2710042547 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a14003d10 con 0x7f2a1c1072e0 2024-03-26T14:05:16.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.777+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c1a18d0 con 0x7f2a1c1072e0 2024-03-26T14:05:16.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.777+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c1a1e50 con 0x7f2a1c1072e0 2024-03-26T14:05:16.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.780+0000 7f2a19600640 1 -- 172.21.15.137:0/2710042547 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2a14046020 con 0x7f2a1c1072e0 2024-03-26T14:05:16.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.780+0000 7f2a25c6a740 1 Processor -- start 2024-03-26T14:05:16.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 -- start start 2024-03-26T14:05:16.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382de040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382412c0 con 0x5611382de580 2024-03-26T14:05:16.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611381e9c90 con 0x5611382dc9f0 2024-03-26T14:05:16.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113823d360 con 0x5611382ddbc0 2024-03-26T14:05:16.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32944/0 (socket says 172.21.15.137:32944) 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.781+0000 7f2a13400640 1 -- 172.21.15.137:0/1972945073 learned_addr learned my addr 172.21.15.137:0/1972945073 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382de040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 -- 172.21.15.137:0/1972945073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 msgr2=0x5611382de040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382de040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 -- 172.21.15.137:0/1972945073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 msgr2=0x5611382dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 -- 172.21.15.137:0/1972945073 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56113820e080 con 0x5611382de580 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a19600640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a000a2900 0x7f2a000a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a13400640 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e4e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a040155d0 tx=0x7f2a0400c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b97bb073ec8b92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.782+0000 7f2a19600640 1 -- 172.21.15.137:0/2710042547 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2a140f11d0 con 0x7f2a1c1072e0 2024-03-26T14:05:16.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.783+0000 7f2a1be00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a000a2900 0x7f2a000a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.783+0000 7f2a11600640 1 -- 172.21.15.137:0/1972945073 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0401d070 con 0x5611382de580 2024-03-26T14:05:16.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.783+0000 7f2a11600640 1 -- 172.21.15.137:0/1972945073 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a04018b20 con 0x5611382de580 2024-03-26T14:05:16.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.783+0000 7f2a11600640 1 -- 172.21.15.137:0/1972945073 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400f040 con 0x5611382de580 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1972945073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 msgr2=0x5611382e4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a1be00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a000a2900 0x7f2a000a4dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005fd0 tx=0x7f2a0c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e4e10 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2a040155d0 tx=0x7f2a0400c1f0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1972945073 shutdown_connections 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e4e10 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382de040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1972945073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1972945073 >> 172.21.15.137:0/1972945073 conn(0x5611382d7de0 msgr2=0x5611382d81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1972945073 shutdown_connections 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1972945073 wait complete. 2024-03-26T14:05:16.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.784+0000 7f2a25c6a740 1 -- start start 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 Processor -- start 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382e5590 con 0x5611382ddbc0 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611381fd0d0 con 0x5611382de580 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611382e5710 con 0x5611382dc9f0 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a25c6a740 10 client.0 _ll_register_callbacks cb 0x561138201e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.785+0000 7f2a13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32948/0 (socket says 172.21.15.137:32948) 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 -- 172.21.15.137:0/3383131348 learned_addr learned my addr 172.21.15.137:0/3383131348 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a21000640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a1be00640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 -- 172.21.15.137:0/3383131348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 msgr2=0x5611382e6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382e6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 -- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 msgr2=0x5611382ee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 -- 172.21.15.137:0/3383131348 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561137f553f0 con 0x5611382ddbc0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a1be00640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a13400640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382e7230 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a04006e20 tx=0x7f2a0400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4626ec11b20e801d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.786+0000 7f2a0a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a0a000640 1 -- 172.21.15.137:0/3383131348 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0401d070 con 0x5611382ddbc0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a0a000640 1 -- 172.21.15.137:0/3383131348 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2a04018680 con 0x5611382ddbc0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a04018b20 con 0x5611382ddbc0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a0a000640 1 -- 172.21.15.137:0/3383131348 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a04030050 con 0x5611382ddbc0 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a25c6a740 10 client.16194 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:16.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56113801f7d0 con 0x5611382ddbc0 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.787+0000 7f2a0a000640 1 -- 172.21.15.137:0/3383131348 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a04018b20 con 0x5611382ddbc0 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a0a000640 1 -- 172.21.15.137:0/3383131348 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2a04015050 con 0x5611382ddbc0 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a25c6a740 10 client.16194 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a25c6a740 10 client.16194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a25c6a740 20 client.16194 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5611382f23d0 0x5611382f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a25c6a740 4 client.16194 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56113824e800 con 0x5611382f23d0 2024-03-26T14:05:16.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.788+0000 7f2a1be00640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5611382f23d0 0x5611382f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.789+0000 7f2a0a000640 10 client.16194 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.789+0000 7f2a1be00640 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5611382f23d0 0x5611382f47c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acf313ebe17844d0 server_cookie=c6e817c8c345d5f9 in_seq=0 out_seq=0 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.790+0000 7f2a0a000640 1 -- 172.21.15.137:0/3383131348 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56113824e800 con 0x5611382f23d0 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.790+0000 7f2a0a000640 10 client.16194 handle_command_reply: tid=0 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.790+0000 7f2a25c6a740 1 client.16194 shutdown 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.790+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 msgr2=0x5611382e7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382e7230 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2a04006e20 tx=0x7f2a0400ca70 comp rx=0 tx=0).stop 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 shutdown_connections 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5611382f23d0 0x5611382f47c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382e7230 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/3383131348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382e6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a0a000640 0 client.16194 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 >> 172.21.15.137:0/3383131348 conn(0x5611382d7de0 msgr2=0x5611382d7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 shutdown_connections 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 1 -- 172.21.15.137:0/3383131348 wait complete. 2024-03-26T14:05:16.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.791+0000 7f2a25c6a740 20 client.16194 trim_cache size 0 max 16384 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 Processor -- start 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 -- start start 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382d81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1ae7f0 con 0x5611382de580 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1ac2f0 con 0x5611382dc9f0 2024-03-26T14:05:16.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.808+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1aa370 con 0x5611382ddbc0 2024-03-26T14:05:16.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382d81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382d81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:59860/0 (socket says 172.21.15.137:59860) 2024-03-26T14:05:16.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 -- 172.21.15.137:0/1503292563 learned_addr learned my addr 172.21.15.137:0/1503292563 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a21000640 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a13400640 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 -- 172.21.15.137:0/1503292563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 msgr2=0x5611382ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 -- 172.21.15.137:0/1503292563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 msgr2=0x5611382e0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a1be00640 1 -- 172.21.15.137:0/1503292563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611382dc5f0 con 0x5611382dc9f0 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.809+0000 7f2a13400640 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.810+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382d81c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5611382f4ea0 tx=0x7f2a0c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6062dc3274286b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.810+0000 7f2a10c00640 1 -- 172.21.15.137:0/1503292563 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c010dc0 con 0x5611382dc9f0 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.810+0000 7f2a10c00640 1 -- 172.21.15.137:0/1503292563 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a0c028680 con 0x5611382dc9f0 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.810+0000 7f2a10c00640 1 -- 172.21.15.137:0/1503292563 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c004260 con 0x5611382dc9f0 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1503292563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 msgr2=0x5611382d81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382d81c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5611382f4ea0 tx=0x7f2a0c015000 comp rx=0 tx=0).stop 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1503292563 shutdown_connections 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382de580 0x5611382e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382ddbc0 0x5611382ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1503292563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382dc9f0 0x5611382d81c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1503292563 >> 172.21.15.137:0/1503292563 conn(0x5611382d7de0 msgr2=0x5611382db8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1503292563 shutdown_connections 2024-03-26T14:05:16.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1503292563 wait complete. 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.811+0000 7f2a25c6a740 1 -- start start 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 Processor -- start 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382ea460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1aa970 con 0x5611382ddbc0 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1abc90 con 0x5611382de580 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1aafd0 con 0x5611382dc9f0 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.812+0000 7f2a25c6a740 10 client.0 _ll_register_callbacks cb 0x7f2a0401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32964/0 (socket says 172.21.15.137:32964) 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 -- 172.21.15.137:0/1901339836 learned_addr learned my addr 172.21.15.137:0/1901339836 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a21000640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382ea460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 -- 172.21.15.137:0/1901339836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 msgr2=0x5611382ea460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382ea460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 -- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 msgr2=0x5611382ed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 -- 172.21.15.137:0/1901339836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611382edf40 con 0x5611382ddbc0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a13400640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382ea9a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400ca40 tx=0x7f2a0400c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc188d9d7235c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.813+0000 7f2a0a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a0a000640 1 -- 172.21.15.137:0/1901339836 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0401d040 con 0x5611382ddbc0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a0a000640 1 -- 172.21.15.137:0/1901339836 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2a0400f070 con 0x5611382ddbc0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a0c028680 con 0x5611382ddbc0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a0a000640 1 -- 172.21.15.137:0/1901339836 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a04006e50 con 0x5611382ddbc0 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a25c6a740 10 client.16200 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:16.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56113834e5b0 con 0x5611382ddbc0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a0a000640 1 -- 172.21.15.137:0/1901339836 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a0c028680 con 0x5611382ddbc0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.814+0000 7f2a0a000640 1 -- 172.21.15.137:0/1901339836 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2a04015040 con 0x5611382ddbc0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a25c6a740 10 client.16200 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a25c6a740 10 client.16200 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a25c6a740 20 client.16200 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56113834eef0 0x561138351310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a25c6a740 4 client.16200 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561138248b50 con 0x56113834eef0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.815+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56113834eef0 0x561138351310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a1be00640 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56113834eef0 0x561138351310 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66932e86775bf7d7 server_cookie=f4ec3d702fbe2163 in_seq=0 out_seq=0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a0a000640 10 client.16200 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a0a000640 1 -- 172.21.15.137:0/1901339836 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561138248b50 con 0x56113834eef0 2024-03-26T14:05:16.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a0a000640 10 client.16200 handle_command_reply: tid=0 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a25c6a740 1 client.16200 shutdown 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 msgr2=0x5611382ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.816+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382ea9a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400ca40 tx=0x7f2a0400c470 comp rx=0 tx=0).stop 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 shutdown_connections 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56113834eef0 0x561138351310 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611382de580 0x5611382ed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611382ddbc0 0x5611382ea9a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 --2- 172.21.15.137:0/1901339836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5611382dc9f0 0x5611382ea460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a0a000640 0 client.16200 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 >> 172.21.15.137:0/1901339836 conn(0x5611382d7de0 msgr2=0x5611382db8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 shutdown_connections 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 1 -- 172.21.15.137:0/1901339836 wait complete. 2024-03-26T14:05:16.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.817+0000 7f2a25c6a740 20 client.16200 trim_cache size 0 max 16384 2024-03-26T14:05:16.820 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:05:16.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a000a2900 msgr2=0x7f2a000a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a000a2900 0x7f2a000a4dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005fd0 tx=0x7f2a0c009680 comp rx=0 tx=0).stop 2024-03-26T14:05:16.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 msgr2=0x7f2a1c1a0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1a0fd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2a14036040 tx=0x7f2a14004270 comp rx=0 tx=0).stop 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 shutdown_connections 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2a000a2900 0x7f2a000a4dc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2a1c1072e0 0x7f2a1c1a0fd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a1c1049b0 0x7f2a1c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 --2- 172.21.15.137:0/2710042547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2a1c102080 0x7f2a1c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 >> 172.21.15.137:0/2710042547 conn(0x7f2a1c0fba10 msgr2=0x7f2a1c101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 shutdown_connections 2024-03-26T14:05:16.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:16.818+0000 7f2a21a00640 1 -- 172.21.15.137:0/2710042547 wait complete. 2024-03-26T14:05:16.832 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:16 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2794196229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:05:16.834 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:05:16.834 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:05:16.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:16 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2794196229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:05:16.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:16 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2794196229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 -- 172.21.15.137:0/388070320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d741038c0 msgr2=0x7f7d74103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 --2- 172.21.15.137:0/388070320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d741038c0 0x7f7d74103ce0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009900 tx=0x7f7d6402bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 -- 172.21.15.137:0/388070320 shutdown_connections 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 --2- 172.21.15.137:0/388070320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d74104ab0 0x7f7d74104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 --2- 172.21.15.137:0/388070320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d74105470 0x7f7d7410bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 --2- 172.21.15.137:0/388070320 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d741038c0 0x7f7d74103ce0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 -- 172.21.15.137:0/388070320 >> 172.21.15.137:0/388070320 conn(0x7f7d740fee50 msgr2=0x7f7d74101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 -- 172.21.15.137:0/388070320 shutdown_connections 2024-03-26T14:05:17.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.038+0000 7f7d7b200640 1 -- 172.21.15.137:0/388070320 wait complete. 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.039+0000 7f7d7b200640 1 Processor -- start 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.039+0000 7f7d7b200640 1 -- start start 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d7b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d741038c0 0x7f7d74199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d7b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d74104ab0 0x7f7d7419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d7b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 0x7f7d741a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d7b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7410f440 con 0x7f7d74105470 2024-03-26T14:05:17.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d7b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7410f2c0 con 0x7f7d741038c0 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d7b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7410f5c0 con 0x7f7d74104ab0 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d79200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 0x7f7d741a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d79200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 0x7f7d741a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32982/0 (socket says 172.21.15.137:32982) 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d79200640 1 -- 172.21.15.137:0/2909970186 learned_addr learned my addr 172.21.15.137:0/2909970186 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d741038c0 0x7f7d74199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.040+0000 7f7d73400640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d74104ab0 0x7f7d7419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d79200640 1 -- 172.21.15.137:0/2909970186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d74104ab0 msgr2=0x7f7d7419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d79200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d74104ab0 0x7f7d7419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d79200640 1 -- 172.21.15.137:0/2909970186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d741038c0 msgr2=0x7f7d74199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d79200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d741038c0 0x7f7d74199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d79200640 1 -- 172.21.15.137:0/2909970186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d741a1700 con 0x7f7d74105470 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d741038c0 0x7f7d74199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d79200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 0x7f7d741a1090 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00eaa0 tx=0x7f7d6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d70c00640 1 -- 172.21.15.137:0/2909970186 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00cdb0 con 0x7f7d74105470 2024-03-26T14:05:17.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d70c00640 1 -- 172.21.15.137:0/2909970186 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d6c005020 con 0x7f7d74105470 2024-03-26T14:05:17.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d741a19f0 con 0x7f7d74105470 2024-03-26T14:05:17.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.041+0000 7f7d70c00640 1 -- 172.21.15.137:0/2909970186 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0116d0 con 0x7f7d74105470 2024-03-26T14:05:17.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.042+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d741a1f90 con 0x7f7d74105470 2024-03-26T14:05:17.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.045+0000 7f7d70c00640 1 -- 172.21.15.137:0/2909970186 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7d6c0051d0 con 0x7f7d74105470 2024-03-26T14:05:17.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.045+0000 7f7d7dbfd740 1 Processor -- start 2024-03-26T14:05:17.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.045+0000 7f7d7dbfd740 1 -- start start 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5903c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5904e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c590bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c58682c0 con 0x5618c59053a0 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c5810c90 con 0x5618c5903810 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c5864360 con 0x5618c59049e0 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c590bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c590bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32984/0 (socket says 172.21.15.137:32984) 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d73400640 1 -- 172.21.15.137:0/2975159078 learned_addr learned my addr 172.21.15.137:0/2975159078 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d73400640 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5903c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.046+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5904e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73400640 1 -- 172.21.15.137:0/2975159078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 msgr2=0x5618c5904e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73400640 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5904e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73400640 1 -- 172.21.15.137:0/2975159078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 msgr2=0x5618c5903c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73400640 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5903c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73400640 1 -- 172.21.15.137:0/2975159078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618c5835080 con 0x5618c59053a0 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d70c00640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d580a2880 0x7f7d580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d70c00640 1 -- 172.21.15.137:0/2909970186 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7d6c0cef90 con 0x7f7d74105470 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d580a2880 0x7f7d580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d73400640 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c590bc30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000ce90 tx=0x7f7d6000c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236116a568a7a248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.047+0000 7f7d69600640 1 -- 172.21.15.137:0/2975159078 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60005020 con 0x5618c59053a0 2024-03-26T14:05:17.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.048+0000 7f7d69600640 1 -- 172.21.15.137:0/2975159078 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d600047a0 con 0x5618c59053a0 2024-03-26T14:05:17.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.048+0000 7f7d69600640 1 -- 172.21.15.137:0/2975159078 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000fa40 con 0x5618c59053a0 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.048+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2975159078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 msgr2=0x5618c590bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.048+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d580a2880 0x7f7d580a4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d6402c240 tx=0x7f7d6402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.048+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c590bc30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000ce90 tx=0x7f7d6000c530 comp rx=0 tx=0).stop 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2975159078 shutdown_connections 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c590bc30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5904e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2975159078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5903c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2975159078 >> 172.21.15.137:0/2975159078 conn(0x5618c58fec00 msgr2=0x5618c58ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2975159078 shutdown_connections 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2975159078 wait complete. 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 -- start start 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.049+0000 7f7d7dbfd740 1 Processor -- start 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c590da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59049e0 0x5618c590dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5915000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c590c3b0 con 0x5618c59053a0 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c58240d0 con 0x5618c59049e0 2024-03-26T14:05:17.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618c590c530 con 0x5618c5903810 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d7dbfd740 10 client.0 _ll_register_callbacks cb 0x5618c5828e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d79200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5915000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d79200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5915000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:32994/0 (socket says 172.21.15.137:32994) 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d79200640 1 -- 172.21.15.137:0/1398426458 learned_addr learned my addr 172.21.15.137:0/1398426458 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d73400640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59049e0 0x5618c590dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.050+0000 7f7d73e00640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c590da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d79200640 1 -- 172.21.15.137:0/1398426458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 msgr2=0x5618c590da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d79200640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c590da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d79200640 1 -- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59049e0 msgr2=0x5618c590dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d79200640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59049e0 0x5618c590dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d79200640 1 -- 172.21.15.137:0/1398426458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618c557c3f0 con 0x5618c59053a0 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d73400640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59049e0 0x5618c590dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:17.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d79200640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5915000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c09b4f0 tx=0x7f7d6c09af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be94e05f45dc8ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d5e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d5e000640 1 -- 172.21.15.137:0/1398426458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0b4060 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d5e000640 1 -- 172.21.15.137:0/1398426458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7d6c0b4200 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.051+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d600047a0 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d5e000640 1 -- 172.21.15.137:0/1398426458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6c01f040 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 10 client.16212 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5618c56467d0 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d5e000640 1 -- 172.21.15.137:0/1398426458 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d600047a0 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d5e000640 1 -- 172.21.15.137:0/1398426458 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7d6c00f040 con 0x5618c59053a0 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 10 client.16212 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 10 client.16212 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:05:17.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 20 client.16212 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5919590 0x5618c591b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 4 client.16212 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.052+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618c5875800 con 0x5618c5919590 2024-03-26T14:05:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.053+0000 7f7d73e00640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5919590 0x5618c591b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.053+0000 7f7d73e00640 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5919590 0x5618c591b9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11d351e5c9feb26c server_cookie=6995f915c3d1b543 in_seq=0 out_seq=0 2024-03-26T14:05:17.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.053+0000 7f7d5e000640 10 client.16212 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:17.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d5e000640 1 -- 172.21.15.137:0/1398426458 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5618c5875800 con 0x5618c5919590 2024-03-26T14:05:17.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d5e000640 10 client.16212 handle_command_reply: tid=0 2024-03-26T14:05:17.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 client.16212 shutdown 2024-03-26T14:05:17.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 msgr2=0x5618c5915000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5915000 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c09b4f0 tx=0x7f7d6c09af80 comp rx=0 tx=0).stop 2024-03-26T14:05:17.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 shutdown_connections 2024-03-26T14:05:17.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5919590 0x5618c591b9d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59049e0 0x5618c590dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5915000 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/1398426458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c590da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 >> 172.21.15.137:0/1398426458 conn(0x5618c58fec00 msgr2=0x5618c58fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.055+0000 7f7d5e000640 0 client.16212 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:17.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.056+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 shutdown_connections 2024-03-26T14:05:17.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.056+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/1398426458 wait complete. 2024-03-26T14:05:17.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.056+0000 7f7d7dbfd740 20 client.16212 trim_cache size 0 max 16384 2024-03-26T14:05:17.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.075+0000 7f7d7dbfd740 1 Processor -- start 2024-03-26T14:05:17.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.075+0000 7f7d7dbfd740 1 -- start start 2024-03-26T14:05:17.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c58fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 0x5618c5914fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59053a0 0x5618c58fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ae9f0 con 0x5618c59049e0 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ac4f0 con 0x5618c59053a0 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741aa570 con 0x5618c5903810 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 0x5618c5914fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 0x5618c5914fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33010/0 (socket says 172.21.15.137:33010) 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d73400640 1 -- 172.21.15.137:0/4063694643 learned_addr learned my addr 172.21.15.137:0/4063694643 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d79200640 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59053a0 0x5618c58fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.076+0000 7f7d73e00640 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c58fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73400640 1 -- 172.21.15.137:0/4063694643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 msgr2=0x5618c58fd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73400640 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c58fd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73400640 1 -- 172.21.15.137:0/4063694643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59053a0 msgr2=0x5618c58fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73400640 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59053a0 0x5618c58fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73400640 1 -- 172.21.15.137:0/4063694643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618c58ff670 con 0x5618c59049e0 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73e00640 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c58fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d73400640 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 0x5618c5914fc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7d60013700 tx=0x7f7d60005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c62fa573adce2e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d68c00640 1 -- 172.21.15.137:0/4063694643 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000c350 con 0x5618c59049e0 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d68c00640 1 -- 172.21.15.137:0/4063694643 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d60005070 con 0x5618c59049e0 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.077+0000 7f7d68c00640 1 -- 172.21.15.137:0/4063694643 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6000f930 con 0x5618c59049e0 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/4063694643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 msgr2=0x5618c5914fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 0x5618c5914fc0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7d60013700 tx=0x7f7d60005a70 comp rx=0 tx=0).stop 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/4063694643 shutdown_connections 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c59053a0 0x5618c58fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59049e0 0x5618c5914fc0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/4063694643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c5903810 0x5618c58fd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/4063694643 >> 172.21.15.137:0/4063694643 conn(0x5618c58fec00 msgr2=0x5618c590da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/4063694643 shutdown_connections 2024-03-26T14:05:17.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.078+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/4063694643 wait complete. 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 -- start start 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 Processor -- start 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5910810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5910d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5913d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741aab70 con 0x5618c59053a0 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741abe90 con 0x5618c5903810 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d741ab1d0 con 0x5618c59049e0 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.079+0000 7f7d7dbfd740 10 client.0 _ll_register_callbacks cb 0x7f7d6c009270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5913d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5913d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33024/0 (socket says 172.21.15.137:33024) 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 -- 172.21.15.137:0/2155696141 learned_addr learned my addr 172.21.15.137:0/2155696141 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5910810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d73400640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5910d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 -- 172.21.15.137:0/2155696141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 msgr2=0x5618c5910d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5910d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 -- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 msgr2=0x5618c5910810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5910810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.080+0000 7f7d79200640 1 -- 172.21.15.137:0/2155696141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618c5914360 con 0x5618c59053a0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d79200640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5913d70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c09f190 tx=0x7f7d6c09aff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafcae315a307bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d5e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d5e000640 1 -- 172.21.15.137:0/2155696141 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c09abb0 con 0x5618c59053a0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d5e000640 1 -- 172.21.15.137:0/2155696141 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7d6c084af0 con 0x5618c59053a0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d60005070 con 0x5618c59053a0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d5e000640 1 -- 172.21.15.137:0/2155696141 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6c09f6b0 con 0x5618c59053a0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d7dbfd740 10 client.16218 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5618c5974f20 con 0x5618c59053a0 2024-03-26T14:05:17.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.081+0000 7f7d5e000640 1 -- 172.21.15.137:0/2155696141 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d60005070 con 0x5618c59053a0 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d5e000640 1 -- 172.21.15.137:0/2155696141 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7d6c09a420 con 0x5618c59053a0 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d7dbfd740 10 client.16218 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d7dbfd740 10 client.16218 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d7dbfd740 20 client.16218 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5975790 0x5618c5977c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d7dbfd740 4 client.16218 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5618c586fb50 con 0x5618c5975790 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.082+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5975790 0x5618c5977c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.083+0000 7f7d73e00640 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5975790 0x5618c5977c50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4624735b6d92797f server_cookie=e36c432223a9893e in_seq=0 out_seq=0 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.083+0000 7f7d5e000640 10 client.16218 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d5e000640 1 -- 172.21.15.137:0/2155696141 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5618c586fb50 con 0x5618c5975790 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d5e000640 10 client.16218 handle_command_reply: tid=0 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 client.16218 shutdown 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 msgr2=0x5618c5913d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5913d70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c09f190 tx=0x7f7d6c09aff0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 shutdown_connections 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x5618c5975790 0x5618c5977c50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618c59053a0 0x5618c5913d70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618c59049e0 0x5618c5910d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 --2- 172.21.15.137:0/2155696141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618c5903810 0x5618c5910810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 >> 172.21.15.137:0/2155696141 conn(0x5618c58fec00 msgr2=0x5618c5901e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d5e000640 0 client.16218 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 shutdown_connections 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 1 -- 172.21.15.137:0/2155696141 wait complete. 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.084+0000 7f7d7dbfd740 20 client.16218 trim_cache size 0 max 16384 2024-03-26T14:05:17.089 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:05:17.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d580a2880 msgr2=0x7f7d580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d580a2880 0x7f7d580a4d40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d6402c240 tx=0x7f7d6402c7e0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 msgr2=0x7f7d741a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 0x7f7d741a1090 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00eaa0 tx=0x7f7d6c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 shutdown_connections 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d580a2880 0x7f7d580a4d40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d74105470 0x7f7d741a1090 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d74104ab0 0x7f7d7419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 --2- 172.21.15.137:0/2909970186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d741038c0 0x7f7d74199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 >> 172.21.15.137:0/2909970186 conn(0x7f7d740fee50 msgr2=0x7f7d74107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 shutdown_connections 2024-03-26T14:05:17.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.085+0000 7f7d7b200640 1 -- 172.21.15.137:0/2909970186 wait complete. 2024-03-26T14:05:17.101 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:05:17.101 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:05:17.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 -- 172.21.15.137:0/4182875062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a681015a0 msgr2=0x7f1a681019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 --2- 172.21.15.137:0/4182875062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a681015a0 0x7f1a681019c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009900 tx=0x7f1a5802bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 -- 172.21.15.137:0/4182875062 shutdown_connections 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 --2- 172.21.15.137:0/4182875062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a68102790 0x7f1a68102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 --2- 172.21.15.137:0/4182875062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a68103150 0x7f1a681099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 --2- 172.21.15.137:0/4182875062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a681015a0 0x7f1a681019c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 -- 172.21.15.137:0/4182875062 >> 172.21.15.137:0/4182875062 conn(0x7f1a680fcb10 msgr2=0x7f1a680fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 -- 172.21.15.137:0/4182875062 shutdown_connections 2024-03-26T14:05:17.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.303+0000 7f1a67200640 1 -- 172.21.15.137:0/4182875062 wait complete. 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.304+0000 7f1a67200640 1 Processor -- start 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.304+0000 7f1a67200640 1 -- start start 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a67200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a681015a0 0x7f1a68075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a67200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 0x7f1a68073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a67200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a68103150 0x7f1a68073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a67200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6810d0e0 con 0x7f1a68102790 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a67200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6810cf60 con 0x7f1a68103150 2024-03-26T14:05:17.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a67200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6810d260 con 0x7f1a681015a0 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 0x7f1a68073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 0x7f1a68073a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33032/0 (socket says 172.21.15.137:33032) 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 -- 172.21.15.137:0/3462647823 learned_addr learned my addr 172.21.15.137:0/3462647823 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65e00640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a681015a0 0x7f1a68075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a66800640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a68103150 0x7f1a68073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 -- 172.21.15.137:0/3462647823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a681015a0 msgr2=0x7f1a68075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a681015a0 0x7f1a68075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 -- 172.21.15.137:0/3462647823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a68103150 msgr2=0x7f1a68073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a68103150 0x7f1a68073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.305+0000 7f1a65400640 1 -- 172.21.15.137:0/3462647823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68074860 con 0x7f1a68102790 2024-03-26T14:05:17.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.306+0000 7f1a65400640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 0x7f1a68073a90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400e9f0 tx=0x7f1a5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.306+0000 7f1a5ea00640 1 -- 172.21.15.137:0/3462647823 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400cd80 con 0x7f1a68102790 2024-03-26T14:05:17.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.306+0000 7f1a5ea00640 1 -- 172.21.15.137:0/3462647823 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a540108e0 con 0x7f1a68102790 2024-03-26T14:05:17.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.306+0000 7f1a5ea00640 1 -- 172.21.15.137:0/3462647823 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54018920 con 0x7f1a68102790 2024-03-26T14:05:17.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.306+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a681a18e0 con 0x7f1a68102790 2024-03-26T14:05:17.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.306+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a681a1de0 con 0x7f1a68102790 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 Processor -- start 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 -- start start 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd27bc0 0x5636ccd28040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd2ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ccc8b2c0 con 0x5636ccd28580 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ccc33c90 con 0x5636ccd27bc0 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ccc87360 con 0x5636ccd269f0 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.310+0000 7f1a5ea00640 1 -- 172.21.15.137:0/3462647823 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1a54010a90 con 0x7f1a68102790 2024-03-26T14:05:17.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd2ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd27bc0 0x5636ccd28040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd2ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33044/0 (socket says 172.21.15.137:33044) 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 -- 172.21.15.137:0/3784050225 learned_addr learned my addr 172.21.15.137:0/3784050225 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a66800640 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 -- 172.21.15.137:0/3784050225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 msgr2=0x5636ccd26e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd26e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 -- 172.21.15.137:0/3784050225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd27bc0 msgr2=0x5636ccd28040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd27bc0 0x5636ccd28040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.311+0000 7f1a65400640 1 -- 172.21.15.137:0/3784050225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ccc58080 con 0x5636ccd28580 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.312+0000 7f1a65400640 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd2ee10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1a540d7460 tx=0x7f1a540d7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626a1bbbbedea3e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.312+0000 7f1a5ea00640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1a4c0a2800 0x7f1a4c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.312+0000 7f1a5ea00640 1 -- 172.21.15.137:0/3462647823 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1a540cf2a0 con 0x7f1a68102790 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.312+0000 7f1a53400640 1 -- 172.21.15.137:0/3784050225 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540e2070 con 0x5636ccd28580 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.313+0000 7f1a53400640 1 -- 172.21.15.137:0/3784050225 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a54010de0 con 0x5636ccd28580 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.313+0000 7f1a53400640 1 -- 172.21.15.137:0/3784050225 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5409bf70 con 0x5636ccd28580 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.313+0000 7f1a66800640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1a4c0a2800 0x7f1a4c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.313+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3784050225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 msgr2=0x5636ccd2ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.313+0000 7f1a66800640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1a4c0a2800 0x7f1a4c0a4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5636ccd27ac0 tx=0x7f1a60009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.313+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd2ee10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1a540d7460 tx=0x7f1a540d7920 comp rx=0 tx=0).stop 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3784050225 shutdown_connections 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd27bc0 0x5636ccd28040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd2ee10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3784050225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3784050225 >> 172.21.15.137:0/3784050225 conn(0x5636ccd21de0 msgr2=0x5636ccd221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3784050225 shutdown_connections 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3784050225 wait complete. 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.314+0000 7f1a6fb77740 1 -- start start 2024-03-26T14:05:17.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 Processor -- start 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd30c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd31190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd28580 0x5636ccd381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ccd2f590 con 0x5636ccd27bc0 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ccc470d0 con 0x5636ccd269f0 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636ccd2f710 con 0x5636ccd28580 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a6fb77740 10 client.0 _ll_register_callbacks cb 0x5636ccc4be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:17.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd31190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd31190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33052/0 (socket says 172.21.15.137:33052) 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a65400640 1 -- 172.21.15.137:0/1900176139 learned_addr learned my addr 172.21.15.137:0/1900176139 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.315+0000 7f1a65e00640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd30c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a66800640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd28580 0x5636ccd381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a65400640 1 -- 172.21.15.137:0/1900176139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd28580 msgr2=0x5636ccd381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a65400640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd28580 0x5636ccd381d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a65400640 1 -- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 msgr2=0x5636ccd30c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a65400640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd30c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a65400640 1 -- 172.21.15.137:0/1900176139 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636cc99f3f0 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a65400640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd31190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1a540a3ae0 tx=0x7f1a540d2af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4285f3af466c486b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a47e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a47e00640 1 -- 172.21.15.137:0/1900176139 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54097b10 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a47e00640 1 -- 172.21.15.137:0/1900176139 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1a54099b70 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.316+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636ccc98800 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a47e00640 1 -- 172.21.15.137:0/1900176139 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a5409bf70 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 10 client.16230 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5636cca697d0 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a47e00640 1 -- 172.21.15.137:0/1900176139 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5636ccc98800 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a47e00640 1 -- 172.21.15.137:0/1900176139 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1a540860f0 con 0x5636ccd27bc0 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 10 client.16230 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 20 client.16230 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd3c330 0x5636ccd3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 4 client.16230 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:17.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.317+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636ccc92b50 con 0x5636ccd3c330 2024-03-26T14:05:17.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.318+0000 7f1a65e00640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd3c330 0x5636ccd3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.319+0000 7f1a47e00640 10 client.16230 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:17.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.319+0000 7f1a65e00640 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd3c330 0x5636ccd3e720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62fdc443f8cb3571 server_cookie=ee3d36db30c7b2d0 in_seq=0 out_seq=0 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a47e00640 1 -- 172.21.15.137:0/1900176139 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5636ccc92b50 con 0x5636ccd3c330 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a47e00640 10 client.16230 handle_command_reply: tid=0 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 client.16230 shutdown 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 msgr2=0x5636ccd31190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd31190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1a540a3ae0 tx=0x7f1a540d2af0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 shutdown_connections 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd3c330 0x5636ccd3e720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd28580 0x5636ccd381d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd31190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/1900176139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd30c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a47e00640 0 client.16230 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.320+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 >> 172.21.15.137:0/1900176139 conn(0x5636ccd21de0 msgr2=0x5636ccd21160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.321+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 shutdown_connections 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.321+0000 7f1a6fb77740 1 -- 172.21.15.137:0/1900176139 wait complete. 2024-03-26T14:05:17.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.321+0000 7f1a6fb77740 20 client.16230 trim_cache size 0 max 16384 2024-03-26T14:05:17.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.335+0000 7f1a6fb77740 1 Processor -- start 2024-03-26T14:05:17.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.335+0000 7f1a6fb77740 1 -- start start 2024-03-26T14:05:17.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.335+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.335+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd27bc0 0x5636ccd38190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681ae820 con 0x5636ccd28580 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681ac320 con 0x5636ccd269f0 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681aa3a0 con 0x5636ccd27bc0 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd20e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33068/0 (socket says 172.21.15.137:33068) 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 -- 172.21.15.137:0/2631093961 learned_addr learned my addr 172.21.15.137:0/2631093961 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a65e00640 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a65400640 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd27bc0 0x5636ccd38190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 -- 172.21.15.137:0/2631093961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd27bc0 msgr2=0x5636ccd38190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd27bc0 0x5636ccd38190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 -- 172.21.15.137:0/2631093961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 msgr2=0x5636ccd208d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd208d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a66800640 1 -- 172.21.15.137:0/2631093961 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636ccd2e780 con 0x5636ccd28580 2024-03-26T14:05:17.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.336+0000 7f1a65e00640 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:17.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.337+0000 7f1a66800640 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd20e10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009fd0 tx=0x7f1a60015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2838f078c3b86ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.337+0000 7f1a52a00640 1 -- 172.21.15.137:0/2631093961 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6001fda0 con 0x5636ccd28580 2024-03-26T14:05:17.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.337+0000 7f1a52a00640 1 -- 172.21.15.137:0/2631093961 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a6001d930 con 0x5636ccd28580 2024-03-26T14:05:17.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.337+0000 7f1a52a00640 1 -- 172.21.15.137:0/2631093961 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60027aa0 con 0x5636ccd28580 2024-03-26T14:05:17.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 -- 172.21.15.137:0/2631093961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 msgr2=0x5636ccd20e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd20e10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009fd0 tx=0x7f1a60015000 comp rx=0 tx=0).stop 2024-03-26T14:05:17.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 -- 172.21.15.137:0/2631093961 shutdown_connections 2024-03-26T14:05:17.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd28580 0x5636ccd20e10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd27bc0 0x5636ccd38190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/2631093961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd269f0 0x5636ccd208d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 -- 172.21.15.137:0/2631093961 >> 172.21.15.137:0/2631093961 conn(0x5636ccd21de0 msgr2=0x5636ccd30c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 -- 172.21.15.137:0/2631093961 shutdown_connections 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.338+0000 7f1a6fb77740 1 -- 172.21.15.137:0/2631093961 wait complete. 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.339+0000 7f1a6fb77740 1 -- start start 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.339+0000 7f1a6fb77740 1 Processor -- start 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd33bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd34110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd28580 0x5636ccd370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681aa9a0 con 0x5636ccd27bc0 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681abcc0 con 0x5636ccd28580 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681ab000 con 0x5636ccd269f0 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a6fb77740 10 client.0 _ll_register_callbacks cb 0x7f1a540e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd34110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a65400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd34110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33070/0 (socket says 172.21.15.137:33070) 2024-03-26T14:05:17.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a65400640 1 -- 172.21.15.137:0/3946159974 learned_addr learned my addr 172.21.15.137:0/3946159974 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a66800640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd28580 0x5636ccd370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a65e00640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd33bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a65400640 1 -- 172.21.15.137:0/3946159974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 msgr2=0x5636ccd33bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.340+0000 7f1a65400640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd33bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a65400640 1 -- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd28580 msgr2=0x5636ccd370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a65400640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd28580 0x5636ccd370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a65400640 1 -- 172.21.15.137:0/3946159974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636ccd376d0 con 0x5636ccd27bc0 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a66800640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd28580 0x5636ccd370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a65400640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd34110 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c6a0 tx=0x7f1a5400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae1f98f08080637 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a47e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:17.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a47e00640 1 -- 172.21.15.137:0/3946159974 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540d6070 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a47e00640 1 -- 172.21.15.137:0/3946159974 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1a540d2850 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a6001d930 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a47e00640 1 -- 172.21.15.137:0/3946159974 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a54097b10 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a6fb77740 10 client.16236 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.341+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5636ccd97b90 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a47e00640 1 -- 172.21.15.137:0/3946159974 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1a6001d930 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a47e00640 1 -- 172.21.15.137:0/3946159974 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1a5409bad0 con 0x5636ccd27bc0 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a6fb77740 10 client.16236 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a6fb77740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a6fb77740 20 client.16236 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd981c0 0x5636ccd9a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a6fb77740 4 client.16236 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:05:17.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.342+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636ccc10500 con 0x5636ccd981c0 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.343+0000 7f1a65e00640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd981c0 0x5636ccd9a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.343+0000 7f1a65e00640 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd981c0 0x5636ccd9a5b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c67d49ff7d30db1 server_cookie=f141a0a95232f71a in_seq=0 out_seq=0 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.343+0000 7f1a47e00640 10 client.16236 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a47e00640 1 -- 172.21.15.137:0/3946159974 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636ccc10500 con 0x5636ccd981c0 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a47e00640 10 client.16236 handle_command_reply: tid=0 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 client.16236 shutdown 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 msgr2=0x5636ccd34110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd34110 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c6a0 tx=0x7f1a5400c7d0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 shutdown_connections 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5636ccd981c0 0x5636ccd9a5b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636ccd28580 0x5636ccd370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5636ccd27bc0 0x5636ccd34110 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 --2- 172.21.15.137:0/3946159974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636ccd269f0 0x5636ccd33bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a47e00640 0 client.16236 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 >> 172.21.15.137:0/3946159974 conn(0x5636ccd21de0 msgr2=0x5636ccd25040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 shutdown_connections 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 1 -- 172.21.15.137:0/3946159974 wait complete. 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.344+0000 7f1a6fb77740 20 client.16236 trim_cache size 0 max 16384 2024-03-26T14:05:17.346 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1a4c0a2800 msgr2=0x7f1a4c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1a4c0a2800 0x7f1a4c0a4cc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x5636ccd27ac0 tx=0x7f1a60009740 comp rx=0 tx=0).stop 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 msgr2=0x7f1a68073a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 0x7f1a68073a90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400e9f0 tx=0x7f1a5400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 shutdown_connections 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1a4c0a2800 0x7f1a4c0a4cc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a68103150 0x7f1a68073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1a68102790 0x7f1a68073a90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 --2- 172.21.15.137:0/3462647823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a681015a0 0x7f1a68075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.345+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 >> 172.21.15.137:0/3462647823 conn(0x7f1a680fcb10 msgr2=0x7f1a68105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.346+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 shutdown_connections 2024-03-26T14:05:17.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:17.346+0000 7f1a67200640 1 -- 172.21.15.137:0/3462647823 wait complete. 2024-03-26T14:05:17.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:17 smithi137 ceph-mon[31539]: pgmap v546: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:17.922 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:17 smithi179 ceph-mon[35472]: pgmap v546: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:17.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:17 smithi159 ceph-mon[35501]: pgmap v546: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:18.364 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:05:18.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:05:18.364 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:05:18.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.626+0000 7fbfa1a00640 1 -- 172.21.15.137:0/1361358774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c075660 msgr2=0x7fbf9c073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.626+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/1361358774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c075660 0x7fbf9c073ab0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009900 tx=0x7fbf8c02bcc0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 -- 172.21.15.137:0/1361358774 shutdown_connections 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/1361358774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 0x7fbf9c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/1361358774 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c074080 0x7fbf9c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/1361358774 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c075660 0x7fbf9c073ab0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 -- 172.21.15.137:0/1361358774 >> 172.21.15.137:0/1361358774 conn(0x7fbf9c0fcc40 msgr2=0x7fbf9c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 -- 172.21.15.137:0/1361358774 shutdown_connections 2024-03-26T14:05:18.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.627+0000 7fbfa1a00640 1 -- 172.21.15.137:0/1361358774 wait complete. 2024-03-26T14:05:18.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.628+0000 7fbfa1a00640 1 Processor -- start 2024-03-26T14:05:18.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.628+0000 7fbfa1a00640 1 -- start start 2024-03-26T14:05:18.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbfa1a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 0x7fbf9c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbfa1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c075660 0x7fbf9c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbfa1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 0x7fbf9c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbfa1a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c10f410 con 0x7fbf9c074080 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbfa1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c10f290 con 0x7fbf9c109560 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbfa1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c10f590 con 0x7fbf9c075660 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbf9aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 0x7fbf9c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbf9aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 0x7fbf9c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41924/0 (socket says 172.21.15.137:41924) 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.629+0000 7fbf9aa00640 1 -- 172.21.15.137:0/3832161940 learned_addr learned my addr 172.21.15.137:0/3832161940 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9a000640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c075660 0x7fbf9c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9aa00640 1 -- 172.21.15.137:0/3832161940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c075660 msgr2=0x7fbf9c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c075660 0x7fbf9c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9aa00640 1 -- 172.21.15.137:0/3832161940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 msgr2=0x7fbf9c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 0x7fbf9c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 0x7fbf9c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9aa00640 1 -- 172.21.15.137:0/3832161940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c1a1750 con 0x7fbf9c074080 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9a000640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c075660 0x7fbf9c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:18.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 0x7fbf9c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:18.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 0x7fbf9c199b30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000c00 tx=0x7fbf8c02dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.630+0000 7fbf93400640 1 -- 172.21.15.137:0/3832161940 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c043020 con 0x7fbf9c074080 2024-03-26T14:05:18.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.631+0000 7fbf93400640 1 -- 172.21.15.137:0/3832161940 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf8c02de20 con 0x7fbf9c074080 2024-03-26T14:05:18.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.631+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c1a19e0 con 0x7fbf9c074080 2024-03-26T14:05:18.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.631+0000 7fbf93400640 1 -- 172.21.15.137:0/3832161940 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c003c30 con 0x7fbf9c074080 2024-03-26T14:05:18.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.631+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c1a1f30 con 0x7fbf9c074080 2024-03-26T14:05:18.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.634+0000 7fbfa47b2740 1 Processor -- start 2024-03-26T14:05:18.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 -- start start 2024-03-26T14:05:18.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a30b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a30e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937a26beb0 con 0x55937a30bc00 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937a214a60 con 0x55937a306a90 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937a267f50 con 0x55937a3060a0 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a30e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a30e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41932/0 (socket says 172.21.15.137:41932) 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbf9a000640 1 -- 172.21.15.137:0/1926012903 learned_addr learned my addr 172.21.15.137:0/1926012903 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.635+0000 7fbf9b400640 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf9a000640 1 -- 172.21.15.137:0/1926012903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 msgr2=0x55937a3064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf9a000640 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf9a000640 1 -- 172.21.15.137:0/1926012903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 msgr2=0x55937a30b590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf9a000640 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a30b590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf9a000640 1 -- 172.21.15.137:0/1926012903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55937a0487e0 con 0x55937a30bc00 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf93400640 1 -- 172.21.15.137:0/3832161940 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbf8c046020 con 0x7fbf9c074080 2024-03-26T14:05:18.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf9a000640 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a30e090 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009c20 tx=0x7fbf8800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3397df0aa10990c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.636+0000 7fbf87e00640 1 -- 172.21.15.137:0/1926012903 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88016070 con 0x55937a30bc00 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.637+0000 7fbf87e00640 1 -- 172.21.15.137:0/1926012903 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf88011920 con 0x55937a30bc00 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.637+0000 7fbf87e00640 1 -- 172.21.15.137:0/1926012903 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880107a0 con 0x55937a30bc00 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.637+0000 7fbf93400640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf800a2800 0x7fbf800a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.637+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1926012903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 msgr2=0x55937a30e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.637+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a30e090 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009c20 tx=0x7fbf8800e6c0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.637+0000 7fbf93400640 1 -- 172.21.15.137:0/3832161940 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf8c0f11f0 con 0x7fbf9c074080 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1926012903 shutdown_connections 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a30b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a30e090 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1926012903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1926012903 >> 172.21.15.137:0/1926012903 conn(0x55937a301610 msgr2=0x55937a301a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf800a2800 0x7fbf800a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1926012903 shutdown_connections 2024-03-26T14:05:18.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1926012903 wait complete. 2024-03-26T14:05:18.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbfa47b2740 1 -- start start 2024-03-26T14:05:18.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.638+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf800a2800 0x7fbf800a4cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55937a3072f0 tx=0x7fbf940098e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 Processor -- start 2024-03-26T14:05:18.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a310820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a317860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937a30ebb0 con 0x55937a30bc00 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937a227eb0 con 0x55937a306a90 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55937a30ed30 con 0x55937a3060a0 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.639+0000 7fbfa47b2740 10 client.0 _ll_register_callbacks cb 0x55937a21c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a310820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a310820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:57274/0 (socket says 172.21.15.137:57274) 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a317860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9a000640 1 -- 172.21.15.137:0/3413370169 learned_addr learned my addr 172.21.15.137:0/3413370169 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9b400640 1 -- 172.21.15.137:0/3413370169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 msgr2=0x55937a3102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9b400640 1 -- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 msgr2=0x55937a310820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a310820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9b400640 1 -- 172.21.15.137:0/3413370169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55937a0a8f70 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.640+0000 7fbf9a000640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a310820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a317860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005fd0 tx=0x7fbf94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f9b0ddf5b17096 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbf84c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbf84c00640 1 -- 172.21.15.137:0/3413370169 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94023070 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbf84c00640 1 -- 172.21.15.137:0/3413370169 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf940119b0 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf88011920 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbf84c00640 1 -- 172.21.15.137:0/3413370169 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf9402f050 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbfa47b2740 10 client.16248 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.641+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559379f40170 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbf84c00640 1 -- 172.21.15.137:0/3413370169 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf88011920 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbf84c00640 1 -- 172.21.15.137:0/3413370169 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbf9400f040 con 0x55937a30bc00 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbfa47b2740 10 client.16248 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbfa47b2740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:18.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbfa47b2740 20 client.16248 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:18.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a31c1b0 0x55937a31e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbfa47b2740 4 client.16248 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:18.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.642+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55937a2793f0 con 0x55937a31c1b0 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.643+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a31c1b0 0x55937a31e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.644+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a31c1b0 0x55937a31e5f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b5a3707b2476ec server_cookie=8bfb327401704ca9 in_seq=0 out_seq=0 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.644+0000 7fbf84c00640 10 client.16248 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbf84c00640 1 -- 172.21.15.137:0/3413370169 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55937a2793f0 con 0x55937a31c1b0 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbf84c00640 10 client.16248 handle_command_reply: tid=0 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 client.16248 shutdown 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 msgr2=0x55937a317860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a317860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005fd0 tx=0x7fbf94004ab0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 shutdown_connections 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a31c1b0 0x55937a31e5f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a306a90 0x55937a310820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a30bc00 0x55937a317860 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.645+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3413370169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.646+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 >> 172.21.15.137:0/3413370169 conn(0x55937a301610 msgr2=0x55937a300830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.646+0000 7fbf84c00640 0 client.16248 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.646+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 shutdown_connections 2024-03-26T14:05:18.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.646+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3413370169 wait complete. 2024-03-26T14:05:18.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.646+0000 7fbfa47b2740 20 client.16248 trim_cache size 0 max 16384 2024-03-26T14:05:18.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.669+0000 7fbfa47b2740 1 Processor -- start 2024-03-26T14:05:18.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.669+0000 7fbfa47b2740 1 -- start start 2024-03-26T14:05:18.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a300bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a317820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1ae940 con 0x55937a306a90 2024-03-26T14:05:18.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1ac440 con 0x55937a30bc00 2024-03-26T14:05:18.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1aa4c0 con 0x55937a3060a0 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a300bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbf9b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a317820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a300bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41960/0 (socket says 172.21.15.137:41960) 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbf9a000640 1 -- 172.21.15.137:0/1684761247 learned_addr learned my addr 172.21.15.137:0/1684761247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.670+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9a000640 1 -- 172.21.15.137:0/1684761247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 msgr2=0x55937a3006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9a000640 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9a000640 1 -- 172.21.15.137:0/1684761247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 msgr2=0x55937a317820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9a000640 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a317820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9a000640 1 -- 172.21.15.137:0/1684761247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55937a2fcc20 con 0x55937a306a90 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:18.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.671+0000 7fbf9a000640 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a300bf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009b70 tx=0x7fbf88009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa90bb9b5def55c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbf87400640 1 -- 172.21.15.137:0/1684761247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88016040 con 0x55937a306a90 2024-03-26T14:05:18.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbf87400640 1 -- 172.21.15.137:0/1684761247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf88011780 con 0x55937a306a90 2024-03-26T14:05:18.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbf87400640 1 -- 172.21.15.137:0/1684761247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf880048e0 con 0x55937a306a90 2024-03-26T14:05:18.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1684761247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 msgr2=0x55937a300bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a300bf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009b70 tx=0x7fbf88009370 comp rx=0 tx=0).stop 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1684761247 shutdown_connections 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.672+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a317820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a300bf0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/1684761247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a3006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1684761247 >> 172.21.15.137:0/1684761247 conn(0x55937a301610 msgr2=0x55937a2fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1684761247 shutdown_connections 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 -- 172.21.15.137:0/1684761247 wait complete. 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 -- start start 2024-03-26T14:05:18.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.673+0000 7fbfa47b2740 1 Processor -- start 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a317460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a315a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a315fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1aaac0 con 0x55937a306a90 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1abde0 con 0x55937a30bc00 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1ab120 con 0x55937a3060a0 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbfa47b2740 10 client.0 _ll_register_callbacks cb 0x7fbf94016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a315a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a315a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41972/0 (socket says 172.21.15.137:41972) 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 -- 172.21.15.137:0/3279338794 learned_addr learned my addr 172.21.15.137:0/3279338794 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a317460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a315fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 -- 172.21.15.137:0/3279338794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 msgr2=0x55937a317460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a317460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 -- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 msgr2=0x55937a315fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a315fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9a000640 1 -- 172.21.15.137:0/3279338794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55937a3164f0 con 0x55937a306a90 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.674+0000 7fbf9b400640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a315fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbf9a000640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a315a70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800e700 tx=0x7fbf88009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325118520ae3b49d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbf84c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbf84c00640 1 -- 172.21.15.137:0/3279338794 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88031020 con 0x55937a306a90 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf88011780 con 0x55937a306a90 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbf84c00640 1 -- 172.21.15.137:0/3279338794 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf88005450 con 0x55937a306a90 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbf84c00640 1 -- 172.21.15.137:0/3279338794 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf8803c050 con 0x55937a306a90 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbfa47b2740 10 client.16254 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.675+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55937a3787a0 con 0x55937a306a90 2024-03-26T14:05:18.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbf84c00640 1 -- 172.21.15.137:0/3279338794 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf88011780 con 0x55937a306a90 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbf84c00640 1 -- 172.21.15.137:0/3279338794 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbf8800f040 con 0x55937a306a90 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbfa47b2740 10 client.16254 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbfa47b2740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbfa47b2740 20 client.16254 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a378ea0 0x55937a37b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbfa47b2740 4 client.16254 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.676+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55937a273740 con 0x55937a378ea0 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.677+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a378ea0 0x55937a37b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.678+0000 7fbf9aa00640 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a378ea0 0x55937a37b2c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76245655beb0b84a server_cookie=fb07d547926b10cc in_seq=0 out_seq=0 2024-03-26T14:05:18.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.678+0000 7fbf84c00640 10 client.16254 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbf84c00640 1 -- 172.21.15.137:0/3279338794 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55937a273740 con 0x55937a378ea0 2024-03-26T14:05:18.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbf84c00640 10 client.16254 handle_command_reply: tid=0 2024-03-26T14:05:18.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 client.16254 shutdown 2024-03-26T14:05:18.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 msgr2=0x55937a315a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a315a70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800e700 tx=0x7fbf88009990 comp rx=0 tx=0).stop 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 shutdown_connections 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55937a378ea0 0x55937a37b2c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55937a30bc00 0x55937a315fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55937a306a90 0x55937a315a70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbf84c00640 0 client.16254 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 --2- 172.21.15.137:0/3279338794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55937a3060a0 0x55937a317460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 >> 172.21.15.137:0/3279338794 conn(0x55937a301610 msgr2=0x55937a300830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.681+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 shutdown_connections 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa47b2740 1 -- 172.21.15.137:0/3279338794 wait complete. 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa47b2740 20 client.16254 trim_cache size 0 max 16384 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "38f345a4-dc06-4436-8feb-d0acd2cc8c07", 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:05:18.681 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:05:18.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf800a2800 msgr2=0x7fbf800a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf800a2800 0x7fbf800a4cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55937a3072f0 tx=0x7fbf940098e0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 msgr2=0x7fbf9c199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 0x7fbf9c199b30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000c00 tx=0x7fbf8c02dc30 comp rx=0 tx=0).stop 2024-03-26T14:05:18.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 shutdown_connections 2024-03-26T14:05:18.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf800a2800 0x7fbf800a4cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf9c109560 0x7fbf9c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf9c075660 0x7fbf9c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 --2- 172.21.15.137:0/3832161940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf9c074080 0x7fbf9c199b30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.682+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 >> 172.21.15.137:0/3832161940 conn(0x7fbf9c0fcc40 msgr2=0x7fbf9c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.683+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 shutdown_connections 2024-03-26T14:05:18.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.683+0000 7fbfa1a00640 1 -- 172.21.15.137:0/3832161940 wait complete. 2024-03-26T14:05:18.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:05:18.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 -- 172.21.15.137:0/1594979443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c109430 msgr2=0x7fef8c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 --2- 172.21.15.137:0/1594979443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c109430 0x7fef8c10b8d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fef84009ca0 tx=0x7fef8402c180 comp rx=0 tx=0).stop 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 -- 172.21.15.137:0/1594979443 shutdown_connections 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 --2- 172.21.15.137:0/1594979443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8c0740a0 0x7fef8c074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 --2- 172.21.15.137:0/1594979443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c109430 0x7fef8c10b8d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 --2- 172.21.15.137:0/1594979443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef8c075680 0x7fef8c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 -- 172.21.15.137:0/1594979443 >> 172.21.15.137:0/1594979443 conn(0x7fef8c0fcb10 msgr2=0x7fef8c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 -- 172.21.15.137:0/1594979443 shutdown_connections 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.907+0000 7fef91200640 1 -- 172.21.15.137:0/1594979443 wait complete. 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 Processor -- start 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 -- start start 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8c0740a0 0x7fef8c199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 0x7fef8c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef8c109430 0x7fef8c1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c10f2d0 con 0x7fef8c075680 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c10f150 con 0x7fef8c0740a0 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.908+0000 7fef91200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c10f450 con 0x7fef8c109430 2024-03-26T14:05:18.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 0x7fef8c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 0x7fef8c199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41980/0 (socket says 172.21.15.137:41980) 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 -- 172.21.15.137:0/1263557478 learned_addr learned my addr 172.21.15.137:0/1263557478 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8b400640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8c0740a0 0x7fef8c199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8be00640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef8c109430 0x7fef8c1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 -- 172.21.15.137:0/1263557478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef8c109430 msgr2=0x7fef8c1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef8c109430 0x7fef8c1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 -- 172.21.15.137:0/1263557478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8c0740a0 msgr2=0x7fef8c199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8c0740a0 0x7fef8c199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 -- 172.21.15.137:0/1263557478 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c1a1560 con 0x7fef8c075680 2024-03-26T14:05:18.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.909+0000 7fef8aa00640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 0x7fef8c199e80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fef7800d950 tx=0x7fef7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.910+0000 7fef83e00640 1 -- 172.21.15.137:0/1263557478 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78017030 con 0x7fef8c075680 2024-03-26T14:05:18.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.910+0000 7fef83e00640 1 -- 172.21.15.137:0/1263557478 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef78004020 con 0x7fef8c075680 2024-03-26T14:05:18.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.910+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8c1a1850 con 0x7fef8c075680 2024-03-26T14:05:18.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.910+0000 7fef83e00640 1 -- 172.21.15.137:0/1263557478 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef780047a0 con 0x7fef8c075680 2024-03-26T14:05:18.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.910+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8c1a1df0 con 0x7fef8c075680 2024-03-26T14:05:18.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 Processor -- start 2024-03-26T14:05:18.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 -- start start 2024-03-26T14:05:18.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef83e00640 1 -- 172.21.15.137:0/1263557478 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fef780041d0 con 0x7fef8c075680 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce92d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce932540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce935040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ce892250 con 0x55c3ce92d050 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ce83aa30 con 0x55c3ce92da40 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.914+0000 7fef95411740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ce88e2f0 con 0x55c3ce932bb0 2024-03-26T14:05:18.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce92d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce92d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41984/0 (socket says 172.21.15.137:41984) 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 -- 172.21.15.137:0/3675059416 learned_addr learned my addr 172.21.15.137:0/3675059416 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8b400640 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce932540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8aa00640 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce935040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 -- 172.21.15.137:0/3675059416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 msgr2=0x55c3ce935040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce935040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 -- 172.21.15.137:0/3675059416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 msgr2=0x55c3ce932540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce932540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8be00640 1 -- 172.21.15.137:0/3675059416 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3ce825690 con 0x55c3ce92d050 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.915+0000 7fef8aa00640 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce935040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:18.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.916+0000 7fef8be00640 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce92d470 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fef84032520 tx=0x7fef84032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b103c6e61c5ff9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.916+0000 7fef83e00640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fef700a2880 0x7fef700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.916+0000 7fef83e00640 1 -- 172.21.15.137:0/1263557478 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fef780ce8a0 con 0x7fef8c075680 2024-03-26T14:05:18.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.916+0000 7fef80c00640 1 -- 172.21.15.137:0/3675059416 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84043020 con 0x55c3ce92d050 2024-03-26T14:05:18.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.916+0000 7fef80c00640 1 -- 172.21.15.137:0/3675059416 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef84003e20 con 0x55c3ce92d050 2024-03-26T14:05:18.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.916+0000 7fef80c00640 1 -- 172.21.15.137:0/3675059416 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84039040 con 0x55c3ce92d050 2024-03-26T14:05:18.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef8b400640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fef700a2880 0x7fef700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 -- 172.21.15.137:0/3675059416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 msgr2=0x55c3ce92d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef8b400640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fef700a2880 0x7fef700a4d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55c3ce92de40 tx=0x7fef7c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce92d470 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fef84032520 tx=0x7fef84032340 comp rx=0 tx=0).stop 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 -- 172.21.15.137:0/3675059416 shutdown_connections 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce932540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce935040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 --2- 172.21.15.137:0/3675059416 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce92d470 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 -- 172.21.15.137:0/3675059416 >> 172.21.15.137:0/3675059416 conn(0x55c3ce8f5200 msgr2=0x55c3ce9289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.917+0000 7fef95411740 1 -- 172.21.15.137:0/3675059416 shutdown_connections 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.918+0000 7fef95411740 1 -- 172.21.15.137:0/3675059416 wait complete. 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.918+0000 7fef95411740 1 -- start start 2024-03-26T14:05:18.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.918+0000 7fef95411740 1 Processor -- start 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce937030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce937570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce93e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ce935b00 con 0x55c3ce92d050 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ce84de80 con 0x55c3ce92da40 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ce935c80 con 0x55c3ce932bb0 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef95411740 10 client.0 _ll_register_callbacks cb 0x55c3ce860130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef8b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce937030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef8aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce937570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef8b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce937030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41992/0 (socket says 172.21.15.137:41992) 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef8b400640 1 -- 172.21.15.137:0/49146231 learned_addr learned my addr 172.21.15.137:0/49146231 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.919+0000 7fef8be00640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce93e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8b400640 1 -- 172.21.15.137:0/49146231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 msgr2=0x55c3ce93e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8b400640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce93e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8b400640 1 -- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 msgr2=0x55c3ce937570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8b400640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce937570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8b400640 1 -- 172.21.15.137:0/49146231 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3ce6707d0 con 0x55c3ce92d050 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8aa00640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce937570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef8b400640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce937030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fef7c003ea0 tx=0x7fef7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a4719a29ad5889 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef75600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef75600640 1 -- 172.21.15.137:0/49146231 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01b660 con 0x55c3ce92d050 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef75600640 1 -- 172.21.15.137:0/49146231 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fef7c01b800 con 0x55c3ce92d050 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.920+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef84003e20 con 0x55c3ce92d050 2024-03-26T14:05:18.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef75600640 1 -- 172.21.15.137:0/49146231 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef7c01ba00 con 0x55c3ce92d050 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 10 client.16266 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c3ce5a63f0 con 0x55c3ce92d050 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef75600640 1 -- 172.21.15.137:0/49146231 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef84003e20 con 0x55c3ce92d050 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef75600640 1 -- 172.21.15.137:0/49146231 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fef7c024070 con 0x55c3ce92d050 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 10 client.16266 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 20 client.16266 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce942f00 0x55c3ce945360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 4 client.16266 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.921+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3ce89f790 con 0x55c3ce942f00 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.922+0000 7fef8aa00640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce942f00 0x55c3ce945360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.922+0000 7fef75600640 10 client.16266 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.922+0000 7fef8aa00640 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce942f00 0x55c3ce945360 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75187b6d603e7682 server_cookie=64aca26bf105e6bb in_seq=0 out_seq=0 2024-03-26T14:05:18.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.923+0000 7fef75600640 1 -- 172.21.15.137:0/49146231 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c3ce89f790 con 0x55c3ce942f00 2024-03-26T14:05:18.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.923+0000 7fef75600640 10 client.16266 handle_command_reply: tid=0 2024-03-26T14:05:18.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.923+0000 7fef95411740 1 client.16266 shutdown 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 msgr2=0x55c3ce937030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce937030 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fef7c003ea0 tx=0x7fef7c0046c0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 shutdown_connections 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce942f00 0x55c3ce945360 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce937570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce93e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 --2- 172.21.15.137:0/49146231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce937030 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef75600640 0 client.16266 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 >> 172.21.15.137:0/49146231 conn(0x55c3ce8f5200 msgr2=0x55c3ce927bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 shutdown_connections 2024-03-26T14:05:18.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 1 -- 172.21.15.137:0/49146231 wait complete. 2024-03-26T14:05:18.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.924+0000 7fef95411740 20 client.16266 trim_cache size 0 max 16384 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.948+0000 7fef95411740 1 Processor -- start 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.948+0000 7fef95411740 1 -- start start 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef95411740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce9374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef95411740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce93e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef95411740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce92c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef95411740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1ae760 con 0x55c3ce92d050 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef95411740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1ac260 con 0x55c3ce92da40 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef95411740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1aa2e0 con 0x55c3ce932bb0 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce9374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce9374d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41998/0 (socket says 172.21.15.137:41998) 2024-03-26T14:05:18.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 -- 172.21.15.137:0/4147649754 learned_addr learned my addr 172.21.15.137:0/4147649754 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8aa00640 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce93e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 -- 172.21.15.137:0/4147649754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 msgr2=0x55c3ce92c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce92c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 -- 172.21.15.137:0/4147649754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 msgr2=0x55c3ce93e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce93e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.949+0000 7fef8b400640 1 -- 172.21.15.137:0/4147649754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3ce5660f0 con 0x55c3ce92d050 2024-03-26T14:05:18.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.950+0000 7fef8b400640 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce9374d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef7c026f40 tx=0x7fef7c026f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee89fdf710dc9537 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.950+0000 7fef77e00640 1 -- 172.21.15.137:0/4147649754 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c01bda0 con 0x55c3ce92d050 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.950+0000 7fef77e00640 1 -- 172.21.15.137:0/4147649754 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef7c039070 con 0x55c3ce92d050 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.950+0000 7fef77e00640 1 -- 172.21.15.137:0/4147649754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c026440 con 0x55c3ce92d050 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 -- 172.21.15.137:0/4147649754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 msgr2=0x55c3ce9374d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce9374d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef7c026f40 tx=0x7fef7c026f70 comp rx=0 tx=0).stop 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 -- 172.21.15.137:0/4147649754 shutdown_connections 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92da40 0x55c3ce93e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce932bb0 0x55c3ce92c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 --2- 172.21.15.137:0/4147649754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce92d050 0x55c3ce9374d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 -- 172.21.15.137:0/4147649754 >> 172.21.15.137:0/4147649754 conn(0x55c3ce8f5200 msgr2=0x55c3ce92bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 -- 172.21.15.137:0/4147649754 shutdown_connections 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 -- 172.21.15.137:0/4147649754 wait complete. 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.951+0000 7fef95411740 1 -- start start 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 Processor -- start 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92d050 0x55c3ce93be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce92da40 0x55c3ce93c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 0x55c3ce93d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1aa8e0 con 0x55c3ce932bb0 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1abc00 con 0x55c3ce92d050 2024-03-26T14:05:18.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8c1aaf40 con 0x55c3ce92da40 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef95411740 10 client.0 _ll_register_callbacks cb 0x7fef7c0043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef8be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 0x55c3ce93d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef8be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 0x55c3ce93d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42010/0 (socket says 172.21.15.137:42010) 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef8be00640 1 -- 172.21.15.137:0/1408870487 learned_addr learned my addr 172.21.15.137:0/1408870487 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.952+0000 7fef8b400640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92d050 0x55c3ce93be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8aa00640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce92da40 0x55c3ce93c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8be00640 1 -- 172.21.15.137:0/1408870487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce92da40 msgr2=0x55c3ce93c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8be00640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce92da40 0x55c3ce93c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8be00640 1 -- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92d050 msgr2=0x55c3ce93be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8be00640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92d050 0x55c3ce93be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8be00640 1 -- 172.21.15.137:0/1408870487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3ce93d970 con 0x55c3ce932bb0 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef8be00640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 0x55c3ce93d380 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fef8402c790 tx=0x7fef84003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be41dcdcb4aca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:18.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef75600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef75600640 1 -- 172.21.15.137:0/1408870487 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84043070 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef75600640 1 -- 172.21.15.137:0/1408870487 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fef840326f0 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.953+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef7c039070 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef75600640 1 -- 172.21.15.137:0/1408870487 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef84054050 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 10 client.16272 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c3ce99e8e0 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef75600640 1 -- 172.21.15.137:0/1408870487 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef7c039070 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef75600640 1 -- 172.21.15.137:0/1408870487 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fef840044e0 con 0x55c3ce932bb0 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 10 client.16272 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 20 client.16272 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce99ee20 0x55c3ce9a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:18.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 4 client.16272 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:05:18.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.954+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c3ce899ae0 con 0x55c3ce99ee20 2024-03-26T14:05:18.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.955+0000 7fef8b400640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce99ee20 0x55c3ce9a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.955+0000 7fef8b400640 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce99ee20 0x55c3ce9a1240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb149e33a102aea server_cookie=ac875e3ba45c6ddc in_seq=0 out_seq=0 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.955+0000 7fef75600640 10 client.16272 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef75600640 1 -- 172.21.15.137:0/1408870487 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c3ce899ae0 con 0x55c3ce99ee20 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef75600640 10 client.16272 handle_command_reply: tid=0 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 client.16272 shutdown 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 msgr2=0x55c3ce93d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 0x55c3ce93d380 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fef8402c790 tx=0x7fef84003eb0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 shutdown_connections 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c3ce99ee20 0x55c3ce9a1240 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c3ce932bb0 0x55c3ce93d380 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c3ce92da40 0x55c3ce93c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 --2- 172.21.15.137:0/1408870487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c3ce92d050 0x55c3ce93be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 >> 172.21.15.137:0/1408870487 conn(0x55c3ce8f5200 msgr2=0x55c3ce9372e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef75600640 0 client.16272 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 shutdown_connections 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 1 -- 172.21.15.137:0/1408870487 wait complete. 2024-03-26T14:05:18.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.956+0000 7fef95411740 20 client.16272 trim_cache size 0 max 16384 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: "38f345a4-dc06-4436-8feb-d0acd2cc8c07": { 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "38f345a4-dc06-4436-8feb-d0acd2cc8c07", 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:05:18.960 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fef700a2880 msgr2=0x7fef700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fef700a2880 0x7fef700a4d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55c3ce92de40 tx=0x7fef7c00b390 comp rx=0 tx=0).stop 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 msgr2=0x7fef8c199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 0x7fef8c199e80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fef7800d950 tx=0x7fef7800de10 comp rx=0 tx=0).stop 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 shutdown_connections 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fef700a2880 0x7fef700a4d40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef8c109430 0x7fef8c1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fef8c075680 0x7fef8c199e80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 --2- 172.21.15.137:0/1263557478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef8c0740a0 0x7fef8c199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:18.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 >> 172.21.15.137:0/1263557478 conn(0x7fef8c0fcb10 msgr2=0x7fef8c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:18.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 shutdown_connections 2024-03-26T14:05:18.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:18.957+0000 7fef91200640 1 -- 172.21.15.137:0/1263557478 wait complete. 2024-03-26T14:05:18.975 INFO:tasks.cephfs.filesystem:scrub status for tag:38f345a4-dc06-4436-8feb-d0acd2cc8c07 - {'path': '/', 'tag': '38f345a4-dc06-4436-8feb-d0acd2cc8c07', 'options': 'recursive,force'} 2024-03-26T14:05:19.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:19 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:05:19.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:19 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:19.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:19 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:05:19.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:19 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:19.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:19 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:05:19.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:19 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:19.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:19 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:05:19.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:19 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:19.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:19 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:05:19.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:19 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:19.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:19 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:05:19.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:19 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:20.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:20 smithi179 ceph-mon[35472]: pgmap v547: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:20.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:20 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:05:20.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:20 smithi159 ceph-mon[35501]: pgmap v547: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:20.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:20 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:05:20.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:20 smithi137 ceph-mon[31539]: pgmap v547: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:20.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:20 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:05:21.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:05:21.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:05:21.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:21 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:05:21.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:05:21.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:05:21.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:21 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:05:21.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:05:21.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:05:21.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:21 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:05:22.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:22 smithi179 ceph-mon[35472]: pgmap v548: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:05:22.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:22 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:05:22.435 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:22 smithi137 ceph-mon[31539]: pgmap v548: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:05:22.435 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:22 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:05:22.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:22 smithi159 ceph-mon[35501]: pgmap v548: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:05:22.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:22 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:05:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:23 smithi179 ceph-mon[35472]: pgmap v549: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:05:23.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:23 smithi159 ceph-mon[35501]: pgmap v549: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:05:23.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:23 smithi137 ceph-mon[31539]: pgmap v549: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:05:25.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:25 smithi179 ceph-mon[35472]: pgmap v550: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:25.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:25 smithi159 ceph-mon[35501]: pgmap v550: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:25.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:25 smithi137 ceph-mon[31539]: pgmap v550: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:27.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:27 smithi179 ceph-mon[35472]: pgmap v551: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:05:27.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:27 smithi159 ceph-mon[35501]: pgmap v551: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:05:27.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:27 smithi137 ceph-mon[31539]: pgmap v551: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:05:29.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:29 smithi179 ceph-mon[35472]: pgmap v552: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:29.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:29 smithi159 ceph-mon[35501]: pgmap v552: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:29.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:29 smithi137 ceph-mon[31539]: pgmap v552: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:31.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:31 smithi179 ceph-mon[35472]: pgmap v553: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:31.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:31 smithi159 ceph-mon[35501]: pgmap v553: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:31.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:31 smithi137 ceph-mon[31539]: pgmap v553: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:33.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:33 smithi179 ceph-mon[35472]: pgmap v554: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:05:33.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:33 smithi159 ceph-mon[35501]: pgmap v554: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:05:33.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:33 smithi137 ceph-mon[31539]: pgmap v554: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:05:35.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:35 smithi179 ceph-mon[35472]: pgmap v555: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:35.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:35 smithi159 ceph-mon[35501]: pgmap v555: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:35.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:35 smithi137 ceph-mon[31539]: pgmap v555: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:37.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:37 smithi179 ceph-mon[35472]: pgmap v556: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:37.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:37 smithi159 ceph-mon[35501]: pgmap v556: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:37.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:37 smithi137 ceph-mon[31539]: pgmap v556: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:39.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:39 smithi179 ceph-mon[35472]: pgmap v557: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:39.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:39 smithi159 ceph-mon[35501]: pgmap v557: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:39.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:39 smithi137 ceph-mon[31539]: pgmap v557: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:41.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:41 smithi179 ceph-mon[35472]: pgmap v558: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:41.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:41 smithi159 ceph-mon[35501]: pgmap v558: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:41.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:41 smithi137 ceph-mon[31539]: pgmap v558: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:43.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:43 smithi179 ceph-mon[35472]: pgmap v559: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:43.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:43 smithi159 ceph-mon[35501]: pgmap v559: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:43.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:43 smithi137 ceph-mon[31539]: pgmap v559: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:45.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:45 smithi179 ceph-mon[35472]: pgmap v560: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:45.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:45 smithi159 ceph-mon[35501]: pgmap v560: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:45.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:45 smithi137 ceph-mon[31539]: pgmap v560: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:47.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:47 smithi179 ceph-mon[35472]: pgmap v561: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:47.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:47 smithi159 ceph-mon[35501]: pgmap v561: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:47.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:47 smithi137 ceph-mon[31539]: pgmap v561: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:05:48.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:05:48.976 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 -- 172.21.15.137:0/3003495824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c0ffe70 msgr2=0x7f606c100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 --2- 172.21.15.137:0/3003495824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c0ffe70 0x7f606c100290 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 -- 172.21.15.137:0/3003495824 shutdown_connections 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 --2- 172.21.15.137:0/3003495824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 0x7f606c100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 --2- 172.21.15.137:0/3003495824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606c101170 0x7f606c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 --2- 172.21.15.137:0/3003495824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c0ffe70 0x7f606c100290 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 -- 172.21.15.137:0/3003495824 >> 172.21.15.137:0/3003495824 conn(0x7f606c0fba10 msgr2=0x7f606c0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 -- 172.21.15.137:0/3003495824 shutdown_connections 2024-03-26T14:05:49.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.202+0000 7f6072400640 1 -- 172.21.15.137:0/3003495824 wait complete. 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.203+0000 7f6072400640 1 Processor -- start 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.203+0000 7f6072400640 1 -- start start 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6072400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606c0ffe70 0x7f606c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6072400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 0x7f606c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6072400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 0x7f606c1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6072400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c10f300 con 0x7f606c101170 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6072400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c10f180 con 0x7f606c1007d0 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6072400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c10f480 con 0x7f606c0ffe70 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6071a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 0x7f606c1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6071a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 0x7f606c1a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36650/0 (socket says 172.21.15.137:36650) 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6071a00640 1 -- 172.21.15.137:0/3736531912 learned_addr learned my addr 172.21.15.137:0/3736531912 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:49.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f606be00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 0x7f606c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.204+0000 7f6071000640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606c0ffe70 0x7f606c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6071a00640 1 -- 172.21.15.137:0/3736531912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606c0ffe70 msgr2=0x7f606c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6071a00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606c0ffe70 0x7f606c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6071a00640 1 -- 172.21.15.137:0/3736531912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 msgr2=0x7f606c199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6071a00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 0x7f606c199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6071a00640 1 -- 172.21.15.137:0/3736531912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c1a15c0 con 0x7f606c101170 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f606be00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 0x7f606c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6071a00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 0x7f606c1a0f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f606400eaa0 tx=0x7f606400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6069600640 1 -- 172.21.15.137:0/3736531912 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400cdb0 con 0x7f606c101170 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6069600640 1 -- 172.21.15.137:0/3736531912 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6064005020 con 0x7f606c101170 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c1a18b0 con 0x7f606c101170 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6069600640 1 -- 172.21.15.137:0/3736531912 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640116d0 con 0x7f606c101170 2024-03-26T14:05:49.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.205+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c1a1e00 con 0x7f606c101170 2024-03-26T14:05:49.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 Processor -- start 2024-03-26T14:05:49.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6069600640 1 -- 172.21.15.137:0/3736531912 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f60640051d0 con 0x7f606c101170 2024-03-26T14:05:49.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 -- start start 2024-03-26T14:05:49.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4050 0x5635ce6b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ce619250 con 0x5635ce6b4a40 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ce5c1a30 con 0x5635ce6b9bb0 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.209+0000 7f6076622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ce6152f0 con 0x5635ce6b4050 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6b9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36656/0 (socket says 172.21.15.137:36656) 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 -- 172.21.15.137:0/2318696948 learned_addr learned my addr 172.21.15.137:0/2318696948 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f606be00640 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f606be00640 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4050 0x5635ce6b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 -- 172.21.15.137:0/2318696948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4050 msgr2=0x5635ce6b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4050 0x5635ce6b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 -- 172.21.15.137:0/2318696948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 msgr2=0x5635ce6bc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6bc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.210+0000 7f6071000640 1 -- 172.21.15.137:0/2318696948 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635ce5ac690 con 0x5635ce6b4a40 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f6069600640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2800 0x7f60540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f6069600640 1 -- 172.21.15.137:0/3736531912 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f60640ceff0 con 0x7f606c101170 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f606be00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2800 0x7f60540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f6071000640 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6b9540 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6060005ab0 tx=0x7f6060009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a232b794dbe9a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f605e000640 1 -- 172.21.15.137:0/2318696948 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060045020 con 0x5635ce6b4a40 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f605e000640 1 -- 172.21.15.137:0/2318696948 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f606002ca20 con 0x5635ce6b4a40 2024-03-26T14:05:49.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.211+0000 7f605e000640 1 -- 172.21.15.137:0/2318696948 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60600046c0 con 0x5635ce6b4a40 2024-03-26T14:05:49.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 -- 172.21.15.137:0/2318696948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 msgr2=0x5635ce6b9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6b9540 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6060005ab0 tx=0x7f6060009560 comp rx=0 tx=0).stop 2024-03-26T14:05:49.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f606be00640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2800 0x7f60540a4cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x5635ce6b52a0 tx=0x7f6058012040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 -- 172.21.15.137:0/2318696948 shutdown_connections 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6bc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6b9540 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 --2- 172.21.15.137:0/2318696948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4050 0x5635ce6b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.212+0000 7f6076622740 1 -- 172.21.15.137:0/2318696948 >> 172.21.15.137:0/2318696948 conn(0x5635ce67c200 msgr2=0x5635ce6af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.213+0000 7f6076622740 1 -- 172.21.15.137:0/2318696948 shutdown_connections 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.213+0000 7f6076622740 1 -- 172.21.15.137:0/2318696948 wait complete. 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.213+0000 7f6076622740 1 -- start start 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.213+0000 7f6076622740 1 Processor -- start 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ce6bcb00 con 0x5635ce6b9bb0 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ce5d4e80 con 0x5635ce6b4050 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635ce6bcc80 con 0x5635ce6b4a40 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6076622740 10 client.0 _ll_register_callbacks cb 0x5635ce5e7130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6071a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6071a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36660/0 (socket says 172.21.15.137:36660) 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6071a00640 1 -- 172.21.15.137:0/4053817123 learned_addr learned my addr 172.21.15.137:0/4053817123 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f6071000640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.214+0000 7f606be00640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071a00640 1 -- 172.21.15.137:0/4053817123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 msgr2=0x5635ce6be590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071a00640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6be590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071a00640 1 -- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 msgr2=0x5635ce6be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071a00640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071a00640 1 -- 172.21.15.137:0/4053817123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635ce3f77d0 con 0x5635ce6b9bb0 2024-03-26T14:05:49.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071000640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:49.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6071a00640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f606409b510 tx=0x7f606409bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd42cafc833080a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6052a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:49.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6052a00640 1 -- 172.21.15.137:0/4053817123 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60640b4060 con 0x5635ce6b9bb0 2024-03-26T14:05:49.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6052a00640 1 -- 172.21.15.137:0/4053817123 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f60640b4200 con 0x5635ce6b9bb0 2024-03-26T14:05:49.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.215+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f606002ca20 con 0x5635ce6b9bb0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6052a00640 1 -- 172.21.15.137:0/4053817123 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606401f040 con 0x5635ce6b9bb0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 10 client.16284 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5635ce32d3f0 con 0x5635ce6b9bb0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6052a00640 1 -- 172.21.15.137:0/4053817123 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f606002ca20 con 0x5635ce6b9bb0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6052a00640 1 -- 172.21.15.137:0/4053817123 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f606400f040 con 0x5635ce6b9bb0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 10 client.16284 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 20 client.16284 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce6c9d40 0x5635ce6cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 4 client.16284 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.216+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635ce626790 con 0x5635ce6c9d40 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.217+0000 7f6071000640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce6c9d40 0x5635ce6cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.218+0000 7f6071000640 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce6c9d40 0x5635ce6cc1d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4f302ac88d61f4 server_cookie=e3d32700680201d3 in_seq=0 out_seq=0 2024-03-26T14:05:49.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.218+0000 7f6052a00640 10 client.16284 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:49.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.219+0000 7f6052a00640 1 -- 172.21.15.137:0/4053817123 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5635ce626790 con 0x5635ce6c9d40 2024-03-26T14:05:49.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.219+0000 7f6052a00640 10 client.16284 handle_command_reply: tid=0 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.219+0000 7f6076622740 1 client.16284 shutdown 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 msgr2=0x5635ce6c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f606409b510 tx=0x7f606409bdc0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 shutdown_connections 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce6c9d40 0x5635ce6cc1d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5560 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6be590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 --2- 172.21.15.137:0/4053817123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6052a00640 0 client.16284 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 >> 172.21.15.137:0/4053817123 conn(0x5635ce67c200 msgr2=0x5635ce6aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 shutdown_connections 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 1 -- 172.21.15.137:0/4053817123 wait complete. 2024-03-26T14:05:49.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.220+0000 7f6076622740 20 client.16284 trim_cache size 0 max 16384 2024-03-26T14:05:49.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 Processor -- start 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 -- start start 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 0x5635ce6aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1ae750 con 0x5635ce6b4050 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1ac250 con 0x5635ce6b9bb0 2024-03-26T14:05:49.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.239+0000 7f6076622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1aa2d0 con 0x5635ce6b4a40 2024-03-26T14:05:49.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 0x5635ce6aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 0x5635ce6aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36666/0 (socket says 172.21.15.137:36666) 2024-03-26T14:05:49.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 -- 172.21.15.137:0/1295682616 learned_addr learned my addr 172.21.15.137:0/1295682616 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:49.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f606be00640 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 -- 172.21.15.137:0/1295682616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 msgr2=0x5635ce6aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 -- 172.21.15.137:0/1295682616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 msgr2=0x5635ce6c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 -- 172.21.15.137:0/1295682616 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635ce2ed0f0 con 0x5635ce6b4050 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.240+0000 7f6071000640 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 0x5635ce6aea40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb3bfb3fb124d04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.241+0000 7f605d600640 1 -- 172.21.15.137:0/1295682616 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060052070 con 0x5635ce6b4050 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.241+0000 7f605d600640 1 -- 172.21.15.137:0/1295682616 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6060005620 con 0x5635ce6b4050 2024-03-26T14:05:49.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.241+0000 7f605d600640 1 -- 172.21.15.137:0/1295682616 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060045040 con 0x5635ce6b4050 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 -- 172.21.15.137:0/1295682616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 msgr2=0x5635ce6aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 0x5635ce6aea40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6060009900 tx=0x7f606002bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 -- 172.21.15.137:0/1295682616 shutdown_connections 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b4a40 0x5635ce6aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 --2- 172.21.15.137:0/1295682616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4050 0x5635ce6aea40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 -- 172.21.15.137:0/1295682616 >> 172.21.15.137:0/1295682616 conn(0x5635ce67c200 msgr2=0x5635ce6ae330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 -- 172.21.15.137:0/1295682616 shutdown_connections 2024-03-26T14:05:49.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 -- 172.21.15.137:0/1295682616 wait complete. 2024-03-26T14:05:49.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.242+0000 7f6076622740 1 -- start start 2024-03-26T14:05:49.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 Processor -- start 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6c3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1aa8d0 con 0x5635ce6b4a40 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1abbf0 con 0x5635ce6b4050 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1aaf30 con 0x5635ce6b9bb0 2024-03-26T14:05:49.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6076622740 10 client.0 _ll_register_callbacks cb 0x7f6064097160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f606be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f606be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36668/0 (socket says 172.21.15.137:36668) 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f606be00640 1 -- 172.21.15.137:0/624897938 learned_addr learned my addr 172.21.15.137:0/624897938 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.243+0000 7f6071000640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f6071a00640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f606be00640 1 -- 172.21.15.137:0/624897938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b9bb0 msgr2=0x5635ce6c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f606be00640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f606be00640 1 -- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 msgr2=0x5635ce6c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f606be00640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f606be00640 1 -- 172.21.15.137:0/624897938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635ce6c4b50 con 0x5635ce6b4a40 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f606be00640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6c3550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f60580038a0 tx=0x7f60580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d432faf4ed2b53f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.244+0000 7f6052a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6052a00640 1 -- 172.21.15.137:0/624897938 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605800fc60 con 0x5635ce6b4a40 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6052a00640 1 -- 172.21.15.137:0/624897938 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f605800fe00 con 0x5635ce6b4a40 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6060005620 con 0x5635ce6b4a40 2024-03-26T14:05:49.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6052a00640 1 -- 172.21.15.137:0/624897938 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6058035050 con 0x5635ce6b4a40 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6076622740 10 client.16290 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5635ce725ae0 con 0x5635ce6b4a40 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6052a00640 1 -- 172.21.15.137:0/624897938 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6060005620 con 0x5635ce6b4a40 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6052a00640 1 -- 172.21.15.137:0/624897938 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f605802a020 con 0x5635ce6b4a40 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6076622740 10 client.16290 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6076622740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.245+0000 7f6076622740 20 client.16290 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.246+0000 7f6076622740 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce7264f0 0x5635ce728910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.246+0000 7f6076622740 4 client.16290 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:05:49.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.246+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5635ce620ae0 con 0x5635ce7264f0 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.246+0000 7f6071000640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce7264f0 0x5635ce728910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6071000640 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce7264f0 0x5635ce728910 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca4eda42d47e7eba server_cookie=9d478e719495efd4 in_seq=0 out_seq=0 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6052a00640 10 client.16290 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6052a00640 1 -- 172.21.15.137:0/624897938 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5635ce620ae0 con 0x5635ce7264f0 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6052a00640 10 client.16290 handle_command_reply: tid=0 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6076622740 1 client.16290 shutdown 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 msgr2=0x5635ce6c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.247+0000 7f6076622740 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6c3550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f60580038a0 tx=0x7f60580046c0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 shutdown_connections 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5635ce7264f0 0x5635ce728910 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635ce6b9bb0 0x5635ce6c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5635ce6b4a40 0x5635ce6c3550 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 --2- 172.21.15.137:0/624897938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635ce6b4050 0x5635ce6c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 >> 172.21.15.137:0/624897938 conn(0x5635ce67c200 msgr2=0x5635ce6aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6052a00640 0 client.16290 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 shutdown_connections 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 1 -- 172.21.15.137:0/624897938 wait complete. 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6076622740 20 client.16290 trim_cache size 0 max 16384 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:05:49.249 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.248+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2800 msgr2=0x7f60540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2800 0x7f60540a4cc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x5635ce6b52a0 tx=0x7f6058012040 comp rx=0 tx=0).stop 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 msgr2=0x7f606c1a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 0x7f606c1a0f50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f606400eaa0 tx=0x7f606400ef60 comp rx=0 tx=0).stop 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 shutdown_connections 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f60540a2800 0x7f60540a4cc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f606c1007d0 0x7f606c199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f606c101170 0x7f606c1a0f50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 --2- 172.21.15.137:0/3736531912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f606c0ffe70 0x7f606c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 >> 172.21.15.137:0/3736531912 conn(0x7f606c0fba10 msgr2=0x7f606c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 shutdown_connections 2024-03-26T14:05:49.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.249+0000 7f6072400640 1 -- 172.21.15.137:0/3736531912 wait complete. 2024-03-26T14:05:49.265 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:05:49.265 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 -- 172.21.15.137:0/4154308209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96981030b0 msgr2=0x7f9698109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 --2- 172.21.15.137:0/4154308209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96981030b0 0x7f9698109940 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9690009ca0 tx=0x7f969002c0f0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 -- 172.21.15.137:0/4154308209 shutdown_connections 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 --2- 172.21.15.137:0/4154308209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981026f0 0x7f9698102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 --2- 172.21.15.137:0/4154308209 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96981030b0 0x7f9698109940 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 --2- 172.21.15.137:0/4154308209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9698101500 0x7f9698101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 -- 172.21.15.137:0/4154308209 >> 172.21.15.137:0/4154308209 conn(0x7f96980fca70 msgr2=0x7f96980feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 -- 172.21.15.137:0/4154308209 shutdown_connections 2024-03-26T14:05:49.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.496+0000 7f9697e00640 1 -- 172.21.15.137:0/4154308209 wait complete. 2024-03-26T14:05:49.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.497+0000 7f9697e00640 1 Processor -- start 2024-03-26T14:05:49.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.497+0000 7f9697e00640 1 -- start start 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 0x7f96981998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96981026f0 0x7f9698199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981030b0 0x7f96981a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969810d010 con 0x7f9698101500 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969810ce90 con 0x7f96981030b0 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969810d190 con 0x7f96981026f0 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9696a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 0x7f96981998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9696a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 0x7f96981998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36688/0 (socket says 172.21.15.137:36688) 2024-03-26T14:05:49.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9696a00640 1 -- 172.21.15.137:0/3447760543 learned_addr learned my addr 172.21.15.137:0/3447760543 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:49.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9697400640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981030b0 0x7f96981a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.498+0000 7f9696000640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96981026f0 0x7f9698199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9696a00640 1 -- 172.21.15.137:0/3447760543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96981026f0 msgr2=0x7f9698199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9696a00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96981026f0 0x7f9698199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9696a00640 1 -- 172.21.15.137:0/3447760543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981030b0 msgr2=0x7f96981a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9696a00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981030b0 0x7f96981a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9696a00640 1 -- 172.21.15.137:0/3447760543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96981a1510 con 0x7f9698101500 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9697400640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981030b0 0x7f96981a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9696a00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 0x7f96981998f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f968800d770 tx=0x7f968800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688017070 con 0x7f9698101500 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9688009c30 con 0x7f9698101500 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96981a1800 con 0x7f9698101500 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688003d00 con 0x7f9698101500 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.499+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96981a1da0 con 0x7f9698101500 2024-03-26T14:05:49.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.501+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96981a1fa0 con 0x7f9698101500 2024-03-26T14:05:49.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.503+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9688003ea0 con 0x7f9698101500 2024-03-26T14:05:49.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.504+0000 7f969ca00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f967c0a2880 0x7f967c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:49.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.505+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f96880cebf0 con 0x7f9698101500 2024-03-26T14:05:49.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.505+0000 7f9696000640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f967c0a2880 0x7f967c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:49.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.505+0000 7f9696000640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f967c0a2880 0x7f967c0a4d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f969819ae00 tx=0x7f968400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:49.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.510+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f968809b880 con 0x7f9698101500 2024-03-26T14:05:49.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:49 smithi179 ceph-mon[35472]: pgmap v562: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:49.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:49 smithi159 ceph-mon[35501]: pgmap v562: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:49.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.755+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96981a2170 con 0x7f9698101500 2024-03-26T14:05:49.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.757+0000 7f969ca00640 1 -- 172.21.15.137:0/3447760543 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f9688009de0 con 0x7f9698101500 2024-03-26T14:05:49.756 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:05:49.756 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:05:49.756 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:05:49.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.759+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f967c0a2880 msgr2=0x7f967c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f967c0a2880 0x7f967c0a4d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f969819ae00 tx=0x7f968400b590 comp rx=0 tx=0).stop 2024-03-26T14:05:49.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 msgr2=0x7f96981998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 0x7f96981998f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f968800d770 tx=0x7f968800dc30 comp rx=0 tx=0).stop 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 shutdown_connections 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f967c0a2880 0x7f967c0a4d40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96981030b0 0x7f96981a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96981026f0 0x7f9698199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 --2- 172.21.15.137:0/3447760543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9698101500 0x7f96981998f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 >> 172.21.15.137:0/3447760543 conn(0x7f96980fca70 msgr2=0x7f96980738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 shutdown_connections 2024-03-26T14:05:49.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:49.760+0000 7f9697e00640 1 -- 172.21.15.137:0/3447760543 wait complete. 2024-03-26T14:05:49.775 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:49 smithi137 ceph-mon[31539]: pgmap v562: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:49.778 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:05:49.778 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 -- 172.21.15.137:0/821573207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194102790 msgr2=0x7f4194102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 --2- 172.21.15.137:0/821573207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194102790 0x7f4194102c10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f4180009900 tx=0x7f418002bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 -- 172.21.15.137:0/821573207 shutdown_connections 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 --2- 172.21.15.137:0/821573207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4194103150 0x7f41941099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 --2- 172.21.15.137:0/821573207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194102790 0x7f4194102c10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 --2- 172.21.15.137:0/821573207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41941015a0 0x7f41941019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 -- 172.21.15.137:0/821573207 >> 172.21.15.137:0/821573207 conn(0x7f41940fcb10 msgr2=0x7f41940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 -- 172.21.15.137:0/821573207 shutdown_connections 2024-03-26T14:05:50.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.005+0000 7f4199200640 1 -- 172.21.15.137:0/821573207 wait complete. 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 Processor -- start 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 -- start start 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41941015a0 0x7f41941977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4194102790 0x7f4194197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 0x7f419419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410d120 con 0x7f4194103150 2024-03-26T14:05:50.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410cfa0 con 0x7f4194102790 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.006+0000 7f4199200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419410d2a0 con 0x7f41941015a0 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 0x7f419419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 0x7f419419ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36692/0 (socket says 172.21.15.137:36692) 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 -- 172.21.15.137:0/2729088977 learned_addr learned my addr 172.21.15.137:0/2729088977 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4192a00640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4194102790 0x7f4194197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193400640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41941015a0 0x7f41941977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 -- 172.21.15.137:0/2729088977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41941015a0 msgr2=0x7f41941977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41941015a0 0x7f41941977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 -- 172.21.15.137:0/2729088977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4194102790 msgr2=0x7f4194197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4194102790 0x7f4194197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.007+0000 7f4193e00640 1 -- 172.21.15.137:0/2729088977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419419f3e0 con 0x7f4194103150 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.008+0000 7f4193e00640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 0x7f419419ed70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f418c00eaa0 tx=0x7f418c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.008+0000 7f418be00640 1 -- 172.21.15.137:0/2729088977 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00cdb0 con 0x7f4194103150 2024-03-26T14:05:50.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.008+0000 7f418be00640 1 -- 172.21.15.137:0/2729088977 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f418c005020 con 0x7f4194103150 2024-03-26T14:05:50.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.008+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419419f6d0 con 0x7f4194103150 2024-03-26T14:05:50.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.008+0000 7f418be00640 1 -- 172.21.15.137:0/2729088977 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0116d0 con 0x7f4194103150 2024-03-26T14:05:50.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.008+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419419fc40 con 0x7f4194103150 2024-03-26T14:05:50.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.012+0000 7f419d611740 1 Processor -- start 2024-03-26T14:05:50.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f418be00640 1 -- 172.21.15.137:0/2729088977 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f418c0051d0 con 0x7f4194103150 2024-03-26T14:05:50.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 -- start start 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f4196fe00 0x563f41970220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41971450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 0x563f41978220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f418d02c0 con 0x563f41971990 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41878c90 con 0x563f41970fd0 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.013+0000 7f419d611740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f418cc360 con 0x563f4196fe00 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 0x563f41978220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 0x563f41978220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36702/0 (socket says 172.21.15.137:36702) 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 -- 172.21.15.137:0/2215281772 learned_addr learned my addr 172.21.15.137:0/2215281772 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4193400640 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41971450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f4196fe00 0x563f41970220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 -- 172.21.15.137:0/2215281772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f4196fe00 msgr2=0x563f41970220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f4196fe00 0x563f41970220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 -- 172.21.15.137:0/2215281772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 msgr2=0x563f41971450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41971450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f4192a00640 1 -- 172.21.15.137:0/2215281772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f4189d080 con 0x563f41971990 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.014+0000 7f418be00640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41780a2880 0x7f41780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.015+0000 7f418be00640 1 -- 172.21.15.137:0/2729088977 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f418c0ceec0 con 0x7f4194103150 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.015+0000 7f4192a00640 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 0x563f41978220 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4180006010 tx=0x7f418002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e6c84c6d387783 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.015+0000 7f4193400640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41780a2880 0x7f41780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.015+0000 7f4188c00640 1 -- 172.21.15.137:0/2215281772 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418004a070 con 0x563f41971990 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.015+0000 7f4188c00640 1 -- 172.21.15.137:0/2215281772 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4180005050 con 0x563f41971990 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.015+0000 7f4188c00640 1 -- 172.21.15.137:0/2215281772 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180004920 con 0x563f41971990 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 -- 172.21.15.137:0/2215281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 msgr2=0x563f41978220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f4193400640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41780a2880 0x7f41780a4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f41941025f0 tx=0x7f4184005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 0x563f41978220 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4180006010 tx=0x7f418002c610 comp rx=0 tx=0).stop 2024-03-26T14:05:50.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 -- 172.21.15.137:0/2215281772 shutdown_connections 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41971450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f41971990 0x563f41978220 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 --2- 172.21.15.137:0/2215281772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f4196fe00 0x563f41970220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 -- 172.21.15.137:0/2215281772 >> 172.21.15.137:0/2215281772 conn(0x563f41966de0 msgr2=0x563f419671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 -- 172.21.15.137:0/2215281772 shutdown_connections 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.016+0000 7f419d611740 1 -- 172.21.15.137:0/2215281772 wait complete. 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.017+0000 7f419d611740 1 -- start start 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.017+0000 7f419d611740 1 Processor -- start 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41970fd0 0x563f4197a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41971990 0x563f419815d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41978940 con 0x563f4196fe00 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f4188c0d0 con 0x563f41971990 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41978ac0 con 0x563f41970fd0 2024-03-26T14:05:50.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f419d611740 10 client.0 _ll_register_callbacks cb 0x563f41890e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4193400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4193400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36708/0 (socket says 172.21.15.137:36708) 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4193400640 1 -- 172.21.15.137:0/3904185769 learned_addr learned my addr 172.21.15.137:0/3904185769 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4193e00640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41971990 0x563f419815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4192a00640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41970fd0 0x563f4197a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4193400640 1 -- 172.21.15.137:0/3904185769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41970fd0 msgr2=0x563f4197a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.018+0000 7f4193400640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41970fd0 0x563f4197a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f4193400640 1 -- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41971990 msgr2=0x563f419815d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f4193400640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41971990 0x563f419815d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f4193400640 1 -- 172.21.15.137:0/3904185769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f415e43f0 con 0x563f4196fe00 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f4193e00640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41971990 0x563f419815d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f4193400640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197a050 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4184004630 tx=0x7f4184004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffb54537acdcf27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f417d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f417d600640 1 -- 172.21.15.137:0/3904185769 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418401f630 con 0x563f4196fe00 2024-03-26T14:05:50.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f417d600640 1 -- 172.21.15.137:0/3904185769 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f418401f7d0 con 0x563f4196fe00 2024-03-26T14:05:50.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.019+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4180005050 con 0x563f4196fe00 2024-03-26T14:05:50.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f417d600640 1 -- 172.21.15.137:0/3904185769 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f418401f9d0 con 0x563f4196fe00 2024-03-26T14:05:50.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 10 client.16308 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:50.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563f416ae7d0 con 0x563f4196fe00 2024-03-26T14:05:50.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f417d600640 1 -- 172.21.15.137:0/3904185769 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4180005050 con 0x563f4196fe00 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f417d600640 1 -- 172.21.15.137:0/3904185769 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4184027070 con 0x563f4196fe00 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 10 client.16308 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 10 client.16308 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 20 client.16308 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f41985bb0 0x563f41988050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 4 client.16308 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.020+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f418dd800 con 0x563f41985bb0 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.021+0000 7f4192a00640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f41985bb0 0x563f41988050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.022+0000 7f4192a00640 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f41985bb0 0x563f41988050 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4319161cbb924529 server_cookie=9df93220aeb1ce20 in_seq=0 out_seq=0 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.022+0000 7f417d600640 10 client.16308 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f417d600640 1 -- 172.21.15.137:0/3904185769 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f418dd800 con 0x563f41985bb0 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f417d600640 10 client.16308 handle_command_reply: tid=0 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 client.16308 shutdown 2024-03-26T14:05:50.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 msgr2=0x563f4197a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197a050 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4184004630 tx=0x7f4184004730 comp rx=0 tx=0).stop 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 shutdown_connections 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f41985bb0 0x563f41988050 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41971990 0x563f419815d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41970fd0 0x563f4197a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 --2- 172.21.15.137:0/3904185769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197a050 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 >> 172.21.15.137:0/3904185769 conn(0x563f41966de0 msgr2=0x563f41966160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.024+0000 7f417d600640 0 client.16308 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.025+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 shutdown_connections 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.025+0000 7f419d611740 1 -- 172.21.15.137:0/3904185769 wait complete. 2024-03-26T14:05:50.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.025+0000 7f419d611740 20 client.16308 trim_cache size 0 max 16384 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 Processor -- start 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 -- start start 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f41974070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41981560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f419745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941aa3e0 con 0x563f4196fe00 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a7ee0 con 0x563f41970fd0 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f419d611740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a5f60 con 0x563f41971990 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f4193400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f41974070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f4193400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f41974070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36720/0 (socket says 172.21.15.137:36720) 2024-03-26T14:05:50.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.044+0000 7f4193400640 1 -- 172.21.15.137:0/4071057223 learned_addr learned my addr 172.21.15.137:0/4071057223 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4193400640 1 -- 172.21.15.137:0/4071057223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 msgr2=0x563f419745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4193400640 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f419745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4192a00640 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41981560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4193400640 1 -- 172.21.15.137:0/4071057223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 msgr2=0x563f41981560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4193400640 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41981560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4193400640 1 -- 172.21.15.137:0/4071057223 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f41970a50 con 0x563f4196fe00 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f4193400640 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f41974070 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4184012d50 tx=0x7f418401f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247ddc627428c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f417fe00640 1 -- 172.21.15.137:0/4071057223 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418401fd70 con 0x563f4196fe00 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f417fe00640 1 -- 172.21.15.137:0/4071057223 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41840054d0 con 0x563f4196fe00 2024-03-26T14:05:50.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.045+0000 7f417fe00640 1 -- 172.21.15.137:0/4071057223 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184029440 con 0x563f4196fe00 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 -- 172.21.15.137:0/4071057223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 msgr2=0x563f41974070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f41974070 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4184012d50 tx=0x7f418401f8f0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 -- 172.21.15.137:0/4071057223 shutdown_connections 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f41981560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f419745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 --2- 172.21.15.137:0/4071057223 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f41974070 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 -- 172.21.15.137:0/4071057223 >> 172.21.15.137:0/4071057223 conn(0x563f41966de0 msgr2=0x563f4196a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 -- 172.21.15.137:0/4071057223 shutdown_connections 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.046+0000 7f419d611740 1 -- 172.21.15.137:0/4071057223 wait complete. 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 -- start start 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 Processor -- start 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f4197da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f41980aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a6560 con 0x563f4196fe00 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a7880 con 0x563f41970fd0 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41941a6bc0 con 0x563f41971990 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.047+0000 7f419d611740 10 client.0 _ll_register_callbacks cb 0x7f41840044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36724/0 (socket says 172.21.15.137:36724) 2024-03-26T14:05:50.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 -- 172.21.15.137:0/3515239494 learned_addr learned my addr 172.21.15.137:0/3515239494 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4192a00640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f4197da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193e00640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f41980aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 -- 172.21.15.137:0/3515239494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 msgr2=0x563f41980aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f41980aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 -- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 msgr2=0x563f4197da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f4197da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 -- 172.21.15.137:0/3515239494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f41981090 con 0x563f4196fe00 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f4193400640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197d540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4184029e40 tx=0x7f41840056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45365316ce6ad08 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.048+0000 7f417d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f417d600640 1 -- 172.21.15.137:0/3515239494 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184023040 con 0x563f4196fe00 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f417d600640 1 -- 172.21.15.137:0/3515239494 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4184004800 con 0x563f4196fe00 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41840054d0 con 0x563f4196fe00 2024-03-26T14:05:50.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f417d600640 1 -- 172.21.15.137:0/3515239494 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4184048050 con 0x563f4196fe00 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f419d611740 10 client.16314 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563f419e1d40 con 0x563f4196fe00 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.049+0000 7f417d600640 1 -- 172.21.15.137:0/3515239494 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41840054d0 con 0x563f4196fe00 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f417d600640 1 -- 172.21.15.137:0/3515239494 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4184027070 con 0x563f4196fe00 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f419d611740 10 client.16314 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f419d611740 10 client.16314 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f419d611740 20 client.16314 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f419d611740 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f419e2630 0x563f419e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f419d611740 4 client.16314 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:05:50.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f418d7b50 con 0x563f419e2630 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.050+0000 7f4192a00640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f419e2630 0x563f419e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.051+0000 7f4192a00640 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f419e2630 0x563f419e4a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2da0d6e97e784e19 server_cookie=ba89cc876f814233 in_seq=0 out_seq=0 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.051+0000 7f417d600640 10 client.16314 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f417d600640 1 -- 172.21.15.137:0/3515239494 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f418d7b50 con 0x563f419e2630 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f417d600640 10 client.16314 handle_command_reply: tid=0 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 client.16314 shutdown 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 msgr2=0x563f4197d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197d540 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4184029e40 tx=0x7f41840056f0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 shutdown_connections 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563f419e2630 0x563f419e4a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f41970fd0 0x563f4197da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f41971990 0x563f41980aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 --2- 172.21.15.137:0/3515239494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f4196fe00 0x563f4197d540 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 >> 172.21.15.137:0/3515239494 conn(0x563f41966de0 msgr2=0x563f419658d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f417d600640 0 client.16314 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:05:50.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 shutdown_connections 2024-03-26T14:05:50.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 1 -- 172.21.15.137:0/3515239494 wait complete. 2024-03-26T14:05:50.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.052+0000 7f419d611740 20 client.16314 trim_cache size 0 max 16384 2024-03-26T14:05:50.054 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41780a2880 msgr2=0x7f41780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41780a2880 0x7f41780a4d40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f41941025f0 tx=0x7f4184005f90 comp rx=0 tx=0).stop 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 msgr2=0x7f419419ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 0x7f419419ed70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f418c00eaa0 tx=0x7f418c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 shutdown_connections 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41780a2880 0x7f41780a4d40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4194102790 0x7f4194197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4194103150 0x7f419419ed70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 --2- 172.21.15.137:0/2729088977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41941015a0 0x7f41941977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 >> 172.21.15.137:0/2729088977 conn(0x7f41940fcb10 msgr2=0x7f4194105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 shutdown_connections 2024-03-26T14:05:50.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.053+0000 7f4199200640 1 -- 172.21.15.137:0/2729088977 wait complete. 2024-03-26T14:05:50.070 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:05:50.070 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:05:50.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.296+0000 7fa1fbe00640 1 -- 172.21.15.137:0/4199257087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 msgr2=0x7fa1fc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.296+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/4199257087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1043f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009900 tx=0x7fa1ec02bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 -- 172.21.15.137:0/4199257087 shutdown_connections 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/4199257087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc104930 0x7fa1fc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/4199257087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc107260 0x7fa1fc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/4199257087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1043f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 -- 172.21.15.137:0/4199257087 >> 172.21.15.137:0/4199257087 conn(0x7fa1fc0fba10 msgr2=0x7fa1fc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 -- 172.21.15.137:0/4199257087 shutdown_connections 2024-03-26T14:05:50.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.297+0000 7fa1fbe00640 1 -- 172.21.15.137:0/4199257087 wait complete. 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 Processor -- start 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 -- start start 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc104930 0x7fa1fc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc107260 0x7fa1fc1a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc10d110 con 0x7fa1fc102000 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc10cf90 con 0x7fa1fc107260 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1fbe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc10d290 con 0x7fa1fc104930 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36730/0 (socket says 172.21.15.137:36730) 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.298+0000 7fa1faa00640 1 -- 172.21.15.137:0/2389562008 learned_addr learned my addr 172.21.15.137:0/2389562008 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc107260 0x7fa1fc1a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1fa000640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc104930 0x7fa1fc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1faa00640 1 -- 172.21.15.137:0/2389562008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc104930 msgr2=0x7fa1fc199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1faa00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc104930 0x7fa1fc199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1faa00640 1 -- 172.21.15.137:0/2389562008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc107260 msgr2=0x7fa1fc1a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1faa00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc107260 0x7fa1fc1a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1faa00640 1 -- 172.21.15.137:0/2389562008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc1a1600 con 0x7fa1fc102000 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc107260 0x7fa1fc1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.299+0000 7fa1faa00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1999e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.300+0000 7fa1f3400640 1 -- 172.21.15.137:0/2389562008 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec043020 con 0x7fa1fc102000 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.300+0000 7fa1f3400640 1 -- 172.21.15.137:0/2389562008 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1ec0054c0 con 0x7fa1fc102000 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.300+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc1a1890 con 0x7fa1fc102000 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.300+0000 7fa1f3400640 1 -- 172.21.15.137:0/2389562008 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec003d70 con 0x7fa1fc102000 2024-03-26T14:05:50.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.300+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc1a1e30 con 0x7fa1fc102000 2024-03-26T14:05:50.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 Processor -- start 2024-03-26T14:05:50.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 -- start start 2024-03-26T14:05:50.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc5119f0 0x563edc511e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc513040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc519e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edc4762c0 con 0x563edc513580 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edc41ec90 con 0x563edc512bc0 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.303+0000 7fa204836740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edc472360 con 0x563edc5119f0 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc519e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc519e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36738/0 (socket says 172.21.15.137:36738) 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 -- 172.21.15.137:0/1024809698 learned_addr learned my addr 172.21.15.137:0/1024809698 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1f3400640 1 -- 172.21.15.137:0/2389562008 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa1ec046020 con 0x7fa1fc102000 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fb400640 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc5119f0 0x563edc511e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 -- 172.21.15.137:0/1024809698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc5119f0 msgr2=0x563edc511e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc5119f0 0x563edc511e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 -- 172.21.15.137:0/1024809698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 msgr2=0x563edc513040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc513040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fa000640 1 -- 172.21.15.137:0/1024809698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563edc443080 con 0x563edc513580 2024-03-26T14:05:50.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.304+0000 7fa1fb400640 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc5119f0 0x563edc511e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:50.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.305+0000 7fa1fa000640 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc519e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009b10 tx=0x7fa1e800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5b49e74935e6b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.305+0000 7fa1e7e00640 1 -- 172.21.15.137:0/1024809698 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8016070 con 0x563edc513580 2024-03-26T14:05:50.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.305+0000 7fa1e7e00640 1 -- 172.21.15.137:0/1024809698 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1e80054d0 con 0x563edc513580 2024-03-26T14:05:50.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.305+0000 7fa1e7e00640 1 -- 172.21.15.137:0/1024809698 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8011470 con 0x563edc513580 2024-03-26T14:05:50.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.305+0000 7fa1f3400640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1e00a2880 0x7fa1e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa1f3400640 1 -- 172.21.15.137:0/2389562008 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa1ec0f13e0 con 0x7fa1fc102000 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1e00a2880 0x7fa1e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 -- 172.21.15.137:0/1024809698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 msgr2=0x563edc519e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc519e10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009b10 tx=0x7fa1e800e2f0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 -- 172.21.15.137:0/1024809698 shutdown_connections 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc513040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc519e10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 --2- 172.21.15.137:0/1024809698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc5119f0 0x563edc511e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.306+0000 7fa204836740 1 -- 172.21.15.137:0/1024809698 >> 172.21.15.137:0/1024809698 conn(0x563edc50cde0 msgr2=0x563edc50d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.307+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1e00a2880 0x7fa1e00a4d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563edc512ac0 tx=0x7fa1f40095b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.307+0000 7fa204836740 1 -- 172.21.15.137:0/1024809698 shutdown_connections 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.307+0000 7fa204836740 1 -- 172.21.15.137:0/1024809698 wait complete. 2024-03-26T14:05:50.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.307+0000 7fa204836740 1 -- start start 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.307+0000 7fa204836740 1 Processor -- start 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc51c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc513580 0x563edc523240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edc51a590 con 0x563edc5119f0 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edc4320d0 con 0x563edc512bc0 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563edc51a710 con 0x563edc513580 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa204836740 10 client.0 _ll_register_callbacks cb 0x563edc436e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36744/0 (socket says 172.21.15.137:36744) 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 -- 172.21.15.137:0/3676072971 learned_addr learned my addr 172.21.15.137:0/3676072971 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc51c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1fb400640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc513580 0x563edc523240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 -- 172.21.15.137:0/3676072971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc513580 msgr2=0x563edc523240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc513580 0x563edc523240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 -- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 msgr2=0x563edc51c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc51c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1faa00640 1 -- 172.21.15.137:0/3676072971 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563edc18a3f0 con 0x563edc5119f0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.308+0000 7fa1fb400640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc513580 0x563edc523240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa1faa00640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51bcc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0b9f30 tx=0x7fa1ec0bd610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ed6d39a9890a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa1e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3676072971 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec004c50 con 0x563edc5119f0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3676072971 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa1ec0eec20 con 0x563edc5119f0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1e80054d0 con 0x563edc5119f0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3676072971 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1ec0be1d0 con 0x563edc5119f0 2024-03-26T14:05:50.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa204836740 10 client.16326 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563edc2547d0 con 0x563edc5119f0 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.309+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3676072971 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1e80054d0 con 0x563edc5119f0 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3676072971 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa1ec0c5de0 con 0x563edc5119f0 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa204836740 10 client.16326 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa204836740 10 client.16326 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa204836740 20 client.16326 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa204836740 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc527600 0x563edc529a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa204836740 4 client.16326 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563edc483800 con 0x563edc527600 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.310+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc527600 0x563edc529a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.311+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc527600 0x563edc529a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb2a32ce32362ef6 server_cookie=40f780bb6e6c304a in_seq=0 out_seq=0 2024-03-26T14:05:50.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.311+0000 7fa1e4c00640 10 client.16326 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:50.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3676072971 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563edc483800 con 0x563edc527600 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa1e4c00640 10 client.16326 handle_command_reply: tid=0 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 client.16326 shutdown 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 msgr2=0x563edc51bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51bcc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0b9f30 tx=0x7fa1ec0bd610 comp rx=0 tx=0).stop 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 shutdown_connections 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc527600 0x563edc529a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc512bc0 0x563edc51c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc513580 0x563edc523240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa1e4c00640 0 client.16326 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 --2- 172.21.15.137:0/3676072971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51bcc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 >> 172.21.15.137:0/3676072971 conn(0x563edc50cde0 msgr2=0x563edc50c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 shutdown_connections 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 1 -- 172.21.15.137:0/3676072971 wait complete. 2024-03-26T14:05:50.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.313+0000 7fa204836740 20 client.16326 trim_cache size 0 max 16384 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.328+0000 7fa204836740 1 Processor -- start 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.328+0000 7fa204836740 1 -- start start 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.328+0000 7fa204836740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc5119f0 0x563edc515c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.328+0000 7fa204836740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc5231d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa204836740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc5161a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa204836740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1ae7a0 con 0x563edc513580 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa204836740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1ac2a0 con 0x563edc5119f0 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa204836740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1aa320 con 0x563edc512bc0 2024-03-26T14:05:50.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa1fb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc5161a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc5119f0 0x563edc515c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa1fb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc5161a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36754/0 (socket says 172.21.15.137:36754) 2024-03-26T14:05:50.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa1fb400640 1 -- 172.21.15.137:0/2778213991 learned_addr learned my addr 172.21.15.137:0/2778213991 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.329+0000 7fa1fa000640 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc5231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1fb400640 1 -- 172.21.15.137:0/2778213991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 msgr2=0x563edc5231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc5231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1fb400640 1 -- 172.21.15.137:0/2778213991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc5119f0 msgr2=0x563edc515c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc5119f0 0x563edc515c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1fb400640 1 -- 172.21.15.137:0/2778213991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563edc512640 con 0x563edc513580 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1fb400640 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc5161a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4012750 tx=0x7fa1f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44238f44e498c3ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.330+0000 7fa1e7400640 1 -- 172.21.15.137:0/2778213991 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401e820 con 0x563edc513580 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.331+0000 7fa1e7400640 1 -- 172.21.15.137:0/2778213991 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1f401ee70 con 0x563edc513580 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.331+0000 7fa1e7400640 1 -- 172.21.15.137:0/2778213991 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401c410 con 0x563edc513580 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.331+0000 7fa204836740 1 -- 172.21.15.137:0/2778213991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 msgr2=0x563edc5161a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.331+0000 7fa204836740 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc5161a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4012750 tx=0x7fa1f4014000 comp rx=0 tx=0).stop 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 -- 172.21.15.137:0/2778213991 shutdown_connections 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc513580 0x563edc5161a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc5231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 --2- 172.21.15.137:0/2778213991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc5119f0 0x563edc515c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 -- 172.21.15.137:0/2778213991 >> 172.21.15.137:0/2778213991 conn(0x563edc50cde0 msgr2=0x563edc5108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 -- 172.21.15.137:0/2778213991 shutdown_connections 2024-03-26T14:05:50.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 -- 172.21.15.137:0/2778213991 wait complete. 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 -- start start 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.332+0000 7fa204836740 1 Processor -- start 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc51f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc513580 0x563edc522830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1aa920 con 0x563edc5119f0 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1abc40 con 0x563edc513580 2024-03-26T14:05:50.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1aaf80 con 0x563edc512bc0 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa204836740 10 client.0 _ll_register_callbacks cb 0x7fa1ec0bd8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa1faa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36758/0 (socket says 172.21.15.137:36758) 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.333+0000 7fa1faa00640 1 -- 172.21.15.137:0/3140102796 learned_addr learned my addr 172.21.15.137:0/3140102796 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc51f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1fb400640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc513580 0x563edc522830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1faa00640 1 -- 172.21.15.137:0/3140102796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 msgr2=0x563edc51f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1faa00640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc51f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1faa00640 1 -- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc513580 msgr2=0x563edc522830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1faa00640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc513580 0x563edc522830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1faa00640 1 -- 172.21.15.137:0/3140102796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563edc522e20 con 0x563edc5119f0 2024-03-26T14:05:50.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1fb400640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc513580 0x563edc522830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc51f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1faa00640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51f2d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0ba4a0 tx=0x7fa1ec02cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b308b423355a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3140102796 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0dc2a0 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3140102796 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa1ec005010 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.334+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1f401ee70 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3140102796 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1ec0bdad0 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 10 client.16332 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563edc583860 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3140102796 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1f401ee70 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3140102796 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa1ec0b9820 con 0x563edc5119f0 2024-03-26T14:05:50.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 10 client.16332 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 20 client.16332 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc583e00 0x563edc586240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 4 client.16332 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.335+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563edc47db50 con 0x563edc583e00 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.336+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc583e00 0x563edc586240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa1fa000640 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc583e00 0x563edc586240 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36ec052aa70a51 server_cookie=f47cf6509e2e31b0 in_seq=0 out_seq=0 2024-03-26T14:05:50.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa1e4c00640 10 client.16332 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:50.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa1e4c00640 1 -- 172.21.15.137:0/3140102796 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563edc47db50 con 0x563edc583e00 2024-03-26T14:05:50.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa1e4c00640 10 client.16332 handle_command_reply: tid=0 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 client.16332 shutdown 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 msgr2=0x563edc51f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51f2d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0ba4a0 tx=0x7fa1ec02cdd0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 shutdown_connections 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x563edc583e00 0x563edc586240 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563edc513580 0x563edc522830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563edc512bc0 0x563edc51f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 --2- 172.21.15.137:0/3140102796 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563edc5119f0 0x563edc51f2d0 secure :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0ba4a0 tx=0x7fa1ec02cdd0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa1e4c00640 0 client.16332 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.337+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 >> 172.21.15.137:0/3140102796 conn(0x563edc50cde0 msgr2=0x563edc50b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 shutdown_connections 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa204836740 1 -- 172.21.15.137:0/3140102796 wait complete. 2024-03-26T14:05:50.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa204836740 20 client.16332 trim_cache size 0 max 16384 2024-03-26T14:05:50.341 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1e00a2880 msgr2=0x7fa1e00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1e00a2880 0x7fa1e00a4d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x563edc512ac0 tx=0x7fa1f40095b0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 msgr2=0x7fa1fc1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1999e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec000c00 tx=0x7fa1ec0042f0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.338+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 shutdown_connections 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1e00a2880 0x7fa1e00a4d40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1fc107260 0x7fa1fc1a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1fc104930 0x7fa1fc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 --2- 172.21.15.137:0/2389562008 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa1fc102000 0x7fa1fc1999e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 >> 172.21.15.137:0/2389562008 conn(0x7fa1fc0fba10 msgr2=0x7fa1fc101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 shutdown_connections 2024-03-26T14:05:50.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.339+0000 7fa1fbe00640 1 -- 172.21.15.137:0/2389562008 wait complete. 2024-03-26T14:05:50.356 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:05:50.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:05:50.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 -- 172.21.15.137:0/1438094465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c0107440 msgr2=0x7f33c0109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 --2- 172.21.15.137:0/1438094465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c0107440 0x7f33c0109830 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f33b8009900 tx=0x7f33b802bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 -- 172.21.15.137:0/1438094465 shutdown_connections 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 --2- 172.21.15.137:0/1438094465 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c0107440 0x7f33c0109830 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 --2- 172.21.15.137:0/1438094465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0104b10 0x7f33c0106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 --2- 172.21.15.137:0/1438094465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c01021e0 0x7f33c01045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.566+0000 7f33c5a00640 1 -- 172.21.15.137:0/1438094465 >> 172.21.15.137:0/1438094465 conn(0x7f33c00fbb30 msgr2=0x7f33c00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.567+0000 7f33c5a00640 1 -- 172.21.15.137:0/1438094465 shutdown_connections 2024-03-26T14:05:50.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.567+0000 7f33c5a00640 1 -- 172.21.15.137:0/1438094465 wait complete. 2024-03-26T14:05:50.781 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:50 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3447760543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:05:50.781 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:50 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3447760543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 Processor -- start 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 -- start start 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 0x7f33c0199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0104b10 0x7f33c019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0107440 0x7f33c01a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010d1e0 con 0x7f33c01021e0 2024-03-26T14:05:50.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010d060 con 0x7f33c0104b10 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33c5a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010d360 con 0x7f33c0107440 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33bea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 0x7f33c0199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33bea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 0x7f33c0199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36786/0 (socket says 172.21.15.137:36786) 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.568+0000 7f33bea00640 1 -- 172.21.15.137:0/2757329674 learned_addr learned my addr 172.21.15.137:0/2757329674 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33b6000640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0104b10 0x7f33c019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bea00640 1 -- 172.21.15.137:0/2757329674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0107440 msgr2=0x7f33c01a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bf400640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0107440 0x7f33c01a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bea00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0107440 0x7f33c01a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bea00640 1 -- 172.21.15.137:0/2757329674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0104b10 msgr2=0x7f33c019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bea00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0104b10 0x7f33c019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bea00640 1 -- 172.21.15.137:0/2757329674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c01a1760 con 0x7f33c01021e0 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33b6000640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0104b10 0x7f33c019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bf400640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0107440 0x7f33c01a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.569+0000 7f33bea00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 0x7f33c0199b40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009d50 tx=0x7f33b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.570+0000 7f33b7e00640 1 -- 172.21.15.137:0/2757329674 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0003b70 con 0x7f33c01021e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.570+0000 7f33b7e00640 1 -- 172.21.15.137:0/2757329674 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33b0003d10 con 0x7f33c01021e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.570+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c01a1a50 con 0x7f33c01021e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.570+0000 7f33b7e00640 1 -- 172.21.15.137:0/2757329674 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b00108f0 con 0x7f33c01021e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.570+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c01a1fa0 con 0x7f33c01021e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.573+0000 7f33c8772740 1 Processor -- start 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.573+0000 7f33c8772740 1 -- start start 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33c8772740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bb6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33c8772740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bb7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33c8772740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bbec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33c8772740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23b1b2c0 con 0x55bb23bb6810 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33c8772740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23ac3c90 con 0x55bb23bb79e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33c8772740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23b17360 con 0x55bb23bb83a0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33b7e00640 1 -- 172.21.15.137:0/2757329674 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f33b0003ec0 con 0x7f33c01021e0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bb6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bb6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36790/0 (socket says 172.21.15.137:36790) 2024-03-26T14:05:50.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 -- 172.21.15.137:0/3224463843 learned_addr learned my addr 172.21.15.137:0/3224463843 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33b6000640 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bbec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 -- 172.21.15.137:0/3224463843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 msgr2=0x55bb23bbec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bbec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 -- 172.21.15.137:0/3224463843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 msgr2=0x55bb23bb7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bb7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.574+0000 7f33bf400640 1 -- 172.21.15.137:0/3224463843 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb23ae8080 con 0x55bb23bb6810 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.575+0000 7f33bf400640 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bb6c30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005ab0 tx=0x7f33b8009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baff8758d03571c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.575+0000 7f33afe00640 1 -- 172.21.15.137:0/3224463843 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8043020 con 0x55bb23bb6810 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.575+0000 7f33afe00640 1 -- 172.21.15.137:0/3224463843 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33b8004040 con 0x55bb23bb6810 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.575+0000 7f33afe00640 1 -- 172.21.15.137:0/3224463843 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80046c0 con 0x55bb23bb6810 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33b7e00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f33a40a2800 0x7f33a40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33b7e00640 1 -- 172.21.15.137:0/2757329674 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f33b00cf650 con 0x7f33c01021e0 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33bea00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f33a40a2800 0x7f33a40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 -- 172.21.15.137:0/3224463843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 msgr2=0x55bb23bb6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bb6c30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005ab0 tx=0x7f33b8009670 comp rx=0 tx=0).stop 2024-03-26T14:05:50.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 -- 172.21.15.137:0/3224463843 shutdown_connections 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bb7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bbec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 --2- 172.21.15.137:0/3224463843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bb6c30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.576+0000 7f33c8772740 1 -- 172.21.15.137:0/3224463843 >> 172.21.15.137:0/3224463843 conn(0x55bb23bb1c00 msgr2=0x55bb23bb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.577+0000 7f33c8772740 1 -- 172.21.15.137:0/3224463843 shutdown_connections 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.577+0000 7f33c8772740 1 -- 172.21.15.137:0/3224463843 wait complete. 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.577+0000 7f33bea00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f33a40a2800 0x7f33a40a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f33b00cf8f0 tx=0x7f33b0009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.577+0000 7f33c8772740 1 -- start start 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.577+0000 7f33c8772740 1 Processor -- start 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb79e0 0x55bb23bc0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 0x55bb23bc7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23bbf350 con 0x55bb23bb83a0 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23ad70d0 con 0x55bb23bb6810 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb23bbf4d0 con 0x55bb23bb79e0 2024-03-26T14:05:50.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33c8772740 10 client.0 _ll_register_callbacks cb 0x55bb23adbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 0x55bb23bc7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 0x55bb23bc7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36796/0 (socket says 172.21.15.137:36796) 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33bf400640 1 -- 172.21.15.137:0/3713931397 learned_addr learned my addr 172.21.15.137:0/3713931397 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33b6000640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb79e0 0x55bb23bc0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.578+0000 7f33bf400640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33bf400640 1 -- 172.21.15.137:0/3713931397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb79e0 msgr2=0x55bb23bc0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33bf400640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb79e0 0x55bb23bc0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33bf400640 1 -- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 msgr2=0x55bb23bc0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33bf400640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33bf400640 1 -- 172.21.15.137:0/3713931397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb2382f3f0 con 0x55bb23bb83a0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33b6000640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb79e0 0x55bb23bc0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33bf400640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 0x55bb23bc7fb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f33b802c6a0 tx=0x7f33b80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f047a9782232aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33acc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33acc00640 1 -- 172.21.15.137:0/3713931397 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8043030 con 0x55bb23bb83a0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33acc00640 1 -- 172.21.15.137:0/3713931397 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f33b80048a0 con 0x55bb23bb83a0 2024-03-26T14:05:50.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.579+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33b8004040 con 0x55bb23bb83a0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33acc00640 1 -- 172.21.15.137:0/3713931397 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33b8004aa0 con 0x55bb23bb83a0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 10 client.16344 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb238f97d0 con 0x55bb23bb83a0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33acc00640 1 -- 172.21.15.137:0/3713931397 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33b8004040 con 0x55bb23bb83a0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33acc00640 1 -- 172.21.15.137:0/3713931397 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f33b8034440 con 0x55bb23bb83a0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 10 client.16344 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 20 client.16344 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23bcc110 0x55bb23bce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 4 client.16344 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.580+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb23b28800 con 0x55bb23bcc110 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.581+0000 7f33b6000640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23bcc110 0x55bb23bce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.582+0000 7f33b6000640 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23bcc110 0x55bb23bce500 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac1dd058823e40d2 server_cookie=2348f2288704b63 in_seq=0 out_seq=0 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.582+0000 7f33acc00640 10 client.16344 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33acc00640 1 -- 172.21.15.137:0/3713931397 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bb23b28800 con 0x55bb23bcc110 2024-03-26T14:05:50.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33acc00640 10 client.16344 handle_command_reply: tid=0 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 client.16344 shutdown 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 msgr2=0x55bb23bc7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 0x55bb23bc7fb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f33b802c6a0 tx=0x7f33b80058b0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 shutdown_connections 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23bcc110 0x55bb23bce500 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb83a0 0x55bb23bc7fb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb79e0 0x55bb23bc0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 --2- 172.21.15.137:0/3713931397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 >> 172.21.15.137:0/3713931397 conn(0x55bb23bb1c00 msgr2=0x55bb23bb0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33acc00640 0 client.16344 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 shutdown_connections 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 1 -- 172.21.15.137:0/3713931397 wait complete. 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.584+0000 7f33c8772740 20 client.16344 trim_cache size 0 max 16384 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 Processor -- start 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 -- start start 2024-03-26T14:05:50.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bd2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bc7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bd31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ae9a0 con 0x55bb23bb6810 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ac4a0 con 0x55bb23bb79e0 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33c8772740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01aa520 con 0x55bb23bb83a0 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bd2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bd2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36808/0 (socket says 172.21.15.137:36808) 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.603+0000 7f33bf400640 1 -- 172.21.15.137:0/1612222720 learned_addr learned my addr 172.21.15.137:0/1612222720 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33b6000640 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bc7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33bf400640 1 -- 172.21.15.137:0/1612222720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 msgr2=0x55bb23bd31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33bf400640 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bd31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33bf400640 1 -- 172.21.15.137:0/1612222720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 msgr2=0x55bb23bc7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33bf400640 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bc7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33bf400640 1 -- 172.21.15.137:0/1612222720 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb23bb2970 con 0x55bb23bb6810 2024-03-26T14:05:50.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33b6000640 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bc7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33bf400640 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bd2c60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005940 tx=0x7f33b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b3b7afbef6c96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33af400640 1 -- 172.21.15.137:0/1612222720 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8063070 con 0x55bb23bb6810 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33af400640 1 -- 172.21.15.137:0/1612222720 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33b8034e50 con 0x55bb23bb6810 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.604+0000 7f33af400640 1 -- 172.21.15.137:0/1612222720 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8043070 con 0x55bb23bb6810 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 -- 172.21.15.137:0/1612222720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 msgr2=0x55bb23bd2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bd2c60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f33b8005940 tx=0x7f33b8009260 comp rx=0 tx=0).stop 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 -- 172.21.15.137:0/1612222720 shutdown_connections 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb79e0 0x55bb23bc7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bd31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 --2- 172.21.15.137:0/1612222720 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb6810 0x55bb23bd2c60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 -- 172.21.15.137:0/1612222720 >> 172.21.15.137:0/1612222720 conn(0x55bb23bb1c00 msgr2=0x55bb23bb56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 -- 172.21.15.137:0/1612222720 shutdown_connections 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.605+0000 7f33c8772740 1 -- 172.21.15.137:0/1612222720 wait complete. 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.606+0000 7f33c8772740 1 -- start start 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.606+0000 7f33c8772740 1 Processor -- start 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 0x55bb23bc5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bc6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01aab20 con 0x55bb23bb79e0 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01abe40 con 0x55bb23bb6810 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c01ab180 con 0x55bb23bb83a0 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33c8772740 10 client.0 _ll_register_callbacks cb 0x7f33b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 0x55bb23bc5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 0x55bb23bc5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36820/0 (socket says 172.21.15.137:36820) 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 -- 172.21.15.137:0/925414695 learned_addr learned my addr 172.21.15.137:0/925414695 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33bf400640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33bf400640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bc6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 -- 172.21.15.137:0/925414695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 msgr2=0x55bb23bc6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bc6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 -- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 msgr2=0x55bb23bc5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.607+0000 7f33b6000640 1 -- 172.21.15.137:0/925414695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb23bc75d0 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33b6000640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 0x55bb23bc5fd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f33a8000c00 tx=0x7f33a8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944a6d8cf3ce66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33acc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33acc00640 1 -- 172.21.15.137:0/925414695 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a801e070 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33acc00640 1 -- 172.21.15.137:0/925414695 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f33a8005250 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33b8034e50 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33acc00640 1 -- 172.21.15.137:0/925414695 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33a802a050 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33c8772740 10 client.16350 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.608+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bb23c27cd0 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33acc00640 1 -- 172.21.15.137:0/925414695 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33b8034e50 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33acc00640 1 -- 172.21.15.137:0/925414695 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f33a800f040 con 0x55bb23bb79e0 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33c8772740 10 client.16350 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33c8772740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33c8772740 20 client.16350 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33c8772740 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23c28560 0x55bb23c2a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33c8772740 4 client.16350 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:05:50.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.609+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bb23b22b50 con 0x55bb23c28560 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.610+0000 7f33bf400640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23c28560 0x55bb23c2a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.610+0000 7f33bf400640 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23c28560 0x55bb23c2a980 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4110b1c85a035c1 server_cookie=62686e21628c65b9 in_seq=0 out_seq=0 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.610+0000 7f33acc00640 10 client.16350 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33acc00640 1 -- 172.21.15.137:0/925414695 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bb23b22b50 con 0x55bb23c28560 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33acc00640 10 client.16350 handle_command_reply: tid=0 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 client.16350 shutdown 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 msgr2=0x55bb23bc5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 0x55bb23bc5fd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f33a8000c00 tx=0x7f33a8012820 comp rx=0 tx=0).stop 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 shutdown_connections 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bb23c28560 0x55bb23c2a980 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bb23bb83a0 0x55bb23bc6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bb23bb79e0 0x55bb23bc5fd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 --2- 172.21.15.137:0/925414695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb23bb6810 0x55bb23bc5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 >> 172.21.15.137:0/925414695 conn(0x55bb23bb1c00 msgr2=0x55bb23bbac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33acc00640 0 client.16350 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 shutdown_connections 2024-03-26T14:05:50.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 1 -- 172.21.15.137:0/925414695 wait complete. 2024-03-26T14:05:50.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.611+0000 7f33c8772740 20 client.16350 trim_cache size 0 max 16384 2024-03-26T14:05:50.805 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:05:50.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f33a40a2800 msgr2=0x7f33a40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f33a40a2800 0x7f33a40a4cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f33b00cf8f0 tx=0x7f33b0009200 comp rx=0 tx=0).stop 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 msgr2=0x7f33c0199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 0x7f33c0199b40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009d50 tx=0x7f33b000c6a0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 shutdown_connections 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f33a40a2800 0x7f33a40a4cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0104b10 0x7f33c019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33c0107440 0x7f33c01a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 --2- 172.21.15.137:0/2757329674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f33c01021e0 0x7f33c0199b40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 >> 172.21.15.137:0/2757329674 conn(0x7f33c00fbb30 msgr2=0x7f33c0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.612+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 shutdown_connections 2024-03-26T14:05:50.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:50.613+0000 7f33c5a00640 1 -- 172.21.15.137:0/2757329674 wait complete. 2024-03-26T14:05:50.806 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:50 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3447760543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:05:51.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:51 smithi179 ceph-mon[35472]: pgmap v563: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:51.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:51 smithi159 ceph-mon[35501]: pgmap v563: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:51.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:51 smithi137 ceph-mon[31539]: pgmap v563: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-26T14:05:51.807 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:05:51.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:05:51.808 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 -- 172.21.15.137:0/509986888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4108db0 msgr2=0x7f84f4109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 --2- 172.21.15.137:0/509986888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4108db0 0x7f84f4109230 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009860 tx=0x7f84e002bb70 comp rx=0 tx=0).stop 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 -- 172.21.15.137:0/509986888 shutdown_connections 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 --2- 172.21.15.137:0/509986888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f4109770 0x7f84f4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 --2- 172.21.15.137:0/509986888 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4108db0 0x7f84f4109230 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 --2- 172.21.15.137:0/509986888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f4107bc0 0x7f84f4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 -- 172.21.15.137:0/509986888 >> 172.21.15.137:0/509986888 conn(0x7f84f4076b70 msgr2=0x7f84f4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 -- 172.21.15.137:0/509986888 shutdown_connections 2024-03-26T14:05:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.064+0000 7f84f8c00640 1 -- 172.21.15.137:0/509986888 wait complete. 2024-03-26T14:05:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.065+0000 7f84f8c00640 1 Processor -- start 2024-03-26T14:05:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.065+0000 7f84f8c00640 1 -- start start 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 0x7f84f419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f8c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f4108db0 0x7f84f419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f4109770 0x7f84f41a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f8c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41136d0 con 0x7f84f4107bc0 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f8c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4113550 con 0x7f84f4109770 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f8c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4113850 con 0x7f84f4108db0 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 0x7f84f419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 0x7f84f419ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36832/0 (socket says 172.21.15.137:36832) 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f3400640 1 -- 172.21.15.137:0/2767785150 learned_addr learned my addr 172.21.15.137:0/2767785150 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f3e00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f4109770 0x7f84f41a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.066+0000 7f84f2a00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f4108db0 0x7f84f419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84f3400640 1 -- 172.21.15.137:0/2767785150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f4108db0 msgr2=0x7f84f419e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84f3400640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f4108db0 0x7f84f419e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84f3400640 1 -- 172.21.15.137:0/2767785150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f4109770 msgr2=0x7f84f41a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84f3400640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f4109770 0x7f84f41a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84f3400640 1 -- 172.21.15.137:0/2767785150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f41a5a10 con 0x7f84f4107bc0 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84f3400640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 0x7f84f419ddf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f84e400e9a0 tx=0x7f84e400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.067+0000 7f84ebe00640 1 -- 172.21.15.137:0/2767785150 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e400cd80 con 0x7f84f4107bc0 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.068+0000 7f84ebe00640 1 -- 172.21.15.137:0/2767785150 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84e40108e0 con 0x7f84f4107bc0 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.068+0000 7f84ebe00640 1 -- 172.21.15.137:0/2767785150 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4018950 con 0x7f84f4107bc0 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.068+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f41a5d00 con 0x7f84f4107bc0 2024-03-26T14:05:52.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.068+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f41a61c0 con 0x7f84f4107bc0 2024-03-26T14:05:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.071+0000 7f84ebe00640 1 -- 172.21.15.137:0/2767785150 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f84e4010a90 con 0x7f84f4107bc0 2024-03-26T14:05:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.071+0000 7f84fcfa1740 1 Processor -- start 2024-03-26T14:05:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 -- start start 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e917386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e9173d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e917402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e9169deb0 con 0x560e9173de10 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e91646a60 con 0x560e917382b0 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e91699f50 con 0x560e91738ca0 2024-03-26T14:05:52.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e917402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e917402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36836/0 (socket says 172.21.15.137:36836) 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84f2a00640 1 -- 172.21.15.137:0/1748550831 learned_addr learned my addr 172.21.15.137:0/1748550831 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.072+0000 7f84f3e00640 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e917386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f3400640 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e9173d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f2a00640 1 -- 172.21.15.137:0/1748550831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 msgr2=0x560e9173d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f2a00640 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e9173d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f2a00640 1 -- 172.21.15.137:0/1748550831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 msgr2=0x560e917386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f2a00640 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e917386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f2a00640 1 -- 172.21.15.137:0/1748550831 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9147a7e0 con 0x560e9173de10 2024-03-26T14:05:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f3e00640 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e917386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84ebe00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84d80a2880 0x7f84d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84f2a00640 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e917402a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000c00 tx=0x7f84e0005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774cc50a45071727 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.073+0000 7f84ebe00640 1 -- 172.21.15.137:0/2767785150 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84e40cf210 con 0x7f84f4107bc0 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84dfe00640 1 -- 172.21.15.137:0/1748550831 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0039040 con 0x560e9173de10 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84f3e00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84d80a2880 0x7f84d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84dfe00640 1 -- 172.21.15.137:0/1748550831 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84e0043070 con 0x560e9173de10 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84dfe00640 1 -- 172.21.15.137:0/1748550831 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0005440 con 0x560e9173de10 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84fcfa1740 1 -- 172.21.15.137:0/1748550831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 msgr2=0x560e917402a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e917402a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f84e0000c00 tx=0x7f84e0005da0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.074+0000 7f84f3e00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84d80a2880 0x7f84d80a4d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x560e91739500 tx=0x7f84ec009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 -- 172.21.15.137:0/1748550831 shutdown_connections 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e917402a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e9173d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/1748550831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e917386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 -- 172.21.15.137:0/1748550831 >> 172.21.15.137:0/1748550831 conn(0x560e91733820 msgr2=0x560e91733c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 -- 172.21.15.137:0/1748550831 shutdown_connections 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 -- 172.21.15.137:0/1748550831 wait complete. 2024-03-26T14:05:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 -- start start 2024-03-26T14:05:52.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.075+0000 7f84fcfa1740 1 Processor -- start 2024-03-26T14:05:52.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e917382b0 0x560e917423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 0x560e91742910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9173de10 0x560e91749950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e91740d60 con 0x560e91738ca0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e91659eb0 con 0x560e9173de10 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e91740ee0 con 0x560e917382b0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84fcfa1740 10 client.0 _ll_register_callbacks cb 0x560e9164e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 0x560e91742910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84f2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 0x560e91742910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36844/0 (socket says 172.21.15.137:36844) 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84f2a00640 1 -- 172.21.15.137:0/3121740249 learned_addr learned my addr 172.21.15.137:0/3121740249 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84f3e00640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9173de10 0x560e91749950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.076+0000 7f84f3400640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e917382b0 0x560e917423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84f2a00640 1 -- 172.21.15.137:0/3121740249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e917382b0 msgr2=0x560e917423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84f2a00640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e917382b0 0x560e917423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84f2a00640 1 -- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9173de10 msgr2=0x560e91749950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84f2a00640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9173de10 0x560e91749950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84f2a00640 1 -- 172.21.15.137:0/3121740249 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e914daf70 con 0x560e91738ca0 2024-03-26T14:05:52.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84f2a00640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 0x560e91742910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f84e002f020 tx=0x7f84e00345c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83432bff511b5a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84dcc00640 1 -- 172.21.15.137:0/3121740249 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0039050 con 0x560e91738ca0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84dcc00640 1 -- 172.21.15.137:0/3121740249 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84e00051c0 con 0x560e91738ca0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.077+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84e0043070 con 0x560e91738ca0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84dcc00640 1 -- 172.21.15.137:0/3121740249 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e00053c0 con 0x560e91738ca0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84fcfa1740 10 client.16362 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e91372170 con 0x560e91738ca0 2024-03-26T14:05:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84dcc00640 1 -- 172.21.15.137:0/3121740249 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84e0043070 con 0x560e91738ca0 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84dcc00640 1 -- 172.21.15.137:0/3121740249 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f84e00047e0 con 0x560e91738ca0 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84fcfa1740 10 client.16362 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84fcfa1740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.078+0000 7f84fcfa1740 20 client.16362 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.079+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e9174de40 0x560e91750230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.079+0000 7f84fcfa1740 4 client.16362 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.079+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e916ab3f0 con 0x560e9174de40 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.079+0000 7f84f3400640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e9174de40 0x560e91750230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.080+0000 7f84dcc00640 10 client.16362 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.080+0000 7f84f3400640 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e9174de40 0x560e91750230 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a235e73f78ed0a6f server_cookie=aa413ede1a0a8200 in_seq=0 out_seq=0 2024-03-26T14:05:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84dcc00640 1 -- 172.21.15.137:0/3121740249 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560e916ab3f0 con 0x560e9174de40 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84dcc00640 10 client.16362 handle_command_reply: tid=0 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 client.16362 shutdown 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 msgr2=0x560e91742910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 0x560e91742910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f84e002f020 tx=0x7f84e00345c0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 shutdown_connections 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e9174de40 0x560e91750230 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9173de10 0x560e91749950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e91738ca0 0x560e91742910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/3121740249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e917382b0 0x560e917423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 >> 172.21.15.137:0/3121740249 conn(0x560e91733820 msgr2=0x560e91732a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.081+0000 7f84dcc00640 0 client.16362 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.082+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 shutdown_connections 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.082+0000 7f84fcfa1740 1 -- 172.21.15.137:0/3121740249 wait complete. 2024-03-26T14:05:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.082+0000 7f84fcfa1740 20 client.16362 trim_cache size 0 max 16384 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 Processor -- start 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 -- start start 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91732900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e91749910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91754a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41b2ae0 con 0x560e9173de10 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41b05e0 con 0x560e917382b0 2024-03-26T14:05:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.099+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41ae660 con 0x560e91738ca0 2024-03-26T14:05:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91754a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91732900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91754a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36854/0 (socket says 172.21.15.137:36854) 2024-03-26T14:05:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 -- 172.21.15.137:0/2455943078 learned_addr learned my addr 172.21.15.137:0/2455943078 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f2a00640 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e91749910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 -- 172.21.15.137:0/2455943078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 msgr2=0x560e91749910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e91749910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 -- 172.21.15.137:0/2455943078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 msgr2=0x560e91732900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91732900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f3e00640 1 -- 172.21.15.137:0/2455943078 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e9173a750 con 0x560e9173de10 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.100+0000 7f84f2a00640 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e91749910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.101+0000 7f84f3e00640 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91754a10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f84ec010670 tx=0x7f84ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cbb949ecb7c8c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.101+0000 7f84df400640 1 -- 172.21.15.137:0/2455943078 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec02d070 con 0x560e9173de10 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.101+0000 7f84df400640 1 -- 172.21.15.137:0/2455943078 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84ec01cc30 con 0x560e9173de10 2024-03-26T14:05:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.101+0000 7f84df400640 1 -- 172.21.15.137:0/2455943078 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec028cd0 con 0x560e9173de10 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 -- 172.21.15.137:0/2455943078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 msgr2=0x560e91754a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91754a10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f84ec010670 tx=0x7f84ec019000 comp rx=0 tx=0).stop 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 -- 172.21.15.137:0/2455943078 shutdown_connections 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91754a10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e91749910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/2455943078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91732900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 -- 172.21.15.137:0/2455943078 >> 172.21.15.137:0/2455943078 conn(0x560e91733820 msgr2=0x560e917321f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 -- 172.21.15.137:0/2455943078 shutdown_connections 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 -- 172.21.15.137:0/2455943078 wait complete. 2024-03-26T14:05:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.102+0000 7f84fcfa1740 1 -- start start 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 Processor -- start 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91748290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e917487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91748d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41aec60 con 0x560e9173de10 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41aff80 con 0x560e917382b0 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f41af2c0 con 0x560e91738ca0 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84fcfa1740 10 client.0 _ll_register_callbacks cb 0x7f84e4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91748d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91748d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36858/0 (socket says 172.21.15.137:36858) 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.103+0000 7f84f3e00640 1 -- 172.21.15.137:0/882125703 learned_addr learned my addr 172.21.15.137:0/882125703 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3400640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91748290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f2a00640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e917487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3e00640 1 -- 172.21.15.137:0/882125703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 msgr2=0x560e917487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3e00640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e917487d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3e00640 1 -- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 msgr2=0x560e91748290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3e00640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91748290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3e00640 1 -- 172.21.15.137:0/882125703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e917a9710 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3400640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91748290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84f3e00640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91748d10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84ec011010 tx=0x7f84ec0285a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1f33db2491d097 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.104+0000 7f84dcc00640 1 -- 172.21.15.137:0/882125703 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec02d030 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84dcc00640 1 -- 172.21.15.137:0/882125703 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84ec004c50 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84ec01cc30 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84dcc00640 1 -- 172.21.15.137:0/882125703 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84ec03e050 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 10 client.16368 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560e917aaa50 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84dcc00640 1 -- 172.21.15.137:0/882125703 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84ec01cc30 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84dcc00640 1 -- 172.21.15.137:0/882125703 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f84ec027070 con 0x560e9173de10 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 10 client.16368 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 20 client.16368 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e917ab3b0 0x560e917ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 4 client.16368 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:05:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.105+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560e916a5740 con 0x560e917ab3b0 2024-03-26T14:05:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.106+0000 7f84f3400640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e917ab3b0 0x560e917ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.107+0000 7f84dcc00640 10 client.16368 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.107+0000 7f84f3400640 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e917ab3b0 0x560e917ad7d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d225a21480d4bf79 server_cookie=5bc163e9fc09045b in_seq=0 out_seq=0 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84dcc00640 1 -- 172.21.15.137:0/882125703 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560e916a5740 con 0x560e917ab3b0 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84dcc00640 10 client.16368 handle_command_reply: tid=0 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 client.16368 shutdown 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 msgr2=0x560e91748d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91748d10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84ec011010 tx=0x7f84ec0285a0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 shutdown_connections 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560e917ab3b0 0x560e917ad7d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560e9173de10 0x560e91748d10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e91738ca0 0x560e917487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 --2- 172.21.15.137:0/882125703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e917382b0 0x560e91748290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 >> 172.21.15.137:0/882125703 conn(0x560e91733820 msgr2=0x560e91739fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84dcc00640 0 client.16368 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.109+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 shutdown_connections 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84fcfa1740 1 -- 172.21.15.137:0/882125703 wait complete. 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84fcfa1740 20 client.16368 trim_cache size 0 max 16384 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "80ac444a-f58b-41c2-80ee-3c2b13d237ba", 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:05:52.110 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84d80a2880 msgr2=0x7f84d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84d80a2880 0x7f84d80a4d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x560e91739500 tx=0x7f84ec009600 comp rx=0 tx=0).stop 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 msgr2=0x7f84f419ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 0x7f84f419ddf0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f84e400e9a0 tx=0x7f84e400ee60 comp rx=0 tx=0).stop 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 shutdown_connections 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84d80a2880 0x7f84d80a4d40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84f4109770 0x7f84f41a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84f4108db0 0x7f84f419e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.110+0000 7f84f8c00640 1 --2- 172.21.15.137:0/2767785150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4107bc0 0x7f84f419ddf0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.111+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 >> 172.21.15.137:0/2767785150 conn(0x7f84f4076b70 msgr2=0x7f84f410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.111+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 shutdown_connections 2024-03-26T14:05:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.111+0000 7f84f8c00640 1 -- 172.21.15.137:0/2767785150 wait complete. 2024-03-26T14:05:52.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:05:52.127 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:05:52.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 -- 172.21.15.137:0/3951305110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8102790 msgr2=0x7f2fb8102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3951305110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8102790 0x7f2fb8102c10 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009900 tx=0x7f2fa002bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:52.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 -- 172.21.15.137:0/3951305110 shutdown_connections 2024-03-26T14:05:52.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3951305110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fb8103150 0x7f2fb81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3951305110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8102790 0x7f2fb8102c10 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3951305110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fb81015a0 0x7f2fb81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 -- 172.21.15.137:0/3951305110 >> 172.21.15.137:0/3951305110 conn(0x7f2fb80fcb10 msgr2=0x7f2fb80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.348+0000 7f2fbe600640 1 -- 172.21.15.137:0/3951305110 shutdown_connections 2024-03-26T14:05:52.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.349+0000 7f2fbe600640 1 -- 172.21.15.137:0/3951305110 wait complete. 2024-03-26T14:05:52.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.350+0000 7f2fbe600640 1 Processor -- start 2024-03-26T14:05:52.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.350+0000 7f2fbe600640 1 -- start start 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbe600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fb81015a0 0x7f2fb81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbe600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fb8102790 0x7f2fb8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbe600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 0x7f2fb81a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbe600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb810d0e0 con 0x7f2fb8103150 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbe600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb810cf60 con 0x7f2fb8102790 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbe600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb810d260 con 0x7f2fb81015a0 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbdc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 0x7f2fb81a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbdc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 0x7f2fb81a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36878/0 (socket says 172.21.15.137:36878) 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbdc00640 1 -- 172.21.15.137:0/3099235422 learned_addr learned my addr 172.21.15.137:0/3099235422 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fbd200640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fb81015a0 0x7f2fb81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.351+0000 7f2fafe00640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fb8102790 0x7f2fb8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.352+0000 7f2fbdc00640 1 -- 172.21.15.137:0/3099235422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fb81015a0 msgr2=0x7f2fb81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.352+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fb81015a0 0x7f2fb81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.352+0000 7f2fbdc00640 1 -- 172.21.15.137:0/3099235422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fb8102790 msgr2=0x7f2fb8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.352+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fb8102790 0x7f2fb8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.352+0000 7f2fbdc00640 1 -- 172.21.15.137:0/3099235422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb81a15c0 con 0x7f2fb8103150 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.352+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 0x7f2fb81a0f50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000eaa0 tx=0x7f2fb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.353+0000 7f2fb6000640 1 -- 172.21.15.137:0/3099235422 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000cdb0 con 0x7f2fb8103150 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.353+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb81a18b0 con 0x7f2fb8103150 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.353+0000 7f2fb6000640 1 -- 172.21.15.137:0/3099235422 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fb0005020 con 0x7f2fb8103150 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.353+0000 7f2fb6000640 1 -- 172.21.15.137:0/3099235422 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00116d0 con 0x7f2fb8103150 2024-03-26T14:05:52.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.353+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb81a1dd0 con 0x7f2fb8103150 2024-03-26T14:05:52.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.356+0000 7f2fc2968740 1 Processor -- start 2024-03-26T14:05:52.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fb6000640 1 -- 172.21.15.137:0/3099235422 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2fb00051d0 con 0x7f2fb8103150 2024-03-26T14:05:52.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 -- start start 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89d050 0x55dfcc89d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc802250 con 0x55dfcc89da40 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc7aaa30 con 0x55dfcc8a2bb0 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.357+0000 7f2fc2968740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc7fe2f0 con 0x55dfcc89d050 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8a2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36894/0 (socket says 172.21.15.137:36894) 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 -- 172.21.15.137:0/4278703175 learned_addr learned my addr 172.21.15.137:0/4278703175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fafe00640 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fafe00640 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89d050 0x55dfcc89d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 -- 172.21.15.137:0/4278703175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89d050 msgr2=0x55dfcc89d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89d050 0x55dfcc89d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 -- 172.21.15.137:0/4278703175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 msgr2=0x55dfcc8a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.358+0000 7f2fbd200640 1 -- 172.21.15.137:0/4278703175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcc795690 con 0x55dfcc89da40 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fb6000640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f9c0a2800 0x7f2f9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fbd200640 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8a2540 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800bcb0 tx=0x7f2fa80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2001d92236582 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fb6000640 1 -- 172.21.15.137:0/3099235422 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2fb00cefd0 con 0x7f2fb8103150 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fafe00640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f9c0a2800 0x7f2f9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fae000640 1 -- 172.21.15.137:0/4278703175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8005020 con 0x55dfcc89da40 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fae000640 1 -- 172.21.15.137:0/4278703175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fa80047a0 con 0x55dfcc89da40 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.359+0000 7f2fae000640 1 -- 172.21.15.137:0/4278703175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011640 con 0x55dfcc89da40 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 -- 172.21.15.137:0/4278703175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 msgr2=0x55dfcc8a2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fafe00640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f9c0a2800 0x7f2f9c0a4cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009230 tx=0x7f2fa0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8a2540 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800bcb0 tx=0x7f2fa80107a0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 -- 172.21.15.137:0/4278703175 shutdown_connections 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8a2540 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 --2- 172.21.15.137:0/4278703175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89d050 0x55dfcc89d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 -- 172.21.15.137:0/4278703175 >> 172.21.15.137:0/4278703175 conn(0x55dfcc865200 msgr2=0x55dfcc8989c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 -- 172.21.15.137:0/4278703175 shutdown_connections 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.360+0000 7f2fc2968740 1 -- 172.21.15.137:0/4278703175 wait complete. 2024-03-26T14:05:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 -- start start 2024-03-26T14:05:52.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 Processor -- start 2024-03-26T14:05:52.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc8a7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89da40 0x55dfcc8a7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc8a5b60 con 0x55dfcc89d050 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc7bde80 con 0x55dfcc8a2bb0 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcc8a5ce0 con 0x55dfcc89da40 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.361+0000 7f2fc2968740 10 client.0 _ll_register_callbacks cb 0x55dfcc7d0130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc8a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc8a7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36904/0 (socket says 172.21.15.137:36904) 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 -- 172.21.15.137:0/2363016609 learned_addr learned my addr 172.21.15.137:0/2363016609 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fafe00640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89da40 0x55dfcc8a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 -- 172.21.15.137:0/2363016609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89da40 msgr2=0x55dfcc8a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89da40 0x55dfcc8a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 -- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 msgr2=0x55dfcc8ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 -- 172.21.15.137:0/2363016609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfcc5e07d0 con 0x55dfcc89d050 2024-03-26T14:05:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.362+0000 7f2fbd200640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc8a7000 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011010 tx=0x7f2fa8004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f83942350f02e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fa6a00640 1 -- 172.21.15.137:0/2363016609 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa802b060 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fa80047a0 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fa6a00640 1 -- 172.21.15.137:0/2363016609 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2fa8005180 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fa6a00640 1 -- 172.21.15.137:0/2363016609 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa8032050 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fc2968740 10 client.16380 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.363+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dfcc5163f0 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fa6a00640 1 -- 172.21.15.137:0/2363016609 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fa80047a0 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fa6a00640 1 -- 172.21.15.137:0/2363016609 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2fa8011440 con 0x55dfcc89d050 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fc2968740 10 client.16380 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fc2968740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fc2968740 20 client.16380 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fc2968740 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc8b2b20 0x55dfcc8b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fc2968740 4 client.16380 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:05:52.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.364+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfcc80f790 con 0x55dfcc8b2b20 2024-03-26T14:05:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.365+0000 7f2fafe00640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc8b2b20 0x55dfcc8b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.366+0000 7f2fafe00640 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc8b2b20 0x55dfcc8b4f10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633dcee550beecf server_cookie=2918f860a851dad9 in_seq=0 out_seq=0 2024-03-26T14:05:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.366+0000 7f2fa6a00640 10 client.16380 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.367+0000 7f2fa6a00640 1 -- 172.21.15.137:0/2363016609 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dfcc80f790 con 0x55dfcc8b2b20 2024-03-26T14:05:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.367+0000 7f2fa6a00640 10 client.16380 handle_command_reply: tid=0 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.367+0000 7f2fc2968740 1 client.16380 shutdown 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 msgr2=0x55dfcc8a7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc8a7000 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8011010 tx=0x7f2fa8004b80 comp rx=0 tx=0).stop 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 shutdown_connections 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc8b2b20 0x55dfcc8b4f10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc89da40 0x55dfcc8a7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 --2- 172.21.15.137:0/2363016609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc8a7000 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 >> 172.21.15.137:0/2363016609 conn(0x55dfcc865200 msgr2=0x55dfcc897bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fa6a00640 0 client.16380 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 shutdown_connections 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 1 -- 172.21.15.137:0/2363016609 wait complete. 2024-03-26T14:05:52.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.368+0000 7f2fc2968740 20 client.16380 trim_cache size 0 max 16384 2024-03-26T14:05:52.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.391+0000 7f2fc2968740 1 Processor -- start 2024-03-26T14:05:52.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.391+0000 7f2fc2968740 1 -- start start 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc89e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89da40 0x55dfcc89eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fc2968740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81ae740 con 0x55dfcc89d050 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fc2968740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81ac240 con 0x55dfcc89da40 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fc2968740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81aa2c0 con 0x55dfcc8a2bb0 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc89e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc89e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36910/0 (socket says 172.21.15.137:36910) 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 -- 172.21.15.137:0/3997712659 learned_addr learned my addr 172.21.15.137:0/3997712659 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fafe00640 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89da40 0x55dfcc89eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 -- 172.21.15.137:0/3997712659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 msgr2=0x55dfcc8ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 -- 172.21.15.137:0/3997712659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89da40 msgr2=0x55dfcc89eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89da40 0x55dfcc89eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.392+0000 7f2fbd200640 1 -- 172.21.15.137:0/3997712659 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfcc4d60f0 con 0x55dfcc89d050 2024-03-26T14:05:52.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.393+0000 7f2fbd200640 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc89e9b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800bbd0 tx=0x7f2fa8010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1e2ce49db4a33d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.393+0000 7f2fad600640 1 -- 172.21.15.137:0/3997712659 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa802b040 con 0x55dfcc89d050 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.393+0000 7f2fad600640 1 -- 172.21.15.137:0/3997712659 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fa800b390 con 0x55dfcc89d050 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.393+0000 7f2fad600640 1 -- 172.21.15.137:0/3997712659 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011980 con 0x55dfcc89d050 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 -- 172.21.15.137:0/3997712659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 msgr2=0x55dfcc89e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc89e9b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800bbd0 tx=0x7f2fa8010dd0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 -- 172.21.15.137:0/3997712659 shutdown_connections 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89da40 0x55dfcc89eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 --2- 172.21.15.137:0/3997712659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89d050 0x55dfcc89e9b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 -- 172.21.15.137:0/3997712659 >> 172.21.15.137:0/3997712659 conn(0x55dfcc865200 msgr2=0x55dfcc89e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 -- 172.21.15.137:0/3997712659 shutdown_connections 2024-03-26T14:05:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 -- 172.21.15.137:0/3997712659 wait complete. 2024-03-26T14:05:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.394+0000 7f2fc2968740 1 -- start start 2024-03-26T14:05:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 Processor -- start 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89d050 0x55dfcc8ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8ac920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81aa8c0 con 0x55dfcc89da40 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81abbe0 con 0x55dfcc89d050 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81aaf20 con 0x55dfcc8a2bb0 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fc2968740 10 client.0 _ll_register_callbacks cb 0x7f2fa800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:05:52.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.395+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36922/0 (socket says 172.21.15.137:36922) 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fbd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89d050 0x55dfcc8ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 -- 172.21.15.137:0/1507566649 learned_addr learned my addr 172.21.15.137:0/1507566649 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fbdc00640 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 -- 172.21.15.137:0/1507566649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 msgr2=0x55dfcc8ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 -- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89d050 msgr2=0x55dfcc8ac3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89d050 0x55dfcc8ac3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 -- 172.21.15.137:0/1507566649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfcc8adf20 con 0x55dfcc89da40 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fafe00640 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8ac920 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009900 tx=0x7f2fa002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f58bc7b781b1699 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.396+0000 7f2fa6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fa6a00640 1 -- 172.21.15.137:0/1507566649 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0053dd0 con 0x55dfcc89da40 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fa6a00640 1 -- 172.21.15.137:0/1507566649 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2fa00531a0 con 0x55dfcc89da40 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dfcc809ae0 con 0x55dfcc89da40 2024-03-26T14:05:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fa6a00640 1 -- 172.21.15.137:0/1507566649 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fa0005040 con 0x55dfcc89da40 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fc2968740 10 client.16386 fetch_fsmap learned FSMap version 24 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dfcc90ea10 con 0x55dfcc89da40 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.397+0000 7f2fa6a00640 1 -- 172.21.15.137:0/1507566649 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55dfcc809ae0 con 0x55dfcc89da40 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fa6a00640 1 -- 172.21.15.137:0/1507566649 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2fa0004ae0 con 0x55dfcc89da40 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fc2968740 10 client.16386 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fc2968740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fc2968740 20 client.16386 populate_metadata read hostname 'smithi137' 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fc2968740 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc90f420 0x55dfcc911840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fc2968740 4 client.16386 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:05:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dfcc7873a0 con 0x55dfcc90f420 2024-03-26T14:05:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.398+0000 7f2fbd200640 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc90f420 0x55dfcc911840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:05:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fbd200640 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc90f420 0x55dfcc911840 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa83c89ed9c52e10 server_cookie=59cc401e79db8706 in_seq=0 out_seq=0 2024-03-26T14:05:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fa6a00640 10 client.16386 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fa6a00640 1 -- 172.21.15.137:0/1507566649 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dfcc7873a0 con 0x55dfcc90f420 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fa6a00640 10 client.16386 handle_command_reply: tid=0 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fc2968740 1 client.16386 shutdown 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 msgr2=0x55dfcc8ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.399+0000 7f2fc2968740 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8ac920 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009900 tx=0x7f2fa002bc30 comp rx=0 tx=0).stop 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 shutdown_connections 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dfcc90f420 0x55dfcc911840 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dfcc8a2bb0 0x55dfcc8ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dfcc89da40 0x55dfcc8ac920 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 --2- 172.21.15.137:0/1507566649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dfcc89d050 0x55dfcc8ac3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fa6a00640 0 client.16386 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 >> 172.21.15.137:0/1507566649 conn(0x55dfcc865200 msgr2=0x55dfcc89ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 shutdown_connections 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 1 -- 172.21.15.137:0/1507566649 wait complete. 2024-03-26T14:05:52.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.400+0000 7f2fc2968740 20 client.16386 trim_cache size 0 max 16384 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: "80ac444a-f58b-41c2-80ee-3c2b13d237ba": { 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "80ac444a-f58b-41c2-80ee-3c2b13d237ba", 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:05:52.401 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f9c0a2800 msgr2=0x7f2f9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f9c0a2800 0x7f2f9c0a4cc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009230 tx=0x7f2fa0009260 comp rx=0 tx=0).stop 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 msgr2=0x7f2fb81a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 0x7f2fb81a0f50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000eaa0 tx=0x7f2fb000ef60 comp rx=0 tx=0).stop 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 shutdown_connections 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f9c0a2800 0x7f2f9c0a4cc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fb8102790 0x7f2fb8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2fb8103150 0x7f2fb81a0f50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 --2- 172.21.15.137:0/3099235422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fb81015a0 0x7f2fb81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 >> 172.21.15.137:0/3099235422 conn(0x7f2fb80fcb10 msgr2=0x7f2fb80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 shutdown_connections 2024-03-26T14:05:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:05:52.401+0000 7f2fbe600640 1 -- 172.21.15.137:0/3099235422 wait complete. 2024-03-26T14:05:52.419 INFO:tasks.cephfs.filesystem:scrub status for tag:80ac444a-f58b-41c2-80ee-3c2b13d237ba - {'path': '/', 'tag': '80ac444a-f58b-41c2-80ee-3c2b13d237ba', 'options': 'recursive,force'} 2024-03-26T14:05:53.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:53 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:05:53.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:53 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:53.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:53 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:05:53.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:53 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:53.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:53 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:05:53.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:53 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:53.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:53 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:05:53.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:53 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:53 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:05:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:53 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:53 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:05:53.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:53 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:05:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:54 smithi179 ceph-mon[35472]: pgmap v564: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:54.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:54 smithi159 ceph-mon[35501]: pgmap v564: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:54.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:54 smithi137 ceph-mon[31539]: pgmap v564: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:05:55.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:55 smithi179 ceph-mon[35472]: pgmap v565: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:55.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:55 smithi159 ceph-mon[35501]: pgmap v565: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:55.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:55 smithi137 ceph-mon[31539]: pgmap v565: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:05:57.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:57 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:05:57.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:57 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:05:57.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:57 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:05:58.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:58 smithi179 ceph-mon[35472]: pgmap v566: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:05:58.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:58 smithi159 ceph-mon[35501]: pgmap v566: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:05:58.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:58 smithi137 ceph-mon[31539]: pgmap v566: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:05:59.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:05:59 smithi179 ceph-mon[35472]: pgmap v567: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:59.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:05:59 smithi159 ceph-mon[35501]: pgmap v567: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:05:59.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:05:59 smithi137 ceph-mon[31539]: pgmap v567: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:01.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:01 smithi179 ceph-mon[35472]: pgmap v568: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:01.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:01 smithi159 ceph-mon[35501]: pgmap v568: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:01.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:01 smithi137 ceph-mon[31539]: pgmap v568: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:03.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:03 smithi179 ceph-mon[35472]: pgmap v569: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:06:03.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:03 smithi159 ceph-mon[35501]: pgmap v569: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:06:03.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:03 smithi137 ceph-mon[31539]: pgmap v569: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:06:05.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:05 smithi179 ceph-mon[35472]: pgmap v570: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:05.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:05 smithi159 ceph-mon[35501]: pgmap v570: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:05.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:05 smithi137 ceph-mon[31539]: pgmap v570: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:07.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:07 smithi179 ceph-mon[35472]: pgmap v571: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:06:07.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:07 smithi159 ceph-mon[35501]: pgmap v571: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:06:07.783 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:07 smithi137 ceph-mon[31539]: pgmap v571: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:06:09.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:09 smithi179 ceph-mon[35472]: pgmap v572: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:06:09.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:09 smithi159 ceph-mon[35501]: pgmap v572: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:06:09.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:09 smithi137 ceph-mon[31539]: pgmap v572: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:06:11.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:11 smithi179 ceph-mon[35472]: pgmap v573: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-26T14:06:11.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:11 smithi159 ceph-mon[35501]: pgmap v573: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-26T14:06:11.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:11 smithi137 ceph-mon[31539]: pgmap v573: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-26T14:06:13.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:13 smithi179 ceph-mon[35472]: pgmap v574: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:13.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:13 smithi159 ceph-mon[35501]: pgmap v574: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:13.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:13 smithi137 ceph-mon[31539]: pgmap v574: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:15.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:15 smithi179 ceph-mon[35472]: pgmap v575: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:06:15.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:15 smithi159 ceph-mon[35501]: pgmap v575: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:06:15.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:15 smithi137 ceph-mon[31539]: pgmap v575: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:06:17.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:17 smithi179 ceph-mon[35472]: pgmap v576: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:17.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:17 smithi159 ceph-mon[35501]: pgmap v576: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:17.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:17 smithi137 ceph-mon[31539]: pgmap v576: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:19.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:19 smithi179 ceph-mon[35472]: pgmap v577: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:19.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:19 smithi159 ceph-mon[35501]: pgmap v577: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:19.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:19 smithi137 ceph-mon[31539]: pgmap v577: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:21.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:21 smithi179 ceph-mon[35472]: pgmap v578: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:21.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:21 smithi159 ceph-mon[35501]: pgmap v578: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:21.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:21 smithi137 ceph-mon[31539]: pgmap v578: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:22.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:06:22.420 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:06:22.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.640+0000 7f2e5fa00640 1 -- 172.21.15.137:0/3570919507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581016b0 msgr2=0x7f2e58101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.640+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/3570919507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581016b0 0x7f2e58101ad0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009900 tx=0x7f2e4c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 -- 172.21.15.137:0/3570919507 shutdown_connections 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/3570919507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e581028a0 0x7f2e58102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/3570919507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e58103260 0x7f2e58109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/3570919507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581016b0 0x7f2e58101ad0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 -- 172.21.15.137:0/3570919507 >> 172.21.15.137:0/3570919507 conn(0x7f2e580fcc40 msgr2=0x7f2e580ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 -- 172.21.15.137:0/3570919507 shutdown_connections 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.641+0000 7f2e5fa00640 1 -- 172.21.15.137:0/3570919507 wait complete. 2024-03-26T14:06:22.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.642+0000 7f2e5fa00640 1 Processor -- start 2024-03-26T14:06:22.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.642+0000 7f2e5fa00640 1 -- start start 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5fa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e581016b0 0x7f2e58197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5fa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 0x7f2e58197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5fa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e58103260 0x7f2e5819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5fa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5810d230 con 0x7f2e581028a0 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5fa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5810d0b0 con 0x7f2e58103260 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5fa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5810d3b0 con 0x7f2e581016b0 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 0x7f2e58197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 0x7f2e58197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46630/0 (socket says 172.21.15.137:46630) 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 -- 172.21.15.137:0/2759912438 learned_addr learned my addr 172.21.15.137:0/2759912438 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5d000640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e581016b0 0x7f2e58197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e5da00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e58103260 0x7f2e5819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 -- 172.21.15.137:0/2759912438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e581016b0 msgr2=0x7f2e58197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e581016b0 0x7f2e58197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 -- 172.21.15.137:0/2759912438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e58103260 msgr2=0x7f2e5819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e58103260 0x7f2e5819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.643+0000 7f2e57e00640 1 -- 172.21.15.137:0/2759912438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5819f550 con 0x7f2e581028a0 2024-03-26T14:06:22.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.644+0000 7f2e57e00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 0x7f2e58197e70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400e9f0 tx=0x7f2e4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.644+0000 7f2e55600640 1 -- 172.21.15.137:0/2759912438 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4400cd80 con 0x7f2e581028a0 2024-03-26T14:06:22.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.644+0000 7f2e55600640 1 -- 172.21.15.137:0/2759912438 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e440108e0 con 0x7f2e581028a0 2024-03-26T14:06:22.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.644+0000 7f2e55600640 1 -- 172.21.15.137:0/2759912438 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44018920 con 0x7f2e581028a0 2024-03-26T14:06:22.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.644+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5819f840 con 0x7f2e581028a0 2024-03-26T14:06:22.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.644+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5819fd40 con 0x7f2e581028a0 2024-03-26T14:06:22.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.649+0000 7f2e627f2740 1 Processor -- start 2024-03-26T14:06:22.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.649+0000 7f2e627f2740 1 -- start start 2024-03-26T14:06:22.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.649+0000 7f2e55600640 1 -- 172.21.15.137:0/2759912438 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2e44010a90 con 0x7f2e581028a0 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ab7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e627f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb1a13250 con 0x55fdb1ab7df0 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e627f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb19bba30 con 0x55fdb1ab2290 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e627f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb1a0f2f0 con 0x55fdb1ab2c80 2024-03-26T14:06:22.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e5da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e5da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37410/0 (socket says 172.21.15.137:37410) 2024-03-26T14:06:22.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.650+0000 7f2e5da00640 1 -- 172.21.15.137:0/4218996364 learned_addr learned my addr 172.21.15.137:0/4218996364 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:22.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e57e00640 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ab7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e55600640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e400a2800 0x7f2e400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e5da00640 1 -- 172.21.15.137:0/4218996364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 msgr2=0x55fdb1ab7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e5da00640 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ab7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e5da00640 1 -- 172.21.15.137:0/4218996364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab7df0 msgr2=0x55fdb1aba280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e5da00640 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aba280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e55600640 1 -- 172.21.15.137:0/2759912438 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2e440cf280 con 0x7f2e581028a0 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.651+0000 7f2e5da00640 1 -- 172.21.15.137:0/4218996364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdb19a6690 con 0x55fdb1ab2290 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.652+0000 7f2e5da00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e400a2800 0x7f2e400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.652+0000 7f2e5da00640 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab26b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e500103f0 tx=0x7f2e500108b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4768ca85441a882 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.652+0000 7f2e4a000640 1 -- 172.21.15.137:0/4218996364 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5001d910 con 0x55fdb1ab2290 2024-03-26T14:06:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.652+0000 7f2e5da00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e400a2800 0x7f2e400a4cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2e50021000 tx=0x7f2e5001e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.652+0000 7f2e4a000640 1 -- 172.21.15.137:0/4218996364 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e5001c440 con 0x55fdb1ab2290 2024-03-26T14:06:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.652+0000 7f2e4a000640 1 -- 172.21.15.137:0/4218996364 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50011640 con 0x55fdb1ab2290 2024-03-26T14:06:22.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.653+0000 7f2e627f2740 1 -- 172.21.15.137:0/4218996364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 msgr2=0x55fdb1ab26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.653+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab26b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2e500103f0 tx=0x7f2e500108b0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 -- 172.21.15.137:0/4218996364 shutdown_connections 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aba280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ab7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4218996364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab26b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 -- 172.21.15.137:0/4218996364 >> 172.21.15.137:0/4218996364 conn(0x55fdb1a76200 msgr2=0x55fdb1aa97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 -- 172.21.15.137:0/4218996364 shutdown_connections 2024-03-26T14:06:22.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 -- 172.21.15.137:0/4218996364 wait complete. 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.654+0000 7f2e627f2740 1 -- start start 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 Processor -- start 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1abc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1abc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb1abada0 con 0x55fdb1ab2290 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb19cee80 con 0x55fdb1ab2c80 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb1abaf20 con 0x55fdb1ab7df0 2024-03-26T14:06:22.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.655+0000 7f2e627f2740 10 client.0 _ll_register_callbacks cb 0x55fdb19e1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:22.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1abc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1abc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1abc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37426/0 (socket says 172.21.15.137:37426) 2024-03-26T14:06:22.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e57e00640 1 -- 172.21.15.137:0/1916907952 learned_addr learned my addr 172.21.15.137:0/1916907952 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e5da00640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e57e00640 1 -- 172.21.15.137:0/1916907952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 msgr2=0x55fdb1ac37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e57e00640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.656+0000 7f2e57e00640 1 -- 172.21.15.137:0/1916907952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 msgr2=0x55fdb1abc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e57e00640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1abc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e57e00640 1 -- 172.21.15.137:0/1916907952 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdb17f17d0 con 0x55fdb1ab2c80 2024-03-26T14:06:22.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e5d000640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1abc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e57e00640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1abc770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400c6a0 tx=0x7f2e4409b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9297a103f73bf911 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e3ea00640 1 -- 172.21.15.137:0/1916907952 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44086270 con 0x55fdb1ab2c80 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e5001c440 con 0x55fdb1ab2c80 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.657+0000 7f2e3ea00640 1 -- 172.21.15.137:0/1916907952 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2e4409cb80 con 0x55fdb1ab2c80 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e3ea00640 1 -- 172.21.15.137:0/1916907952 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e4409bf50 con 0x55fdb1ab2c80 2024-03-26T14:06:22.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e627f2740 10 client.36050 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fdb17273f0 con 0x55fdb1ab2c80 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e3ea00640 1 -- 172.21.15.137:0/1916907952 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e5001c440 con 0x55fdb1ab2c80 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e3ea00640 1 -- 172.21.15.137:0/1916907952 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2e440d2b50 con 0x55fdb1ab2c80 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e627f2740 10 client.36050 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.658+0000 7f2e627f2740 10 client.36050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.659+0000 7f2e627f2740 20 client.36050 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.659+0000 7f2e627f2740 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1ac7ed0 0x55fdb1aca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.659+0000 7f2e627f2740 4 client.36050 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:22.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.659+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdb1a20790 con 0x55fdb1ac7ed0 2024-03-26T14:06:22.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.659+0000 7f2e5d000640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1ac7ed0 0x55fdb1aca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.660+0000 7f2e5d000640 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1ac7ed0 0x55fdb1aca330 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76dfb246d51a31b server_cookie=f39faef922b5cb02 in_seq=0 out_seq=0 2024-03-26T14:06:22.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.660+0000 7f2e3ea00640 10 client.36050 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:22.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e3ea00640 1 -- 172.21.15.137:0/1916907952 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fdb1a20790 con 0x55fdb1ac7ed0 2024-03-26T14:06:22.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e3ea00640 10 client.36050 handle_command_reply: tid=0 2024-03-26T14:06:22.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e627f2740 1 client.36050 shutdown 2024-03-26T14:06:22.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 msgr2=0x55fdb1abc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e627f2740 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1abc770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400c6a0 tx=0x7f2e4409b390 comp rx=0 tx=0).stop 2024-03-26T14:06:22.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 shutdown_connections 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.661+0000 7f2e627f2740 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1ac7ed0 0x55fdb1aca330 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1abc770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 1 --2- 172.21.15.137:0/1916907952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1abc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e3ea00640 0 client.36050 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 >> 172.21.15.137:0/1916907952 conn(0x55fdb1a76200 msgr2=0x55fdb1aa8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 shutdown_connections 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 1 -- 172.21.15.137:0/1916907952 wait complete. 2024-03-26T14:06:22.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.662+0000 7f2e627f2740 20 client.36050 trim_cache size 0 max 16384 2024-03-26T14:06:22.672 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:22 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:06:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 Processor -- start 2024-03-26T14:06:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 -- start start 2024-03-26T14:06:22.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aa8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581aa630 con 0x55fdb1ab2290 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581a8130 con 0x55fdb1ab7df0 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.682+0000 7f2e627f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581a61b0 con 0x55fdb1ab2c80 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46670/0 (socket says 172.21.15.137:46670) 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 -- 172.21.15.137:0/4228289053 learned_addr learned my addr 172.21.15.137:0/4228289053 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5da00640 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aa8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e57e00640 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 -- 172.21.15.137:0/4228289053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 msgr2=0x55fdb1ac3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 -- 172.21.15.137:0/4228289053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab7df0 msgr2=0x55fdb1aa8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aa8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.683+0000 7f2e5d000640 1 -- 172.21.15.137:0/4228289053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdb16e70f0 con 0x55fdb1ab2290 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.684+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab7250 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02ce10 tx=0x7f2e4c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce714c5da6d74fc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.684+0000 7f2e49600640 1 -- 172.21.15.137:0/4228289053 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c04e020 con 0x55fdb1ab2290 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.684+0000 7f2e49600640 1 -- 172.21.15.137:0/4228289053 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c0054f0 con 0x55fdb1ab2290 2024-03-26T14:06:22.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.684+0000 7f2e49600640 1 -- 172.21.15.137:0/4228289053 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c048050 con 0x55fdb1ab2290 2024-03-26T14:06:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 -- 172.21.15.137:0/4228289053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 msgr2=0x55fdb1ab7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab7250 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02ce10 tx=0x7f2e4c005a20 comp rx=0 tx=0).stop 2024-03-26T14:06:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 -- 172.21.15.137:0/4228289053 shutdown_connections 2024-03-26T14:06:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab7df0 0x55fdb1aa8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4228289053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ab7250 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 -- 172.21.15.137:0/4228289053 >> 172.21.15.137:0/4228289053 conn(0x55fdb1a76200 msgr2=0x55fdb1aad2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 -- 172.21.15.137:0/4228289053 shutdown_connections 2024-03-26T14:06:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.685+0000 7f2e627f2740 1 -- 172.21.15.137:0/4228289053 wait complete. 2024-03-26T14:06:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.686+0000 7f2e627f2740 1 -- start start 2024-03-26T14:06:22.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.686+0000 7f2e627f2740 1 Processor -- start 2024-03-26T14:06:22.685 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:22 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:06:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ac2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581a67b0 con 0x55fdb1ab2290 2024-03-26T14:06:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581a7ad0 con 0x55fdb1ab2c80 2024-03-26T14:06:22.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e581a6e10 con 0x55fdb1ab7df0 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e627f2740 10 client.0 _ll_register_callbacks cb 0x7f2e44009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ac2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e5d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ac2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46682/0 (socket says 172.21.15.137:46682) 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e5d000640 1 -- 172.21.15.137:0/4110580883 learned_addr learned my addr 172.21.15.137:0/4110580883 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.687+0000 7f2e57e00640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5da00640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5d000640 1 -- 172.21.15.137:0/4110580883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 msgr2=0x55fdb1ac15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5d000640 1 -- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 msgr2=0x55fdb1ac3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5d000640 1 -- 172.21.15.137:0/4110580883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdb1ab1c60 con 0x55fdb1ab2290 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e57e00640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.688+0000 7f2e5d000640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ac2f50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02c170 tx=0x7f2e4c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11264d4b8ea25ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.689+0000 7f2e3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.689+0000 7f2e3ea00640 1 -- 172.21.15.137:0/4110580883 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0040c0 con 0x55fdb1ab2290 2024-03-26T14:06:22.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.689+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e4c0054f0 con 0x55fdb1ab2290 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.689+0000 7f2e3ea00640 1 -- 172.21.15.137:0/4110580883 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2e4c004260 con 0x55fdb1ab2290 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.689+0000 7f2e3ea00640 1 -- 172.21.15.137:0/4110580883 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e4c004840 con 0x55fdb1ab2290 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 10 client.16404 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fdb1b1dda0 con 0x55fdb1ab2290 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e3ea00640 1 -- 172.21.15.137:0/4110580883 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e4c0054f0 con 0x55fdb1ab2290 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e3ea00640 1 -- 172.21.15.137:0/4110580883 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2e4c003960 con 0x55fdb1ab2290 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 10 client.16404 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 20 client.16404 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:22.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1b1e680 0x55fdb1b20aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 4 client.16404 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:06:22.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.690+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fdb1a1aae0 con 0x55fdb1b1e680 2024-03-26T14:06:22.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.691+0000 7f2e57e00640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1b1e680 0x55fdb1b20aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e57e00640 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1b1e680 0x55fdb1b20aa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb54b7d50091efea server_cookie=4bf12e3fde6372ff in_seq=0 out_seq=0 2024-03-26T14:06:22.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e3ea00640 10 client.16404 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e3ea00640 1 -- 172.21.15.137:0/4110580883 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fdb1a1aae0 con 0x55fdb1b1e680 2024-03-26T14:06:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e3ea00640 10 client.16404 handle_command_reply: tid=0 2024-03-26T14:06:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 client.16404 shutdown 2024-03-26T14:06:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 msgr2=0x55fdb1ac2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ac2f50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02c170 tx=0x7f2e4c0056a0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 shutdown_connections 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fdb1b1e680 0x55fdb1b20aa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdb1ab2c80 0x55fdb1ac3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fdb1ab7df0 0x55fdb1ac15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 --2- 172.21.15.137:0/4110580883 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fdb1ab2290 0x55fdb1ac2f50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e3ea00640 0 client.16404 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.692+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 >> 172.21.15.137:0/4110580883 conn(0x55fdb1a76200 msgr2=0x55fdb1ab3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.693+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 shutdown_connections 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.693+0000 7f2e627f2740 1 -- 172.21.15.137:0/4110580883 wait complete. 2024-03-26T14:06:22.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.693+0000 7f2e627f2740 20 client.16404 trim_cache size 0 max 16384 2024-03-26T14:06:22.693 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:06:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:06:22.693 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:06:22.693 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:06:22.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e400a2800 msgr2=0x7f2e400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e400a2800 0x7f2e400a4cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2e50021000 tx=0x7f2e5001e000 comp rx=0 tx=0).stop 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 msgr2=0x7f2e58197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 0x7f2e58197e70 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400e9f0 tx=0x7f2e4400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 shutdown_connections 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e400a2800 0x7f2e400a4cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e58103260 0x7f2e5819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e581028a0 0x7f2e58197e70 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 --2- 172.21.15.137:0/2759912438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e581016b0 0x7f2e58197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 >> 172.21.15.137:0/2759912438 conn(0x7f2e580fcc40 msgr2=0x7f2e58105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 shutdown_connections 2024-03-26T14:06:22.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.694+0000 7f2e5fa00640 1 -- 172.21.15.137:0/2759912438 wait complete. 2024-03-26T14:06:22.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:22 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:06:22.710 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:06:22.710 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 -- 172.21.15.137:0/3731913537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 msgr2=0x7f942c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 --2- 172.21.15.137:0/3731913537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 0x7f942c10bd20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9424009ca0 tx=0x7f942402c100 comp rx=0 tx=0).stop 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 -- 172.21.15.137:0/3731913537 shutdown_connections 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 --2- 172.21.15.137:0/3731913537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 0x7f942c10bd20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 --2- 172.21.15.137:0/3731913537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c104ad0 0x7f942c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 --2- 172.21.15.137:0/3731913537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f942c1038e0 0x7f942c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 -- 172.21.15.137:0/3731913537 >> 172.21.15.137:0/3731913537 conn(0x7f942c0fee90 msgr2=0x7f942c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 -- 172.21.15.137:0/3731913537 shutdown_connections 2024-03-26T14:06:22.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.932+0000 7f9430a00640 1 -- 172.21.15.137:0/3731913537 wait complete. 2024-03-26T14:06:22.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.933+0000 7f9430a00640 1 Processor -- start 2024-03-26T14:06:22.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.933+0000 7f9430a00640 1 -- start start 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9430a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 0x7f942c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9430a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f942c104ad0 0x7f942c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9430a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 0x7f942c1a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9430a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10f410 con 0x7f942c1038e0 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9430a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10f290 con 0x7f942c105490 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9430a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c10f590 con 0x7f942c104ad0 2024-03-26T14:06:22.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9429e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 0x7f942c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9429e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 0x7f942c199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46712/0 (socket says 172.21.15.137:46712) 2024-03-26T14:06:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f9429e00640 1 -- 172.21.15.137:0/2710285628 learned_addr learned my addr 172.21.15.137:0/2710285628 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.934+0000 7f942a800640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 0x7f942c1a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9423e00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f942c104ad0 0x7f942c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9429e00640 1 -- 172.21.15.137:0/2710285628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f942c104ad0 msgr2=0x7f942c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9429e00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f942c104ad0 0x7f942c199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9429e00640 1 -- 172.21.15.137:0/2710285628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 msgr2=0x7f942c1a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9429e00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 0x7f942c1a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9429e00640 1 -- 172.21.15.137:0/2710285628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c1a16d0 con 0x7f942c1038e0 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9429e00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 0x7f942c199ab0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e9c0 tx=0x7f941c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00cd80 con 0x7f942c1038e0 2024-03-26T14:06:22.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c1a19c0 con 0x7f942c1038e0 2024-03-26T14:06:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f941c0035f0 con 0x7f942c1038e0 2024-03-26T14:06:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.935+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c010690 con 0x7f942c1038e0 2024-03-26T14:06:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.936+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c1a1ec0 con 0x7f942c1038e0 2024-03-26T14:06:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.937+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c10ac20 con 0x7f942c1038e0 2024-03-26T14:06:22.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.939+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f941c0037a0 con 0x7f942c1038e0 2024-03-26T14:06:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.940+0000 7f9422a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94100a2800 0x7f94100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.940+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f941c0cf430 con 0x7f942c1038e0 2024-03-26T14:06:22.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.940+0000 7f9423e00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94100a2800 0x7f94100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:22.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.941+0000 7f9423e00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94100a2800 0x7f94100a4cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f942c19afc0 tx=0x7f9414005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:22.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:22.945+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f941c09c100 con 0x7f942c1038e0 2024-03-26T14:06:23.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.219+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f942c02d070 con 0x7f942c1038e0 2024-03-26T14:06:23.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.220+0000 7f9422a00640 1 -- 172.21.15.137:0/2710285628 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f941c09ba00 con 0x7f942c1038e0 2024-03-26T14:06:23.220 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:06:23.220 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:06:23.220 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:06:23.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94100a2800 msgr2=0x7f94100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94100a2800 0x7f94100a4cc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f942c19afc0 tx=0x7f9414005f90 comp rx=0 tx=0).stop 2024-03-26T14:06:23.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 msgr2=0x7f942c199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 0x7f942c199ab0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e9c0 tx=0x7f941c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:06:23.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 shutdown_connections 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f94100a2800 0x7f94100a4cc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f942c105490 0x7f942c1a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f942c104ad0 0x7f942c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 --2- 172.21.15.137:0/2710285628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f942c1038e0 0x7f942c199ab0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.223+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 >> 172.21.15.137:0/2710285628 conn(0x7f942c0fee90 msgr2=0x7f942c107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.224+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 shutdown_connections 2024-03-26T14:06:23.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.224+0000 7f9430a00640 1 -- 172.21.15.137:0/2710285628 wait complete. 2024-03-26T14:06:23.243 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:06:23.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 -- 172.21.15.137:0/10149454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364102760 msgr2=0x7f8364104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 --2- 172.21.15.137:0/10149454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364102760 0x7f8364104bd0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f834c009900 tx=0x7f834c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 -- 172.21.15.137:0/10149454 shutdown_connections 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 --2- 172.21.15.137:0/10149454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364105110 0x7f8364105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 --2- 172.21.15.137:0/10149454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364102760 0x7f8364104bd0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 --2- 172.21.15.137:0/10149454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83640ffdf0 0x7f8364102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 -- 172.21.15.137:0/10149454 >> 172.21.15.137:0/10149454 conn(0x7f83640fba10 msgr2=0x7f83640fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 -- 172.21.15.137:0/10149454 shutdown_connections 2024-03-26T14:06:23.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.475+0000 7f8363e00640 1 -- 172.21.15.137:0/10149454 wait complete. 2024-03-26T14:06:23.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.476+0000 7f8363e00640 1 Processor -- start 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.476+0000 7f8363e00640 1 -- start start 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.476+0000 7f8363e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83640ffdf0 0x7f836419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.476+0000 7f8363e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364102760 0x7f836419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.476+0000 7f8363e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 0x7f83641a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641136d0 con 0x7f8364105110 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8364113550 con 0x7f8364102760 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8364113850 con 0x7f83640ffdf0 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 0x7f83641a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 0x7f83641a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46740/0 (socket says 172.21.15.137:46740) 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 -- 172.21.15.137:0/3515225615 learned_addr learned my addr 172.21.15.137:0/3515225615 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f835a000640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364102760 0x7f836419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8362a00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83640ffdf0 0x7f836419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 -- 172.21.15.137:0/3515225615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83640ffdf0 msgr2=0x7f836419dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83640ffdf0 0x7f836419dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 -- 172.21.15.137:0/3515225615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364102760 msgr2=0x7f836419e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364102760 0x7f836419e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f8363400640 1 -- 172.21.15.137:0/3515225615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83641a5960 con 0x7f8364105110 2024-03-26T14:06:23.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.477+0000 7f835a000640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364102760 0x7f836419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:23.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.478+0000 7f8363400640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 0x7f83641a52f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e970 tx=0x7f835c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.478+0000 7f835be00640 1 -- 172.21.15.137:0/3515225615 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c00ccc0 con 0x7f8364105110 2024-03-26T14:06:23.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.478+0000 7f835be00640 1 -- 172.21.15.137:0/3515225615 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f835c00ce60 con 0x7f8364105110 2024-03-26T14:06:23.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.478+0000 7f835be00640 1 -- 172.21.15.137:0/3515225615 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c010430 con 0x7f8364105110 2024-03-26T14:06:23.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.478+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83641a5c50 con 0x7f8364105110 2024-03-26T14:06:23.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.478+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83641a6120 con 0x7f8364105110 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.482+0000 7f836c42a740 1 Processor -- start 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.482+0000 7f835be00640 1 -- 172.21.15.137:0/3515225615 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f835c004ff0 con 0x7f8364105110 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.482+0000 7f836c42a740 1 -- start start 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.482+0000 7f836c42a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.482+0000 7f836c42a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2ee040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f836c42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f2f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f836c42a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39f2512c0 con 0x55b39f2edbc0 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f836c42a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39f1f9c90 con 0x55b39f2ec9f0 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f836c42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39f24d360 con 0x55b39f2ee580 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2ee040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46754/0 (socket says 172.21.15.137:46754) 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 -- 172.21.15.137:0/1699387587 learned_addr learned my addr 172.21.15.137:0/1699387587 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f835a000640 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f835a000640 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f2f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 -- 172.21.15.137:0/1699387587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 msgr2=0x55b39f2f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f2f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 -- 172.21.15.137:0/1699387587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 msgr2=0x55b39f2ece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.483+0000 7f8362a00640 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2ece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f8362a00640 1 -- 172.21.15.137:0/1699387587 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b39f21e080 con 0x55b39f2edbc0 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f835a000640 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:23.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f835be00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83480a2800 0x7f83480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f8362a00640 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2ee040 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f835400f410 tx=0x7f8354009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2ece42031416b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f835be00640 1 -- 172.21.15.137:0/3515225615 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f835c0cf300 con 0x7f8364105110 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f835a000640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83480a2800 0x7f83480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f8353e00640 1 -- 172.21.15.137:0/1699387587 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354004d20 con 0x55b39f2edbc0 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f8353e00640 1 -- 172.21.15.137:0/1699387587 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83540119b0 con 0x55b39f2edbc0 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.484+0000 7f8353e00640 1 -- 172.21.15.137:0/1699387587 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354005520 con 0x55b39f2edbc0 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 -- 172.21.15.137:0/1699387587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 msgr2=0x55b39f2ee040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f835a000640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83480a2800 0x7f83480a4cc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f834c006010 tx=0x7f834c02c5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2ee040 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f835400f410 tx=0x7f8354009f90 comp rx=0 tx=0).stop 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 -- 172.21.15.137:0/1699387587 shutdown_connections 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f2f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2ee040 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 --2- 172.21.15.137:0/1699387587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.485+0000 7f836c42a740 1 -- 172.21.15.137:0/1699387587 >> 172.21.15.137:0/1699387587 conn(0x55b39f2e7de0 msgr2=0x55b39f2e81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.486+0000 7f836c42a740 1 -- 172.21.15.137:0/1699387587 shutdown_connections 2024-03-26T14:06:23.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.486+0000 7f836c42a740 1 -- 172.21.15.137:0/1699387587 wait complete. 2024-03-26T14:06:23.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.486+0000 7f836c42a740 1 -- start start 2024-03-26T14:06:23.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.486+0000 7f836c42a740 1 Processor -- start 2024-03-26T14:06:23.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 0x55b39f2f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ee580 0x55b39f2fe130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39f2f5590 con 0x55b39f2ec9f0 2024-03-26T14:06:23.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39f20d0d0 con 0x55b39f2ee580 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b39f2f5710 con 0x55b39f2edbc0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f836c42a740 10 client.0 _ll_register_callbacks cb 0x55b39f211e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f8362a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 0x55b39f2f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f8362a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 0x55b39f2f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46762/0 (socket says 172.21.15.137:46762) 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f8362a00640 1 -- 172.21.15.137:0/2977498698 learned_addr learned my addr 172.21.15.137:0/2977498698 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f8363400640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ee580 0x55b39f2fe130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.487+0000 7f835a000640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8362a00640 1 -- 172.21.15.137:0/2977498698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 msgr2=0x55b39f2f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8362a00640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8362a00640 1 -- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ee580 msgr2=0x55b39f2fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8362a00640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ee580 0x55b39f2fe130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8362a00640 1 -- 172.21.15.137:0/2977498698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b39ef653f0 con 0x55b39f2ec9f0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8362a00640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 0x55b39f2f6bb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f80 tx=0x7f8354011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bacdae1f3f9495e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8350c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8350c00640 1 -- 172.21.15.137:0/2977498698 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540117d0 con 0x55b39f2ec9f0 2024-03-26T14:06:23.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f8350c00640 1 -- 172.21.15.137:0/2977498698 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8354004a40 con 0x55b39f2ec9f0 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.488+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83540119b0 con 0x55b39f2ec9f0 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f8350c00640 1 -- 172.21.15.137:0/2977498698 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8354004c40 con 0x55b39f2ec9f0 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 10 client.16422 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b39f02f7d0 con 0x55b39f2ec9f0 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f8350c00640 1 -- 172.21.15.137:0/2977498698 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83540119b0 con 0x55b39f2ec9f0 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f8350c00640 1 -- 172.21.15.137:0/2977498698 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8354005710 con 0x55b39f2ec9f0 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 10 client.16422 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 10 client.16422 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 20 client.16422 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f302430 0x55b39f304880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 4 client.16422 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.489+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b39f25e800 con 0x55b39f302430 2024-03-26T14:06:23.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.490+0000 7f835a000640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f302430 0x55b39f304880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.491+0000 7f835a000640 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f302430 0x55b39f304880 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbfcda37dac51deb server_cookie=88e7a28c9b253a16 in_seq=0 out_seq=0 2024-03-26T14:06:23.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.491+0000 7f8350c00640 10 client.16422 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:23.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f8350c00640 1 -- 172.21.15.137:0/2977498698 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b39f25e800 con 0x55b39f302430 2024-03-26T14:06:23.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f8350c00640 10 client.16422 handle_command_reply: tid=0 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 client.16422 shutdown 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 msgr2=0x55b39f2f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 0x55b39f2f6bb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f80 tx=0x7f8354011d90 comp rx=0 tx=0).stop 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 shutdown_connections 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f302430 0x55b39f304880 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ee580 0x55b39f2fe130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 --2- 172.21.15.137:0/2977498698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ec9f0 0x55b39f2f6bb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 >> 172.21.15.137:0/2977498698 conn(0x55b39f2e7de0 msgr2=0x55b39f2e7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.492+0000 7f8350c00640 0 client.16422 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.493+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 shutdown_connections 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.493+0000 7f836c42a740 1 -- 172.21.15.137:0/2977498698 wait complete. 2024-03-26T14:06:23.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.493+0000 7f836c42a740 20 client.16422 trim_cache size 0 max 16384 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.512+0000 7f836c42a740 1 Processor -- start 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.512+0000 7f836c42a740 1 -- start start 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f836c42a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f309140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f836c42a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f836c42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f309680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f836c42a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641b2b40 con 0x55b39f2edbc0 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f836c42a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641b0640 con 0x55b39f2ec9f0 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f836c42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641ae6c0 con 0x55b39f2ee580 2024-03-26T14:06:23.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2fe0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46776/0 (socket says 172.21.15.137:46776) 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 -- 172.21.15.137:0/985925702 learned_addr learned my addr 172.21.15.137:0/985925702 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f8362a00640 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f309140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f8363400640 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f309680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 -- 172.21.15.137:0/985925702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 msgr2=0x55b39f309680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f309680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 -- 172.21.15.137:0/985925702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 msgr2=0x55b39f309140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f309140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.513+0000 7f835a000640 1 -- 172.21.15.137:0/985925702 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b39f2e8d50 con 0x55b39f2edbc0 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.514+0000 7f835a000640 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2fe0c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f834c02ce10 tx=0x7f834c03a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49989270d90b381b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.514+0000 7f8353400640 1 -- 172.21.15.137:0/985925702 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c004360 con 0x55b39f2edbc0 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.514+0000 7f8353400640 1 -- 172.21.15.137:0/985925702 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f834c059070 con 0x55b39f2edbc0 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.514+0000 7f8353400640 1 -- 172.21.15.137:0/985925702 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c004a60 con 0x55b39f2edbc0 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 -- 172.21.15.137:0/985925702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 msgr2=0x55b39f2fe0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2fe0c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f834c02ce10 tx=0x7f834c03a000 comp rx=0 tx=0).stop 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 -- 172.21.15.137:0/985925702 shutdown_connections 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2ee580 0x55b39f309680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2edbc0 0x55b39f2fe0c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 --2- 172.21.15.137:0/985925702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f309140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 -- 172.21.15.137:0/985925702 >> 172.21.15.137:0/985925702 conn(0x55b39f2e7de0 msgr2=0x55b39f2e7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 -- 172.21.15.137:0/985925702 shutdown_connections 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 -- 172.21.15.137:0/985925702 wait complete. 2024-03-26T14:06:23.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.515+0000 7f836c42a740 1 -- start start 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 Processor -- start 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 0x55b39f2fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641aecc0 con 0x55b39f2ee580 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641affe0 con 0x55b39f2ec9f0 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641af320 con 0x55b39f2edbc0 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f836c42a740 10 client.0 _ll_register_callbacks cb 0x7f8354015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f8363400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 0x55b39f2fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.516+0000 7f8363400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 0x55b39f2fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46792/0 (socket says 172.21.15.137:46792) 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 -- 172.21.15.137:0/2451566914 learned_addr learned my addr 172.21.15.137:0/2451566914 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8362a00640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f835a000640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 -- 172.21.15.137:0/2451566914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 msgr2=0x55b39f2fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 -- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 msgr2=0x55b39f2fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 -- 172.21.15.137:0/2451566914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b39f2fdd10 con 0x55b39f2ee580 2024-03-26T14:06:23.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8363400640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 0x55b39f2fd720 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c6a0 tx=0x7f835c09b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27040066d35d3c72 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8350c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8350c00640 1 -- 172.21.15.137:0/2451566914 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0b3070 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f8350c00640 1 -- 172.21.15.137:0/2451566914 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f835c0052e0 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.517+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f834c059070 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f8350c00640 1 -- 172.21.15.137:0/2451566914 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f835c084e60 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 10 client.16428 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b39f35e7a0 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f8350c00640 1 -- 172.21.15.137:0/2451566914 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f834c059070 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f8350c00640 1 -- 172.21.15.137:0/2451566914 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f835c09bb30 con 0x55b39f2ee580 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 10 client.16428 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 10 client.16428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 20 client.16428 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:23.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f35f120 0x55b39f361540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 4 client.16428 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.518+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b39f258b50 con 0x55b39f35f120 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.519+0000 7f8362a00640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f35f120 0x55b39f361540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.519+0000 7f8362a00640 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f35f120 0x55b39f361540 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=852c80f644dd7d41 server_cookie=49f3ba0418a5239c in_seq=0 out_seq=0 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.519+0000 7f8350c00640 10 client.16428 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f8350c00640 1 -- 172.21.15.137:0/2451566914 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b39f258b50 con 0x55b39f35f120 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f8350c00640 10 client.16428 handle_command_reply: tid=0 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 client.16428 shutdown 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 msgr2=0x55b39f2fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 0x55b39f2fd720 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c6a0 tx=0x7f835c09b490 comp rx=0 tx=0).stop 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 shutdown_connections 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b39f35f120 0x55b39f361540 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b39f2ee580 0x55b39f2fd720 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b39f2edbc0 0x55b39f2fa700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 --2- 172.21.15.137:0/2451566914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b39f2ec9f0 0x55b39f2fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f8350c00640 0 client.16428 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 >> 172.21.15.137:0/2451566914 conn(0x55b39f2e7de0 msgr2=0x55b39f2f0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 shutdown_connections 2024-03-26T14:06:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 1 -- 172.21.15.137:0/2451566914 wait complete. 2024-03-26T14:06:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.520+0000 7f836c42a740 20 client.16428 trim_cache size 0 max 16384 2024-03-26T14:06:23.522 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:06:23.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83480a2800 msgr2=0x7f83480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83480a2800 0x7f83480a4cc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f834c006010 tx=0x7f834c02c5f0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 msgr2=0x7f83641a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 0x7f83641a52f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f835c00e970 tx=0x7f835c00ed70 comp rx=0 tx=0).stop 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 shutdown_connections 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83480a2800 0x7f83480a4cc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8364102760 0x7f836419e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8364105110 0x7f83641a52f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 --2- 172.21.15.137:0/3515225615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83640ffdf0 0x7f836419dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 >> 172.21.15.137:0/3515225615 conn(0x7f83640fba10 msgr2=0x7f8364100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 shutdown_connections 2024-03-26T14:06:23.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.521+0000 7f8363e00640 1 -- 172.21.15.137:0/3515225615 wait complete. 2024-03-26T14:06:23.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:06:23.533 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:06:23.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:06:23.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:23 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2710285628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:06:23.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:23 smithi137 ceph-mon[31539]: pgmap v579: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:23.536 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:06:23.537 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:06:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:06:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:06:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:06:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:23 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2710285628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:06:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:23 smithi179 ceph-mon[35472]: pgmap v579: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:23.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:06:23.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:06:23.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:06:23.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:23 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2710285628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:06:23.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:23 smithi159 ceph-mon[35501]: pgmap v579: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 -- 172.21.15.137:0/139553123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 msgr2=0x7eff2c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 --2- 172.21.15.137:0/139553123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c101c10 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7eff18009900 tx=0x7eff1802bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 -- 172.21.15.137:0/139553123 shutdown_connections 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 --2- 172.21.15.137:0/139553123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff2c102150 0x7eff2c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 --2- 172.21.15.137:0/139553123 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c101c10 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 --2- 172.21.15.137:0/139553123 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c1005a0 0x7eff2c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 -- 172.21.15.137:0/139553123 >> 172.21.15.137:0/139553123 conn(0x7eff2c0fbb30 msgr2=0x7eff2c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 -- 172.21.15.137:0/139553123 shutdown_connections 2024-03-26T14:06:23.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.758+0000 7eff32400640 1 -- 172.21.15.137:0/139553123 wait complete. 2024-03-26T14:06:23.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.759+0000 7eff32400640 1 Processor -- start 2024-03-26T14:06:23.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.759+0000 7eff32400640 1 -- start start 2024-03-26T14:06:23.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff32400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c1005a0 0x7eff2c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff32400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff32400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff2c102150 0x7eff2c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff32400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c068cd0 con 0x7eff2c101790 2024-03-26T14:06:23.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff32400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c068b50 con 0x7eff2c102150 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff32400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c068e50 con 0x7eff2c1005a0 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c19a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46824/0 (socket says 172.21.15.137:46824) 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff2aa00640 1 -- 172.21.15.137:0/1885157568 learned_addr learned my addr 172.21.15.137:0/1885157568 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff2be00640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff2c102150 0x7eff2c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.760+0000 7eff2b400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c1005a0 0x7eff2c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.761+0000 7eff2aa00640 1 -- 172.21.15.137:0/1885157568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c1005a0 msgr2=0x7eff2c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.761+0000 7eff2aa00640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c1005a0 0x7eff2c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.761+0000 7eff2aa00640 1 -- 172.21.15.137:0/1885157568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff2c102150 msgr2=0x7eff2c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.761+0000 7eff2aa00640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff2c102150 0x7eff2c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.761+0000 7eff2aa00640 1 -- 172.21.15.137:0/1885157568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c1a1760 con 0x7eff2c101790 2024-03-26T14:06:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.761+0000 7eff2aa00640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c19a080 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7eff18000c00 tx=0x7eff180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.762+0000 7eff23e00640 1 -- 172.21.15.137:0/1885157568 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18043020 con 0x7eff2c101790 2024-03-26T14:06:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.762+0000 7eff23e00640 1 -- 172.21.15.137:0/1885157568 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff18004aa0 con 0x7eff2c101790 2024-03-26T14:06:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.762+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c1a19f0 con 0x7eff2c101790 2024-03-26T14:06:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.762+0000 7eff23e00640 1 -- 172.21.15.137:0/1885157568 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18003d70 con 0x7eff2c101790 2024-03-26T14:06:23.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.762+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1a1f40 con 0x7eff2c101790 2024-03-26T14:06:23.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 Processor -- start 2024-03-26T14:06:23.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 -- start start 2024-03-26T14:06:23.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff23e00640 1 -- 172.21.15.137:0/1885157568 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7eff18046020 con 0x7eff2c101790 2024-03-26T14:06:23.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 0x555c08ac7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac89e0 0x555c08ac8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac93a0 0x555c08acfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c08a2c2c0 con 0x555c08ac7810 2024-03-26T14:06:23.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c089d4c90 con 0x555c08ac89e0 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.766+0000 7eff351a6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c08a28360 con 0x555c08ac93a0 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 0x555c08ac7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 0x555c08ac7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46836/0 (socket says 172.21.15.137:46836) 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 -- 172.21.15.137:0/2270275219 learned_addr learned my addr 172.21.15.137:0/2270275219 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2b400640 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac89e0 0x555c08ac8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2aa00640 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac93a0 0x555c08acfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 -- 172.21.15.137:0/2270275219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac93a0 msgr2=0x555c08acfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac93a0 0x555c08acfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 -- 172.21.15.137:0/2270275219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac89e0 msgr2=0x555c08ac8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac89e0 0x555c08ac8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.767+0000 7eff2be00640 1 -- 172.21.15.137:0/2270275219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c089f9080 con 0x555c08ac7810 2024-03-26T14:06:23.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff2be00640 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 0x555c08ac7c30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7eff24009fd0 tx=0x7eff240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac90008b91f0b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff23e00640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff100a2800 0x7eff100a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff20c00640 1 -- 172.21.15.137:0/2270275219 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24004d20 con 0x555c08ac7810 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff20c00640 1 -- 172.21.15.137:0/2270275219 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff24004050 con 0x555c08ac7810 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff23e00640 1 -- 172.21.15.137:0/1885157568 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff180f11b0 con 0x7eff2c101790 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff20c00640 1 -- 172.21.15.137:0/2270275219 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24005280 con 0x555c08ac7810 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.768+0000 7eff2b400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff100a2800 0x7eff100a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 -- 172.21.15.137:0/2270275219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 msgr2=0x555c08ac7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff2b400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff100a2800 0x7eff100a4cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1015f0 tx=0x7eff1c00b4d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 0x555c08ac7c30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7eff24009fd0 tx=0x7eff240118d0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 -- 172.21.15.137:0/2270275219 shutdown_connections 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac89e0 0x555c08ac8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac93a0 0x555c08acfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 --2- 172.21.15.137:0/2270275219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac7810 0x555c08ac7c30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 -- 172.21.15.137:0/2270275219 >> 172.21.15.137:0/2270275219 conn(0x555c08ac2c00 msgr2=0x555c08ac3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.769+0000 7eff351a6740 1 -- 172.21.15.137:0/2270275219 shutdown_connections 2024-03-26T14:06:23.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.770+0000 7eff351a6740 1 -- 172.21.15.137:0/2270275219 wait complete. 2024-03-26T14:06:23.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.770+0000 7eff351a6740 1 -- start start 2024-03-26T14:06:23.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.770+0000 7eff351a6740 1 Processor -- start 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08ad1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08ad8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c08ad03b0 con 0x555c08ac89e0 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c089e80d0 con 0x555c08ac93a0 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c08ad0530 con 0x555c08ac7810 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff351a6740 10 client.0 _ll_register_callbacks cb 0x555c089ece80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:23.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46848/0 (socket says 172.21.15.137:46848) 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff2aa00640 1 -- 172.21.15.137:0/122575481 learned_addr learned my addr 172.21.15.137:0/122575481 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff2b400640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08ad1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.771+0000 7eff2be00640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08ad8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff2aa00640 1 -- 172.21.15.137:0/122575481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 msgr2=0x555c08ad1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff2aa00640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08ad1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff2aa00640 1 -- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 msgr2=0x555c08ad8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff2aa00640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08ad8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff2aa00640 1 -- 172.21.15.137:0/122575481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c087403f0 con 0x555c08ac89e0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff2aa00640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad1fa0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7eff180ba270 tx=0x7eff180b9b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23c62afebf75ae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff15600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff15600640 1 -- 172.21.15.137:0/122575481 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180bd200 con 0x555c08ac89e0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff15600640 1 -- 172.21.15.137:0/122575481 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff180bebd0 con 0x555c08ac89e0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.772+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff24004050 con 0x555c08ac89e0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff15600640 1 -- 172.21.15.137:0/122575481 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff180044e0 con 0x555c08ac89e0 2024-03-26T14:06:23.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 10 client.16440 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c0880a7d0 con 0x555c08ac89e0 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff15600640 1 -- 172.21.15.137:0/122575481 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff24004050 con 0x555c08ac89e0 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff15600640 1 -- 172.21.15.137:0/122575481 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7eff180c5bb0 con 0x555c08ac89e0 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 10 client.16440 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 10 client.16440 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 20 client.16440 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08add470 0x555c08adf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 4 client.16440 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:23.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.773+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c08a39800 con 0x555c08add470 2024-03-26T14:06:23.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.774+0000 7eff2b400640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08add470 0x555c08adf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.775+0000 7eff2b400640 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08add470 0x555c08adf8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bcc850919324cf0a server_cookie=83a524720f653188 in_seq=0 out_seq=0 2024-03-26T14:06:23.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.775+0000 7eff15600640 10 client.16440 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff15600640 1 -- 172.21.15.137:0/122575481 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555c08a39800 con 0x555c08add470 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff15600640 10 client.16440 handle_command_reply: tid=0 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 client.16440 shutdown 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 msgr2=0x555c08ad1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad1fa0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7eff180ba270 tx=0x7eff180b9b40 comp rx=0 tx=0).stop 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 shutdown_connections 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08add470 0x555c08adf8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08ad8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad1fa0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 --2- 172.21.15.137:0/122575481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08ad1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff15600640 0 client.16440 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 >> 172.21.15.137:0/122575481 conn(0x555c08ac2c00 msgr2=0x555c08ac1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 shutdown_connections 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 1 -- 172.21.15.137:0/122575481 wait complete. 2024-03-26T14:06:23.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.776+0000 7eff351a6740 20 client.16440 trim_cache size 0 max 16384 2024-03-26T14:06:23.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 Processor -- start 2024-03-26T14:06:23.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 -- start start 2024-03-26T14:06:23.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08acb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08acbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ae9b0 con 0x555c08ac89e0 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ac4b0 con 0x555c08ac93a0 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff351a6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1aa530 con 0x555c08ac7810 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46858/0 (socket says 172.21.15.137:46858) 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.790+0000 7eff2aa00640 1 -- 172.21.15.137:0/77716495 learned_addr learned my addr 172.21.15.137:0/77716495 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2be00640 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08acbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2b400640 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08acb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2aa00640 1 -- 172.21.15.137:0/77716495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 msgr2=0x555c08acb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2aa00640 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08acb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2aa00640 1 -- 172.21.15.137:0/77716495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 msgr2=0x555c08acbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2aa00640 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08acbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2aa00640 1 -- 172.21.15.137:0/77716495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c08adff30 con 0x555c08ac89e0 2024-03-26T14:06:23.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2be00640 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08acbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff2aa00640 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad8fa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7eff1802c6a0 tx=0x7eff180b9c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127925b0d54eccc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff17e00640 1 -- 172.21.15.137:0/77716495 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18004c50 con 0x555c08ac89e0 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff17e00640 1 -- 172.21.15.137:0/77716495 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff180bdfa0 con 0x555c08ac89e0 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.791+0000 7eff17e00640 1 -- 172.21.15.137:0/77716495 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180c5bb0 con 0x555c08ac89e0 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 -- 172.21.15.137:0/77716495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 msgr2=0x555c08ad8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad8fa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7eff1802c6a0 tx=0x7eff180b9c70 comp rx=0 tx=0).stop 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 -- 172.21.15.137:0/77716495 shutdown_connections 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac93a0 0x555c08acbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac89e0 0x555c08ad8fa0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 --2- 172.21.15.137:0/77716495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac7810 0x555c08acb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 -- 172.21.15.137:0/77716495 >> 172.21.15.137:0/77716495 conn(0x555c08ac2c00 msgr2=0x555c08ac5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 -- 172.21.15.137:0/77716495 shutdown_connections 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.792+0000 7eff351a6740 1 -- 172.21.15.137:0/77716495 wait complete. 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.793+0000 7eff351a6740 1 -- start start 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.793+0000 7eff351a6740 1 Processor -- start 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.793+0000 7eff351a6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac7810 0x555c08ad87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.793+0000 7eff351a6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac89e0 0x555c08ad8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff351a6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 0x555c08ad5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff351a6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1aab30 con 0x555c08ac93a0 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff351a6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1abe50 con 0x555c08ac7810 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff351a6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c1ab190 con 0x555c08ac89e0 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff351a6740 10 client.0 _ll_register_callbacks cb 0x7eff18009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 0x555c08ad5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 0x555c08ad5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46864/0 (socket says 172.21.15.137:46864) 2024-03-26T14:06:23.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 -- 172.21.15.137:0/1049557656 learned_addr learned my addr 172.21.15.137:0/1049557656 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2b400640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac7810 0x555c08ad87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2aa00640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac89e0 0x555c08ad8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 -- 172.21.15.137:0/1049557656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac89e0 msgr2=0x555c08ad8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac89e0 0x555c08ad8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 -- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac7810 msgr2=0x555c08ad87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac7810 0x555c08ad87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2be00640 1 -- 172.21.15.137:0/1049557656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c08ad5980 con 0x555c08ac93a0 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2aa00640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac89e0 0x555c08ad8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:23.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.794+0000 7eff2b400640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac7810 0x555c08ad87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:23.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff2be00640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 0x555c08ad5290 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7eff24006010 tx=0x7eff24003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a217293034c82b54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:23.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff15600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff15600640 1 -- 172.21.15.137:0/1049557656 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24028070 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff180bdfa0 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff15600640 1 -- 172.21.15.137:0/1049557656 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7eff24004c90 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff15600640 1 -- 172.21.15.137:0/1049557656 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff24034050 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff351a6740 10 client.16446 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.795+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c08b399b0 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff15600640 1 -- 172.21.15.137:0/1049557656 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff180bdfa0 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff15600640 1 -- 172.21.15.137:0/1049557656 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7eff240051d0 con 0x555c08ac93a0 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff351a6740 10 client.16446 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff351a6740 10 client.16446 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff351a6740 20 client.16446 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff351a6740 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08b3a170 0x555c08b3c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff351a6740 4 client.16446 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:06:23.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.796+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555c08a33b50 con 0x555c08b3a170 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.797+0000 7eff2b400640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08b3a170 0x555c08b3c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.797+0000 7eff2b400640 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08b3a170 0x555c08b3c590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5d038e1bb4f3114 server_cookie=2b0ce5350b050fc7 in_seq=0 out_seq=0 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.797+0000 7eff15600640 10 client.16446 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.797+0000 7eff15600640 1 -- 172.21.15.137:0/1049557656 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555c08a33b50 con 0x555c08b3a170 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.797+0000 7eff15600640 10 client.16446 handle_command_reply: tid=0 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.797+0000 7eff351a6740 1 client.16446 shutdown 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 msgr2=0x555c08ad5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 0x555c08ad5290 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7eff24006010 tx=0x7eff24003e90 comp rx=0 tx=0).stop 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 shutdown_connections 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x555c08b3a170 0x555c08b3c590 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555c08ac93a0 0x555c08ad5290 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555c08ac89e0 0x555c08ad8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 --2- 172.21.15.137:0/1049557656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c08ac7810 0x555c08ad87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 >> 172.21.15.137:0/1049557656 conn(0x555c08ac2c00 msgr2=0x555c08ac1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff15600640 0 client.16446 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:23.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 shutdown_connections 2024-03-26T14:06:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 1 -- 172.21.15.137:0/1049557656 wait complete. 2024-03-26T14:06:23.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.798+0000 7eff351a6740 20 client.16446 trim_cache size 0 max 16384 2024-03-26T14:06:23.800 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff100a2800 msgr2=0x7eff100a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff100a2800 0x7eff100a4cc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff2c1015f0 tx=0x7eff1c00b4d0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 msgr2=0x7eff2c19a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c19a080 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7eff18000c00 tx=0x7eff180042f0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 shutdown_connections 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7eff100a2800 0x7eff100a4cc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff2c102150 0x7eff2c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff2c101790 0x7eff2c19a080 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 --2- 172.21.15.137:0/1885157568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff2c1005a0 0x7eff2c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 >> 172.21.15.137:0/1885157568 conn(0x7eff2c0fbb30 msgr2=0x7eff2c10bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 shutdown_connections 2024-03-26T14:06:23.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:23.799+0000 7eff32400640 1 -- 172.21.15.137:0/1885157568 wait complete. 2024-03-26T14:06:23.817 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:06:23.817 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 -- 172.21.15.137:0/3317032180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 msgr2=0x7f5ae0103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3317032180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0103be0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009900 tx=0x7f5ad002bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 -- 172.21.15.137:0/3317032180 shutdown_connections 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3317032180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 0x7f5ae010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3317032180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3317032180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0103be0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 -- 172.21.15.137:0/3317032180 >> 172.21.15.137:0/3317032180 conn(0x7f5ae00fed50 msgr2=0x7f5ae0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 -- 172.21.15.137:0/3317032180 shutdown_connections 2024-03-26T14:06:24.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.025+0000 7f5adfe00640 1 -- 172.21.15.137:0/3317032180 wait complete. 2024-03-26T14:06:24.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.026+0000 7f5adfe00640 1 Processor -- start 2024-03-26T14:06:24.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 -- start start 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 0x7f5ae01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010f340 con 0x7f5ae01049b0 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010f1c0 con 0x7f5ae01037c0 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adfe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae010f4c0 con 0x7f5ae0105370 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5ade000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5adea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5ade000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46876/0 (socket says 172.21.15.137:46876) 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.027+0000 7f5ade000640 1 -- 172.21.15.137:0/3267866550 learned_addr learned my addr 172.21.15.137:0/3267866550 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5adf400640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 0x7f5ae01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ade000640 1 -- 172.21.15.137:0/3267866550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 msgr2=0x7f5ae01a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ade000640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 0x7f5ae01a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ade000640 1 -- 172.21.15.137:0/3267866550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 msgr2=0x7f5ae0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ade000640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ade000640 1 -- 172.21.15.137:0/3267866550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae01a1630 con 0x7f5ae01049b0 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5adf400640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 0x7f5ae01a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5adea00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ade000640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0199f50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00e9f0 tx=0x7f5acc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ad7400640 1 -- 172.21.15.137:0/3267866550 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc00cd80 con 0x7f5ae01049b0 2024-03-26T14:06:24.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.028+0000 7f5ad7400640 1 -- 172.21.15.137:0/3267866550 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5acc0108e0 con 0x7f5ae01049b0 2024-03-26T14:06:24.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.029+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae01a1920 con 0x7f5ae01049b0 2024-03-26T14:06:24.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.029+0000 7f5ad7400640 1 -- 172.21.15.137:0/3267866550 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc018920 con 0x7f5ae01049b0 2024-03-26T14:06:24.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.029+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae01a1e70 con 0x7f5ae01049b0 2024-03-26T14:06:24.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.033+0000 7f5ae8410740 1 Processor -- start 2024-03-26T14:06:24.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.033+0000 7f5ae8410740 1 -- start start 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.033+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a3ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.033+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3bbc0 0x563ff1a3c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3c580 0x563ff1a42e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5ae8410740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff199f2c0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5ae8410740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff1947c90 con 0x563ff1a3bbc0 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5ae8410740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff199b360 con 0x563ff1a3c580 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5ad7400640 1 -- 172.21.15.137:0/3267866550 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5acc010a90 con 0x7f5ae01049b0 2024-03-26T14:06:24.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a3ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3bbc0 0x563ff1a3c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a3ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46886/0 (socket says 172.21.15.137:46886) 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 -- 172.21.15.137:0/1248927182 learned_addr learned my addr 172.21.15.137:0/1248927182 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5ade000640 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3c580 0x563ff1a42e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 -- 172.21.15.137:0/1248927182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3c580 msgr2=0x563ff1a42e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3c580 0x563ff1a42e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 -- 172.21.15.137:0/1248927182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3bbc0 msgr2=0x563ff1a3c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.034+0000 7f5adf400640 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3bbc0 0x563ff1a3c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.035+0000 7f5adf400640 1 -- 172.21.15.137:0/1248927182 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff196c080 con 0x563ff1a3a9f0 2024-03-26T14:06:24.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.035+0000 7f5adf400640 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a3ae10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009f90 tx=0x7f5ad800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32edb9715e9d66d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:24.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.035+0000 7f5acb400640 1 -- 172.21.15.137:0/1248927182 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8003b70 con 0x563ff1a3a9f0 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.035+0000 7f5acb400640 1 -- 172.21.15.137:0/1248927182 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad8003d10 con 0x563ff1a3a9f0 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.035+0000 7f5acb400640 1 -- 172.21.15.137:0/1248927182 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80118f0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.035+0000 7f5ad7400640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ac40a2880 0x7f5ac40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.036+0000 7f5ad7400640 1 -- 172.21.15.137:0/3267866550 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5acc0cf210 con 0x7f5ae01049b0 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.036+0000 7f5adea00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ac40a2880 0x7f5ac40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.036+0000 7f5ae8410740 1 -- 172.21.15.137:0/1248927182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 msgr2=0x563ff1a3ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.036+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a3ae10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009f90 tx=0x7f5ad800f7f0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.036+0000 7f5ae8410740 1 -- 172.21.15.137:0/1248927182 shutdown_connections 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.036+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3bbc0 0x563ff1a3c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3c580 0x563ff1a42e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1248927182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a3ae10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5ae8410740 1 -- 172.21.15.137:0/1248927182 >> 172.21.15.137:0/1248927182 conn(0x563ff1a35de0 msgr2=0x563ff1a361e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5adea00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ac40a2880 0x7f5ac40a4d40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5ad002c240 tx=0x7f5ad002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5ae8410740 1 -- 172.21.15.137:0/1248927182 shutdown_connections 2024-03-26T14:06:24.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5ae8410740 1 -- 172.21.15.137:0/1248927182 wait complete. 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.037+0000 7f5ae8410740 1 -- start start 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 Processor -- start 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3a9f0 0x563ff1a44c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 0x563ff1a451d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a4c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff1a43590 con 0x563ff1a3bbc0 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff195b0d0 con 0x563ff1a3c580 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff1a43710 con 0x563ff1a3a9f0 2024-03-26T14:06:24.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ae8410740 10 client.0 _ll_register_callbacks cb 0x563ff195fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ade000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 0x563ff1a451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5adf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a4c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.038+0000 7f5ade000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 0x563ff1a451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46900/0 (socket says 172.21.15.137:46900) 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 -- 172.21.15.137:0/3047633782 learned_addr learned my addr 172.21.15.137:0/3047633782 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5adea00640 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3a9f0 0x563ff1a44c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 -- 172.21.15.137:0/3047633782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3a9f0 msgr2=0x563ff1a44c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3a9f0 0x563ff1a44c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 -- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 msgr2=0x563ff1a4c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a4c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 -- 172.21.15.137:0/3047633782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff16b33f0 con 0x563ff1a3bbc0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5ade000640 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 0x563ff1a451d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5acc09b730 tx=0x7f5acc09b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c7c4e0717f26ee server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5abfe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5abfe00640 1 -- 172.21.15.137:0/3047633782 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc09bea0 con 0x563ff1a3bbc0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.039+0000 7f5abfe00640 1 -- 172.21.15.137:0/3047633782 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5acc010430 con 0x563ff1a3bbc0 2024-03-26T14:06:24.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ad8003d10 con 0x563ff1a3bbc0 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5abfe00640 1 -- 172.21.15.137:0/3047633782 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5acc09b7a0 con 0x563ff1a3bbc0 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5ae8410740 10 client.16458 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563ff177d7d0 con 0x563ff1a3bbc0 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5abfe00640 1 -- 172.21.15.137:0/3047633782 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad8003d10 con 0x563ff1a3bbc0 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5abfe00640 1 -- 172.21.15.137:0/3047633782 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5acc0974f0 con 0x563ff1a3bbc0 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5ae8410740 10 client.16458 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5ae8410740 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.040+0000 7f5ae8410740 20 client.16458 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.041+0000 7f5ae8410740 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1a50740 0x563ff1a52b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.041+0000 7f5ae8410740 4 client.16458 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:24.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.041+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ff19ac800 con 0x563ff1a50740 2024-03-26T14:06:24.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.041+0000 7f5adea00640 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1a50740 0x563ff1a52b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.042+0000 7f5adea00640 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1a50740 0x563ff1a52b60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f944f9dc5bf3a3 server_cookie=43a24b489971286e in_seq=0 out_seq=0 2024-03-26T14:06:24.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.042+0000 7f5abfe00640 10 client.16458 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.043+0000 7f5abfe00640 1 -- 172.21.15.137:0/3047633782 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563ff19ac800 con 0x563ff1a50740 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.043+0000 7f5abfe00640 10 client.16458 handle_command_reply: tid=0 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.043+0000 7f5ae8410740 1 client.16458 shutdown 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 msgr2=0x563ff1a451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 0x563ff1a451d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5acc09b730 tx=0x7f5acc09b2e0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 shutdown_connections 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1a50740 0x563ff1a52b60 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a4c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3bbc0 0x563ff1a451d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 --2- 172.21.15.137:0/3047633782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3a9f0 0x563ff1a44c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5abfe00640 0 client.16458 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:24.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 >> 172.21.15.137:0/3047633782 conn(0x563ff1a35de0 msgr2=0x563ff1a35160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:24.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 shutdown_connections 2024-03-26T14:06:24.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 1 -- 172.21.15.137:0/3047633782 wait complete. 2024-03-26T14:06:24.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.044+0000 7f5ae8410740 20 client.16458 trim_cache size 0 max 16384 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 Processor -- start 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 -- start start 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a34fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a57420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01ae850 con 0x563ff1a3a9f0 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01ac350 con 0x563ff1a3c580 2024-03-26T14:06:24.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.061+0000 7f5ae8410740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01aa3d0 con 0x563ff1a3bbc0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a34fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a34fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46904/0 (socket says 172.21.15.137:46904) 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 -- 172.21.15.137:0/578461257 learned_addr learned my addr 172.21.15.137:0/578461257 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adf400640 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a57420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 -- 172.21.15.137:0/578461257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 msgr2=0x563ff1a4c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5ade000640 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 -- 172.21.15.137:0/578461257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 msgr2=0x563ff1a57420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a57420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.062+0000 7f5adea00640 1 -- 172.21.15.137:0/578461257 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff1a4f130 con 0x563ff1a3a9f0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.063+0000 7f5adea00640 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a34fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00096c0 tx=0x7f5ad0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73126d07b15a90d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.063+0000 7f5acaa00640 1 -- 172.21.15.137:0/578461257 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00542d0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.063+0000 7f5acaa00640 1 -- 172.21.15.137:0/578461257 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad005a070 con 0x563ff1a3a9f0 2024-03-26T14:06:24.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.063+0000 7f5acaa00640 1 -- 172.21.15.137:0/578461257 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00035f0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 -- 172.21.15.137:0/578461257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 msgr2=0x563ff1a34fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a34fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00096c0 tx=0x7f5ad0041000 comp rx=0 tx=0).stop 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 -- 172.21.15.137:0/578461257 shutdown_connections 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a57420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 --2- 172.21.15.137:0/578461257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a34fe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 -- 172.21.15.137:0/578461257 >> 172.21.15.137:0/578461257 conn(0x563ff1a35de0 msgr2=0x563ff1a348d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 -- 172.21.15.137:0/578461257 shutdown_connections 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 -- 172.21.15.137:0/578461257 wait complete. 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 -- start start 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.064+0000 7f5ae8410740 1 Processor -- start 2024-03-26T14:06:24.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a48510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01aa9d0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01abcf0 con 0x563ff1a3c580 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01ab030 con 0x563ff1a3bbc0 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5ae8410740 10 client.0 _ll_register_callbacks cb 0x7f5acc097a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5adea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5adea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a4ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46918/0 (socket says 172.21.15.137:46918) 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5adea00640 1 -- 172.21.15.137:0/1492076599 learned_addr learned my addr 172.21.15.137:0/1492076599 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.065+0000 7f5adf400640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a48510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5ade000640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5adea00640 1 -- 172.21.15.137:0/1492076599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 msgr2=0x563ff1a4bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5adea00640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5adea00640 1 -- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 msgr2=0x563ff1a48510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5adea00640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a48510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5adea00640 1 -- 172.21.15.137:0/1492076599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff1a48c50 con 0x563ff1a3a9f0 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5adea00640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a4ba70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad0005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b82c463de4e5bf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5abfe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5abfe00640 1 -- 172.21.15.137:0/1492076599 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0063060 con 0x563ff1a3a9f0 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5abfe00640 1 -- 172.21.15.137:0/1492076599 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5ad00048d0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.066+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ad005a070 con 0x563ff1a3a9f0 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5abfe00640 1 -- 172.21.15.137:0/1492076599 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ad0004ad0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 10 client.16464 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563ff1aaccd0 con 0x563ff1a3a9f0 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5abfe00640 1 -- 172.21.15.137:0/1492076599 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5ad005a070 con 0x563ff1a3a9f0 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5abfe00640 1 -- 172.21.15.137:0/1492076599 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5ad0034600 con 0x563ff1a3a9f0 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 10 client.16464 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 20 client.16464 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:24.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1aad660 0x563ff1aafa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 4 client.16464 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.067+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563ff19a6b50 con 0x563ff1aad660 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.068+0000 7f5ade000640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1aad660 0x563ff1aafa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ade000640 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1aad660 0x563ff1aafa80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d59e5f67ce6819 server_cookie=548425f88da2809f in_seq=0 out_seq=0 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5abfe00640 10 client.16464 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5abfe00640 1 -- 172.21.15.137:0/1492076599 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563ff19a6b50 con 0x563ff1aad660 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5abfe00640 10 client.16464 handle_command_reply: tid=0 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 client.16464 shutdown 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 msgr2=0x563ff1a4ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a4ba70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad0005520 comp rx=0 tx=0).stop 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 shutdown_connections 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563ff1aad660 0x563ff1aafa80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ff1a3c580 0x563ff1a48510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563ff1a3bbc0 0x563ff1a4bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 --2- 172.21.15.137:0/1492076599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563ff1a3a9f0 0x563ff1a4ba70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 >> 172.21.15.137:0/1492076599 conn(0x563ff1a35de0 msgr2=0x563ff1a35160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:24.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.069+0000 7f5abfe00640 0 client.16464 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:24.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 shutdown_connections 2024-03-26T14:06:24.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5ae8410740 1 -- 172.21.15.137:0/1492076599 wait complete. 2024-03-26T14:06:24.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5ae8410740 20 client.16464 trim_cache size 0 max 16384 2024-03-26T14:06:24.071 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ac40a2880 msgr2=0x7f5ac40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ac40a2880 0x7f5ac40a4d40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5ad002c240 tx=0x7f5ad002c7e0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 msgr2=0x7f5ae0199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.070+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0199f50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00e9f0 tx=0x7f5acc00eeb0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 shutdown_connections 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ac40a2880 0x7f5ac40a4d40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ae0105370 0x7f5ae01a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5ae01049b0 0x7f5ae0199f50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 --2- 172.21.15.137:0/3267866550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ae01037c0 0x7f5ae0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 >> 172.21.15.137:0/3267866550 conn(0x7f5ae00fed50 msgr2=0x7f5ae0107cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 shutdown_connections 2024-03-26T14:06:24.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:24.071+0000 7f5adfe00640 1 -- 172.21.15.137:0/3267866550 wait complete. 2024-03-26T14:06:25.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:06:25.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:06:25.087 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:06:25.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.318+0000 7f6826200640 1 -- 172.21.15.137:0/3615052006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68200ffe50 msgr2=0x7f6820100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.318+0000 7f6826200640 1 --2- 172.21.15.137:0/3615052006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68200ffe50 0x7f6820100270 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6814009900 tx=0x7f681402bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 -- 172.21.15.137:0/3615052006 shutdown_connections 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 --2- 172.21.15.137:0/3615052006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68201007b0 0x7f6820100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 --2- 172.21.15.137:0/3615052006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6820101150 0x7f68201015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 --2- 172.21.15.137:0/3615052006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68200ffe50 0x7f6820100270 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 -- 172.21.15.137:0/3615052006 >> 172.21.15.137:0/3615052006 conn(0x7f68200fba10 msgr2=0x7f68200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 -- 172.21.15.137:0/3615052006 shutdown_connections 2024-03-26T14:06:25.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.319+0000 7f6826200640 1 -- 172.21.15.137:0/3615052006 wait complete. 2024-03-26T14:06:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 Processor -- start 2024-03-26T14:06:25.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 -- start start 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68200ffe50 0x7f6820199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 0x7f6820199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6820101150 0x7f68201a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682010f2c0 con 0x7f68201007b0 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682010f140 con 0x7f68200ffe50 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f6826200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f682010f440 con 0x7f6820101150 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f681fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 0x7f6820199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f681fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 0x7f6820199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46950/0 (socket says 172.21.15.137:46950) 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.320+0000 7f681fe00640 1 -- 172.21.15.137:0/2310268688 learned_addr learned my addr 172.21.15.137:0/2310268688 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f6824e00640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68200ffe50 0x7f6820199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f6825800640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6820101150 0x7f68201a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681fe00640 1 -- 172.21.15.137:0/2310268688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6820101150 msgr2=0x7f68201a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681fe00640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6820101150 0x7f68201a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681fe00640 1 -- 172.21.15.137:0/2310268688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68200ffe50 msgr2=0x7f6820199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681fe00640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68200ffe50 0x7f6820199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681fe00640 1 -- 172.21.15.137:0/2310268688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68201a1590 con 0x7f68201007b0 2024-03-26T14:06:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681fe00640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 0x7f6820199eb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d950 tx=0x7f680c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681d600640 1 -- 172.21.15.137:0/2310268688 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c017070 con 0x7f68201007b0 2024-03-26T14:06:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681d600640 1 -- 172.21.15.137:0/2310268688 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f680c009c30 con 0x7f68201007b0 2024-03-26T14:06:25.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.321+0000 7f681d600640 1 -- 172.21.15.137:0/2310268688 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c00f620 con 0x7f68201007b0 2024-03-26T14:06:25.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.322+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68201a1880 con 0x7f68201007b0 2024-03-26T14:06:25.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.322+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68201a1d80 con 0x7f68201007b0 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.325+0000 7f682a473740 1 Processor -- start 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 -- start start 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f681d600640 1 -- 172.21.15.137:0/2310268688 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f680c0109a0 con 0x7f68201007b0 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c97582b0 0x5586c97586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c975d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c975de10 0x5586c97602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c96bdeb0 con 0x5586c9758ca0 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9666a60 con 0x5586c97582b0 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f682a473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c96b9f50 con 0x5586c975de10 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f6824e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c975d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f6824e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c975d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46952/0 (socket says 172.21.15.137:46952) 2024-03-26T14:06:25.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.326+0000 7f6824e00640 1 -- 172.21.15.137:0/3779602473 learned_addr learned my addr 172.21.15.137:0/3779602473 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6825800640 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c97582b0 0x5586c97586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f681fe00640 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c975de10 0x5586c97602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6824e00640 1 -- 172.21.15.137:0/3779602473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c975de10 msgr2=0x5586c97602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6824e00640 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c975de10 0x5586c97602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6824e00640 1 -- 172.21.15.137:0/3779602473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c97582b0 msgr2=0x5586c97586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6824e00640 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c97582b0 0x5586c97586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6824e00640 1 -- 172.21.15.137:0/3779602473 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586c949a7e0 con 0x5586c9758ca0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6825800640 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c97582b0 0x5586c97586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.327+0000 7f6824e00640 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c975d7a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6814005ab0 tx=0x7f6814005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2a237e925bb51d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.328+0000 7f681d600640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68080a2880 0x7f68080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.328+0000 7f6812000640 1 -- 172.21.15.137:0/3779602473 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814044020 con 0x5586c9758ca0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.328+0000 7f681d600640 1 -- 172.21.15.137:0/2310268688 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f680c0ceaf0 con 0x7f68201007b0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.328+0000 7f6812000640 1 -- 172.21.15.137:0/3779602473 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6814005040 con 0x5586c9758ca0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.328+0000 7f6812000640 1 -- 172.21.15.137:0/3779602473 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6814004920 con 0x5586c9758ca0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.328+0000 7f6825800640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68080a2880 0x7f68080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f6825800640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68080a2880 0x7f68080a4d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x5586c9759500 tx=0x7f6818008470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 -- 172.21.15.137:0/3779602473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 msgr2=0x5586c975d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c975d7a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6814005ab0 tx=0x7f6814005e00 comp rx=0 tx=0).stop 2024-03-26T14:06:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 -- 172.21.15.137:0/3779602473 shutdown_connections 2024-03-26T14:06:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c975de10 0x5586c97602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c975d7a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 --2- 172.21.15.137:0/3779602473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c97582b0 0x5586c97586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 -- 172.21.15.137:0/3779602473 >> 172.21.15.137:0/3779602473 conn(0x5586c9753820 msgr2=0x5586c9753c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 -- 172.21.15.137:0/3779602473 shutdown_connections 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.329+0000 7f682a473740 1 -- 172.21.15.137:0/3779602473 wait complete. 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 -- start start 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 Processor -- start 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9762460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c97699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9760d60 con 0x5586c975de10 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9679eb0 con 0x5586c9758ca0 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586c9760ee0 con 0x5586c97582b0 2024-03-26T14:06:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.330+0000 7f682a473740 10 client.0 _ll_register_callbacks cb 0x5586c966e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c97699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c97699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46958/0 (socket says 172.21.15.137:46958) 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 -- 172.21.15.137:0/4013858964 learned_addr learned my addr 172.21.15.137:0/4013858964 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f681fe00640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6824e00640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9762460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 -- 172.21.15.137:0/4013858964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 msgr2=0x5586c9762460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9762460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 -- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 msgr2=0x5586c97629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f6825800640 1 -- 172.21.15.137:0/4013858964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586c94faf70 con 0x5586c975de10 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.331+0000 7f681fe00640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f6825800640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c97699e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6818010e30 tx=0x7f6818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7149bb627a354e6c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f6806a00640 1 -- 172.21.15.137:0/4013858964 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681801d420 con 0x5586c975de10 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f6806a00640 1 -- 172.21.15.137:0/4013858964 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f681801d5c0 con 0x5586c975de10 2024-03-26T14:06:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6814005040 con 0x5586c975de10 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f6806a00640 1 -- 172.21.15.137:0/4013858964 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f681802e050 con 0x5586c975de10 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f682a473740 10 client.16476 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.332+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5586c9392170 con 0x5586c975de10 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f6806a00640 1 -- 172.21.15.137:0/4013858964 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6814005040 con 0x5586c975de10 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f6806a00640 1 -- 172.21.15.137:0/4013858964 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f681801b7d0 con 0x5586c975de10 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f682a473740 10 client.16476 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f682a473740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f682a473740 20 client.16476 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f682a473740 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c976e350 0x5586c97707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f682a473740 4 client.16476 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.333+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586c96cb3f0 con 0x5586c976e350 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.334+0000 7f6824e00640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c976e350 0x5586c97707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.335+0000 7f6806a00640 10 client.16476 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.335+0000 7f6824e00640 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c976e350 0x5586c97707b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c690a11b6843009e server_cookie=5e3ae2a0deff4ccb in_seq=0 out_seq=0 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.336+0000 7f6806a00640 1 -- 172.21.15.137:0/4013858964 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5586c96cb3f0 con 0x5586c976e350 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.336+0000 7f6806a00640 10 client.16476 handle_command_reply: tid=0 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.336+0000 7f682a473740 1 client.16476 shutdown 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 msgr2=0x5586c97699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c97699e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6818010e30 tx=0x7f6818004a40 comp rx=0 tx=0).stop 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 shutdown_connections 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c976e350 0x5586c97707b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c97699e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 --2- 172.21.15.137:0/4013858964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9762460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 >> 172.21.15.137:0/4013858964 conn(0x5586c9753820 msgr2=0x5586c9752a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f6806a00640 0 client.16476 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 shutdown_connections 2024-03-26T14:06:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 1 -- 172.21.15.137:0/4013858964 wait complete. 2024-03-26T14:06:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.337+0000 7f682a473740 20 client.16476 trim_cache size 0 max 16384 2024-03-26T14:06:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 Processor -- start 2024-03-26T14:06:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 -- start start 2024-03-26T14:06:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9752900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c9775070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201ae7b0 con 0x5586c975de10 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201ac2b0 con 0x5586c9758ca0 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.359+0000 7f682a473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201aa330 con 0x5586c97582b0 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c9775070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c9775070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46968/0 (socket says 172.21.15.137:46968) 2024-03-26T14:06:25.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 -- 172.21.15.137:0/3637407155 learned_addr learned my addr 172.21.15.137:0/3637407155 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f681fe00640 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6824e00640 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9752900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 -- 172.21.15.137:0/3637407155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 msgr2=0x5586c9752900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9752900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 -- 172.21.15.137:0/3637407155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 msgr2=0x5586c97699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f6825800640 1 -- 172.21.15.137:0/3637407155 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586c9762250 con 0x5586c975de10 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.360+0000 7f681fe00640 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.361+0000 7f6825800640 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c9775070 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6818010e00 tx=0x7f681801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a9af6bf809ab6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.361+0000 7f6811600640 1 -- 172.21.15.137:0/3637407155 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818038020 con 0x5586c975de10 2024-03-26T14:06:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.361+0000 7f6811600640 1 -- 172.21.15.137:0/3637407155 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6818008bc0 con 0x5586c975de10 2024-03-26T14:06:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.361+0000 7f6811600640 1 -- 172.21.15.137:0/3637407155 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818034a80 con 0x5586c975de10 2024-03-26T14:06:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 -- 172.21.15.137:0/3637407155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 msgr2=0x5586c9775070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c9775070 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6818010e00 tx=0x7f681801c430 comp rx=0 tx=0).stop 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 -- 172.21.15.137:0/3637407155 shutdown_connections 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c9758ca0 0x5586c97699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c975de10 0x5586c9775070 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 --2- 172.21.15.137:0/3637407155 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c9752900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 -- 172.21.15.137:0/3637407155 >> 172.21.15.137:0/3637407155 conn(0x5586c9753820 msgr2=0x5586c97521f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.362+0000 7f682a473740 1 -- 172.21.15.137:0/3637407155 shutdown_connections 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.363+0000 7f682a473740 1 -- 172.21.15.137:0/3637407155 wait complete. 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.363+0000 7f682a473740 1 -- start start 2024-03-26T14:06:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.363+0000 7f682a473740 1 Processor -- start 2024-03-26T14:06:25.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c97695f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c9767c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c975de10 0x5586c9768140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201aa930 con 0x5586c9758ca0 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201abc50 con 0x5586c975de10 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201aaf90 con 0x5586c97582b0 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f682a473740 10 client.0 _ll_register_callbacks cb 0x7f681800ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c9767c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c9767c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46974/0 (socket says 172.21.15.137:46974) 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 -- 172.21.15.137:0/2236557409 learned_addr learned my addr 172.21.15.137:0/2236557409 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f6824e00640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c97695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f6825800640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c975de10 0x5586c9768140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 -- 172.21.15.137:0/2236557409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 msgr2=0x5586c97695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c97695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 -- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c975de10 msgr2=0x5586c9768140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c975de10 0x5586c9768140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.364+0000 7f681fe00640 1 -- 172.21.15.137:0/2236557409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586c97686b0 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f681fe00640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c9767c00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f680c0058c0 tx=0x7f680c09aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854d790aad57b0d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f6806a00640 1 -- 172.21.15.137:0/2236557409 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c020040 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f6806a00640 1 -- 172.21.15.137:0/2236557409 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f680c09a4d0 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6818008bc0 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f6806a00640 1 -- 172.21.15.137:0/2236557409 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f680c09f1d0 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f682a473740 10 client.16482 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.365+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5586c97caab0 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f6806a00640 1 -- 172.21.15.137:0/2236557409 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6818008bc0 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f6806a00640 1 -- 172.21.15.137:0/2236557409 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f680c0b9b80 con 0x5586c9758ca0 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f682a473740 10 client.16482 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f682a473740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f682a473740 20 client.16482 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f682a473740 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c97cb3c0 0x5586c97cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f682a473740 4 client.16482 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:06:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.366+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5586c96c5740 con 0x5586c97cb3c0 2024-03-26T14:06:25.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.367+0000 7f6824e00640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c97cb3c0 0x5586c97cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.367+0000 7f6824e00640 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c97cb3c0 0x5586c97cd7e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311f524e3a2c6a3b server_cookie=51507dcd31a59e43 in_seq=0 out_seq=0 2024-03-26T14:06:25.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.367+0000 7f6806a00640 10 client.16482 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f6806a00640 1 -- 172.21.15.137:0/2236557409 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5586c96c5740 con 0x5586c97cb3c0 2024-03-26T14:06:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f6806a00640 10 client.16482 handle_command_reply: tid=0 2024-03-26T14:06:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 client.16482 shutdown 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 msgr2=0x5586c9767c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c9767c00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f680c0058c0 tx=0x7f680c09aaa0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 shutdown_connections 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5586c97cb3c0 0x5586c97cd7e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586c975de10 0x5586c9768140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5586c9758ca0 0x5586c9767c00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 --2- 172.21.15.137:0/2236557409 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586c97582b0 0x5586c97695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.370+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 >> 172.21.15.137:0/2236557409 conn(0x5586c9753820 msgr2=0x5586c9752a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.371+0000 7f6806a00640 0 client.16482 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.371+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 shutdown_connections 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.371+0000 7f682a473740 1 -- 172.21.15.137:0/2236557409 wait complete. 2024-03-26T14:06:25.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.371+0000 7f682a473740 20 client.16482 trim_cache size 0 max 16384 2024-03-26T14:06:25.371 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:06:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:06:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "fe07bfe6-b00f-4ede-ae33-a8cd627fdf06", 2024-03-26T14:06:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:06:25.371 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68080a2880 msgr2=0x7f68080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68080a2880 0x7f68080a4d40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x5586c9759500 tx=0x7f6818008470 comp rx=0 tx=0).stop 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 msgr2=0x7f6820199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 0x7f6820199eb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d950 tx=0x7f680c00de10 comp rx=0 tx=0).stop 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 shutdown_connections 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68080a2880 0x7f68080a4d40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6820101150 0x7f68201a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68201007b0 0x7f6820199eb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 --2- 172.21.15.137:0/2310268688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68200ffe50 0x7f6820199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 >> 172.21.15.137:0/2310268688 conn(0x7f68200fba10 msgr2=0x7f68200fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 shutdown_connections 2024-03-26T14:06:25.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.372+0000 7f6826200640 1 -- 172.21.15.137:0/2310268688 wait complete. 2024-03-26T14:06:25.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:06:25.391 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:06:25.465 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:25 smithi137 ceph-mon[31539]: pgmap v580: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:25.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 -- 172.21.15.137:0/686319640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 msgr2=0x7f5afc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 --2- 172.21.15.137:0/686319640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc1043f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009900 tx=0x7f5af002bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 -- 172.21.15.137:0/686319640 shutdown_connections 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 --2- 172.21.15.137:0/686319640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5afc104930 0x7f5afc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 --2- 172.21.15.137:0/686319640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5afc107260 0x7f5afc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 --2- 172.21.15.137:0/686319640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc1043f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.606+0000 7f5b03200640 1 -- 172.21.15.137:0/686319640 >> 172.21.15.137:0/686319640 conn(0x7f5afc0fba10 msgr2=0x7f5afc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.607+0000 7f5b03200640 1 -- 172.21.15.137:0/686319640 shutdown_connections 2024-03-26T14:06:25.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.607+0000 7f5b03200640 1 -- 172.21.15.137:0/686319640 wait complete. 2024-03-26T14:06:25.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 Processor -- start 2024-03-26T14:06:25.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 -- start start 2024-03-26T14:06:25.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5afc104930 0x7f5afc199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5afc107260 0x7f5afc1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc10d0c0 con 0x7f5afc102000 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc10cf40 con 0x7f5afc107260 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b03200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc10d240 con 0x7f5afc104930 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.608+0000 7f5b01e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46994/0 (socket says 172.21.15.137:46994) 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 -- 172.21.15.137:0/428999748 learned_addr learned my addr 172.21.15.137:0/428999748 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b02800640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5afc107260 0x7f5afc1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01400640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5afc104930 0x7f5afc199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 -- 172.21.15.137:0/428999748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5afc104930 msgr2=0x7f5afc199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5afc104930 0x7f5afc199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 -- 172.21.15.137:0/428999748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5afc107260 msgr2=0x7f5afc1a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5afc107260 0x7f5afc1a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 -- 172.21.15.137:0/428999748 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc1a1650 con 0x7f5afc102000 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.609+0000 7f5b01e00640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc199a30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5af0006010 tx=0x7f5af0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.610+0000 7f5af6a00640 1 -- 172.21.15.137:0/428999748 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0043020 con 0x7f5afc102000 2024-03-26T14:06:25.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.610+0000 7f5af6a00640 1 -- 172.21.15.137:0/428999748 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5af0004440 con 0x7f5afc102000 2024-03-26T14:06:25.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.610+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5afc1a18e0 con 0x7f5afc102000 2024-03-26T14:06:25.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.610+0000 7f5af6a00640 1 -- 172.21.15.137:0/428999748 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0003bf0 con 0x7f5afc102000 2024-03-26T14:06:25.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.610+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc1a1e80 con 0x7f5afc102000 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 Processor -- start 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5af6a00640 1 -- 172.21.15.137:0/428999748 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5af0046020 con 0x7f5afc102000 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 -- start start 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2050 0x55bbf32b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2a40 0x55bbf32b7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf3217250 con 0x55bbf32b7bb0 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf31bfa30 con 0x55bbf32b2a40 2024-03-26T14:06:25.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.614+0000 7f5b07465740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf32132f0 con 0x55bbf32b2050 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ba040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47002/0 (socket says 172.21.15.137:47002) 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 -- 172.21.15.137:0/3249209160 learned_addr learned my addr 172.21.15.137:0/3249209160 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01e00640 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2a40 0x55bbf32b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b02800640 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2050 0x55bbf32b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 -- 172.21.15.137:0/3249209160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2050 msgr2=0x55bbf32b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2050 0x55bbf32b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 -- 172.21.15.137:0/3249209160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2a40 msgr2=0x55bbf32b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2a40 0x55bbf32b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01400640 1 -- 172.21.15.137:0/3249209160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf31aa690 con 0x55bbf32b7bb0 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.615+0000 7f5b01e00640 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2a40 0x55bbf32b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:06:25.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.616+0000 7f5b01400640 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ba040 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009bf0 tx=0x7f5aec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961f074545b28ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.616+0000 7f5aeb000640 1 -- 172.21.15.137:0/3249209160 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec004030 con 0x55bbf32b7bb0 2024-03-26T14:06:25.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.616+0000 7f5af6a00640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ae40a2880 0x7f5ae40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.616+0000 7f5aeb000640 1 -- 172.21.15.137:0/3249209160 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5aec0041d0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.616+0000 7f5aeb000640 1 -- 172.21.15.137:0/3249209160 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0126b0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.616+0000 7f5af6a00640 1 -- 172.21.15.137:0/428999748 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5af00f12c0 con 0x7f5afc102000 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b02800640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ae40a2880 0x7f5ae40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 -- 172.21.15.137:0/3249209160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 msgr2=0x55bbf32ba040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ba040 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009bf0 tx=0x7f5aec0117b0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 -- 172.21.15.137:0/3249209160 shutdown_connections 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2a40 0x55bbf32b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ba040 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 --2- 172.21.15.137:0/3249209160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2050 0x55bbf32b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 -- 172.21.15.137:0/3249209160 >> 172.21.15.137:0/3249209160 conn(0x55bbf327a200 msgr2=0x55bbf32ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 -- 172.21.15.137:0/3249209160 shutdown_connections 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.617+0000 7f5b07465740 1 -- 172.21.15.137:0/3249209160 wait complete. 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.618+0000 7f5b02800640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ae40a2880 0x7f5ae40a4d40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55bbf32b32a0 tx=0x7f5af80085f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.618+0000 7f5b07465740 1 -- start start 2024-03-26T14:06:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.618+0000 7f5b07465740 1 Processor -- start 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 0x55bbf32bc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32bc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf32bab60 con 0x55bbf32b2050 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf31d2e80 con 0x55bbf32b7bb0 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bbf32bace0 con 0x55bbf32b2a40 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b07465740 10 client.0 _ll_register_callbacks cb 0x55bbf31e5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 0x55bbf32bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 0x55bbf32bc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47004/0 (socket says 172.21.15.137:47004) 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 -- 172.21.15.137:0/4148113808 learned_addr learned my addr 172.21.15.137:0/4148113808 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01400640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32bc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b02800640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 -- 172.21.15.137:0/4148113808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 msgr2=0x55bbf32bc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32bc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 -- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b7bb0 msgr2=0x55bbf32c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b01e00640 1 -- 172.21.15.137:0/4148113808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbf2ff57d0 con 0x55bbf32b2050 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.619+0000 7f5b02800640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5b01e00640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 0x55bbf32bc080 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005ab0 tx=0x7f5af00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265d59665bbbb422 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5adf400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5adf400640 1 -- 172.21.15.137:0/4148113808 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00bd620 con 0x55bbf32b2050 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5adf400640 1 -- 172.21.15.137:0/4148113808 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5af00a80e0 con 0x55bbf32b2050 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aec0041d0 con 0x55bbf32b2050 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5adf400640 1 -- 172.21.15.137:0/4148113808 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af00bd1b0 con 0x55bbf32b2050 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5b07465740 10 client.16494 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bbf2f2b3f0 con 0x55bbf32b2050 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.620+0000 7f5adf400640 1 -- 172.21.15.137:0/4148113808 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5aec0041d0 con 0x55bbf32b2050 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5adf400640 1 -- 172.21.15.137:0/4148113808 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5af00bdab0 con 0x55bbf32b2050 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b07465740 10 client.16494 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b07465740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b07465740 20 client.16494 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b07465740 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf32c7ec0 0x55bbf32ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b07465740 4 client.16494 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bbf3224790 con 0x55bbf32c7ec0 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.621+0000 7f5b01400640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf32c7ec0 0x55bbf32ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.622+0000 7f5b01400640 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf32c7ec0 0x55bbf32ca310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf183c62a400fca server_cookie=ce2bc5adfed29244 in_seq=0 out_seq=0 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.622+0000 7f5adf400640 10 client.16494 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5adf400640 1 -- 172.21.15.137:0/4148113808 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bbf3224790 con 0x55bbf32c7ec0 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5adf400640 10 client.16494 handle_command_reply: tid=0 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 client.16494 shutdown 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 msgr2=0x55bbf32bc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 0x55bbf32bc080 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5af0005ab0 tx=0x7f5af00035f0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 shutdown_connections 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf32c7ec0 0x55bbf32ca310 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32bc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 --2- 172.21.15.137:0/4148113808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2050 0x55bbf32bc080 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 >> 172.21.15.137:0/4148113808 conn(0x55bbf327a200 msgr2=0x55bbf32acbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.623+0000 7f5adf400640 0 client.16494 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.624+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 shutdown_connections 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.624+0000 7f5b07465740 1 -- 172.21.15.137:0/4148113808 wait complete. 2024-03-26T14:06:25.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.624+0000 7f5b07465740 20 client.16494 trim_cache size 0 max 16384 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.646+0000 7f5b07465740 1 Processor -- start 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.646+0000 7f5b07465740 1 -- start start 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b07465740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b07465740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 0x55bbf32c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b07465740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b07465740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1ae760 con 0x55bbf32b2a40 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b07465740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1ac260 con 0x55bbf32b2050 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b07465740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1aa2e0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b01400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 0x55bbf32c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b01400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 0x55bbf32c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47008/0 (socket says 172.21.15.137:47008) 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b01400640 1 -- 172.21.15.137:0/2900528050 learned_addr learned my addr 172.21.15.137:0/2900528050 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b01e00640 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b02800640 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b01400640 1 -- 172.21.15.137:0/2900528050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b7bb0 msgr2=0x55bbf32ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.647+0000 7f5b01400640 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5b01400640 1 -- 172.21.15.137:0/2900528050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 msgr2=0x55bbf32bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5b01400640 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5b01400640 1 -- 172.21.15.137:0/2900528050 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bbf2eeb0f0 con 0x55bbf32b2a40 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5b02800640 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5b01400640 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 0x55bbf32c3590 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009980 tx=0x7f5aec003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bc72f04b4aa505 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5aea600640 1 -- 172.21.15.137:0/2900528050 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01f040 con 0x55bbf32b2a40 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5aea600640 1 -- 172.21.15.137:0/2900528050 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5aec0095f0 con 0x55bbf32b2a40 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.648+0000 7f5aea600640 1 -- 172.21.15.137:0/2900528050 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec003ec0 con 0x55bbf32b2a40 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 -- 172.21.15.137:0/2900528050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 msgr2=0x55bbf32c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 0x55bbf32c3590 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009980 tx=0x7f5aec003bf0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 -- 172.21.15.137:0/2900528050 shutdown_connections 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b7bb0 0x55bbf32ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b2a40 0x55bbf32c3590 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 --2- 172.21.15.137:0/2900528050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 -- 172.21.15.137:0/2900528050 >> 172.21.15.137:0/2900528050 conn(0x55bbf327a200 msgr2=0x55bbf32bbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 -- 172.21.15.137:0/2900528050 shutdown_connections 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.649+0000 7f5b07465740 1 -- 172.21.15.137:0/2900528050 wait complete. 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.650+0000 7f5b07465740 1 -- start start 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.650+0000 7f5b07465740 1 Processor -- start 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.650+0000 7f5b07465740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b07465740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b07465740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b07465740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1aa8e0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b07465740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1abc00 con 0x55bbf32b2050 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b07465740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1aaf40 con 0x55bbf32b2a40 2024-03-26T14:06:25.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b07465740 10 client.0 _ll_register_callbacks cb 0x7f5af00057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47012/0 (socket says 172.21.15.137:47012) 2024-03-26T14:06:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 -- 172.21.15.137:0/804375713 learned_addr learned my addr 172.21.15.137:0/804375713 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b01e00640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b01400640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 -- 172.21.15.137:0/804375713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 msgr2=0x55bbf32c16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32c16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 -- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 msgr2=0x55bbf32c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.651+0000 7f5b02800640 1 -- 172.21.15.137:0/804375713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bbf32c2ca0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5b02800640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c26b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5af80088a0 tx=0x7f5af8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b3908d12e1a159 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5adf400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5adf400640 1 -- 172.21.15.137:0/804375713 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af802a060 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5aec0095f0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5adf400640 1 -- 172.21.15.137:0/804375713 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5af80253f0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5adf400640 1 -- 172.21.15.137:0/804375713 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5af8031050 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5b07465740 10 client.16500 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.652+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bbf3323cd0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5adf400640 1 -- 172.21.15.137:0/804375713 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5aec0095f0 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5adf400640 1 -- 172.21.15.137:0/804375713 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5af8024070 con 0x55bbf32b7bb0 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5b07465740 10 client.16500 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5b07465740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5b07465740 20 client.16500 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5b07465740 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf3324690 0x55bbf3326ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5b07465740 4 client.16500 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:06:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.653+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bbf321eae0 con 0x55bbf3324690 2024-03-26T14:06:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.654+0000 7f5b01e00640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf3324690 0x55bbf3326ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.654+0000 7f5b01e00640 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf3324690 0x55bbf3326ab0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45f1b141e5930fc6 server_cookie=7aac85fbbe7074ec in_seq=0 out_seq=0 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.654+0000 7f5adf400640 10 client.16500 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5adf400640 1 -- 172.21.15.137:0/804375713 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bbf321eae0 con 0x55bbf3324690 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5adf400640 10 client.16500 handle_command_reply: tid=0 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 client.16500 shutdown 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 msgr2=0x55bbf32c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c26b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5af80088a0 tx=0x7f5af8004a40 comp rx=0 tx=0).stop 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 shutdown_connections 2024-03-26T14:06:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55bbf3324690 0x55bbf3326ab0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bbf32b7bb0 0x55bbf32c26b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bbf32b2a40 0x55bbf32c16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 --2- 172.21.15.137:0/804375713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bbf32b2050 0x55bbf32c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 >> 172.21.15.137:0/804375713 conn(0x55bbf327a200 msgr2=0x55bbf32ac330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5adf400640 0 client.16500 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 shutdown_connections 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 1 -- 172.21.15.137:0/804375713 wait complete. 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.655+0000 7f5b07465740 20 client.16500 trim_cache size 0 max 16384 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:06:25.656 INFO:teuthology.orchestra.run.smithi137.stdout: "fe07bfe6-b00f-4ede-ae33-a8cd627fdf06": { 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "fe07bfe6-b00f-4ede-ae33-a8cd627fdf06", 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:06:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ae40a2880 msgr2=0x7f5ae40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ae40a2880 0x7f5ae40a4d40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55bbf32b32a0 tx=0x7f5af80085f0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 msgr2=0x7f5afc199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc199a30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5af0006010 tx=0x7f5af0004040 comp rx=0 tx=0).stop 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 shutdown_connections 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5ae40a2880 0x7f5ae40a4d40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5afc107260 0x7f5afc1a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5afc104930 0x7f5afc199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 --2- 172.21.15.137:0/428999748 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5afc102000 0x7f5afc199a30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.656+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 >> 172.21.15.137:0/428999748 conn(0x7f5afc0fba10 msgr2=0x7f5afc1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.657+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 shutdown_connections 2024-03-26T14:06:25.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:25.657+0000 7f5b03200640 1 -- 172.21.15.137:0/428999748 wait complete. 2024-03-26T14:06:25.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:25 smithi179 ceph-mon[35472]: pgmap v580: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:25.674 INFO:tasks.cephfs.filesystem:scrub status for tag:fe07bfe6-b00f-4ede-ae33-a8cd627fdf06 - {'path': '/', 'tag': 'fe07bfe6-b00f-4ede-ae33-a8cd627fdf06', 'options': 'recursive,force'} 2024-03-26T14:06:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:25 smithi159 ceph-mon[35501]: pgmap v580: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:26 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:06:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:26 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:26 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:06:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:26 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:26.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:26 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:06:26.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:26 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:26.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:26 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:06:26.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:26 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:26.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:26 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:06:26.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:26 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:26.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:26 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:06:26.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:26 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:27.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:27 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:06:27.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:27 smithi179 ceph-mon[35472]: pgmap v581: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:06:27.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:27 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:06:27.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:27 smithi159 ceph-mon[35501]: pgmap v581: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:06:27.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:27 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:06:27.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:27 smithi137 ceph-mon[31539]: pgmap v581: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:06:29.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:29 smithi179 ceph-mon[35472]: pgmap v582: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:29.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:29 smithi159 ceph-mon[35501]: pgmap v582: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:29.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:29 smithi137 ceph-mon[31539]: pgmap v582: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:31.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:31 smithi179 ceph-mon[35472]: pgmap v583: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:06:31.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:31 smithi159 ceph-mon[35501]: pgmap v583: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:06:31.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:31 smithi137 ceph-mon[31539]: pgmap v583: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:06:33.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:33 smithi179 ceph-mon[35472]: pgmap v584: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:06:33.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:33 smithi159 ceph-mon[35501]: pgmap v584: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:06:33.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:33 smithi137 ceph-mon[31539]: pgmap v584: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:06:35.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:35 smithi179 ceph-mon[35472]: pgmap v585: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:35.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:35 smithi159 ceph-mon[35501]: pgmap v585: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:35.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:35 smithi137 ceph-mon[31539]: pgmap v585: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:06:37.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:37 smithi179 ceph-mon[35472]: pgmap v586: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:06:37.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:37 smithi159 ceph-mon[35501]: pgmap v586: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:06:37.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:37 smithi137 ceph-mon[31539]: pgmap v586: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-03-26T14:06:39.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:39 smithi179 ceph-mon[35472]: pgmap v587: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:39.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:39 smithi159 ceph-mon[35501]: pgmap v587: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:39.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:39 smithi137 ceph-mon[31539]: pgmap v587: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:41.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:41 smithi179 ceph-mon[35472]: pgmap v588: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:41.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:41 smithi159 ceph-mon[35501]: pgmap v588: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:41.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:41 smithi137 ceph-mon[31539]: pgmap v588: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:43.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:43 smithi179 ceph-mon[35472]: pgmap v589: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:06:43.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:43 smithi159 ceph-mon[35501]: pgmap v589: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:06:43.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:43 smithi137 ceph-mon[31539]: pgmap v589: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:06:45.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:45 smithi179 ceph-mon[35472]: pgmap v590: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:45.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:45 smithi159 ceph-mon[35501]: pgmap v590: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:45.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:45 smithi137 ceph-mon[31539]: pgmap v590: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:47.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:47 smithi179 ceph-mon[35472]: pgmap v591: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:47.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:47 smithi159 ceph-mon[35501]: pgmap v591: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:47.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:47 smithi137 ceph-mon[31539]: pgmap v591: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:49.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:49 smithi179 ceph-mon[35472]: pgmap v592: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:49.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:49 smithi159 ceph-mon[35501]: pgmap v592: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:49.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:49 smithi137 ceph-mon[31539]: pgmap v592: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:51.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:51 smithi179 ceph-mon[35472]: pgmap v593: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:51.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:51 smithi159 ceph-mon[35501]: pgmap v593: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:51.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:51 smithi137 ceph-mon[31539]: pgmap v593: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:06:53.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:53 smithi179 ceph-mon[35472]: pgmap v594: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:06:53.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:53 smithi159 ceph-mon[35501]: pgmap v594: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:06:53.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:53 smithi137 ceph-mon[31539]: pgmap v594: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:06:55.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:55 smithi179 ceph-mon[35472]: pgmap v595: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:55.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:06:55.675 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:06:55.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:55 smithi159 ceph-mon[35501]: pgmap v595: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:55.767 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:55 smithi137 ceph-mon[31539]: pgmap v595: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:06:55.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 -- 172.21.15.137:0/2356085164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28104930 msgr2=0x7f6c28106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/2356085164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28104930 0x7f6c28106d20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009900 tx=0x7f6c1402bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 -- 172.21.15.137:0/2356085164 shutdown_connections 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/2356085164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c28107260 0x7f6c281096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/2356085164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28104930 0x7f6c28106d20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/2356085164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28102000 0x7f6c281043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.919+0000 7f6c2ec00640 1 -- 172.21.15.137:0/2356085164 >> 172.21.15.137:0/2356085164 conn(0x7f6c280fba10 msgr2=0x7f6c280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.920+0000 7f6c2ec00640 1 -- 172.21.15.137:0/2356085164 shutdown_connections 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.920+0000 7f6c2ec00640 1 -- 172.21.15.137:0/2356085164 wait complete. 2024-03-26T14:06:55.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.921+0000 7f6c2ec00640 1 Processor -- start 2024-03-26T14:06:55.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.921+0000 7f6c2ec00640 1 -- start start 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ec00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 0x7f6c28199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c28104930 0x7f6c28199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28107260 0x7f6c281a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ec00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2810d110 con 0x7f6c28102000 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2810cf90 con 0x7f6c28104930 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2810d290 con 0x7f6c28107260 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 0x7f6c28199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 0x7f6c28199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52712/0 (socket says 172.21.15.137:52712) 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2d800640 1 -- 172.21.15.137:0/1860887308 learned_addr learned my addr 172.21.15.137:0/1860887308 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:55.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c28104930 0x7f6c28199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.922+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28107260 0x7f6c281a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2d800640 1 -- 172.21.15.137:0/1860887308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28107260 msgr2=0x7f6c281a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2d800640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28107260 0x7f6c281a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2d800640 1 -- 172.21.15.137:0/1860887308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c28104930 msgr2=0x7f6c28199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2d800640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c28104930 0x7f6c28199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2d800640 1 -- 172.21.15.137:0/1860887308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c281a1630 con 0x7f6c28102000 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28107260 0x7f6c281a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2d800640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 0x7f6c28199a10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d770 tx=0x7f6c1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c26000640 1 -- 172.21.15.137:0/1860887308 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c017070 con 0x7f6c28102000 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c26000640 1 -- 172.21.15.137:0/1860887308 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c1c0054d0 con 0x7f6c28102000 2024-03-26T14:06:55.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c281a1920 con 0x7f6c28102000 2024-03-26T14:06:55.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c26000640 1 -- 172.21.15.137:0/1860887308 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c004cf0 con 0x7f6c28102000 2024-03-26T14:06:55.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.923+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c281a1e40 con 0x7f6c28102000 2024-03-26T14:06:55.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 Processor -- start 2024-03-26T14:06:55.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 -- start start 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c26000640 1 -- 172.21.15.137:0/1860887308 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6c1c004e90 con 0x7f6c28102000 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6d3bb0 0x55dffa6d6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa633250 con 0x55dffa6ce050 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa5dba30 con 0x55dffa6cea40 2024-03-26T14:06:55.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.927+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa62f2f0 con 0x55dffa6d3bb0 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52714/0 (socket says 172.21.15.137:52714) 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 -- 172.21.15.137:0/2849365373 learned_addr learned my addr 172.21.15.137:0/2849365373 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2d800640 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6d3bb0 0x55dffa6d6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 -- 172.21.15.137:0/2849365373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6d3bb0 msgr2=0x55dffa6d6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6d3bb0 0x55dffa6d6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 -- 172.21.15.137:0/2849365373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 msgr2=0x55dffa6d3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6d3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2e200640 1 -- 172.21.15.137:0/2849365373 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dffa5c6690 con 0x55dffa6ce050 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.928+0000 7f6c2d800640 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6d3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:55.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.929+0000 7f6c2e200640 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6ce470 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009ea0 tx=0x7f6c20009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb86c62ecc8cc63 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:55.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.929+0000 7f6c1aa00640 1 -- 172.21.15.137:0/2849365373 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004d20 con 0x55dffa6ce050 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.929+0000 7f6c1aa00640 1 -- 172.21.15.137:0/2849365373 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c200054d0 con 0x55dffa6ce050 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.929+0000 7f6c1aa00640 1 -- 172.21.15.137:0/2849365373 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200118a0 con 0x55dffa6ce050 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.929+0000 7f6c26000640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c100a2880 0x7f6c100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.929+0000 7f6c26000640 1 -- 172.21.15.137:0/1860887308 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6c1c0ce900 con 0x7f6c28102000 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 -- 172.21.15.137:0/2849365373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 msgr2=0x55dffa6ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6ce470 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009ea0 tx=0x7f6c20009f90 comp rx=0 tx=0).stop 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c2d800640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c100a2880 0x7f6c100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 -- 172.21.15.137:0/2849365373 shutdown_connections 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6d3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6d3bb0 0x55dffa6d6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/2849365373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6ce470 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 -- 172.21.15.137:0/2849365373 >> 172.21.15.137:0/2849365373 conn(0x55dffa696200 msgr2=0x55dffa6c99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:55.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 -- 172.21.15.137:0/2849365373 shutdown_connections 2024-03-26T14:06:55.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.930+0000 7f6c32f6f740 1 -- 172.21.15.137:0/2849365373 wait complete. 2024-03-26T14:06:55.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.931+0000 7f6c2d800640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c100a2880 0x7f6c100a4d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c09ab90 tx=0x7f6c1c097240 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:55.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.931+0000 7f6c32f6f740 1 -- start start 2024-03-26T14:06:55.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.931+0000 7f6c32f6f740 1 Processor -- start 2024-03-26T14:06:55.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.931+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6ce050 0x55dffa6d8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 0x55dffa6d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6d3bb0 0x55dffa6df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa6d6b60 con 0x55dffa6cea40 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa5eee80 con 0x55dffa6d3bb0 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dffa6d6ce0 con 0x55dffa6ce050 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c32f6f740 10 client.0 _ll_register_callbacks cb 0x55dffa601130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 0x55dffa6d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 0x55dffa6d85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52728/0 (socket says 172.21.15.137:52728) 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 -- 172.21.15.137:0/1395375111 learned_addr learned my addr 172.21.15.137:0/1395375111 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6d3bb0 0x55dffa6df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6ce050 0x55dffa6d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 -- 172.21.15.137:0/1395375111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6ce050 msgr2=0x55dffa6d8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6ce050 0x55dffa6d8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 -- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6d3bb0 msgr2=0x55dffa6df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6d3bb0 0x55dffa6df5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2ce00640 1 -- 172.21.15.137:0/1395375111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dffa4117d0 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.932+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6ce050 0x55dffa6d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 0x55dffa6d85b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6c28069360 tx=0x7f6c14004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6dbb4772902738 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c0f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c0f400640 1 -- 172.21.15.137:0/1395375111 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14005230 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c0f400640 1 -- 172.21.15.137:0/1395375111 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6c140053d0 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c200054d0 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c0f400640 1 -- 172.21.15.137:0/1395375111 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c140055d0 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c32f6f740 10 client.16512 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.933+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dffa3473f0 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c0f400640 1 -- 172.21.15.137:0/1395375111 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c200054d0 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c0f400640 1 -- 172.21.15.137:0/1395375111 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6c14003900 con 0x55dffa6cea40 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c32f6f740 10 client.16512 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c32f6f740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c32f6f740 20 client.16512 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6e3f50 0x55dffa6e6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c32f6f740 4 client.16512 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:55.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.934+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dffa640790 con 0x55dffa6e3f50 2024-03-26T14:06:55.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.935+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6e3f50 0x55dffa6e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.935+0000 7f6c2e200640 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6e3f50 0x55dffa6e6370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d63371da3e35d01 server_cookie=7be37f85ba06360f in_seq=0 out_seq=0 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.935+0000 7f6c0f400640 10 client.16512 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c0f400640 1 -- 172.21.15.137:0/1395375111 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dffa640790 con 0x55dffa6e3f50 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c0f400640 10 client.16512 handle_command_reply: tid=0 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 client.16512 shutdown 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 msgr2=0x55dffa6d85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 0x55dffa6d85b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f6c28069360 tx=0x7f6c14004250 comp rx=0 tx=0).stop 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 shutdown_connections 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6e3f50 0x55dffa6e6370 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6d3bb0 0x55dffa6df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6cea40 0x55dffa6d85b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/1395375111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6ce050 0x55dffa6d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 >> 172.21.15.137:0/1395375111 conn(0x55dffa696200 msgr2=0x55dffa6c8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c0f400640 0 client.16512 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 shutdown_connections 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.937+0000 7f6c32f6f740 1 -- 172.21.15.137:0/1395375111 wait complete. 2024-03-26T14:06:55.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.938+0000 7f6c32f6f740 20 client.16512 trim_cache size 0 max 16384 2024-03-26T14:06:55.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.959+0000 7f6c32f6f740 1 Processor -- start 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.959+0000 7f6c32f6f740 1 -- start start 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa696200 0x55dffa6c4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6df5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6cea40 0x55dffa6cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281ae7f0 con 0x55dffa6ce050 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281ac2f0 con 0x55dffa696200 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281aa370 con 0x55dffa6cea40 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6df5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6df5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52736/0 (socket says 172.21.15.137:52736) 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c2ce00640 1 -- 172.21.15.137:0/4286389014 learned_addr learned my addr 172.21.15.137:0/4286389014 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.960+0000 7f6c2e200640 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa696200 0x55dffa6c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2e200640 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6cea40 0x55dffa6cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.960 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2ce00640 1 -- 172.21.15.137:0/4286389014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6cea40 msgr2=0x55dffa6cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6cea40 0x55dffa6cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2ce00640 1 -- 172.21.15.137:0/4286389014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa696200 msgr2=0x55dffa6c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa696200 0x55dffa6c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2ce00640 1 -- 172.21.15.137:0/4286389014 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dffa3070f0 con 0x55dffa6ce050 2024-03-26T14:06:55.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2e200640 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6cea40 0x55dffa6cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:55.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6df5b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6c1402c170 tx=0x7f6c1402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f879a24e14d83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:55.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c1a000640 1 -- 172.21.15.137:0/4286389014 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14058070 con 0x55dffa6ce050 2024-03-26T14:06:55.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c1a000640 1 -- 172.21.15.137:0/4286389014 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c14009520 con 0x55dffa6ce050 2024-03-26T14:06:55.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.961+0000 7f6c1a000640 1 -- 172.21.15.137:0/4286389014 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14000e60 con 0x55dffa6ce050 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 -- 172.21.15.137:0/4286389014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 msgr2=0x55dffa6df5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6df5b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6c1402c170 tx=0x7f6c1402bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 -- 172.21.15.137:0/4286389014 shutdown_connections 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa6cea40 0x55dffa6cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6df5b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/4286389014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa696200 0x55dffa6c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 -- 172.21.15.137:0/4286389014 >> 172.21.15.137:0/4286389014 conn(0x55dffa6d7c30 msgr2=0x55dffa6cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.962+0000 7f6c32f6f740 1 -- 172.21.15.137:0/4286389014 shutdown_connections 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 -- 172.21.15.137:0/4286389014 wait complete. 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 -- start start 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 Processor -- start 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa696200 0x55dffa6dd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6dd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281aa970 con 0x55dffa6ce050 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281abc90 con 0x55dffa6cea40 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.963+0000 7f6c32f6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c281aafd0 con 0x55dffa696200 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c32f6f740 10 client.0 _ll_register_callbacks cb 0x7f6c14009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6dd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6dd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52750/0 (socket says 172.21.15.137:52750) 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 -- 172.21.15.137:0/3527544597 learned_addr learned my addr 172.21.15.137:0/3527544597 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2e200640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2e200640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa696200 0x55dffa6dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 -- 172.21.15.137:0/3527544597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa696200 msgr2=0x55dffa6dd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa696200 0x55dffa6dd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 -- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 msgr2=0x55dffa6de8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6de8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2ce00640 1 -- 172.21.15.137:0/3527544597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dffa6deec0 con 0x55dffa6ce050 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.964+0000 7f6c2e200640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa696200 0x55dffa6dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:55.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c2ce00640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6dd9a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c14004650 tx=0x7f6c14034c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9acfca5d170bce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:55.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c0f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c0f400640 1 -- 172.21.15.137:0/3527544597 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14060060 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c0f400640 1 -- 172.21.15.137:0/3527544597 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6c14003a00 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c14009520 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c0f400640 1 -- 172.21.15.137:0/3527544597 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c14067050 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c32f6f740 10 client.16518 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.965+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dffa73fe60 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c0f400640 1 -- 172.21.15.137:0/3527544597 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6c14009520 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c0f400640 1 -- 172.21.15.137:0/3527544597 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6c14048400 con 0x55dffa6ce050 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c32f6f740 10 client.16518 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c32f6f740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c32f6f740 20 client.16518 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6d3bb0 0x55dffa7426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c32f6f740 4 client.16518 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:06:55.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dffa63aae0 con 0x55dffa6d3bb0 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.966+0000 7f6c2e200640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6d3bb0 0x55dffa7426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.967+0000 7f6c2e200640 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6d3bb0 0x55dffa7426e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f420d95d60bd7a14 server_cookie=95b6b6e2c6696bbb in_seq=0 out_seq=0 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.967+0000 7f6c0f400640 10 client.16518 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c0f400640 1 -- 172.21.15.137:0/3527544597 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55dffa63aae0 con 0x55dffa6d3bb0 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c0f400640 10 client.16518 handle_command_reply: tid=0 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 client.16518 shutdown 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 msgr2=0x55dffa6dd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6dd9a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c14004650 tx=0x7f6c14034c80 comp rx=0 tx=0).stop 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 shutdown_connections 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dffa6d3bb0 0x55dffa7426e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dffa6cea40 0x55dffa6de8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dffa6ce050 0x55dffa6dd9a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 --2- 172.21.15.137:0/3527544597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dffa696200 0x55dffa6dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c0f400640 0 client.16518 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:55.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 >> 172.21.15.137:0/3527544597 conn(0x55dffa6d7c30 msgr2=0x55dffa6cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 shutdown_connections 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 1 -- 172.21.15.137:0/3527544597 wait complete. 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.968+0000 7f6c32f6f740 20 client.16518 trim_cache size 0 max 16384 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:06:55.974 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c100a2880 msgr2=0x7f6c100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c100a2880 0x7f6c100a4d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c09ab90 tx=0x7f6c1c097240 comp rx=0 tx=0).stop 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 msgr2=0x7f6c28199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 0x7f6c28199a10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d770 tx=0x7f6c1c00dc30 comp rx=0 tx=0).stop 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 shutdown_connections 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6c100a2880 0x7f6c100a4d40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c28104930 0x7f6c28199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c28107260 0x7f6c281a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 --2- 172.21.15.137:0/1860887308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6c28102000 0x7f6c28199a10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:55.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 >> 172.21.15.137:0/1860887308 conn(0x7f6c280fba10 msgr2=0x7f6c28101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:55.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.969+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 shutdown_connections 2024-03-26T14:06:55.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:55.970+0000 7f6c2ec00640 1 -- 172.21.15.137:0/1860887308 wait complete. 2024-03-26T14:06:55.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:06:55.989 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:06:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 -- 172.21.15.137:0/2210284771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 msgr2=0x7efdac1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2210284771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac1001e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 -- 172.21.15.137:0/2210284771 shutdown_connections 2024-03-26T14:06:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2210284771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdac1010c0 0x7efdac101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2210284771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac100720 0x7efdac100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.199 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2210284771 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac1001e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 -- 172.21.15.137:0/2210284771 >> 172.21.15.137:0/2210284771 conn(0x7efdac0fb980 msgr2=0x7efdac0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 -- 172.21.15.137:0/2210284771 shutdown_connections 2024-03-26T14:06:56.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.200+0000 7efdb0e00640 1 -- 172.21.15.137:0/2210284771 wait complete. 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.201+0000 7efdb0e00640 1 Processor -- start 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.201+0000 7efdb0e00640 1 -- start start 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdb0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdb0e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdac100720 0x7efdac199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdb0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1010c0 0x7efdac1a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdb0e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac10f250 con 0x7efdac0ffdc0 2024-03-26T14:06:56.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdb0e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac10f0d0 con 0x7efdac100720 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdb0e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac10f3d0 con 0x7efdac1010c0 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52768/0 (socket says 172.21.15.137:52768) 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdab400640 1 -- 172.21.15.137:0/2148768503 learned_addr learned my addr 172.21.15.137:0/2148768503 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdaaa00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdac100720 0x7efdac199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdabe00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1010c0 0x7efdac1a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdab400640 1 -- 172.21.15.137:0/2148768503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1010c0 msgr2=0x7efdac1a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.202+0000 7efdab400640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1010c0 0x7efdac1a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efdab400640 1 -- 172.21.15.137:0/2148768503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdac100720 msgr2=0x7efdac199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efdab400640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdac100720 0x7efdac199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efdab400640 1 -- 172.21.15.137:0/2148768503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac1a1560 con 0x7efdac0ffdc0 2024-03-26T14:06:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efdab400640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac199940 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c043020 con 0x7efdac0ffdc0 2024-03-26T14:06:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd9c004aa0 con 0x7efdac0ffdc0 2024-03-26T14:06:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac1a17f0 con 0x7efdac0ffdc0 2024-03-26T14:06:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003d00 con 0x7efdac0ffdc0 2024-03-26T14:06:56.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.203+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac1a1d40 con 0x7efdac0ffdc0 2024-03-26T14:06:56.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.205+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac10ab10 con 0x7efdac0ffdc0 2024-03-26T14:06:56.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.207+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7efd9c046020 con 0x7efdac0ffdc0 2024-03-26T14:06:56.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.208+0000 7efda3e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd900a2800 0x7efd900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.209+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd9c0f1290 con 0x7efdac0ffdc0 2024-03-26T14:06:56.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.209+0000 7efdaaa00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd900a2800 0x7efd900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.210+0000 7efdaaa00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd900a2800 0x7efd900a4cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efd98005fd0 tx=0x7efd98005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.213+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efd9c04d020 con 0x7efdac0ffdc0 2024-03-26T14:06:56.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.457+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdac02d070 con 0x7efdac0ffdc0 2024-03-26T14:06:56.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.459+0000 7efda3e00640 1 -- 172.21.15.137:0/2148768503 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7efd9c0bdf20 con 0x7efdac0ffdc0 2024-03-26T14:06:56.458 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:06:56.458 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:06:56.458 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd900a2800 msgr2=0x7efd900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd900a2800 0x7efd900a4cc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efd98005fd0 tx=0x7efd98005850 comp rx=0 tx=0).stop 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 msgr2=0x7efdac199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac199940 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c0042f0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 shutdown_connections 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd900a2800 0x7efd900a4cc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdac100720 0x7efdac199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efdac1010c0 0x7efdac1a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 --2- 172.21.15.137:0/2148768503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efdac0ffdc0 0x7efdac199940 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 >> 172.21.15.137:0/2148768503 conn(0x7efdac0fb980 msgr2=0x7efdac0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 shutdown_connections 2024-03-26T14:06:56.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.462+0000 7efdb0e00640 1 -- 172.21.15.137:0/2148768503 wait complete. 2024-03-26T14:06:56.479 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:06:56.479 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.706+0000 7f54aa000640 1 -- 172.21.15.137:0/2080921636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 msgr2=0x7f54a4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.706+0000 7f54aa000640 1 --2- 172.21.15.137:0/2080921636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4106dc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5490009900 tx=0x7f549002bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 -- 172.21.15.137:0/2080921636 shutdown_connections 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 --2- 172.21.15.137:0/2080921636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a4107300 0x7f54a41096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 --2- 172.21.15.137:0/2080921636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4106dc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 --2- 172.21.15.137:0/2080921636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a41020a0 0x7f54a4104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 -- 172.21.15.137:0/2080921636 >> 172.21.15.137:0/2080921636 conn(0x7f54a40fba10 msgr2=0x7f54a40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 -- 172.21.15.137:0/2080921636 shutdown_connections 2024-03-26T14:06:56.706 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.707+0000 7f54aa000640 1 -- 172.21.15.137:0/2080921636 wait complete. 2024-03-26T14:06:56.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.708+0000 7f54aa000640 1 Processor -- start 2024-03-26T14:06:56.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.708+0000 7f54aa000640 1 -- start start 2024-03-26T14:06:56.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.708+0000 7f54aa000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a41020a0 0x7f54a41019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.708+0000 7f54aa000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54aa000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a4107300 0x7f54a4100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54aa000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410d0e0 con 0x7f54a41049d0 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54aa000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410cf60 con 0x7f54a4107300 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54aa000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a410d260 con 0x7f54a41020a0 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4100050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52786/0 (socket says 172.21.15.137:52786) 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 -- 172.21.15.137:0/2773309818 learned_addr learned my addr 172.21.15.137:0/2773309818 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a9600640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a4107300 0x7f54a4100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a8c00640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a41020a0 0x7f54a41019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 -- 172.21.15.137:0/2773309818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a41020a0 msgr2=0x7f54a41019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a41020a0 0x7f54a41019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 -- 172.21.15.137:0/2773309818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a4107300 msgr2=0x7f54a4100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a4107300 0x7f54a4100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.709+0000 7f54a3e00640 1 -- 172.21.15.137:0/2773309818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a4100e50 con 0x7f54a41049d0 2024-03-26T14:06:56.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.710+0000 7f54a3e00640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4100050 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f5490004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.710+0000 7f54a1600640 1 -- 172.21.15.137:0/2773309818 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490043020 con 0x7f54a41049d0 2024-03-26T14:06:56.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.710+0000 7f54a1600640 1 -- 172.21.15.137:0/2773309818 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5490004aa0 con 0x7f54a41049d0 2024-03-26T14:06:56.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.710+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a41a1900 con 0x7f54a41049d0 2024-03-26T14:06:56.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.710+0000 7f54a1600640 1 -- 172.21.15.137:0/2773309818 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5490003d00 con 0x7f54a41049d0 2024-03-26T14:06:56.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.710+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a41a1e70 con 0x7f54a41049d0 2024-03-26T14:06:56.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.714+0000 7f54a1600640 1 -- 172.21.15.137:0/2773309818 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5490046020 con 0x7f54a41049d0 2024-03-26T14:06:56.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.714+0000 7f54ae23f740 1 Processor -- start 2024-03-26T14:06:56.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 -- start start 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490422e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490423bc0 0x565490424040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x56549042ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654903872c0 con 0x5654904229f0 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549032fc90 con 0x565490423bc0 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54ae23f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x565490383360 con 0x565490424580 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54a9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490422e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54a9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490422e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52790/0 (socket says 172.21.15.137:52790) 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.715+0000 7f54a9600640 1 -- 172.21.15.137:0/3970271248 learned_addr learned my addr 172.21.15.137:0/3970271248 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a8c00640 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490423bc0 0x565490424040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a3e00640 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x56549042ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a9600640 1 -- 172.21.15.137:0/3970271248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 msgr2=0x56549042ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a9600640 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x56549042ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a9600640 1 -- 172.21.15.137:0/3970271248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490423bc0 msgr2=0x565490424040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a9600640 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490423bc0 0x565490424040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a9600640 1 -- 172.21.15.137:0/3970271248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565490354080 con 0x5654904229f0 2024-03-26T14:06:56.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a3e00640 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x56549042ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:56.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.716+0000 7f54a9600640 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490422e10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f549c009ee0 tx=0x7f549c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efcbe692a812390 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.717+0000 7f5495c00640 1 -- 172.21.15.137:0/3970271248 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c016070 con 0x5654904229f0 2024-03-26T14:06:56.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.717+0000 7f5495c00640 1 -- 172.21.15.137:0/3970271248 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f549c004d20 con 0x5654904229f0 2024-03-26T14:06:56.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.717+0000 7f5495c00640 1 -- 172.21.15.137:0/3970271248 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0127c0 con 0x5654904229f0 2024-03-26T14:06:56.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.717+0000 7f54a1600640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f548c0a2880 0x7f548c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.717+0000 7f54a1600640 1 -- 172.21.15.137:0/2773309818 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f54900f1430 con 0x7f54a41049d0 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.717+0000 7f54a8c00640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f548c0a2880 0x7f548c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 -- 172.21.15.137:0/3970271248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 msgr2=0x565490422e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490422e10 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f549c009ee0 tx=0x7f549c010740 comp rx=0 tx=0).stop 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 -- 172.21.15.137:0/3970271248 shutdown_connections 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490423bc0 0x565490424040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x56549042ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3970271248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490422e10 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 -- 172.21.15.137:0/3970271248 >> 172.21.15.137:0/3970271248 conn(0x56549041dde0 msgr2=0x56549041e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 -- 172.21.15.137:0/3970271248 shutdown_connections 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 -- 172.21.15.137:0/3970271248 wait complete. 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54a8c00640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f548c0a2880 0x7f548c0a4d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5498005fd0 tx=0x7f5498005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.718+0000 7f54ae23f740 1 -- start start 2024-03-26T14:06:56.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 Processor -- start 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654904229f0 0x56549042ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x56549042d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x565490434250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549042b590 con 0x565490423bc0 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654903430d0 con 0x5654904229f0 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549042b710 con 0x565490424580 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54ae23f740 10 client.0 _ll_register_callbacks cb 0x565490347e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x56549042d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.719+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x56549042d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52802/0 (socket says 172.21.15.137:52802) 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 -- 172.21.15.137:0/504783488 learned_addr learned my addr 172.21.15.137:0/504783488 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a8c00640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654904229f0 0x56549042ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a9600640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x565490434250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 -- 172.21.15.137:0/504783488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 msgr2=0x565490434250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x565490434250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 -- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654904229f0 msgr2=0x56549042ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654904229f0 0x56549042ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 -- 172.21.15.137:0/504783488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56549009b3f0 con 0x565490423bc0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54a3e00640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x56549042d210 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f54900b9e60 tx=0x7f54900b9d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401eb2fa3e76f2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f548a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f548a000640 1 -- 172.21.15.137:0/504783488 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900be100 con 0x565490423bc0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f548a000640 1 -- 172.21.15.137:0/504783488 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f54900c5d10 con 0x565490423bc0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.720+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f549c004d20 con 0x565490423bc0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f548a000640 1 -- 172.21.15.137:0/504783488 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54900bda00 con 0x565490423bc0 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 10 client.16536 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:56.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5654901657d0 con 0x565490423bc0 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f548a000640 1 -- 172.21.15.137:0/504783488 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f549c004d20 con 0x565490423bc0 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f548a000640 1 -- 172.21.15.137:0/504783488 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f54900b9750 con 0x565490423bc0 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 10 client.16536 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 20 client.16536 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490438680 0x56549043aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 4 client.16536 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.721+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565490394800 con 0x565490438680 2024-03-26T14:06:56.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.722+0000 7f54a8c00640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490438680 0x56549043aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.723+0000 7f54a8c00640 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490438680 0x56549043aab0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e82c7816de51fa4a server_cookie=22bfa0ea9abd2a3a in_seq=0 out_seq=0 2024-03-26T14:06:56.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.723+0000 7f548a000640 10 client.16536 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:56.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.724+0000 7f548a000640 1 -- 172.21.15.137:0/504783488 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x565490394800 con 0x565490438680 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.724+0000 7f548a000640 10 client.16536 handle_command_reply: tid=0 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.724+0000 7f54ae23f740 1 client.16536 shutdown 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 msgr2=0x56549042d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x56549042d210 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f54900b9e60 tx=0x7f54900b9d60 comp rx=0 tx=0).stop 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 shutdown_connections 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490438680 0x56549043aab0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490424580 0x565490434250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x56549042d210 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 --2- 172.21.15.137:0/504783488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654904229f0 0x56549042ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 >> 172.21.15.137:0/504783488 conn(0x56549041dde0 msgr2=0x56549041d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f548a000640 0 client.16536 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 shutdown_connections 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 1 -- 172.21.15.137:0/504783488 wait complete. 2024-03-26T14:06:56.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.725+0000 7f54ae23f740 20 client.16536 trim_cache size 0 max 16384 2024-03-26T14:06:56.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 Processor -- start 2024-03-26T14:06:56.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 -- start start 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654904229f0 0x56549043f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x565490434210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x56549043f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41ae720 con 0x565490423bc0 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41ac220 con 0x565490424580 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54ae23f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41aa2a0 con 0x5654904229f0 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x565490434210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x565490434210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52818/0 (socket says 172.21.15.137:52818) 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.743+0000 7f54a3e00640 1 -- 172.21.15.137:0/3340068849 learned_addr learned my addr 172.21.15.137:0/3340068849 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a9600640 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x56549043f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a8c00640 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654904229f0 0x56549043f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a3e00640 1 -- 172.21.15.137:0/3340068849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654904229f0 msgr2=0x56549043f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a3e00640 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654904229f0 0x56549043f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a3e00640 1 -- 172.21.15.137:0/3340068849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 msgr2=0x56549043f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a3e00640 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x56549043f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a3e00640 1 -- 172.21.15.137:0/3340068849 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56549043b130 con 0x565490423bc0 2024-03-26T14:06:56.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f54a3e00640 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x565490434210 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5490005ab0 tx=0x7f54900056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6679aacfe40904 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f5495200640 1 -- 172.21.15.137:0/3340068849 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900c1b50 con 0x565490423bc0 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f5495200640 1 -- 172.21.15.137:0/3340068849 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54900d6e50 con 0x565490423bc0 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.744+0000 7f5495200640 1 -- 172.21.15.137:0/3340068849 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900c5d10 con 0x565490423bc0 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 -- 172.21.15.137:0/3340068849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 msgr2=0x565490434210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x565490434210 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5490005ab0 tx=0x7f54900056b0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 -- 172.21.15.137:0/3340068849 shutdown_connections 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x56549043f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565490423bc0 0x565490434210 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 --2- 172.21.15.137:0/3340068849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654904229f0 0x56549043f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 -- 172.21.15.137:0/3340068849 >> 172.21.15.137:0/3340068849 conn(0x56549041dde0 msgr2=0x5654904218d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.745+0000 7f54ae23f740 1 -- 172.21.15.137:0/3340068849 shutdown_connections 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.746+0000 7f54ae23f740 1 -- 172.21.15.137:0/3340068849 wait complete. 2024-03-26T14:06:56.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.746+0000 7f54ae23f740 1 -- start start 2024-03-26T14:06:56.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.746+0000 7f54ae23f740 1 Processor -- start 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490434140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490423bc0 0x565490430be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x565490431120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41aa8a0 con 0x5654904229f0 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41abbc0 con 0x565490424580 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a41aaf00 con 0x565490423bc0 2024-03-26T14:06:56.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54ae23f740 10 client.0 _ll_register_callbacks cb 0x7f54900057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490434140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490434140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52830/0 (socket says 172.21.15.137:52830) 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 -- 172.21.15.137:0/1053513051 learned_addr learned my addr 172.21.15.137:0/1053513051 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a3e00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490423bc0 0x565490430be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a9600640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x565490431120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 -- 172.21.15.137:0/1053513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490423bc0 msgr2=0x565490430be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490423bc0 0x565490430be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 -- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 msgr2=0x565490431120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x565490431120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a8c00640 1 -- 172.21.15.137:0/1053513051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565490493f80 con 0x5654904229f0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a3e00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490423bc0 0x565490430be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.747+0000 7f54a9600640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x565490431120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f54a8c00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490434140 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f549800bf90 tx=0x7f54980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5483d70fe52ab7ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f548a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:56.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f548a000640 1 -- 172.21.15.137:0/1053513051 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498004230 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f548a000640 1 -- 172.21.15.137:0/1053513051 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f54980043d0 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54900d6e50 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f548a000640 1 -- 172.21.15.137:0/1053513051 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5498039050 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f54ae23f740 10 client.16542 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.748+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5654904952e0 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f548a000640 1 -- 172.21.15.137:0/1053513051 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54900d6e50 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f548a000640 1 -- 172.21.15.137:0/1053513051 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5498003930 con 0x5654904229f0 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f54ae23f740 10 client.16542 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f54ae23f740 10 client.16542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f54ae23f740 20 client.16542 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f54ae23f740 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490495c70 0x565490498090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f54ae23f740 4 client.16542 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:06:56.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.749+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56549038eb50 con 0x565490495c70 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.750+0000 7f54a3e00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490495c70 0x565490498090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.750+0000 7f548a000640 10 client.16542 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.750+0000 7f54a3e00640 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490495c70 0x565490498090 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c33a5a0f4af91b5 server_cookie=a727b72fa43afae1 in_seq=0 out_seq=0 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f548a000640 1 -- 172.21.15.137:0/1053513051 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56549038eb50 con 0x565490495c70 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f548a000640 10 client.16542 handle_command_reply: tid=0 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 client.16542 shutdown 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 msgr2=0x565490434140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490434140 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f549800bf90 tx=0x7f54980046c0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 shutdown_connections 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x565490495c70 0x565490498090 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565490424580 0x565490431120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565490423bc0 0x565490430be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 --2- 172.21.15.137:0/1053513051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5654904229f0 0x565490434140 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f548a000640 0 client.16542 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:06:56.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 >> 172.21.15.137:0/1053513051 conn(0x56549041dde0 msgr2=0x56549043f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 shutdown_connections 2024-03-26T14:06:56.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 1 -- 172.21.15.137:0/1053513051 wait complete. 2024-03-26T14:06:56.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.751+0000 7f54ae23f740 20 client.16542 trim_cache size 0 max 16384 2024-03-26T14:06:56.753 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f548c0a2880 msgr2=0x7f548c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f548c0a2880 0x7f548c0a4d40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5498005fd0 tx=0x7f5498005c50 comp rx=0 tx=0).stop 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 msgr2=0x7f54a4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4100050 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f5490000c00 tx=0x7f5490004270 comp rx=0 tx=0).stop 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 shutdown_connections 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f548c0a2880 0x7f548c0a4d40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54a4107300 0x7f54a4100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54a41049d0 0x7f54a4100050 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 --2- 172.21.15.137:0/2773309818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a41020a0 0x7f54a41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.752+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 >> 172.21.15.137:0/2773309818 conn(0x7f54a40fba10 msgr2=0x7f54a4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.753+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 shutdown_connections 2024-03-26T14:06:56.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.753+0000 7f54aa000640 1 -- 172.21.15.137:0/2773309818 wait complete. 2024-03-26T14:06:56.769 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2148768503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:06:56.771 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:06:56.772 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:06:56.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2148768503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:06:56.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2148768503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:06:56.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 -- 172.21.15.137:0/417287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260104930 msgr2=0x7f0260106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 --2- 172.21.15.137:0/417287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260104930 0x7f0260106d20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0248009900 tx=0x7f024802bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 -- 172.21.15.137:0/417287656 shutdown_connections 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 --2- 172.21.15.137:0/417287656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260107260 0x7f02601096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 --2- 172.21.15.137:0/417287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260104930 0x7f0260106d20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 --2- 172.21.15.137:0/417287656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 0x7f02601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 -- 172.21.15.137:0/417287656 >> 172.21.15.137:0/417287656 conn(0x7f02600fba10 msgr2=0x7f02600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.987+0000 7f025f200640 1 -- 172.21.15.137:0/417287656 shutdown_connections 2024-03-26T14:06:56.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.988+0000 7f025f200640 1 -- 172.21.15.137:0/417287656 wait complete. 2024-03-26T14:06:56.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.988+0000 7f025f200640 1 Processor -- start 2024-03-26T14:06:56.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 -- start start 2024-03-26T14:06:56.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 0x7f02601977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260104930 0x7f0260197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 0x7f026019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026010d0c0 con 0x7f0260107260 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026010cf40 con 0x7f0260104930 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026010d240 con 0x7f0260102000 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 0x7f026019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f0255400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260104930 0x7f0260197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 0x7f026019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52852/0 (socket says 172.21.15.137:52852) 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.989+0000 7f025e800640 1 -- 172.21.15.137:0/437736566 learned_addr learned my addr 172.21.15.137:0/437736566 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025de00640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 0x7f02601977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025e800640 1 -- 172.21.15.137:0/437736566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 msgr2=0x7f02601977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025e800640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 0x7f02601977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025e800640 1 -- 172.21.15.137:0/437736566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260104930 msgr2=0x7f0260197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025e800640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260104930 0x7f0260197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025e800640 1 -- 172.21.15.137:0/437736566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026019f3f0 con 0x7f0260107260 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025de00640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 0x7f02601977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f0255400640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260104930 0x7f0260197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.990+0000 7f025e800640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 0x7f026019ed80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f025800eb10 tx=0x7f025800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.991+0000 7f0257400640 1 -- 172.21.15.137:0/437736566 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258004b20 con 0x7f0260107260 2024-03-26T14:06:56.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.991+0000 7f0257400640 1 -- 172.21.15.137:0/437736566 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0258004cc0 con 0x7f0260107260 2024-03-26T14:06:56.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.991+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026019f6e0 con 0x7f0260107260 2024-03-26T14:06:56.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.991+0000 7f0257400640 1 -- 172.21.15.137:0/437736566 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0258010690 con 0x7f0260107260 2024-03-26T14:06:56.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.991+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026019fc00 con 0x7f0260107260 2024-03-26T14:06:56.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 Processor -- start 2024-03-26T14:06:56.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f0257400640 1 -- 172.21.15.137:0/437736566 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0258005020 con 0x7f0260107260 2024-03-26T14:06:56.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 -- start start 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb927220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb928450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb92f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5eb8872c0 con 0x55f5eb928990 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5eb82fc90 con 0x55f5eb927fd0 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.995+0000 7f02677ec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5eb883360 con 0x55f5eb926e00 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f0255400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb92f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb928450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb928450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:42154/0 (socket says 172.21.15.137:42154) 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 -- 172.21.15.137:0/2517111859 learned_addr learned my addr 172.21.15.137:0/2517111859 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:56.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f0255400640 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb927220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 -- 172.21.15.137:0/2517111859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 msgr2=0x55f5eb927220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb927220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 -- 172.21.15.137:0/2517111859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 msgr2=0x55f5eb92f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb92f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f025de00640 1 -- 172.21.15.137:0/2517111859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5eb854080 con 0x55f5eb927fd0 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f0257400640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02440a2880 0x7f02440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.996+0000 7f0255400640 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb927220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.997+0000 7f0257400640 1 -- 172.21.15.137:0/437736566 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f02580cf430 con 0x7f0260107260 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.997+0000 7f025e800640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02440a2880 0x7f02440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.997+0000 7f025de00640 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb928450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0250009d40 tx=0x7f02500117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ef591729c7e04a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.997+0000 7f024f000640 1 -- 172.21.15.137:0/2517111859 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250012650 con 0x55f5eb927fd0 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.997+0000 7f024f000640 1 -- 172.21.15.137:0/2517111859 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0250009e30 con 0x55f5eb927fd0 2024-03-26T14:06:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.997+0000 7f024f000640 1 -- 172.21.15.137:0/2517111859 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025000fbd0 con 0x55f5eb927fd0 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 -- 172.21.15.137:0/2517111859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 msgr2=0x55f5eb928450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb928450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0250009d40 tx=0x7f02500117b0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f025e800640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02440a2880 0x7f02440a4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0258098390 tx=0x7f0258009a70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 -- 172.21.15.137:0/2517111859 shutdown_connections 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb928450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb92f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 --2- 172.21.15.137:0/2517111859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb927220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 -- 172.21.15.137:0/2517111859 >> 172.21.15.137:0/2517111859 conn(0x55f5eb91dde0 msgr2=0x55f5eb91e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 -- 172.21.15.137:0/2517111859 shutdown_connections 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.998+0000 7f02677ec740 1 -- 172.21.15.137:0/2517111859 wait complete. 2024-03-26T14:06:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 -- start start 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 Processor -- start 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb931070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb927fd0 0x55f5eb9315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb938580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5eb92f9a0 con 0x55f5eb928990 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5eb8430d0 con 0x55f5eb926e00 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5eb92fb20 con 0x55f5eb927fd0 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:56.999+0000 7f02677ec740 10 client.0 _ll_register_callbacks cb 0x55f5eb847e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb931070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb938580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb938580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52864/0 (socket says 172.21.15.137:52864) 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 -- 172.21.15.137:0/1427176460 learned_addr learned my addr 172.21.15.137:0/1427176460 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f025de00640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb927fd0 0x55f5eb9315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 -- 172.21.15.137:0/1427176460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb927fd0 msgr2=0x55f5eb9315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb927fd0 0x55f5eb9315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 -- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 msgr2=0x55f5eb931070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb931070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f0255400640 1 -- 172.21.15.137:0/1427176460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5eb59b3f0 con 0x55f5eb928990 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.000+0000 7f025de00640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb927fd0 0x55f5eb9315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f0255400640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb938580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0248005c40 tx=0x7f0248005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da603e3e5faf6a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f0243400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f0243400640 1 -- 172.21.15.137:0/1427176460 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248003e00 con 0x55f5eb928990 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0250009e30 con 0x55f5eb928990 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f0243400640 1 -- 172.21.15.137:0/1427176460 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0248005050 con 0x55f5eb928990 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f0243400640 1 -- 172.21.15.137:0/1427176460 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0248056050 con 0x55f5eb928990 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f02677ec740 10 client.16554 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.001+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f5eb6657d0 con 0x55f5eb928990 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f0243400640 1 -- 172.21.15.137:0/1427176460 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0250009e30 con 0x55f5eb928990 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f0243400640 1 -- 172.21.15.137:0/1427176460 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f024804d070 con 0x55f5eb928990 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f02677ec740 10 client.16554 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f02677ec740 10 client.16554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f02677ec740 20 client.16554 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f02677ec740 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb93c9d0 0x55f5eb93ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f02677ec740 4 client.16554 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.002+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5eb894800 con 0x55f5eb93c9d0 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.003+0000 7f025de00640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb93c9d0 0x55f5eb93ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.003+0000 7f0243400640 10 client.16554 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.003+0000 7f025de00640 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb93c9d0 0x55f5eb93ee40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d28767d3a548a613 server_cookie=22ab91d7d13a15a7 in_seq=0 out_seq=0 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f0243400640 1 -- 172.21.15.137:0/1427176460 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f5eb894800 con 0x55f5eb93c9d0 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f0243400640 10 client.16554 handle_command_reply: tid=0 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 client.16554 shutdown 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 msgr2=0x55f5eb938580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb938580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0248005c40 tx=0x7f0248005b50 comp rx=0 tx=0).stop 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 shutdown_connections 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb93c9d0 0x55f5eb93ee40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb938580 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb927fd0 0x55f5eb9315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f0243400640 0 client.16554 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 --2- 172.21.15.137:0/1427176460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb931070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 >> 172.21.15.137:0/1427176460 conn(0x55f5eb91dde0 msgr2=0x55f5eb91d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 shutdown_connections 2024-03-26T14:06:57.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 1 -- 172.21.15.137:0/1427176460 wait complete. 2024-03-26T14:06:57.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.005+0000 7f02677ec740 20 client.16554 trim_cache size 0 max 16384 2024-03-26T14:06:57.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 Processor -- start 2024-03-26T14:06:57.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 -- start start 2024-03-26T14:06:57.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb921750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb938510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb943750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601aa2b0 con 0x55f5eb928990 2024-03-26T14:06:57.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a7db0 con 0x55f5eb927fd0 2024-03-26T14:06:57.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.023+0000 7f02677ec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a5e30 con 0x55f5eb926e00 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb938510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f0255400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb943750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb938510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:42180/0 (socket says 172.21.15.137:42180) 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 -- 172.21.15.137:0/759854923 learned_addr learned my addr 172.21.15.137:0/759854923 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f0255400640 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb921750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 -- 172.21.15.137:0/759854923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 msgr2=0x55f5eb921750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb921750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 -- 172.21.15.137:0/759854923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 msgr2=0x55f5eb943750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.024+0000 7f025de00640 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb943750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.025+0000 7f025de00640 1 -- 172.21.15.137:0/759854923 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5eb927a50 con 0x55f5eb927fd0 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.025+0000 7f0255400640 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb943750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:06:57.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.025+0000 7f025de00640 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb938510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0250005fd0 tx=0x7f0250009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1363630b7669d95d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.025+0000 7f024e600640 1 -- 172.21.15.137:0/759854923 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02500095f0 con 0x55f5eb927fd0 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.025+0000 7f024e600640 1 -- 172.21.15.137:0/759854923 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f025002b070 con 0x55f5eb927fd0 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.025+0000 7f024e600640 1 -- 172.21.15.137:0/759854923 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025002bae0 con 0x55f5eb927fd0 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 -- 172.21.15.137:0/759854923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 msgr2=0x55f5eb938510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb938510 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0250005fd0 tx=0x7f0250009c40 comp rx=0 tx=0).stop 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 -- 172.21.15.137:0/759854923 shutdown_connections 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb927fd0 0x55f5eb938510 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb928990 0x55f5eb943750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 --2- 172.21.15.137:0/759854923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb926e00 0x55f5eb921750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 -- 172.21.15.137:0/759854923 >> 172.21.15.137:0/759854923 conn(0x55f5eb91dde0 msgr2=0x55f5eb921040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 -- 172.21.15.137:0/759854923 shutdown_connections 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.026+0000 7f02677ec740 1 -- 172.21.15.137:0/759854923 wait complete. 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 -- start start 2024-03-26T14:06:57.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 Processor -- start 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb9341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 0x55f5eb934700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb928990 0x55f5eb937720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a6430 con 0x55f5eb927fd0 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a7750 con 0x55f5eb926e00 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02601a6a90 con 0x55f5eb928990 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.027+0000 7f02677ec740 10 client.0 _ll_register_callbacks cb 0x7f0248009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 0x55f5eb934700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 0x55f5eb934700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52896/0 (socket says 172.21.15.137:52896) 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 -- 172.21.15.137:0/785638481 learned_addr learned my addr 172.21.15.137:0/785638481 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f0255400640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb9341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f0255400640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb928990 0x55f5eb937720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 -- 172.21.15.137:0/785638481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb928990 msgr2=0x55f5eb937720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb928990 0x55f5eb937720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 -- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 msgr2=0x55f5eb9341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb9341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 -- 172.21.15.137:0/785638481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5eb937d10 con 0x55f5eb927fd0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f0255400640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb9341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f025de00640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 0x55f5eb934700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f025000f600 tx=0x7f0250009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8460d3afd1f13e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.028+0000 7f0243400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f0243400640 1 -- 172.21.15.137:0/785638481 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250038070 con 0x55f5eb927fd0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f0243400640 1 -- 172.21.15.137:0/785638481 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f025000fa30 con 0x55f5eb927fd0 2024-03-26T14:06:57.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f025002b070 con 0x55f5eb927fd0 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f0243400640 1 -- 172.21.15.137:0/785638481 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0250046050 con 0x55f5eb927fd0 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f02677ec740 10 client.16560 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f5eb998890 con 0x55f5eb927fd0 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.029+0000 7f0243400640 1 -- 172.21.15.137:0/785638481 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f025002b070 con 0x55f5eb927fd0 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f0243400640 1 -- 172.21.15.137:0/785638481 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0248004360 con 0x55f5eb927fd0 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f02677ec740 10 client.16560 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f02677ec740 10 client.16560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f02677ec740 20 client.16560 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f02677ec740 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb999020 0x55f5eb99b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f02677ec740 4 client.16560 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5eb88eb50 con 0x55f5eb999020 2024-03-26T14:06:57.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.030+0000 7f0255400640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb999020 0x55f5eb99b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.031+0000 7f0255400640 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb999020 0x55f5eb99b440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77fec9099087bc93 server_cookie=598531e409097a54 in_seq=0 out_seq=0 2024-03-26T14:06:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.031+0000 7f0243400640 10 client.16560 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.031+0000 7f0243400640 1 -- 172.21.15.137:0/785638481 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5eb88eb50 con 0x55f5eb999020 2024-03-26T14:06:57.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.031+0000 7f0243400640 10 client.16560 handle_command_reply: tid=0 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.031+0000 7f02677ec740 1 client.16560 shutdown 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 msgr2=0x55f5eb934700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 0x55f5eb934700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f025000f600 tx=0x7f0250009310 comp rx=0 tx=0).stop 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 shutdown_connections 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f5eb999020 0x55f5eb99b440 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5eb928990 0x55f5eb937720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5eb927fd0 0x55f5eb934700 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 --2- 172.21.15.137:0/785638481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5eb926e00 0x55f5eb9341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 >> 172.21.15.137:0/785638481 conn(0x55f5eb91dde0 msgr2=0x55f5eb9218d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f0243400640 0 client.16560 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 shutdown_connections 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 1 -- 172.21.15.137:0/785638481 wait complete. 2024-03-26T14:06:57.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.032+0000 7f02677ec740 20 client.16560 trim_cache size 0 max 16384 2024-03-26T14:06:57.034 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:06:57.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02440a2880 msgr2=0x7f02440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02440a2880 0x7f02440a4d40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0258098390 tx=0x7f0258009a70 comp rx=0 tx=0).stop 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 msgr2=0x7f026019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 0x7f026019ed80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f025800eb10 tx=0x7f025800c2d0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 shutdown_connections 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02440a2880 0x7f02440a4d40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0260104930 0x7f0260197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0260107260 0x7f026019ed80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 --2- 172.21.15.137:0/437736566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0260102000 0x7f02601977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 >> 172.21.15.137:0/437736566 conn(0x7f02600fba10 msgr2=0x7f0260102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 shutdown_connections 2024-03-26T14:06:57.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.033+0000 7f025f200640 1 -- 172.21.15.137:0/437736566 wait complete. 2024-03-26T14:06:57.048 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:06:57.048 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:06:57.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.256+0000 7f24d7200640 1 -- 172.21.15.137:0/2426775477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0105490 msgr2=0x7f24d010bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.256+0000 7f24d7200640 1 --2- 172.21.15.137:0/2426775477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0105490 0x7f24d010bd20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009ca0 tx=0x7f24c802c100 comp rx=0 tx=0).stop 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.256+0000 7f24d7200640 1 -- 172.21.15.137:0/2426775477 shutdown_connections 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.256+0000 7f24d7200640 1 --2- 172.21.15.137:0/2426775477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d0104ad0 0x7f24d0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.256+0000 7f24d7200640 1 --2- 172.21.15.137:0/2426775477 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d0105490 0x7f24d010bd20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.256+0000 7f24d7200640 1 --2- 172.21.15.137:0/2426775477 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d01038e0 0x7f24d0103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.257+0000 7f24d7200640 1 -- 172.21.15.137:0/2426775477 >> 172.21.15.137:0/2426775477 conn(0x7f24d00fee50 msgr2=0x7f24d0101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.257+0000 7f24d7200640 1 -- 172.21.15.137:0/2426775477 shutdown_connections 2024-03-26T14:06:57.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.257+0000 7f24d7200640 1 -- 172.21.15.137:0/2426775477 wait complete. 2024-03-26T14:06:57.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 Processor -- start 2024-03-26T14:06:57.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 -- start start 2024-03-26T14:06:57.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 0x7f24d0199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0104ad0 0x7f24d0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d0105490 0x7f24d01a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d010f3f0 con 0x7f24d01038e0 2024-03-26T14:06:57.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d010f270 con 0x7f24d0105490 2024-03-26T14:06:57.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.258+0000 7f24d7200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d010f570 con 0x7f24d0104ad0 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 0x7f24d0199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 0x7f24d0199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52906/0 (socket says 172.21.15.137:52906) 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 -- 172.21.15.137:0/1893934349 learned_addr learned my addr 172.21.15.137:0/1893934349 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24d5200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d0105490 0x7f24d01a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cf400640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0104ad0 0x7f24d0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 -- 172.21.15.137:0/1893934349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0104ad0 msgr2=0x7f24d0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0104ad0 0x7f24d0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 -- 172.21.15.137:0/1893934349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d0105490 msgr2=0x7f24d01a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d0105490 0x7f24d01a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 -- 172.21.15.137:0/1893934349 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d01a16b0 con 0x7f24d01038e0 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cf400640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0104ad0 0x7f24d0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:57.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.259+0000 7f24cfe00640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 0x7f24d0199a90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.260+0000 7f24ccc00640 1 -- 172.21.15.137:0/1893934349 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0017020 con 0x7f24d01038e0 2024-03-26T14:06:57.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.260+0000 7f24ccc00640 1 -- 172.21.15.137:0/1893934349 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24c0009c30 con 0x7f24d01038e0 2024-03-26T14:06:57.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.260+0000 7f24ccc00640 1 -- 172.21.15.137:0/1893934349 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000f6b0 con 0x7f24d01038e0 2024-03-26T14:06:57.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.260+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d01a19a0 con 0x7f24d01038e0 2024-03-26T14:06:57.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.260+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d01a1ea0 con 0x7f24d01038e0 2024-03-26T14:06:57.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.263+0000 7f24ccc00640 1 -- 172.21.15.137:0/1893934349 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f24c001a020 con 0x7f24d01038e0 2024-03-26T14:06:57.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 Processor -- start 2024-03-26T14:06:57.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 -- start start 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24efc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b24f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b24542c0 con 0x5590b24f13a0 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b23fcc90 con 0x5590b24f09e0 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.264+0000 7f24d9e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b2450360 con 0x5590b24ef810 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b24f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b24f7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52916/0 (socket says 172.21.15.137:52916) 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 -- 172.21.15.137:0/242670554 learned_addr learned my addr 172.21.15.137:0/242670554 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cfe00640 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24ccc00640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24b40a2800 0x7f24b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24d5200640 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 -- 172.21.15.137:0/242670554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 msgr2=0x5590b24efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 -- 172.21.15.137:0/242670554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 msgr2=0x5590b24f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24ccc00640 1 -- 172.21.15.137:0/1893934349 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f24c00cebf0 con 0x7f24d01038e0 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.265+0000 7f24cf400640 1 -- 172.21.15.137:0/242670554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590b2421080 con 0x5590b24f13a0 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.266+0000 7f24d5200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24b40a2800 0x7f24b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.266+0000 7f24cf400640 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b24f7c30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009c50 tx=0x7f24bc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f42bd773044f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.266+0000 7f24c5600640 1 -- 172.21.15.137:0/242670554 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc004030 con 0x5590b24f13a0 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.266+0000 7f24c5600640 1 -- 172.21.15.137:0/242670554 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24bc0041d0 con 0x5590b24f13a0 2024-03-26T14:06:57.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.266+0000 7f24c5600640 1 -- 172.21.15.137:0/242670554 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0126b0 con 0x5590b24f13a0 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 -- 172.21.15.137:0/242670554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 msgr2=0x5590b24f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d5200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24b40a2800 0x7f24b40a4cc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b24f7c30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009c50 tx=0x7f24bc010740 comp rx=0 tx=0).stop 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 -- 172.21.15.137:0/242670554 shutdown_connections 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b24f7c30 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 --2- 172.21.15.137:0/242670554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 -- 172.21.15.137:0/242670554 >> 172.21.15.137:0/242670554 conn(0x5590b24eac00 msgr2=0x5590b24eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 -- 172.21.15.137:0/242670554 shutdown_connections 2024-03-26T14:06:57.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.267+0000 7f24d9e02740 1 -- 172.21.15.137:0/242670554 wait complete. 2024-03-26T14:06:57.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.268+0000 7f24d9e02740 1 -- start start 2024-03-26T14:06:57.268 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.268+0000 7f24d9e02740 1 Processor -- start 2024-03-26T14:06:57.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.268+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f13a0 0x5590b2501060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d9e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b24f83b0 con 0x5590b24ef810 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d9e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b24100d0 con 0x5590b24f09e0 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d9e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590b24f8530 con 0x5590b24f13a0 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d9e02740 10 client.0 _ll_register_callbacks cb 0x5590b2414e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52930/0 (socket says 172.21.15.137:52930) 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 -- 172.21.15.137:0/686224163 learned_addr learned my addr 172.21.15.137:0/686224163 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cf400640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24d5200640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f13a0 0x5590b2501060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 -- 172.21.15.137:0/686224163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f13a0 msgr2=0x5590b2501060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f13a0 0x5590b2501060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 -- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 msgr2=0x5590b24fa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cfe00640 1 -- 172.21.15.137:0/686224163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590b21683f0 con 0x5590b24ef810 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.269+0000 7f24cf400640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:57.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24cfe00640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f9ae0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f24c00cee90 tx=0x7f24c009aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc53d405d5d8173 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24ba000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24ba000640 1 -- 172.21.15.137:0/686224163 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c009b180 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24ba000640 1 -- 172.21.15.137:0/686224163 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f24c0099480 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24bc0041d0 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24ba000640 1 -- 172.21.15.137:0/686224163 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c009f2d0 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24d9e02740 10 client.16572 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5590b22327d0 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.270+0000 7f24ba000640 1 -- 172.21.15.137:0/686224163 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24bc0041d0 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24ba000640 1 -- 172.21.15.137:0/686224163 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f24c0024060 con 0x5590b24ef810 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24d9e02740 10 client.16572 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24d9e02740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24d9e02740 20 client.16572 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24d9e02740 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25055b0 0x5590b2507a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24d9e02740 4 client.16572 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590b2461800 con 0x5590b25055b0 2024-03-26T14:06:57.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.271+0000 7f24cf400640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25055b0 0x5590b2507a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.272+0000 7f24cf400640 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25055b0 0x5590b2507a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b81776393a81e4 server_cookie=ef695e461fb6d9f2 in_seq=0 out_seq=0 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.272+0000 7f24ba000640 10 client.16572 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24ba000640 1 -- 172.21.15.137:0/686224163 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5590b2461800 con 0x5590b25055b0 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24ba000640 10 client.16572 handle_command_reply: tid=0 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 client.16572 shutdown 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 msgr2=0x5590b24f9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f9ae0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f24c00cee90 tx=0x7f24c009aae0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 shutdown_connections 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25055b0 0x5590b2507a20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f13a0 0x5590b2501060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 --2- 172.21.15.137:0/686224163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f9ae0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 >> 172.21.15.137:0/686224163 conn(0x5590b24eac00 msgr2=0x5590b24e9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24ba000640 0 client.16572 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:57.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 shutdown_connections 2024-03-26T14:06:57.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 1 -- 172.21.15.137:0/686224163 wait complete. 2024-03-26T14:06:57.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.274+0000 7f24d9e02740 20 client.16572 trim_cache size 0 max 16384 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 Processor -- start 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 -- start start 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f09e0 0x5590b2501020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 0x5590b24f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01ae960 con 0x5590b24ef810 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01ac460 con 0x5590b24f13a0 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.290+0000 7f24d9e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01aa4e0 con 0x5590b24f09e0 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 0x5590b24f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 0x5590b24f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:42226/0 (socket says 172.21.15.137:42226) 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 -- 172.21.15.137:0/436277387 learned_addr learned my addr 172.21.15.137:0/436277387 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52946/0 (socket says 172.21.15.137:52946) 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24cf400640 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f09e0 0x5590b2501020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 -- 172.21.15.137:0/436277387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f09e0 msgr2=0x5590b2501020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f09e0 0x5590b2501020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 -- 172.21.15.137:0/436277387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 msgr2=0x5590b24f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 -- 172.21.15.137:0/436277387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590b24f0460 con 0x5590b24f13a0 2024-03-26T14:06:57.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.291+0000 7f24d5200640 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 0x5590b24f4060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009ca0 tx=0x7f24c803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5057f0e03a1f69 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.292+0000 7f24c4c00640 1 -- 172.21.15.137:0/436277387 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8039040 con 0x5590b24f13a0 2024-03-26T14:06:57.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.292+0000 7f24c4c00640 1 -- 172.21.15.137:0/436277387 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24c8051070 con 0x5590b24f13a0 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.292+0000 7f24c4c00640 1 -- 172.21.15.137:0/436277387 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c80347f0 con 0x5590b24f13a0 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.292+0000 7f24d9e02740 1 -- 172.21.15.137:0/436277387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 msgr2=0x5590b24f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 0x5590b24f4060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f24c8009ca0 tx=0x7f24c803f000 comp rx=0 tx=0).stop 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 -- 172.21.15.137:0/436277387 shutdown_connections 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f13a0 0x5590b24f4060 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24f09e0 0x5590b2501020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 --2- 172.21.15.137:0/436277387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24ef810 0x5590b24f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 -- 172.21.15.137:0/436277387 >> 172.21.15.137:0/436277387 conn(0x5590b24eac00 msgr2=0x5590b24ee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 -- 172.21.15.137:0/436277387 shutdown_connections 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 -- 172.21.15.137:0/436277387 wait complete. 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 -- start start 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.293+0000 7f24d9e02740 1 Processor -- start 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b25004a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01aaae0 con 0x5590b24f13a0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01abe00 con 0x5590b24f09e0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d01ab140 con 0x5590b24ef810 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d9e02740 10 client.0 _ll_register_callbacks cb 0x7f24c009acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b25004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b25004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:52954/0 (socket says 172.21.15.137:52954) 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 -- 172.21.15.137:0/652247130 learned_addr learned my addr 172.21.15.137:0/652247130 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24cf400640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24cfe00640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 -- 172.21.15.137:0/652247130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 msgr2=0x5590b24fcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24fcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 -- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 msgr2=0x5590b24fd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.294+0000 7f24d5200640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24d5200640 1 -- 172.21.15.137:0/652247130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590b2500a90 con 0x5590b24f13a0 2024-03-26T14:06:57.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24cfe00640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24d5200640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b25004a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005dc0 tx=0x7f24c8004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f48e78259b41cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24ba000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24ba000640 1 -- 172.21.15.137:0/652247130 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8039040 con 0x5590b24f13a0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24ba000640 1 -- 172.21.15.137:0/652247130 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f24c8003a30 con 0x5590b24f13a0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.295+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24c8051070 con 0x5590b24f13a0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24ba000640 1 -- 172.21.15.137:0/652247130 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c8067050 con 0x5590b24f13a0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 10 client.16578 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5590b2561700 con 0x5590b24f13a0 2024-03-26T14:06:57.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24ba000640 1 -- 172.21.15.137:0/652247130 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24c8051070 con 0x5590b24f13a0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24ba000640 1 -- 172.21.15.137:0/652247130 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f24c8034c20 con 0x5590b24f13a0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 10 client.16578 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 20 client.16578 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25620e0 0x5590b2564500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 4 client.16578 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.296+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5590b245bb50 con 0x5590b25620e0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.297+0000 7f24cfe00640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25620e0 0x5590b2564500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.297+0000 7f24cfe00640 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25620e0 0x5590b2564500 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cee759e235adee server_cookie=e88abaa2b59edb23 in_seq=0 out_seq=0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.297+0000 7f24ba000640 10 client.16578 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24ba000640 1 -- 172.21.15.137:0/652247130 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5590b245bb50 con 0x5590b25620e0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24ba000640 10 client.16578 handle_command_reply: tid=0 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 client.16578 shutdown 2024-03-26T14:06:57.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 msgr2=0x5590b25004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b25004a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f24c8005dc0 tx=0x7f24c8004720 comp rx=0 tx=0).stop 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 shutdown_connections 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590b25620e0 0x5590b2564500 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5590b24f09e0 0x5590b24fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5590b24f13a0 0x5590b25004a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 --2- 172.21.15.137:0/652247130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590b24ef810 0x5590b24fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 >> 172.21.15.137:0/652247130 conn(0x5590b24eac00 msgr2=0x5590b24e9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24ba000640 0 client.16578 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 shutdown_connections 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 1 -- 172.21.15.137:0/652247130 wait complete. 2024-03-26T14:06:57.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.298+0000 7f24d9e02740 20 client.16578 trim_cache size 0 max 16384 2024-03-26T14:06:57.301 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:06:57.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24b40a2800 msgr2=0x7f24b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24b40a2800 0x7f24b40a4cc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f24c8006010 tx=0x7f24c802e040 comp rx=0 tx=0).stop 2024-03-26T14:06:57.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 msgr2=0x7f24d0199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:57.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 0x7f24d0199a90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f24c000d950 tx=0x7f24c000de10 comp rx=0 tx=0).stop 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 shutdown_connections 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f24b40a2800 0x7f24b40a4cc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24d0105490 0x7f24d01a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f24d0104ad0 0x7f24d0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 --2- 172.21.15.137:0/1893934349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f24d01038e0 0x7f24d0199a90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 >> 172.21.15.137:0/1893934349 conn(0x7f24d00fee50 msgr2=0x7f24d0107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 shutdown_connections 2024-03-26T14:06:57.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:57.299+0000 7f24d7200640 1 -- 172.21.15.137:0/1893934349 wait complete. 2024-03-26T14:06:57.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:57 smithi137 ceph-mon[31539]: pgmap v596: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:57.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:57 smithi179 ceph-mon[35472]: pgmap v596: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:57.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:57 smithi159 ceph-mon[35501]: pgmap v596: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:06:58.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:06:58.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:06:58.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 -- 172.21.15.137:0/430551844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d0103260 msgr2=0x7ff6d0109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/430551844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d0103260 0x7ff6d0109af0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009ca0 tx=0x7ff6c802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 -- 172.21.15.137:0/430551844 shutdown_connections 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/430551844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6d01028a0 0x7ff6d0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/430551844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d0103260 0x7ff6d0109af0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/430551844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6d01016b0 0x7ff6d0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 -- 172.21.15.137:0/430551844 >> 172.21.15.137:0/430551844 conn(0x7ff6d00fcc40 msgr2=0x7ff6d00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 -- 172.21.15.137:0/430551844 shutdown_connections 2024-03-26T14:06:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.577+0000 7ff6d6e00640 1 -- 172.21.15.137:0/430551844 wait complete. 2024-03-26T14:06:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 Processor -- start 2024-03-26T14:06:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 -- start start 2024-03-26T14:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 0x7ff6d0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6d01028a0 0x7ff6d0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6d0103260 0x7ff6d01a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d010d1e0 con 0x7ff6d01016b0 2024-03-26T14:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d010d060 con 0x7ff6d0103260 2024-03-26T14:06:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.578+0000 7ff6d6e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d010d360 con 0x7ff6d01028a0 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 0x7ff6d0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 0x7ff6d0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33030/0 (socket says 172.21.15.137:33030) 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 -- 172.21.15.137:0/1064351191 learned_addr learned my addr 172.21.15.137:0/1064351191 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6d0103260 0x7ff6d01a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cf400640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6d01028a0 0x7ff6d0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 -- 172.21.15.137:0/1064351191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6d01028a0 msgr2=0x7ff6d0199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6d01028a0 0x7ff6d0199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 -- 172.21.15.137:0/1064351191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6d0103260 msgr2=0x7ff6d01a1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6d0103260 0x7ff6d01a1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 -- 172.21.15.137:0/1064351191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d01a16c0 con 0x7ff6d01016b0 2024-03-26T14:06:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.579+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 0x7ff6d0199aa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000d950 tx=0x7ff6c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.580+0000 7ff6ccc00640 1 -- 172.21.15.137:0/1064351191 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0017070 con 0x7ff6d01016b0 2024-03-26T14:06:58.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.580+0000 7ff6ccc00640 1 -- 172.21.15.137:0/1064351191 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6c0009c30 con 0x7ff6d01016b0 2024-03-26T14:06:58.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.580+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d01a19b0 con 0x7ff6d01016b0 2024-03-26T14:06:58.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.580+0000 7ff6ccc00640 1 -- 172.21.15.137:0/1064351191 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c000f620 con 0x7ff6d01016b0 2024-03-26T14:06:58.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.580+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d01a1f00 con 0x7ff6d01016b0 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.583+0000 7ff6d9a01740 1 Processor -- start 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.583+0000 7ff6d9a01740 1 -- start start 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac25980a0 0x55dac25984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 0x55dac259d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dac24fdeb0 con 0x55dac2598a90 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dac24a6a60 con 0x55dac259dc00 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dac24f9f50 con 0x55dac25980a0 2024-03-26T14:06:58.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6ccc00640 1 -- 172.21.15.137:0/1064351191 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff6c001a070 con 0x7ff6d01016b0 2024-03-26T14:06:58.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 0x55dac259d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 0x55dac259d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33038/0 (socket says 172.21.15.137:33038) 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2649514830 learned_addr learned my addr 172.21.15.137:0/2649514830 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6cf400640 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.584+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac25980a0 0x55dac25984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.585+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2649514830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac25980a0 msgr2=0x55dac25984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.585+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac25980a0 0x55dac25984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.585+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2649514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 msgr2=0x55dac25a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.585+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.585+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2649514830 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dac22da7e0 con 0x55dac2598a90 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.585+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 0x55dac259d590 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4c20 tx=0x7ff6c00d6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8920bd75e46c7e50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.586+0000 7ff6c5600640 1 -- 172.21.15.137:0/2649514830 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00dd070 con 0x55dac2598a90 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.586+0000 7ff6c5600640 1 -- 172.21.15.137:0/2649514830 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6c00d9ad0 con 0x55dac2598a90 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.586+0000 7ff6c5600640 1 -- 172.21.15.137:0/2649514830 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00d0630 con 0x55dac2598a90 2024-03-26T14:06:58.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.586+0000 7ff6ccc00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6b40a2880 0x7ff6b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.586+0000 7ff6ccc00640 1 -- 172.21.15.137:0/1064351191 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff6c00ce8d0 con 0x7ff6d01016b0 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.586+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6b40a2880 0x7ff6b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2649514830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 msgr2=0x55dac259d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 0x55dac259d590 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4c20 tx=0x7ff6c00d6670 comp rx=0 tx=0).stop 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2649514830 shutdown_connections 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac2598a90 0x55dac259d590 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2649514830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac25980a0 0x55dac25984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2649514830 >> 172.21.15.137:0/2649514830 conn(0x55dac2593610 msgr2=0x55dac2593a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2649514830 shutdown_connections 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6b40a2880 0x7ff6b40a4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff6c802c630 tx=0x7ff6c8005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.587+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2649514830 wait complete. 2024-03-26T14:06:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.588+0000 7ff6d9a01740 1 -- start start 2024-03-26T14:06:58.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.588+0000 7ff6d9a01740 1 Processor -- start 2024-03-26T14:06:58.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.588+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac25980a0 0x55dac25a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.588+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac25a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.588+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 0x55dac25a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dac25a0bb0 con 0x55dac259dc00 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dac24b9eb0 con 0x55dac25980a0 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dac25a0d30 con 0x55dac2598a90 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d9a01740 10 client.0 _ll_register_callbacks cb 0x55dac24ae150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 0x55dac25a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 0x55dac25a9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33046/0 (socket says 172.21.15.137:33046) 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 -- 172.21.15.137:0/1488170813 learned_addr learned my addr 172.21.15.137:0/1488170813 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac25980a0 0x55dac25a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6cf400640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac25a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 -- 172.21.15.137:0/1488170813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 msgr2=0x55dac25a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac25a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 -- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac25980a0 msgr2=0x55dac25a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac25980a0 0x55dac25a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6d4e00640 1 -- 172.21.15.137:0/1488170813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dac233af70 con 0x55dac259dc00 2024-03-26T14:06:58.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.589+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac25980a0 0x55dac25a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.590+0000 7ff6d4e00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 0x55dac25a9810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006010 tx=0x7ff6c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcf5ffc7013bc9c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.590+0000 7ff6ba000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.590+0000 7ff6ba000640 1 -- 172.21.15.137:0/1488170813 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8039040 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.590+0000 7ff6ba000640 1 -- 172.21.15.137:0/1488170813 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff6c804c660 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.590+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6c00d9ad0 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.590+0000 7ff6ba000640 1 -- 172.21.15.137:0/1488170813 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c805a050 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 10 client.16590 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dac21d2170 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6ba000640 1 -- 172.21.15.137:0/1488170813 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6c00d9ad0 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6ba000640 1 -- 172.21.15.137:0/1488170813 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff6c8040a00 con 0x55dac259dc00 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 10 client.16590 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 20 client.16590 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac25ae140 0x55dac25b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 4 client.16590 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.591+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dac250b3f0 con 0x55dac25ae140 2024-03-26T14:06:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.592+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac25ae140 0x55dac25b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.592+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac25ae140 0x55dac25b05e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=658612117401ddf1 server_cookie=97ece6f324abc445 in_seq=0 out_seq=0 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.592+0000 7ff6ba000640 10 client.16590 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.594+0000 7ff6ba000640 1 -- 172.21.15.137:0/1488170813 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dac250b3f0 con 0x55dac25ae140 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.594+0000 7ff6ba000640 10 client.16590 handle_command_reply: tid=0 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.594+0000 7ff6d9a01740 1 client.16590 shutdown 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.594+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 msgr2=0x55dac25a9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.594+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 0x55dac25a9810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8006010 tx=0x7ff6c80095c0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 shutdown_connections 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac25ae140 0x55dac25b05e0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac259dc00 0x55dac25a9810 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac25a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/1488170813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac25980a0 0x55dac25a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6ba000640 0 client.16590 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 >> 172.21.15.137:0/1488170813 conn(0x55dac2593610 msgr2=0x55dac2592830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 shutdown_connections 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 1 -- 172.21.15.137:0/1488170813 wait complete. 2024-03-26T14:06:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.595+0000 7ff6d9a01740 20 client.16590 trim_cache size 0 max 16384 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 Processor -- start 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 -- start start 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac2592bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01ae8a0 con 0x55dac25980a0 2024-03-26T14:06:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01ac3a0 con 0x55dac259dc00 2024-03-26T14:06:58.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.618+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01aa420 con 0x55dac2598a90 2024-03-26T14:06:58.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6d4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25926b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33060/0 (socket says 172.21.15.137:33060) 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2243104114 learned_addr learned my addr 172.21.15.137:0/2243104114 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cf400640 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac2592bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2243104114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 msgr2=0x55dac2592bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac2592bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2243104114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 msgr2=0x55dac25a97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.619+0000 7ff6cfe00640 1 -- 172.21.15.137:0/2243104114 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dac25a1360 con 0x55dac25980a0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.620+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25926b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4bf0 tx=0x7ff6c00d46d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed5eb5f014a66fd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.620+0000 7ff6c4c00640 1 -- 172.21.15.137:0/2243104114 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0021030 con 0x55dac25980a0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.620+0000 7ff6c4c00640 1 -- 172.21.15.137:0/2243104114 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6c0009de0 con 0x55dac25980a0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.620+0000 7ff6c4c00640 1 -- 172.21.15.137:0/2243104114 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c00dd070 con 0x55dac25980a0 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2243104114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 msgr2=0x55dac25926b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25926b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4bf0 tx=0x7ff6c00d46d0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2243104114 shutdown_connections 2024-03-26T14:06:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac259dc00 0x55dac25a97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac2598a90 0x55dac2592bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/2243104114 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25926b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2243104114 >> 172.21.15.137:0/2243104114 conn(0x55dac2593610 msgr2=0x55dac2591fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2243104114 shutdown_connections 2024-03-26T14:06:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 -- 172.21.15.137:0/2243104114 wait complete. 2024-03-26T14:06:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 -- start start 2024-03-26T14:06:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.621+0000 7ff6d9a01740 1 Processor -- start 2024-03-26T14:06:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25a9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac2598a90 0x55dac25a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac259dc00 0x55dac25a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01aaa20 con 0x55dac25980a0 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01abd40 con 0x55dac2598a90 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d01ab080 con 0x55dac259dc00 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6d9a01740 10 client.0 _ll_register_callbacks cb 0x7ff6bc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25a9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33072/0 (socket says 172.21.15.137:33072) 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 -- 172.21.15.137:0/3103865032 learned_addr learned my addr 172.21.15.137:0/3103865032 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cf400640 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac2598a90 0x55dac25a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 -- 172.21.15.137:0/3103865032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac259dc00 msgr2=0x55dac25a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac259dc00 0x55dac25a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 -- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac2598a90 msgr2=0x55dac25a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac2598a90 0x55dac25a7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.622+0000 7ff6cfe00640 1 -- 172.21.15.137:0/3103865032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dac25a84f0 con 0x55dac25980a0 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6cfe00640 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25a9410 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4c20 tx=0x7ff6c0097200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02c5bac3fbed954 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6ba000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6ba000640 1 -- 172.21.15.137:0/3103865032 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0021070 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6ba000640 1 -- 172.21.15.137:0/3103865032 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff6c0099160 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6c0009de0 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6ba000640 1 -- 172.21.15.137:0/3103865032 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6c00dd050 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6d9a01740 10 client.16596 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.623+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dac260a760 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6ba000640 1 -- 172.21.15.137:0/3103865032 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6c0009de0 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6ba000640 1 -- 172.21.15.137:0/3103865032 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff6c00cc040 con 0x55dac25980a0 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6d9a01740 10 client.16596 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6d9a01740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6d9a01740 20 client.16596 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac260ac40 0x55dac260d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6d9a01740 4 client.16596 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dac2505740 con 0x55dac260ac40 2024-03-26T14:06:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.624+0000 7ff6cf400640 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac260ac40 0x55dac260d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.625+0000 7ff6ba000640 10 client.16596 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.625+0000 7ff6cf400640 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac260ac40 0x55dac260d060 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4dcfc5f3d2bebf server_cookie=205a6a721e9bbf70 in_seq=0 out_seq=0 2024-03-26T14:06:58.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6ba000640 1 -- 172.21.15.137:0/3103865032 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dac2505740 con 0x55dac260ac40 2024-03-26T14:06:58.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6ba000640 10 client.16596 handle_command_reply: tid=0 2024-03-26T14:06:58.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 client.16596 shutdown 2024-03-26T14:06:58.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 msgr2=0x55dac25a9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25a9410 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00d4c20 tx=0x7ff6c0097200 comp rx=0 tx=0).stop 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 shutdown_connections 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dac260ac40 0x55dac260d060 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dac2598a90 0x55dac25a7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dac259dc00 0x55dac25a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 --2- 172.21.15.137:0/3103865032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dac25980a0 0x55dac25a9410 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6ba000640 0 client.16596 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 >> 172.21.15.137:0/3103865032 conn(0x55dac2593610 msgr2=0x55dac2592830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 shutdown_connections 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 1 -- 172.21.15.137:0/3103865032 wait complete. 2024-03-26T14:06:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.629+0000 7ff6d9a01740 20 client.16596 trim_cache size 0 max 16384 2024-03-26T14:06:58.630 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:06:58.630 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:06:58.630 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "632edb99-94ef-4a42-b179-99c11d3098b4", 2024-03-26T14:06:58.630 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:06:58.630 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6b40a2880 msgr2=0x7ff6b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6b40a2880 0x7ff6b40a4d40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff6c802c630 tx=0x7ff6c8005d80 comp rx=0 tx=0).stop 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 msgr2=0x7ff6d0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 0x7ff6d0199aa0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff6c000d950 tx=0x7ff6c000de10 comp rx=0 tx=0).stop 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 shutdown_connections 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff6b40a2880 0x7ff6b40a4d40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff6d0103260 0x7ff6d01a1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff6d01028a0 0x7ff6d0199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 --2- 172.21.15.137:0/1064351191 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01016b0 0x7ff6d0199aa0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 >> 172.21.15.137:0/1064351191 conn(0x7ff6d00fcc40 msgr2=0x7ff6d00738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 shutdown_connections 2024-03-26T14:06:58.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.630+0000 7ff6d6e00640 1 -- 172.21.15.137:0/1064351191 wait complete. 2024-03-26T14:06:58.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:06:58.649 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 -- 172.21.15.137:0/2711816995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 msgr2=0x7f02c41045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 --2- 172.21.15.137:0/2711816995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c41045b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009900 tx=0x7f02b402bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 -- 172.21.15.137:0/2711816995 shutdown_connections 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 --2- 172.21.15.137:0/2711816995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02c4104af0 0x7f02c4106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 --2- 172.21.15.137:0/2711816995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4107420 0x7f02c4109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 --2- 172.21.15.137:0/2711816995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c41045b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 -- 172.21.15.137:0/2711816995 >> 172.21.15.137:0/2711816995 conn(0x7f02c40fbb30 msgr2=0x7f02c40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 -- 172.21.15.137:0/2711816995 shutdown_connections 2024-03-26T14:06:58.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.872+0000 7f02ca800640 1 -- 172.21.15.137:0/2711816995 wait complete. 2024-03-26T14:06:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 Processor -- start 2024-03-26T14:06:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 -- start start 2024-03-26T14:06:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4104af0 0x7f02c4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02c4107420 0x7f02c419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c410d1e0 con 0x7f02c41021c0 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c410d060 con 0x7f02c4107420 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.873+0000 7f02ca800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c410d360 con 0x7f02c4104af0 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c4197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33086/0 (socket says 172.21.15.137:33086) 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 -- 172.21.15.137:0/2043769983 learned_addr learned my addr 172.21.15.137:0/2043769983 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3e00640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02c4107420 0x7f02c419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c2a00640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4104af0 0x7f02c4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 -- 172.21.15.137:0/2043769983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4104af0 msgr2=0x7f02c4197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4104af0 0x7f02c4197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 -- 172.21.15.137:0/2043769983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02c4107420 msgr2=0x7f02c419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02c4107420 0x7f02c419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 -- 172.21.15.137:0/2043769983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c419f590 con 0x7f02c41021c0 2024-03-26T14:06:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c2a00640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4104af0 0x7f02c4197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.874+0000 7f02c3400640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c4197970 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006010 tx=0x7f02b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.875+0000 7f02bbe00640 1 -- 172.21.15.137:0/2043769983 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4043020 con 0x7f02c41021c0 2024-03-26T14:06:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.875+0000 7f02bbe00640 1 -- 172.21.15.137:0/2043769983 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02b4004440 con 0x7f02c41021c0 2024-03-26T14:06:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.875+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c419f820 con 0x7f02c41021c0 2024-03-26T14:06:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.875+0000 7f02bbe00640 1 -- 172.21.15.137:0/2043769983 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4003bf0 con 0x7f02c41021c0 2024-03-26T14:06:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.875+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c419fdc0 con 0x7f02c41021c0 2024-03-26T14:06:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 Processor -- start 2024-03-26T14:06:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02bbe00640 1 -- 172.21.15.137:0/2043769983 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f02b4046020 con 0x7f02c41021c0 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 -- start start 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaaf7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faaafa280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faaa53250 con 0x557faaaf2290 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.879+0000 7f02cd599740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faa9fba30 con 0x557faaaf7df0 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02cd599740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faaa4f2f0 con 0x557faaaf2c80 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33096/0 (socket says 172.21.15.137:33096) 2024-03-26T14:06:58.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 -- 172.21.15.137:0/1919500044 learned_addr learned my addr 172.21.15.137:0/1919500044 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3400640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaaf7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c2a00640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faaafa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 -- 172.21.15.137:0/1919500044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 msgr2=0x557faaaf7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaaf7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 -- 172.21.15.137:0/1919500044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 msgr2=0x557faaafa280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faaafa280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3e00640 1 -- 172.21.15.137:0/1919500044 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557faa9e6690 con 0x557faaaf2290 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c2a00640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faaafa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.880+0000 7f02c3400640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaaf7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02c3e00640 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf26b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009ee0 tx=0x7f02bc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc20cbd869e601b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02bbe00640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02a80a2880 0x7f02a80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02b8c00640 1 -- 172.21.15.137:0/1919500044 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc016070 con 0x557faaaf2290 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02b8c00640 1 -- 172.21.15.137:0/1919500044 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc004d20 con 0x557faaaf2290 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02b8c00640 1 -- 172.21.15.137:0/1919500044 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc012740 con 0x557faaaf2290 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02bbe00640 1 -- 172.21.15.137:0/2043769983 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f02b40f1210 con 0x7f02c41021c0 2024-03-26T14:06:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.881+0000 7f02c3400640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02a80a2880 0x7f02a80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 -- 172.21.15.137:0/1919500044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 msgr2=0x557faaaf26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf26b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009ee0 tx=0x7f02bc010740 comp rx=0 tx=0).stop 2024-03-26T14:06:58.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02c3400640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02a80a2880 0x7f02a80a4d40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f02b40ee940 tx=0x7f02b402bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 -- 172.21.15.137:0/1919500044 shutdown_connections 2024-03-26T14:06:58.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faaafa280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaaf7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 --2- 172.21.15.137:0/1919500044 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf26b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 -- 172.21.15.137:0/1919500044 >> 172.21.15.137:0/1919500044 conn(0x557faaab6200 msgr2=0x557faaaedc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 -- 172.21.15.137:0/1919500044 shutdown_connections 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.882+0000 7f02cd599740 1 -- 172.21.15.137:0/1919500044 wait complete. 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 -- start start 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 Processor -- start 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2290 0x557faaafc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaafc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 0x557faab03780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faaafada0 con 0x557faaaf7df0 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faaa0ee80 con 0x557faaaf2290 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.883+0000 7f02cd599740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557faaafaf20 con 0x557faaaf2c80 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02cd599740 10 client.0 _ll_register_callbacks cb 0x557faaa21130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 0x557faab03780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 0x557faab03780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33100/0 (socket says 172.21.15.137:33100) 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 -- 172.21.15.137:0/4007034035 learned_addr learned my addr 172.21.15.137:0/4007034035 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2290 0x557faaafc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c2a00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaafc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 -- 172.21.15.137:0/4007034035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 msgr2=0x557faaafc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaafc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 -- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2290 msgr2=0x557faaafc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2290 0x557faaafc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.884+0000 7f02c3e00640 1 -- 172.21.15.137:0/4007034035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557faa8317d0 con 0x557faaaf7df0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02c3e00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 0x557faab03780 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f02bc011010 tx=0x7f02bc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feef7db4be91ccf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02ad600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02ad600640 1 -- 172.21.15.137:0/4007034035 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc016030 con 0x557faaaf7df0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02ad600640 1 -- 172.21.15.137:0/4007034035 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f02bc013d70 con 0x557faaaf7df0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02bc004d20 con 0x557faaaf7df0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02ad600640 1 -- 172.21.15.137:0/4007034035 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02bc0128b0 con 0x557faaaf7df0 2024-03-26T14:06:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02cd599740 10 client.16608 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.885+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557faa7673f0 con 0x557faaaf7df0 2024-03-26T14:06:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02ad600640 1 -- 172.21.15.137:0/4007034035 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc004d20 con 0x557faaaf7df0 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02ad600640 1 -- 172.21.15.137:0/4007034035 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f02bc00f040 con 0x557faaaf7df0 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02cd599740 10 client.16608 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02cd599740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02cd599740 20 client.16608 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02cd599740 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab07c90 0x557faab0a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02cd599740 4 client.16608 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.886+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557faaa60790 con 0x557faab07c90 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.887+0000 7f02c2a00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab07c90 0x557faab0a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.888+0000 7f02c2a00640 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab07c90 0x557faab0a080 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b37a0eb456d162f7 server_cookie=bcd0bc4064ae3b03 in_seq=0 out_seq=0 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.888+0000 7f02ad600640 10 client.16608 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02ad600640 1 -- 172.21.15.137:0/4007034035 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557faaa60790 con 0x557faab07c90 2024-03-26T14:06:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02ad600640 10 client.16608 handle_command_reply: tid=0 2024-03-26T14:06:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 client.16608 shutdown 2024-03-26T14:06:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 msgr2=0x557faab03780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 0x557faab03780 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f02bc011010 tx=0x7f02bc010f40 comp rx=0 tx=0).stop 2024-03-26T14:06:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 shutdown_connections 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab07c90 0x557faab0a080 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf7df0 0x557faab03780 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faaafc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 --2- 172.21.15.137:0/4007034035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2290 0x557faaafc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 >> 172.21.15.137:0/4007034035 conn(0x557faaab6200 msgr2=0x557faaae8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.889+0000 7f02ad600640 0 client.16608 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.890+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 shutdown_connections 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.890+0000 7f02cd599740 1 -- 172.21.15.137:0/4007034035 wait complete. 2024-03-26T14:06:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.890+0000 7f02cd599740 20 client.16608 trim_cache size 0 max 16384 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 Processor -- start 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 -- start start 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2c80 0x557faab03740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf7df0 0x557faaaf0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41aa610 con 0x557faaaf2290 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a8110 con 0x557faaaf2c80 2024-03-26T14:06:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02cd599740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a6190 con 0x557faaaf7df0 2024-03-26T14:06:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33112/0 (socket says 172.21.15.137:33112) 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02c3e00640 1 -- 172.21.15.137:0/1087884786 learned_addr learned my addr 172.21.15.137:0/1087884786 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.909+0000 7f02c2a00640 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2c80 0x557faab03740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.910+0000 7f02c3e00640 1 -- 172.21.15.137:0/1087884786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf7df0 msgr2=0x557faaaf0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.910+0000 7f02c3e00640 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf7df0 0x557faaaf0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.910+0000 7f02c3e00640 1 -- 172.21.15.137:0/1087884786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2c80 msgr2=0x557faab03740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.910+0000 7f02c3e00640 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2c80 0x557faab03740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.910+0000 7f02c3e00640 1 -- 172.21.15.137:0/1087884786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557faa7270f0 con 0x557faaaf2290 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.910+0000 7f02c3e00640 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf3e30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009810 tx=0x7f02bc012780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5c152bf1745385 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.911+0000 7f02afe00640 1 -- 172.21.15.137:0/1087884786 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc016070 con 0x557faaaf2290 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.911+0000 7f02afe00640 1 -- 172.21.15.137:0/1087884786 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc005bd0 con 0x557faaaf2290 2024-03-26T14:06:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.911+0000 7f02afe00640 1 -- 172.21.15.137:0/1087884786 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0135d0 con 0x557faaaf2290 2024-03-26T14:06:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.911+0000 7f02cd599740 1 -- 172.21.15.137:0/1087884786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 msgr2=0x557faaaf3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.911+0000 7f02cd599740 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf3e30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009810 tx=0x7f02bc012780 comp rx=0 tx=0).stop 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 -- 172.21.15.137:0/1087884786 shutdown_connections 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf2c80 0x557faab03740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf7df0 0x557faaaf0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 --2- 172.21.15.137:0/1087884786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faaaf3e30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 -- 172.21.15.137:0/1087884786 >> 172.21.15.137:0/1087884786 conn(0x557faaab6200 msgr2=0x557faaaf3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 -- 172.21.15.137:0/1087884786 shutdown_connections 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 -- 172.21.15.137:0/1087884786 wait complete. 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 -- start start 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.912+0000 7f02cd599740 1 Processor -- start 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faab00b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faab01060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faab02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a6790 con 0x557faaaf2290 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a7ab0 con 0x557faaaf7df0 2024-03-26T14:06:58.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a6df0 con 0x557faaaf2c80 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02cd599740 10 client.0 _ll_register_callbacks cb 0x7f02bc009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faab00b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02c3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faab00b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33122/0 (socket says 172.21.15.137:33122) 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02c3e00640 1 -- 172.21.15.137:0/2063046035 learned_addr learned my addr 172.21.15.137:0/2063046035 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02c2a00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faab01060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.913+0000 7f02c3e00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faab02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02c3e00640 1 -- 172.21.15.137:0/2063046035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 msgr2=0x557faab01060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02c3e00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faab01060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02c3e00640 1 -- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 msgr2=0x557faab02070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02c3e00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faab02070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02c3e00640 1 -- 172.21.15.137:0/2063046035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557faab02660 con 0x557faaaf2290 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02c3e00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faab00b20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f02bc006010 tx=0x7f02bc0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bd61e6dbda9c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:06:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02ad600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:06:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02ad600640 1 -- 172.21.15.137:0/2063046035 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc016030 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02ad600640 1 -- 172.21.15.137:0/2063046035 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f02bc013560 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02bc005bd0 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02ad600640 1 -- 172.21.15.137:0/2063046035 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02bc013e70 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.914+0000 7f02cd599740 10 client.16614 fetch_fsmap learned FSMap version 24 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557faab03970 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02ad600640 1 -- 172.21.15.137:0/2063046035 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f02bc005bd0 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02ad600640 1 -- 172.21.15.137:0/2063046035 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f02bc041070 con 0x557faaaf2290 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 10 client.16614 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 20 client.16614 populate_metadata read hostname 'smithi137' 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab63b10 0x557faab65f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 4 client.16614 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:06:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.915+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557faaa5aae0 con 0x557faab63b10 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.916+0000 7f02c2a00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab63b10 0x557faab65f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02c2a00640 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab63b10 0x557faab65f30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5583ecd6ba55d977 server_cookie=5f109aa7f9f214ac in_seq=0 out_seq=0 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02ad600640 10 client.16614 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02ad600640 1 -- 172.21.15.137:0/2063046035 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557faaa5aae0 con 0x557faab63b10 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02ad600640 10 client.16614 handle_command_reply: tid=0 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 client.16614 shutdown 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 msgr2=0x557faab00b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faab00b20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f02bc006010 tx=0x7f02bc0021f0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 shutdown_connections 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557faab63b10 0x557faab65f30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557faaaf7df0 0x557faab02070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557faaaf2c80 0x557faab01060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 --2- 172.21.15.137:0/2063046035 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557faaaf2290 0x557faab00b20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02ad600640 0 client.16614 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.917+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 >> 172.21.15.137:0/2063046035 conn(0x557faaab6200 msgr2=0x557faaaf3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 shutdown_connections 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02cd599740 1 -- 172.21.15.137:0/2063046035 wait complete. 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02cd599740 20 client.16614 trim_cache size 0 max 16384 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout: "632edb99-94ef-4a42-b179-99c11d3098b4": { 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "632edb99-94ef-4a42-b179-99c11d3098b4", 2024-03-26T14:06:58.919 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:06:58.920 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:06:58.920 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:06:58.920 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:06:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02a80a2880 msgr2=0x7f02a80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02a80a2880 0x7f02a80a4d40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f02b40ee940 tx=0x7f02b402bc30 comp rx=0 tx=0).stop 2024-03-26T14:06:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 msgr2=0x7f02c4197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c4197970 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f02b4006010 tx=0x7f02b4004040 comp rx=0 tx=0).stop 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 shutdown_connections 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f02a80a2880 0x7f02a80a4d40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02c4107420 0x7f02c419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02c4104af0 0x7f02c4197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 --2- 172.21.15.137:0/2043769983 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f02c41021c0 0x7f02c4197970 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.918+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 >> 172.21.15.137:0/2043769983 conn(0x7f02c40fbb30 msgr2=0x7f02c4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.919+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 shutdown_connections 2024-03-26T14:06:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:06:58.919+0000 7f02ca800640 1 -- 172.21.15.137:0/2043769983 wait complete. 2024-03-26T14:06:58.934 INFO:tasks.cephfs.filesystem:scrub status for tag:632edb99-94ef-4a42-b179-99c11d3098b4 - {'path': '/', 'tag': '632edb99-94ef-4a42-b179-99c11d3098b4', 'options': 'recursive,force'} 2024-03-26T14:06:59.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:59 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:06:59.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:59 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:59.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:59 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:06:59.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:06:59 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:59.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:59 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:06:59.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:59 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:59.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:59 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:06:59.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:06:59 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:59.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:59 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:06:59.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:59 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:06:59.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:59 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:06:59.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:06:59 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:00.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:00 smithi179 ceph-mon[35472]: pgmap v597: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:00.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:00 smithi159 ceph-mon[35501]: pgmap v597: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:00.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:00 smithi137 ceph-mon[31539]: pgmap v597: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:01.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:01 smithi179 ceph-mon[35472]: pgmap v598: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:07:01.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:01 smithi159 ceph-mon[35501]: pgmap v598: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:07:01.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:01 smithi137 ceph-mon[31539]: pgmap v598: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:07:02.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:02 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:07:02.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:02 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:07:02.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:02 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:07:03.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:03 smithi179 ceph-mon[35472]: pgmap v599: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:07:03.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:03 smithi159 ceph-mon[35501]: pgmap v599: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:07:03.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:03 smithi137 ceph-mon[31539]: pgmap v599: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:07:05.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:05 smithi179 ceph-mon[35472]: pgmap v600: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:07:05.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:05 smithi159 ceph-mon[35501]: pgmap v600: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:07:05.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:05 smithi137 ceph-mon[31539]: pgmap v600: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:07:07.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:07 smithi179 ceph-mon[35472]: pgmap v601: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:07:07.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:07 smithi159 ceph-mon[35501]: pgmap v601: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:07:07.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:07 smithi137 ceph-mon[31539]: pgmap v601: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 6 op/s 2024-03-26T14:07:09.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:09 smithi179 ceph-mon[35472]: pgmap v602: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:07:09.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:09 smithi159 ceph-mon[35501]: pgmap v602: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:07:09.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:09 smithi137 ceph-mon[31539]: pgmap v602: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:07:11.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:11 smithi179 ceph-mon[35472]: pgmap v603: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:07:11.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:11 smithi159 ceph-mon[35501]: pgmap v603: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:07:11.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:11 smithi137 ceph-mon[31539]: pgmap v603: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:07:13.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:13 smithi179 ceph-mon[35472]: pgmap v604: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:13.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:13 smithi159 ceph-mon[35501]: pgmap v604: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:13 smithi137 ceph-mon[31539]: pgmap v604: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:15.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:15 smithi179 ceph-mon[35472]: pgmap v605: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:15.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:15 smithi159 ceph-mon[35501]: pgmap v605: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:15.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:15 smithi137 ceph-mon[31539]: pgmap v605: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:17.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:17 smithi179 ceph-mon[35472]: pgmap v606: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:17.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:17 smithi159 ceph-mon[35501]: pgmap v606: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:17.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:17 smithi137 ceph-mon[31539]: pgmap v606: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:19.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:19 smithi179 ceph-mon[35472]: pgmap v607: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:19.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:19 smithi159 ceph-mon[35501]: pgmap v607: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:19.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:19 smithi137 ceph-mon[31539]: pgmap v607: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:21.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:21 smithi179 ceph-mon[35472]: pgmap v608: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:07:21.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:21 smithi159 ceph-mon[35501]: pgmap v608: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:07:21.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:21 smithi137 ceph-mon[31539]: pgmap v608: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:07:23.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:23 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:07:23.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:23 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:07:23.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:23 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:07:24.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:24 smithi137 ceph-mon[31539]: pgmap v609: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:24.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:24 smithi179 ceph-mon[35472]: pgmap v609: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:24.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:24 smithi159 ceph-mon[35501]: pgmap v609: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:25.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:07:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:07:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:07:25.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:07:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:07:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:07:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:07:26.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:26 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi137 to 176.7M 2024-03-26T14:07:26.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:26 smithi137 ceph-mon[31539]: Unable to set osd_memory_target on smithi137 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:07:26.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:26 smithi137 ceph-mon[31539]: pgmap v610: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:26 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi137 to 176.7M 2024-03-26T14:07:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:26 smithi179 ceph-mon[35472]: Unable to set osd_memory_target on smithi137 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:07:26.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:26 smithi179 ceph-mon[35472]: pgmap v610: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:26.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:26 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi137 to 176.7M 2024-03-26T14:07:26.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:26 smithi159 ceph-mon[35501]: Unable to set osd_memory_target on smithi137 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:07:26.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:26 smithi159 ceph-mon[35501]: pgmap v610: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:28.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:28 smithi179 ceph-mon[35472]: pgmap v611: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:28.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:28 smithi159 ceph-mon[35501]: pgmap v611: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:28.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:28 smithi137 ceph-mon[31539]: pgmap v611: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:28.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:07:28.936 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:07:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 -- 172.21.15.137:0/1570749253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f541015a0 msgr2=0x7f9f541019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 --2- 172.21.15.137:0/1570749253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f541015a0 0x7f9f541019c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f4802bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 -- 172.21.15.137:0/1570749253 shutdown_connections 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 --2- 172.21.15.137:0/1570749253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f54103150 0x7f9f541099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 --2- 172.21.15.137:0/1570749253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f54102790 0x7f9f54102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 --2- 172.21.15.137:0/1570749253 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f541015a0 0x7f9f541019c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 -- 172.21.15.137:0/1570749253 >> 172.21.15.137:0/1570749253 conn(0x7f9f540fcb10 msgr2=0x7f9f540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 -- 172.21.15.137:0/1570749253 shutdown_connections 2024-03-26T14:07:29.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.169+0000 7f9f5a000640 1 -- 172.21.15.137:0/1570749253 wait complete. 2024-03-26T14:07:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.170+0000 7f9f5a000640 1 Processor -- start 2024-03-26T14:07:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.170+0000 7f9f5a000640 1 -- start start 2024-03-26T14:07:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f5a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f541015a0 0x7f9f54107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f5a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 0x7f9f54107860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f5a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f54103150 0x7f9f54103e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f5a000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5410d0b0 con 0x7f9f54102790 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f5a000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5410cf30 con 0x7f9f54103150 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f5a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5410d230 con 0x7f9f541015a0 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 0x7f9f54107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 0x7f9f54107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55854/0 (socket says 172.21.15.137:55854) 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f53e00640 1 -- 172.21.15.137:0/143007235 learned_addr learned my addr 172.21.15.137:0/143007235 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f58c00640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f541015a0 0x7f9f54107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.171+0000 7f9f59600640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f54103150 0x7f9f54103e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f53e00640 1 -- 172.21.15.137:0/143007235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f541015a0 msgr2=0x7f9f54107320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f53e00640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f541015a0 0x7f9f54107320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f53e00640 1 -- 172.21.15.137:0/143007235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f54103150 msgr2=0x7f9f54103e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f53e00640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f54103150 0x7f9f54103e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f53e00640 1 -- 172.21.15.137:0/143007235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54104750 con 0x7f9f54102790 2024-03-26T14:07:29.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f53e00640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 0x7f9f54107860 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d950 tx=0x7f9f4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f51600640 1 -- 172.21.15.137:0/143007235 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40017070 con 0x7f9f54102790 2024-03-26T14:07:29.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f51600640 1 -- 172.21.15.137:0/143007235 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f40009c30 con 0x7f9f54102790 2024-03-26T14:07:29.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f51600640 1 -- 172.21.15.137:0/143007235 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000f620 con 0x7f9f54102790 2024-03-26T14:07:29.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f541a5e40 con 0x7f9f54102790 2024-03-26T14:07:29.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.172+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a6240 con 0x7f9f54102790 2024-03-26T14:07:29.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.176+0000 7f9f51600640 1 -- 172.21.15.137:0/143007235 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9f400109a0 con 0x7f9f54102790 2024-03-26T14:07:29.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.176+0000 7f9f5e3bf740 1 Processor -- start 2024-03-26T14:07:29.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.176+0000 7f9f5e3bf740 1 -- start start 2024-03-26T14:07:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f99466470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f9946b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f9946e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f993cb250 con 0x562f99466a40 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f99373a30 con 0x562f9946bbb0 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f993c72f0 con 0x562f99466050 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f9946b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f9946b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55862/0 (socket says 172.21.15.137:55862) 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 -- 172.21.15.137:0/4266944906 learned_addr learned my addr 172.21.15.137:0/4266944906 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f59600640 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f99466470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f53e00640 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f9946e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 -- 172.21.15.137:0/4266944906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 msgr2=0x562f99466470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f99466470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 -- 172.21.15.137:0/4266944906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 msgr2=0x562f9946e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.177+0000 7f9f58c00640 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f9946e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f58c00640 1 -- 172.21.15.137:0/4266944906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f9935e690 con 0x562f99466a40 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f58c00640 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f9946b540 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f48005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb97abb8b6d7894 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f51600640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9f3c0a2880 0x7f9f3c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f45c00640 1 -- 172.21.15.137:0/4266944906 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48045020 con 0x562f99466a40 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f45c00640 1 -- 172.21.15.137:0/4266944906 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f4802ca20 con 0x562f99466a40 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f45c00640 1 -- 172.21.15.137:0/4266944906 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480046c0 con 0x562f99466a40 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f51600640 1 -- 172.21.15.137:0/143007235 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9f400cea30 con 0x7f9f54102790 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.178+0000 7f9f59600640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9f3c0a2880 0x7f9f3c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/4266944906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 msgr2=0x562f9946b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f9946b540 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f48005c00 comp rx=0 tx=0).stop 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f59600640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9f3c0a2880 0x7f9f3c0a4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x562f994672a0 tx=0x7f9f4c009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/4266944906 shutdown_connections 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f9946e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f9946b540 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/4266944906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f99466470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/4266944906 >> 172.21.15.137:0/4266944906 conn(0x562f9942e200 msgr2=0x562f994619c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.179+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/4266944906 shutdown_connections 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.180+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/4266944906 wait complete. 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.180+0000 7f9f5e3bf740 1 -- start start 2024-03-26T14:07:29.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.180+0000 7f9f5e3bf740 1 Processor -- start 2024-03-26T14:07:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466050 0x562f9946ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f994704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99477520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9946eb00 con 0x562f99466a40 2024-03-26T14:07:29.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f99386e80 con 0x562f99466050 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f9946ec80 con 0x562f9946bbb0 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f5e3bf740 10 client.0 _ll_register_callbacks cb 0x562f99399130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f994704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f994704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55872/0 (socket says 172.21.15.137:55872) 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 -- 172.21.15.137:0/3144269281 learned_addr learned my addr 172.21.15.137:0/3144269281 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f58c00640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466050 0x562f9946ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f59600640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99477520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 -- 172.21.15.137:0/3144269281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 msgr2=0x562f99477520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99477520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 -- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466050 msgr2=0x562f9946ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466050 0x562f9946ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.181+0000 7f9f53e00640 1 -- 172.21.15.137:0/3144269281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f991a97d0 con 0x562f99466a40 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f53e00640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f994704e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9f400a3230 tx=0x7f9f4009a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436801df3506f333 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f3a000640 1 -- 172.21.15.137:0/3144269281 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40020040 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f3a000640 1 -- 172.21.15.137:0/3144269281 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9f40095b30 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f4802ca20 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f3a000640 1 -- 172.21.15.137:0/3144269281 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4009afc0 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f5e3bf740 10 client.16626 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.182+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562f990df3f0 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f3a000640 1 -- 172.21.15.137:0/3144269281 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f4802ca20 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f3a000640 1 -- 172.21.15.137:0/3144269281 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9f4009d020 con 0x562f99466a40 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f5e3bf740 10 client.16626 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f5e3bf740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f5e3bf740 20 client.16626 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f9947bcd0 0x562f9947e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f5e3bf740 4 client.16626 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f993d8790 con 0x562f9947bcd0 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.183+0000 7f9f58c00640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f9947bcd0 0x562f9947e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.184+0000 7f9f58c00640 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f9947bcd0 0x562f9947e150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=979f7dd2f17fbf10 server_cookie=da2dcf1d823d6750 in_seq=0 out_seq=0 2024-03-26T14:07:29.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.184+0000 7f9f3a000640 10 client.16626 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:29.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f3a000640 1 -- 172.21.15.137:0/3144269281 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562f993d8790 con 0x562f9947bcd0 2024-03-26T14:07:29.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f3a000640 10 client.16626 handle_command_reply: tid=0 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 client.16626 shutdown 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 msgr2=0x562f994704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f994704e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9f400a3230 tx=0x7f9f4009a920 comp rx=0 tx=0).stop 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 shutdown_connections 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f9947bcd0 0x562f9947e150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99477520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f994704e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/3144269281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466050 0x562f9946ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f3a000640 0 client.16626 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.186+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 >> 172.21.15.137:0/3144269281 conn(0x562f9942e200 msgr2=0x562f99460bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.187+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 shutdown_connections 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.187+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/3144269281 wait complete. 2024-03-26T14:07:29.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.187+0000 7f9f5e3bf740 20 client.16626 trim_cache size 0 max 16384 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 Processor -- start 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 -- start start 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 0x562f99460330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466a40 0x562f994774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99460870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541b2af0 con 0x562f99466050 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541b05f0 con 0x562f99466a40 2024-03-26T14:07:29.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541ae670 con 0x562f9946bbb0 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 0x562f99460330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f58c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 0x562f99460330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55884/0 (socket says 172.21.15.137:55884) 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f58c00640 1 -- 172.21.15.137:0/1822597749 learned_addr learned my addr 172.21.15.137:0/1822597749 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.208+0000 7f9f53e00640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466a40 0x562f994774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f58c00640 1 -- 172.21.15.137:0/1822597749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 msgr2=0x562f99460870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f59600640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99460870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f58c00640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99460870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f58c00640 1 -- 172.21.15.137:0/1822597749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466a40 msgr2=0x562f994774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f58c00640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466a40 0x562f994774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f58c00640 1 -- 172.21.15.137:0/1822597749 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f9909f0f0 con 0x562f99466050 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f53e00640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466a40 0x562f994774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f59600640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99460870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f58c00640 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 0x562f99460330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffcfab9136f83b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.209+0000 7f9f45200640 1 -- 172.21.15.137:0/1822597749 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4804e070 con 0x562f99466050 2024-03-26T14:07:29.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f45200640 1 -- 172.21.15.137:0/1822597749 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f4802ca20 con 0x562f99466050 2024-03-26T14:07:29.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f45200640 1 -- 172.21.15.137:0/1822597749 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48045040 con 0x562f99466050 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1822597749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 msgr2=0x562f99460330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 0x562f99460330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009900 tx=0x7f9f48000bc0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1822597749 shutdown_connections 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f99466a40 0x562f994774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f9946bbb0 0x562f99460870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1822597749 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466050 0x562f99460330 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.210+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1822597749 >> 172.21.15.137:0/1822597749 conn(0x562f9942e200 msgr2=0x562f99467d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.211+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1822597749 shutdown_connections 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.211+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1822597749 wait complete. 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.211+0000 7f9f5e3bf740 1 -- start start 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.211+0000 7f9f5e3bf740 1 Processor -- start 2024-03-26T14:07:29.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f994754c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f99475a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f994769c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541aec70 con 0x562f99466a40 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541aff90 con 0x562f9946bbb0 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541af2d0 con 0x562f99466050 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f5e3bf740 10 client.0 _ll_register_callbacks cb 0x7f9f4000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f99475a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f99475a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55894/0 (socket says 172.21.15.137:55894) 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 -- 172.21.15.137:0/1785536812 learned_addr learned my addr 172.21.15.137:0/1785536812 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f59600640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f994769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f58c00640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f994754c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 -- 172.21.15.137:0/1785536812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 msgr2=0x562f994754c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f994754c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 -- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 msgr2=0x562f994769c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f994769c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f53e00640 1 -- 172.21.15.137:0/1785536812 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f99476fb0 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.212+0000 7f9f59600640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f994769c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f53e00640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f99475a00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9f40097420 tx=0x7f9f40095d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0e6c8533b72047 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f3a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f3a000640 1 -- 172.21.15.137:0/1785536812 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40020070 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f3a000640 1 -- 172.21.15.137:0/1785536812 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9f40010460 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f4802ca20 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f3a000640 1 -- 172.21.15.137:0/1785536812 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f4009ad90 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f5e3bf740 10 client.16632 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.213+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562f994d7dd0 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f3a000640 1 -- 172.21.15.137:0/1785536812 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f4802ca20 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f3a000640 1 -- 172.21.15.137:0/1785536812 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9f4009d040 con 0x562f99466a40 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f5e3bf740 10 client.16632 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:29.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f5e3bf740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:29.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f5e3bf740 20 client.16632 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:29.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f994d85e0 0x562f994daa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f5e3bf740 4 client.16632 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:07:29.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f99470870 con 0x562f994d85e0 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.214+0000 7f9f58c00640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f994d85e0 0x562f994daa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.215+0000 7f9f3a000640 10 client.16632 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.215+0000 7f9f58c00640 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f994d85e0 0x562f994daa00 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7309f27559601a server_cookie=de84b1b0394c2342 in_seq=0 out_seq=0 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f3a000640 1 -- 172.21.15.137:0/1785536812 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562f99470870 con 0x562f994d85e0 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f3a000640 10 client.16632 handle_command_reply: tid=0 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 client.16632 shutdown 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 msgr2=0x562f99475a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f99475a00 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9f40097420 tx=0x7f9f40095d40 comp rx=0 tx=0).stop 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 shutdown_connections 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562f994d85e0 0x562f994daa00 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f9946bbb0 0x562f994769c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562f99466a40 0x562f99475a00 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 --2- 172.21.15.137:0/1785536812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562f99466050 0x562f994754c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 >> 172.21.15.137:0/1785536812 conn(0x562f9942e200 msgr2=0x562f99460330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f3a000640 0 client.16632 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 shutdown_connections 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 1 -- 172.21.15.137:0/1785536812 wait complete. 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.216+0000 7f9f5e3bf740 20 client.16632 trim_cache size 0 max 16384 2024-03-26T14:07:29.218 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:07:29.219 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:07:29.219 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:07:29.219 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:07:29.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9f3c0a2880 msgr2=0x7f9f3c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9f3c0a2880 0x7f9f3c0a4d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x562f994672a0 tx=0x7f9f4c009580 comp rx=0 tx=0).stop 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 msgr2=0x7f9f54107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 0x7f9f54107860 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000d950 tx=0x7f9f4000de10 comp rx=0 tx=0).stop 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 shutdown_connections 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9f3c0a2880 0x7f9f3c0a4d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9f54103150 0x7f9f54103e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9f54102790 0x7f9f54107860 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 --2- 172.21.15.137:0/143007235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9f541015a0 0x7f9f54107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 >> 172.21.15.137:0/143007235 conn(0x7f9f540fcb10 msgr2=0x7f9f540738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 shutdown_connections 2024-03-26T14:07:29.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.217+0000 7f9f5a000640 1 -- 172.21.15.137:0/143007235 wait complete. 2024-03-26T14:07:29.234 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:07:29.234 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:07:29.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 -- 172.21.15.137:0/4105377572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 msgr2=0x7f6b0c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4105377572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c110120 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009ca0 tx=0x7f6b0402c100 comp rx=0 tx=0).stop 2024-03-26T14:07:29.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 -- 172.21.15.137:0/4105377572 shutdown_connections 2024-03-26T14:07:29.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4105377572 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c110120 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4105377572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 0x7f6b0c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4105377572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b0c107ce0 0x7f6b0c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 -- 172.21.15.137:0/4105377572 >> 172.21.15.137:0/4105377572 conn(0x7f6b0c076cb0 msgr2=0x7f6b0c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 -- 172.21.15.137:0/4105377572 shutdown_connections 2024-03-26T14:07:29.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.444+0000 7f6b11a00640 1 -- 172.21.15.137:0/4105377572 wait complete. 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 Processor -- start 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 -- start start 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b0c107ce0 0x7f6b0c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 0x7f6b0c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c113810 con 0x7f6b0c109890 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c113690 con 0x7f6b0c107ce0 2024-03-26T14:07:29.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.445+0000 7f6b11a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c113990 con 0x7f6b0c108ed0 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c1a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55912/0 (socket says 172.21.15.137:55912) 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 -- 172.21.15.137:0/4200605491 learned_addr learned my addr 172.21.15.137:0/4200605491 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0aa00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b0c107ce0 0x7f6b0c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0a000640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 0x7f6b0c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 -- 172.21.15.137:0/4200605491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 msgr2=0x7f6b0c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 0x7f6b0c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 -- 172.21.15.137:0/4200605491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b0c107ce0 msgr2=0x7f6b0c19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b0c107ce0 0x7f6b0c19de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0b400640 1 -- 172.21.15.137:0/4200605491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c1a5ab0 con 0x7f6b0c109890 2024-03-26T14:07:29.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.446+0000 7f6b0a000640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 0x7f6b0c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:29.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.447+0000 7f6b0b400640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c1a5440 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6b04032520 tx=0x7f6b04034880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.447+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04039040 con 0x7f6b0c109890 2024-03-26T14:07:29.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.447+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6b04044070 con 0x7f6b0c109890 2024-03-26T14:07:29.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.447+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c1a5d40 con 0x7f6b0c109890 2024-03-26T14:07:29.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.447+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04004430 con 0x7f6b0c109890 2024-03-26T14:07:29.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.447+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1a6220 con 0x7f6b0c109890 2024-03-26T14:07:29.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.448+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c10f000 con 0x7f6b0c109890 2024-03-26T14:07:29.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.450+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6b04003bc0 con 0x7f6b0c109890 2024-03-26T14:07:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.452+0000 7f6b03400640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af00a2880 0x7f6af00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.452+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6b040f1570 con 0x7f6b0c109890 2024-03-26T14:07:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.452+0000 7f6b0aa00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af00a2880 0x7f6af00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.453+0000 7f6b0aa00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af00a2880 0x7f6af00a4d40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005fd0 tx=0x7f6afc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.458+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6b040333a0 con 0x7f6b0c109890 2024-03-26T14:07:29.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.734+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b0c062340 con 0x7f6b0c109890 2024-03-26T14:07:29.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.735+0000 7f6b03400640 1 -- 172.21.15.137:0/4200605491 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f6b040be240 con 0x7f6b0c109890 2024-03-26T14:07:29.735 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:07:29.735 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:07:29.735 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af00a2880 msgr2=0x7f6af00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af00a2880 0x7f6af00a4d40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005fd0 tx=0x7f6afc005850 comp rx=0 tx=0).stop 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 msgr2=0x7f6b0c1a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c1a5440 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6b04032520 tx=0x7f6b04034880 comp rx=0 tx=0).stop 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 shutdown_connections 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6af00a2880 0x7f6af00a4d40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6b0c109890 0x7f6b0c1a5440 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b0c108ed0 0x7f6b0c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 --2- 172.21.15.137:0/4200605491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b0c107ce0 0x7f6b0c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.738+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 >> 172.21.15.137:0/4200605491 conn(0x7f6b0c076cb0 msgr2=0x7f6b0c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.739+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 shutdown_connections 2024-03-26T14:07:29.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.739+0000 7f6b11a00640 1 -- 172.21.15.137:0/4200605491 wait complete. 2024-03-26T14:07:29.756 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:07:29.757 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:07:29.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.992+0000 7ffba9400640 1 -- 172.21.15.137:0/2718039823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4102000 msgr2=0x7ffba41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.992+0000 7ffba9400640 1 --2- 172.21.15.137:0/2718039823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4102000 0x7ffba41043f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009900 tx=0x7ffb9402bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:29.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 -- 172.21.15.137:0/2718039823 shutdown_connections 2024-03-26T14:07:29.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 --2- 172.21.15.137:0/2718039823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4107260 0x7ffba41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 --2- 172.21.15.137:0/2718039823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4104930 0x7ffba4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 --2- 172.21.15.137:0/2718039823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4102000 0x7ffba41043f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 -- 172.21.15.137:0/2718039823 >> 172.21.15.137:0/2718039823 conn(0x7ffba40fba10 msgr2=0x7ffba40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:29.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 -- 172.21.15.137:0/2718039823 shutdown_connections 2024-03-26T14:07:29.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.993+0000 7ffba9400640 1 -- 172.21.15.137:0/2718039823 wait complete. 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 Processor -- start 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 -- start start 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4102000 0x7ffba4101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 0x7ffba40fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4107260 0x7ffba41004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba410d110 con 0x7ffba4104930 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba410cf90 con 0x7ffba4102000 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba9400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba410d290 con 0x7ffba4107260 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.994+0000 7ffba3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 0x7ffba40fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 0x7ffba40fffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55938/0 (socket says 172.21.15.137:55938) 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 -- 172.21.15.137:0/3475426505 learned_addr learned my addr 172.21.15.137:0/3475426505 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4102000 0x7ffba4101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba8a00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4107260 0x7ffba41004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 -- 172.21.15.137:0/3475426505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4107260 msgr2=0x7ffba41004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4107260 0x7ffba41004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 -- 172.21.15.137:0/3475426505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4102000 msgr2=0x7ffba4101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4102000 0x7ffba4101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 -- 172.21.15.137:0/3475426505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4100d80 con 0x7ffba4104930 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba8a00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4107260 0x7ffba41004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4102000 0x7ffba4101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.995+0000 7ffba3400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 0x7ffba40fffb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000d770 tx=0x7ffb9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.996+0000 7ffba0c00640 1 -- 172.21.15.137:0/3475426505 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90017070 con 0x7ffba4104930 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.996+0000 7ffba0c00640 1 -- 172.21.15.137:0/3475426505 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb90009c30 con 0x7ffba4104930 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.996+0000 7ffba0c00640 1 -- 172.21.15.137:0/3475426505 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9000f620 con 0x7ffba4104930 2024-03-26T14:07:29.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.996+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba41a18e0 con 0x7ffba4104930 2024-03-26T14:07:29.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:29.996+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba41a1de0 con 0x7ffba4104930 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 Processor -- start 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 -- start start 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffba0c00640 1 -- 172.21.15.137:0/3475426505 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ffb9001a070 con 0x7ffba4104930 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd68e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd69bc0 0x5606ebd6a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd70e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ebccd2c0 con 0x5606ebd689f0 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ebc75c90 con 0x5606ebd69bc0 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffbad774740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ebcc9360 con 0x5606ebd6a580 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffba8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd68e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffba8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd68e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55954/0 (socket says 172.21.15.137:55954) 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.000+0000 7ffba8a00640 1 -- 172.21.15.137:0/1663730489 learned_addr learned my addr 172.21.15.137:0/1663730489 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba3e00640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd69bc0 0x5606ebd6a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba3400640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd70e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba8a00640 1 -- 172.21.15.137:0/1663730489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 msgr2=0x5606ebd70e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba8a00640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd70e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba8a00640 1 -- 172.21.15.137:0/1663730489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd69bc0 msgr2=0x5606ebd6a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba8a00640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd69bc0 0x5606ebd6a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba8a00640 1 -- 172.21.15.137:0/1663730489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ebc9a080 con 0x5606ebd689f0 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba3400640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd70e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba3e00640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd69bc0 0x5606ebd6a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:30.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffba8a00640 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd68e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009ea0 tx=0x7ffb9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfae610ba450885 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.001+0000 7ffb99200640 1 -- 172.21.15.137:0/1663730489 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c004d20 con 0x5606ebd689f0 2024-03-26T14:07:30.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.002+0000 7ffb99200640 1 -- 172.21.15.137:0/1663730489 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c0054d0 con 0x5606ebd689f0 2024-03-26T14:07:30.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.002+0000 7ffb99200640 1 -- 172.21.15.137:0/1663730489 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c011820 con 0x5606ebd689f0 2024-03-26T14:07:30.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.002+0000 7ffba0c00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb880a2880 0x7ffb880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.002+0000 7ffba0c00640 1 -- 172.21.15.137:0/3475426505 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ffb900ce9c0 con 0x7ffba4104930 2024-03-26T14:07:30.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.002+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb880a2880 0x7ffb880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 -- 172.21.15.137:0/1663730489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 msgr2=0x5606ebd68e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd68e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009ea0 tx=0x7ffb9c009f90 comp rx=0 tx=0).stop 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 -- 172.21.15.137:0/1663730489 shutdown_connections 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd69bc0 0x5606ebd6a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd70e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 --2- 172.21.15.137:0/1663730489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd68e10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 -- 172.21.15.137:0/1663730489 >> 172.21.15.137:0/1663730489 conn(0x5606ebd63de0 msgr2=0x5606ebd641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 -- 172.21.15.137:0/1663730489 shutdown_connections 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffbad774740 1 -- 172.21.15.137:0/1663730489 wait complete. 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.003+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb880a2880 0x7ffb880a4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006010 tx=0x7ffb9402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 -- start start 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 Processor -- start 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd689f0 0x5606ebd72c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd7a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ebd71590 con 0x5606ebd69bc0 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ebc890d0 con 0x5606ebd689f0 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606ebd71710 con 0x5606ebd6a580 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.004+0000 7ffbad774740 10 client.0 _ll_register_callbacks cb 0x5606ebc8de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd73170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55970/0 (socket says 172.21.15.137:55970) 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 -- 172.21.15.137:0/3501454834 learned_addr learned my addr 172.21.15.137:0/3501454834 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd689f0 0x5606ebd72c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba8a00640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd7a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 -- 172.21.15.137:0/3501454834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 msgr2=0x5606ebd7a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd7a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 -- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd689f0 msgr2=0x5606ebd72c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd689f0 0x5606ebd72c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.005+0000 7ffba3400640 1 -- 172.21.15.137:0/3501454834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606eb9e13f0 con 0x5606ebd69bc0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffba3400640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd73170 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ffb900973b0 tx=0x7ffb90009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ab48426b7efaa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffb8d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffb8d600640 1 -- 172.21.15.137:0/3501454834 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90021050 con 0x5606ebd69bc0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffb8d600640 1 -- 172.21.15.137:0/3501454834 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ffb9009a3a0 con 0x5606ebd69bc0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb9c0054d0 con 0x5606ebd69bc0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffb8d600640 1 -- 172.21.15.137:0/3501454834 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb9009f0a0 con 0x5606ebd69bc0 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffbad774740 10 client.16650 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:30.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5606ebaab7d0 con 0x5606ebd69bc0 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.006+0000 7ffb8d600640 1 -- 172.21.15.137:0/3501454834 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb9c0054d0 con 0x5606ebd69bc0 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffb8d600640 1 -- 172.21.15.137:0/3501454834 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ffb90095c50 con 0x5606ebd69bc0 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffbad774740 10 client.16650 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffbad774740 10 client.16650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffbad774740 20 client.16650 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffbad774740 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebd7e650 0x5606ebd80aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffbad774740 4 client.16650 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606ebcda800 con 0x5606ebd7e650 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.007+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebd7e650 0x5606ebd80aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.008+0000 7ffb8d600640 10 client.16650 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:30.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.008+0000 7ffba3e00640 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebd7e650 0x5606ebd80aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce3a85a38f506e3 server_cookie=63df0a8c80e0a0cd in_seq=0 out_seq=0 2024-03-26T14:07:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffb8d600640 1 -- 172.21.15.137:0/3501454834 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5606ebcda800 con 0x5606ebd7e650 2024-03-26T14:07:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffb8d600640 10 client.16650 handle_command_reply: tid=0 2024-03-26T14:07:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 client.16650 shutdown 2024-03-26T14:07:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 msgr2=0x5606ebd73170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd73170 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ffb900973b0 tx=0x7ffb90009de0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 shutdown_connections 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebd7e650 0x5606ebd80aa0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd6a580 0x5606ebd7a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd73170 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffb8d600640 0 client.16650 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 --2- 172.21.15.137:0/3501454834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd689f0 0x5606ebd72c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 >> 172.21.15.137:0/3501454834 conn(0x5606ebd63de0 msgr2=0x5606ebd63160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.010+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 shutdown_connections 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.011+0000 7ffbad774740 1 -- 172.21.15.137:0/3501454834 wait complete. 2024-03-26T14:07:30.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.011+0000 7ffbad774740 20 client.16650 trim_cache size 0 max 16384 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 Processor -- start 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 -- start start 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd689f0 0x5606ebd67750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd7a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd85360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41ae830 con 0x5606ebd69bc0 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41ac330 con 0x5606ebd6a580 2024-03-26T14:07:30.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffbad774740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41aa3b0 con 0x5606ebd689f0 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.026+0000 7ffba3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd7a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd7a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55976/0 (socket says 172.21.15.137:55976) 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 -- 172.21.15.137:0/2042437358 learned_addr learned my addr 172.21.15.137:0/2042437358 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba8a00640 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd85360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 -- 172.21.15.137:0/2042437358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd689f0 msgr2=0x5606ebd67750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd689f0 0x5606ebd67750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 -- 172.21.15.137:0/2042437358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 msgr2=0x5606ebd85360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd85360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 -- 172.21.15.137:0/2042437358 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606ebd69640 con 0x5606ebd69bc0 2024-03-26T14:07:30.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffba3400640 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd7a170 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009f90 tx=0x7ffb900cec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99991c75bfd50535 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffb8fe00640 1 -- 172.21.15.137:0/2042437358 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90021070 con 0x5606ebd69bc0 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.027+0000 7ffb8fe00640 1 -- 172.21.15.137:0/2042437358 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb9009a3a0 con 0x5606ebd69bc0 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffb8fe00640 1 -- 172.21.15.137:0/2042437358 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb90096f00 con 0x5606ebd69bc0 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 -- 172.21.15.137:0/2042437358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 msgr2=0x5606ebd7a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd7a170 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009f90 tx=0x7ffb900cec40 comp rx=0 tx=0).stop 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 -- 172.21.15.137:0/2042437358 shutdown_connections 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd85360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd69bc0 0x5606ebd7a170 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 --2- 172.21.15.137:0/2042437358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd689f0 0x5606ebd67750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 -- 172.21.15.137:0/2042437358 >> 172.21.15.137:0/2042437358 conn(0x5606ebd63de0 msgr2=0x5606ebd67040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 -- 172.21.15.137:0/2042437358 shutdown_connections 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.028+0000 7ffbad774740 1 -- 172.21.15.137:0/2042437358 wait complete. 2024-03-26T14:07:30.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.029+0000 7ffbad774740 1 -- start start 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.029+0000 7ffbad774740 1 Processor -- start 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.029+0000 7ffbad774740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd79ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.029+0000 7ffbad774740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd69bc0 0x5606ebd784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffbad774740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd78a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffbad774740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41aa9b0 con 0x5606ebd689f0 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffbad774740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41abcd0 con 0x5606ebd6a580 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffbad774740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba41ab010 con 0x5606ebd69bc0 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffbad774740 10 client.0 _ll_register_callbacks cb 0x7ffb9000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd79ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd79ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:55992/0 (socket says 172.21.15.137:55992) 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 -- 172.21.15.137:0/1429204434 learned_addr learned my addr 172.21.15.137:0/1429204434 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba8a00640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd78a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3400640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd69bc0 0x5606ebd784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 -- 172.21.15.137:0/1429204434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd69bc0 msgr2=0x5606ebd784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd69bc0 0x5606ebd784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 -- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 msgr2=0x5606ebd78a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd78a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 -- 172.21.15.137:0/1429204434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606ebd79080 con 0x5606ebd689f0 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffba3e00640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd79ec0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009900 tx=0x7ffb9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2a1538f907a026 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.030+0000 7ffb8d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffb8d600640 1 -- 172.21.15.137:0/1429204434 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94005510 con 0x5606ebd689f0 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffb8d600640 1 -- 172.21.15.137:0/1429204434 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ffb940046c0 con 0x5606ebd689f0 2024-03-26T14:07:30.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb9009a3a0 con 0x5606ebd689f0 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffb8d600640 1 -- 172.21.15.137:0/1429204434 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb94063050 con 0x5606ebd689f0 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffbad774740 10 client.16656 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5606ebddae50 con 0x5606ebd689f0 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.031+0000 7ffb8d600640 1 -- 172.21.15.137:0/1429204434 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb9009a3a0 con 0x5606ebd689f0 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffb8d600640 1 -- 172.21.15.137:0/1429204434 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ffb94054a50 con 0x5606ebd689f0 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffbad774740 10 client.16656 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:30.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffbad774740 10 client.16656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffbad774740 20 client.16656 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffbad774740 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebddb7e0 0x5606ebdddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffbad774740 4 client.16656 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5606ebcd4b50 con 0x5606ebddb7e0 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.032+0000 7ffba3400640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebddb7e0 0x5606ebdddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.033+0000 7ffba3400640 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebddb7e0 0x5606ebdddc00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30d482a8dc4af53d server_cookie=bad79edb58a10acb in_seq=0 out_seq=0 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.033+0000 7ffb8d600640 10 client.16656 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.033+0000 7ffb8d600640 1 -- 172.21.15.137:0/1429204434 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5606ebcd4b50 con 0x5606ebddb7e0 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.033+0000 7ffb8d600640 10 client.16656 handle_command_reply: tid=0 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.033+0000 7ffbad774740 1 client.16656 shutdown 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 msgr2=0x5606ebd79ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd79ec0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009900 tx=0x7ffb9402bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 shutdown_connections 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5606ebddb7e0 0x5606ebdddc00 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606ebd6a580 0x5606ebd78a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606ebd69bc0 0x5606ebd784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffb8d600640 0 client.16656 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 --2- 172.21.15.137:0/1429204434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5606ebd689f0 0x5606ebd79ec0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 >> 172.21.15.137:0/1429204434 conn(0x5606ebd63de0 msgr2=0x5606ebd678d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 shutdown_connections 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 1 -- 172.21.15.137:0/1429204434 wait complete. 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.034+0000 7ffbad774740 20 client.16656 trim_cache size 0 max 16384 2024-03-26T14:07:30.036 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb880a2880 msgr2=0x7ffb880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb880a2880 0x7ffb880a4d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffb94006010 tx=0x7ffb9402c7e0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 msgr2=0x7ffba40fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 0x7ffba40fffb0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000d770 tx=0x7ffb9000dc30 comp rx=0 tx=0).stop 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 shutdown_connections 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ffb880a2880 0x7ffb880a4d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffba4107260 0x7ffba41004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffba4104930 0x7ffba40fffb0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 --2- 172.21.15.137:0/3475426505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffba4102000 0x7ffba4101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 >> 172.21.15.137:0/3475426505 conn(0x7ffba40fba10 msgr2=0x7ffba4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 shutdown_connections 2024-03-26T14:07:30.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.035+0000 7ffba9400640 1 -- 172.21.15.137:0/3475426505 wait complete. 2024-03-26T14:07:30.051 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:07:30.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 -- 172.21.15.137:0/452663658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8104ad0 msgr2=0x7f01e8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 --2- 172.21.15.137:0/452663658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8104ad0 0x7f01e8104f50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009900 tx=0x7f01d002bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 -- 172.21.15.137:0/452663658 shutdown_connections 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 --2- 172.21.15.137:0/452663658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8105490 0x7f01e810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 --2- 172.21.15.137:0/452663658 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8104ad0 0x7f01e8104f50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 --2- 172.21.15.137:0/452663658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01e81038e0 0x7f01e8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 -- 172.21.15.137:0/452663658 >> 172.21.15.137:0/452663658 conn(0x7f01e80fee50 msgr2=0x7f01e8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 -- 172.21.15.137:0/452663658 shutdown_connections 2024-03-26T14:07:30.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.270+0000 7f01ece00640 1 -- 172.21.15.137:0/452663658 wait complete. 2024-03-26T14:07:30.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.271+0000 7f01ece00640 1 Processor -- start 2024-03-26T14:07:30.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.271+0000 7f01ece00640 1 -- start start 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01ece00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01e81038e0 0x7f01e8199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01ece00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8104ad0 0x7f01e819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01ece00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 0x7f01e81a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01ece00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e810f410 con 0x7f01e8105490 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01ece00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e810f290 con 0x7f01e81038e0 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01ece00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e810f590 con 0x7f01e8104ad0 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 0x7f01e81a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 0x7f01e81a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56000/0 (socket says 172.21.15.137:56000) 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01e6800640 1 -- 172.21.15.137:0/844963979 learned_addr learned my addr 172.21.15.137:0/844963979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01e5e00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01e81038e0 0x7f01e8199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.272+0000 7f01dfe00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8104ad0 0x7f01e819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01e6800640 1 -- 172.21.15.137:0/844963979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8104ad0 msgr2=0x7f01e819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01e6800640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8104ad0 0x7f01e819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01e6800640 1 -- 172.21.15.137:0/844963979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01e81038e0 msgr2=0x7f01e8199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01e6800640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01e81038e0 0x7f01e8199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01e6800640 1 -- 172.21.15.137:0/844963979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e81a1730 con 0x7f01e8105490 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01dfe00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8104ad0 0x7f01e819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01e6800640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 0x7f01e81a10c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f01e000eaa0 tx=0x7f01e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01dea00640 1 -- 172.21.15.137:0/844963979 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000cdb0 con 0x7f01e8105490 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01dea00640 1 -- 172.21.15.137:0/844963979 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01e0005020 con 0x7f01e8105490 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e81a1a20 con 0x7f01e8105490 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.273+0000 7f01dea00640 1 -- 172.21.15.137:0/844963979 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00116d0 con 0x7f01e8105490 2024-03-26T14:07:30.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.274+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e81a1f70 con 0x7f01e8105490 2024-03-26T14:07:30.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.277+0000 7f01dea00640 1 -- 172.21.15.137:0/844963979 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f01e00051d0 con 0x7f01e8105490 2024-03-26T14:07:30.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.277+0000 7f01efbdf740 1 Processor -- start 2024-03-26T14:07:30.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.277+0000 7f01efbdf740 1 -- start start 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.277+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa02e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01efbdf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf9662c0 con 0x559bbfa01810 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01efbdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf90ec90 con 0x559bbfa029e0 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01efbdf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf962360 con 0x559bbfa033a0 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa01c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56008/0 (socket says 172.21.15.137:56008) 2024-03-26T14:07:30.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01dfe00640 1 -- 172.21.15.137:0/4165925793 learned_addr learned my addr 172.21.15.137:0/4165925793 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01e5e00640 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa02e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.278+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa09c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dfe00640 1 -- 172.21.15.137:0/4165925793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 msgr2=0x559bbfa09c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa09c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dfe00640 1 -- 172.21.15.137:0/4165925793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 msgr2=0x559bbfa02e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa02e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dfe00640 1 -- 172.21.15.137:0/4165925793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bbf933080 con 0x559bbfa01810 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dea00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01cc0a2800 0x7f01cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa01c30 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f01d002c840 tx=0x7f01d002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630cfc8bc6fe10ab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01dea00640 1 -- 172.21.15.137:0/844963979 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f01e00cefe0 con 0x7f01e8105490 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01d7400640 1 -- 172.21.15.137:0/4165925793 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0048070 con 0x559bbfa01810 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01d7400640 1 -- 172.21.15.137:0/4165925793 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01d002cc60 con 0x559bbfa01810 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01e5e00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01cc0a2800 0x7f01cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.279+0000 7f01d7400640 1 -- 172.21.15.137:0/4165925793 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0004a00 con 0x559bbfa01810 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 -- 172.21.15.137:0/4165925793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 msgr2=0x559bbfa01c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01e5e00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01cc0a2800 0x7f01cc0a4cc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01e8104930 tx=0x7f01d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa01c30 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f01d002c840 tx=0x7f01d002c5d0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 -- 172.21.15.137:0/4165925793 shutdown_connections 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa02e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4165925793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa01c30 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 -- 172.21.15.137:0/4165925793 >> 172.21.15.137:0/4165925793 conn(0x559bbf9fcc00 msgr2=0x559bbf9fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.280+0000 7f01efbdf740 1 -- 172.21.15.137:0/4165925793 shutdown_connections 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.281+0000 7f01efbdf740 1 -- 172.21.15.137:0/4165925793 wait complete. 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.281+0000 7f01efbdf740 1 -- start start 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.281+0000 7f01efbdf740 1 Processor -- start 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa0bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa13000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfa0a350 con 0x559bbfa01810 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbf9220d0 con 0x559bbfa029e0 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559bbfa0a4d0 con 0x559bbfa033a0 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01efbdf740 10 client.0 _ll_register_callbacks cb 0x559bbf926e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56018/0 (socket says 172.21.15.137:56018) 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 -- 172.21.15.137:0/4280150591 learned_addr learned my addr 172.21.15.137:0/4280150591 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa0bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e6800640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa13000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 -- 172.21.15.137:0/4280150591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 msgr2=0x559bbfa13000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa13000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 -- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 msgr2=0x559bbfa0bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa0bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.282+0000 7f01e5e00640 1 -- 172.21.15.137:0/4280150591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bbf67a3f0 con 0x559bbfa01810 2024-03-26T14:07:30.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.283+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa0bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.283+0000 7f01e5e00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0ba80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004630 tx=0x7f01d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd75693318c8f57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.283+0000 7f01cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.283+0000 7f01cbe00640 1 -- 172.21.15.137:0/4280150591 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d801f630 con 0x559bbfa01810 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.283+0000 7f01cbe00640 1 -- 172.21.15.137:0/4280150591 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f01d801f7d0 con 0x559bbfa01810 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.283+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01d002cc60 con 0x559bbfa01810 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01cbe00640 1 -- 172.21.15.137:0/4280150591 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d801f9d0 con 0x559bbfa01810 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01efbdf740 10 client.16668 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559bbf7447d0 con 0x559bbfa01810 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01cbe00640 1 -- 172.21.15.137:0/4280150591 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01d002cc60 con 0x559bbfa01810 2024-03-26T14:07:30.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01cbe00640 1 -- 172.21.15.137:0/4280150591 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f01d8027070 con 0x559bbfa01810 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01efbdf740 10 client.16668 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01efbdf740 10 client.16668 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.284+0000 7f01efbdf740 20 client.16668 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.285+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa175e0 0x559bbfa19a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.285+0000 7f01efbdf740 4 client.16668 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.285+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559bbf973800 con 0x559bbfa175e0 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.285+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa175e0 0x559bbfa19a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.286+0000 7f01dfe00640 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa175e0 0x559bbfa19a60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1915214110203b2b server_cookie=4e7c2eb0bbcb279f in_seq=0 out_seq=0 2024-03-26T14:07:30.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.286+0000 7f01cbe00640 10 client.16668 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.287+0000 7f01cbe00640 1 -- 172.21.15.137:0/4280150591 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559bbf973800 con 0x559bbfa175e0 2024-03-26T14:07:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.287+0000 7f01cbe00640 10 client.16668 handle_command_reply: tid=0 2024-03-26T14:07:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.287+0000 7f01efbdf740 1 client.16668 shutdown 2024-03-26T14:07:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 msgr2=0x559bbfa0ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0ba80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f01d8004630 tx=0x7f01d8004730 comp rx=0 tx=0).stop 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 shutdown_connections 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa175e0 0x559bbfa19a60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa0bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa13000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01cbe00640 0 client.16668 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 --2- 172.21.15.137:0/4280150591 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0ba80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 >> 172.21.15.137:0/4280150591 conn(0x559bbf9fcc00 msgr2=0x559bbf9fbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 shutdown_connections 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 1 -- 172.21.15.137:0/4280150591 wait complete. 2024-03-26T14:07:30.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.288+0000 7f01efbdf740 20 client.16668 trim_cache size 0 max 16384 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 Processor -- start 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 -- start start 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa00570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81ae940 con 0x559bbfa01810 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81ac440 con 0x559bbfa029e0 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.305+0000 7f01efbdf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81aa4c0 con 0x559bbfa033a0 2024-03-26T14:07:30.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56032/0 (socket says 172.21.15.137:56032) 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 -- 172.21.15.137:0/2815295892 learned_addr learned my addr 172.21.15.137:0/2815295892 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01dfe00640 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e6800640 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa00570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 -- 172.21.15.137:0/2815295892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 msgr2=0x559bbfa00570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa00570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 -- 172.21.15.137:0/2815295892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 msgr2=0x559bbfa12f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa12f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 -- 172.21.15.137:0/2815295892 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559bbfa173a0 con 0x559bbfa01810 2024-03-26T14:07:30.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.306+0000 7f01e5e00640 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0bf20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f01d8029dc0 tx=0x7f01d8003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160a0859012c4cfc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.307+0000 7f01d6a00640 1 -- 172.21.15.137:0/2815295892 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d801fb70 con 0x559bbfa01810 2024-03-26T14:07:30.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.307+0000 7f01d6a00640 1 -- 172.21.15.137:0/2815295892 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01d801fd10 con 0x559bbfa01810 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.307+0000 7f01d6a00640 1 -- 172.21.15.137:0/2815295892 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d801db20 con 0x559bbfa01810 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 -- 172.21.15.137:0/2815295892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 msgr2=0x559bbfa0bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0bf20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f01d8029dc0 tx=0x7f01d8003fe0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 -- 172.21.15.137:0/2815295892 shutdown_connections 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa029e0 0x559bbfa12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa033a0 0x559bbfa00570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 --2- 172.21.15.137:0/2815295892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa01810 0x559bbfa0bf20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 -- 172.21.15.137:0/2815295892 >> 172.21.15.137:0/2815295892 conn(0x559bbf9fcc00 msgr2=0x559bbf9ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 -- 172.21.15.137:0/2815295892 shutdown_connections 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 -- 172.21.15.137:0/2815295892 wait complete. 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.308+0000 7f01efbdf740 1 -- start start 2024-03-26T14:07:30.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 Processor -- start 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa01810 0x559bbfa0e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 0x559bbfa0ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa033a0 0x559bbfa11bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81aaac0 con 0x559bbfa029e0 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81abde0 con 0x559bbfa033a0 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81ab120 con 0x559bbfa01810 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01efbdf740 10 client.0 _ll_register_callbacks cb 0x7f01d80044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 0x559bbfa0ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01dfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 0x559bbfa0ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56038/0 (socket says 172.21.15.137:56038) 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01dfe00640 1 -- 172.21.15.137:0/33031599 learned_addr learned my addr 172.21.15.137:0/33031599 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.309+0000 7f01e6800640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa033a0 0x559bbfa11bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01e5e00640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa01810 0x559bbfa0e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01dfe00640 1 -- 172.21.15.137:0/33031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa01810 msgr2=0x559bbfa0e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01dfe00640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa01810 0x559bbfa0e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01dfe00640 1 -- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa033a0 msgr2=0x559bbfa11bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01dfe00640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa033a0 0x559bbfa11bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01dfe00640 1 -- 172.21.15.137:0/33031599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559bbfa121c0 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01dfe00640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 0x559bbfa0ebb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f01d0005c40 tx=0x7f01d002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddab9486e0d0c1a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.310+0000 7f01cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01cbe00640 1 -- 172.21.15.137:0/33031599 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0048040 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01cbe00640 1 -- 172.21.15.137:0/33031599 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f01d0053200 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01d801fd10 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01cbe00640 1 -- 172.21.15.137:0/33031599 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01d00439e0 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01efbdf740 10 client.16674 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559bbfa72f30 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01cbe00640 1 -- 172.21.15.137:0/33031599 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f01d801fd10 con 0x559bbfa029e0 2024-03-26T14:07:30.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.311+0000 7f01cbe00640 1 -- 172.21.15.137:0/33031599 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f01d0053820 con 0x559bbfa029e0 2024-03-26T14:07:30.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01efbdf740 10 client.16674 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:30.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01efbdf740 10 client.16674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:07:30.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01efbdf740 20 client.16674 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:30.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01efbdf740 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa732c0 0x559bbfa75760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01efbdf740 4 client.16674 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:07:30.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559bbf96db50 con 0x559bbfa732c0 2024-03-26T14:07:30.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.312+0000 7f01e5e00640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa732c0 0x559bbfa75760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.313+0000 7f01e5e00640 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa732c0 0x559bbfa75760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84bedc3709de209b server_cookie=38ac5db095c8797 in_seq=0 out_seq=0 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.313+0000 7f01cbe00640 10 client.16674 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01cbe00640 1 -- 172.21.15.137:0/33031599 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559bbf96db50 con 0x559bbfa732c0 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01cbe00640 10 client.16674 handle_command_reply: tid=0 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 client.16674 shutdown 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 msgr2=0x559bbfa0ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 0x559bbfa0ebb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f01d0005c40 tx=0x7f01d002c660 comp rx=0 tx=0).stop 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 shutdown_connections 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559bbfa732c0 0x559bbfa75760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559bbfa033a0 0x559bbfa11bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559bbfa029e0 0x559bbfa0ebb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 --2- 172.21.15.137:0/33031599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559bbfa01810 0x559bbfa0e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01cbe00640 0 client.16674 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 >> 172.21.15.137:0/33031599 conn(0x559bbf9fcc00 msgr2=0x559bbfa0bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 shutdown_connections 2024-03-26T14:07:30.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.314+0000 7f01efbdf740 1 -- 172.21.15.137:0/33031599 wait complete. 2024-03-26T14:07:30.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01efbdf740 20 client.16674 trim_cache size 0 max 16384 2024-03-26T14:07:30.316 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01cc0a2800 msgr2=0x7f01cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01cc0a2800 0x7f01cc0a4cc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f01e8104930 tx=0x7f01d8005f90 comp rx=0 tx=0).stop 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 msgr2=0x7f01e81a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 0x7f01e81a10c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f01e000eaa0 tx=0x7f01e000ef60 comp rx=0 tx=0).stop 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 shutdown_connections 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f01cc0a2800 0x7f01cc0a4cc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f01e8105490 0x7f01e81a10c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f01e8104ad0 0x7f01e819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 --2- 172.21.15.137:0/844963979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01e81038e0 0x7f01e8199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.315+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 >> 172.21.15.137:0/844963979 conn(0x7f01e80fee50 msgr2=0x7f01e8107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.316+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 shutdown_connections 2024-03-26T14:07:30.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.316+0000 7f01ece00640 1 -- 172.21.15.137:0/844963979 wait complete. 2024-03-26T14:07:30.335 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:07:30.335 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 -- 172.21.15.137:0/3395577080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14104ab0 msgr2=0x7fdb14106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 --2- 172.21.15.137:0/3395577080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14104ab0 0x7fdb14106ea0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009900 tx=0x7fdb0002bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 -- 172.21.15.137:0/3395577080 shutdown_connections 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 --2- 172.21.15.137:0/3395577080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb141073e0 0x7fdb141097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 --2- 172.21.15.137:0/3395577080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14104ab0 0x7fdb14106ea0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 --2- 172.21.15.137:0/3395577080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb14102180 0x7fdb14104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 -- 172.21.15.137:0/3395577080 >> 172.21.15.137:0/3395577080 conn(0x7fdb140fbb30 msgr2=0x7fdb140fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 -- 172.21.15.137:0/3395577080 shutdown_connections 2024-03-26T14:07:30.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.556+0000 7fdb19600640 1 -- 172.21.15.137:0/3395577080 wait complete. 2024-03-26T14:07:30.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.557+0000 7fdb19600640 1 Processor -- start 2024-03-26T14:07:30.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 -- start start 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 0x7fdb14199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb14104ab0 0x7fdb1419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb141073e0 0x7fdb141a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1410d1f0 con 0x7fdb14102180 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1410d070 con 0x7fdb14104ab0 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb19600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1410d370 con 0x7fdb141073e0 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb12a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 0x7fdb14199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb12a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 0x7fdb14199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56064/0 (socket says 172.21.15.137:56064) 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb12a00640 1 -- 172.21.15.137:0/1179073387 learned_addr learned my addr 172.21.15.137:0/1179073387 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.558+0000 7fdb12000640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb14104ab0 0x7fdb1419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb13400640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb141073e0 0x7fdb141a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb12a00640 1 -- 172.21.15.137:0/1179073387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb141073e0 msgr2=0x7fdb141a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb12a00640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb141073e0 0x7fdb141a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb12a00640 1 -- 172.21.15.137:0/1179073387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb14104ab0 msgr2=0x7fdb1419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb12a00640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb14104ab0 0x7fdb1419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb12a00640 1 -- 172.21.15.137:0/1179073387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb141a1730 con 0x7fdb14102180 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb12a00640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 0x7fdb14199b10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400d950 tx=0x7fdb0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb0b400640 1 -- 172.21.15.137:0/1179073387 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04017070 con 0x7fdb14102180 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb0b400640 1 -- 172.21.15.137:0/1179073387 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb04009c30 con 0x7fdb14102180 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb0b400640 1 -- 172.21.15.137:0/1179073387 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0400f620 con 0x7fdb14102180 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.559+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb141a1a20 con 0x7fdb14102180 2024-03-26T14:07:30.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.560+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb141a1ef0 con 0x7fdb14102180 2024-03-26T14:07:30.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.563+0000 7fdb1c388740 1 Processor -- start 2024-03-26T14:07:30.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.563+0000 7fdb1c388740 1 -- start start 2024-03-26T14:07:30.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.563+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a21810 0x55b652a21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a229e0 0x55b652a22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a29c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb1c388740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6529862c0 con 0x55b652a233a0 2024-03-26T14:07:30.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb1c388740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65292ec90 con 0x55b652a21810 2024-03-26T14:07:30.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb1c388740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b652982360 con 0x55b652a229e0 2024-03-26T14:07:30.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb0b400640 1 -- 172.21.15.137:0/1179073387 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdb040109a0 con 0x7fdb14102180 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb12000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a29c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb12000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a29c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56080/0 (socket says 172.21.15.137:56080) 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb12000640 1 -- 172.21.15.137:0/3563664716 learned_addr learned my addr 172.21.15.137:0/3563664716 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb13400640 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a21810 0x55b652a21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.564+0000 7fdb12a00640 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a229e0 0x55b652a22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.565+0000 7fdb12000640 1 -- 172.21.15.137:0/3563664716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a229e0 msgr2=0x55b652a22e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.565+0000 7fdb12000640 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a229e0 0x55b652a22e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.565+0000 7fdb12000640 1 -- 172.21.15.137:0/3563664716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a21810 msgr2=0x55b652a21c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.565+0000 7fdb12000640 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a21810 0x55b652a21c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.565+0000 7fdb12000640 1 -- 172.21.15.137:0/3563664716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b652953080 con 0x55b652a233a0 2024-03-26T14:07:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.565+0000 7fdb12000640 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a29c30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005ab0 tx=0x7fdb00005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2e9ee0d4c9e04f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.566+0000 7fdb0b400640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaf80a2880 0x7fdaf80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.566+0000 7fdaff400640 1 -- 172.21.15.137:0/3563664716 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00044020 con 0x55b652a233a0 2024-03-26T14:07:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.566+0000 7fdaff400640 1 -- 172.21.15.137:0/3563664716 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb0002cbe0 con 0x55b652a233a0 2024-03-26T14:07:30.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.566+0000 7fdaff400640 1 -- 172.21.15.137:0/3563664716 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00004920 con 0x55b652a233a0 2024-03-26T14:07:30.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.566+0000 7fdb0b400640 1 -- 172.21.15.137:0/1179073387 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdb040ceab0 con 0x7fdb14102180 2024-03-26T14:07:30.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.566+0000 7fdb13400640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaf80a2880 0x7fdaf80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 -- 172.21.15.137:0/3563664716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 msgr2=0x55b652a29c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a29c30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005ab0 tx=0x7fdb00005dd0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb13400640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaf80a2880 0x7fdaf80a4d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55b652a228e0 tx=0x7fdb0c009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 -- 172.21.15.137:0/3563664716 shutdown_connections 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a29c30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a229e0 0x55b652a22e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3563664716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a21810 0x55b652a21c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 -- 172.21.15.137:0/3563664716 >> 172.21.15.137:0/3563664716 conn(0x55b652a1cc00 msgr2=0x55b652a1d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 -- 172.21.15.137:0/3563664716 shutdown_connections 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 -- 172.21.15.137:0/3563664716 wait complete. 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.567+0000 7fdb1c388740 1 -- start start 2024-03-26T14:07:30.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 Processor -- start 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a2b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 0x55b652a2bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a233a0 0x55b652a32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b652a2a350 con 0x55b652a229e0 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6529420d0 con 0x55b652a233a0 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b652a2a4d0 con 0x55b652a21810 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.568+0000 7fdb1c388740 10 client.0 _ll_register_callbacks cb 0x55b652946e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 0x55b652a2bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 0x55b652a2bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56088/0 (socket says 172.21.15.137:56088) 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 -- 172.21.15.137:0/3519756468 learned_addr learned my addr 172.21.15.137:0/3519756468 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb13400640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a233a0 0x55b652a32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12a00640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a2b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 -- 172.21.15.137:0/3519756468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 msgr2=0x55b652a2b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a2b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 -- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a233a0 msgr2=0x55b652a32f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a233a0 0x55b652a32f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 -- 172.21.15.137:0/3519756468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b65269a3f0 con 0x55b652a229e0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdb12000640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 0x55b652a2bf00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdb0002c170 tx=0x7fdb000058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bd86e8ced2f5cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.569+0000 7fdaf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdaf3e00640 1 -- 172.21.15.137:0/3519756468 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00044030 con 0x55b652a229e0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdaf3e00640 1 -- 172.21.15.137:0/3519756468 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdb00003a00 con 0x55b652a229e0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb0002cbe0 con 0x55b652a229e0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdaf3e00640 1 -- 172.21.15.137:0/3519756468 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0005a050 con 0x55b652a229e0 2024-03-26T14:07:30.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdb1c388740 10 client.16686 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b6527647d0 con 0x55b652a229e0 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.570+0000 7fdaf3e00640 1 -- 172.21.15.137:0/3519756468 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb0002cbe0 con 0x55b652a229e0 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdaf3e00640 1 -- 172.21.15.137:0/3519756468 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdb0004b440 con 0x55b652a229e0 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb1c388740 10 client.16686 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb1c388740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb1c388740 20 client.16686 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a370a0 0x55b652a39490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb1c388740 4 client.16686 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:30.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b652993800 con 0x55b652a370a0 2024-03-26T14:07:30.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.571+0000 7fdb12a00640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a370a0 0x55b652a39490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.572+0000 7fdb12a00640 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a370a0 0x55b652a39490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d26d9e1aebab56b server_cookie=46c1e8522df06ce1 in_seq=0 out_seq=0 2024-03-26T14:07:30.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.572+0000 7fdaf3e00640 10 client.16686 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdaf3e00640 1 -- 172.21.15.137:0/3519756468 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b652993800 con 0x55b652a370a0 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdaf3e00640 10 client.16686 handle_command_reply: tid=0 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 client.16686 shutdown 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 msgr2=0x55b652a2bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 0x55b652a2bf00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdb0002c170 tx=0x7fdb000058b0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 shutdown_connections 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a370a0 0x55b652a39490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a233a0 0x55b652a32f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a229e0 0x55b652a2bf00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdaf3e00640 0 client.16686 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 --2- 172.21.15.137:0/3519756468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a2b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 >> 172.21.15.137:0/3519756468 conn(0x55b652a1cc00 msgr2=0x55b652a1beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 shutdown_connections 2024-03-26T14:07:30.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 1 -- 172.21.15.137:0/3519756468 wait complete. 2024-03-26T14:07:30.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.574+0000 7fdb1c388740 20 client.16686 trim_cache size 0 max 16384 2024-03-26T14:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.589+0000 7fdb1c388740 1 Processor -- start 2024-03-26T14:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 -- start start 2024-03-26T14:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a25c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a32f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a26150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141ae910 con 0x55b652a233a0 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141ac410 con 0x55b652a229e0 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb1c388740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141aa490 con 0x55b652a21810 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb12000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a32f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb12000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a32f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:43162/0 (socket says 172.21.15.137:43162) 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb12000640 1 -- 172.21.15.137:0/2209268635 learned_addr learned my addr 172.21.15.137:0/2209268635 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb13400640 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a26150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.590+0000 7fdb12a00640 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a25c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12000640 1 -- 172.21.15.137:0/2209268635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 msgr2=0x55b652a25c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12000640 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a25c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12000640 1 -- 172.21.15.137:0/2209268635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 msgr2=0x55b652a26150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12000640 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a26150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12000640 1 -- 172.21.15.137:0/2209268635 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b652a21410 con 0x55b652a229e0 2024-03-26T14:07:30.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12a00640 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a25c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdb12000640 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a32f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb00000c00 tx=0x7fdb00005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66d0ec08a158d75 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdafea00640 1 -- 172.21.15.137:0/2209268635 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00044070 con 0x55b652a229e0 2024-03-26T14:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdafea00640 1 -- 172.21.15.137:0/2209268635 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb0004ba70 con 0x55b652a229e0 2024-03-26T14:07:30.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.591+0000 7fdafea00640 1 -- 172.21.15.137:0/2209268635 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00005030 con 0x55b652a229e0 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 -- 172.21.15.137:0/2209268635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 msgr2=0x55b652a32f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a32f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb00000c00 tx=0x7fdb00005a20 comp rx=0 tx=0).stop 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 -- 172.21.15.137:0/2209268635 shutdown_connections 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a32f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a26150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 --2- 172.21.15.137:0/2209268635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a25c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 -- 172.21.15.137:0/2209268635 >> 172.21.15.137:0/2209268635 conn(0x55b652a1cc00 msgr2=0x55b652a206f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 -- 172.21.15.137:0/2209268635 shutdown_connections 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.592+0000 7fdb1c388740 1 -- 172.21.15.137:0/2209268635 wait complete. 2024-03-26T14:07:30.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 -- start start 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 Processor -- start 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a30ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a313e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141aaa90 con 0x55b652a233a0 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141abdb0 con 0x55b652a229e0 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb141ab0f0 con 0x55b652a21810 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.593+0000 7fdb1c388740 10 client.0 _ll_register_callbacks cb 0x7fdb0c0094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a313e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb13400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a313e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:43166/0 (socket says 172.21.15.137:43166) 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 -- 172.21.15.137:0/924441104 learned_addr learned my addr 172.21.15.137:0/924441104 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12a00640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a30ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 -- 172.21.15.137:0/924441104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 msgr2=0x55b652a30ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a30ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 -- 172.21.15.137:0/924441104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 msgr2=0x55b652a323f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a323f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 -- 172.21.15.137:0/924441104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b652a329e0 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12a00640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a30ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdb12000640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a313e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb0004ba40 tx=0x7fdb0004b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a57c2bb66058f8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.594+0000 7fdaf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdaf3e00640 1 -- 172.21.15.137:0/924441104 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00062020 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb0004ba70 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdaf3e00640 1 -- 172.21.15.137:0/924441104 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdb00005460 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdaf3e00640 1 -- 172.21.15.137:0/924441104 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0006a050 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdb1c388740 10 client.36329 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b652a93040 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.595+0000 7fdaf3e00640 1 -- 172.21.15.137:0/924441104 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb0004ba70 con 0x55b652a229e0 2024-03-26T14:07:30.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdaf3e00640 1 -- 172.21.15.137:0/924441104 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdb000047c0 con 0x55b652a229e0 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb1c388740 10 client.36329 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb1c388740 10 client.36329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb1c388740 20 client.36329 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb1c388740 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a93990 0x55b652a95db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb1c388740 4 client.36329 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b652a2b7b0 con 0x55b652a93990 2024-03-26T14:07:30.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.596+0000 7fdb12a00640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a93990 0x55b652a95db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:30.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdaf3e00640 10 client.36329 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:30.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdb12a00640 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a93990 0x55b652a95db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3fe4fabca5b4b8 server_cookie=bf956398d32a70e in_seq=0 out_seq=0 2024-03-26T14:07:30.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdaf3e00640 1 -- 172.21.15.137:0/924441104 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b652a2b7b0 con 0x55b652a93990 2024-03-26T14:07:30.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdaf3e00640 10 client.36329 handle_command_reply: tid=0 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdb1c388740 1 client.36329 shutdown 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 msgr2=0x55b652a313e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdb1c388740 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a313e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb0004ba40 tx=0x7fdb0004b910 comp rx=0 tx=0).stop 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 shutdown_connections 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.597+0000 7fdb1c388740 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b652a93990 0x55b652a95db0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b652a229e0 0x55b652a313e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b652a233a0 0x55b652a323f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 1 --2- 172.21.15.137:0/924441104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b652a21810 0x55b652a30ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdaf3e00640 0 client.36329 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 >> 172.21.15.137:0/924441104 conn(0x55b652a1cc00 msgr2=0x55b652a1b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 shutdown_connections 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 1 -- 172.21.15.137:0/924441104 wait complete. 2024-03-26T14:07:30.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb1c388740 20 client.36329 trim_cache size 0 max 16384 2024-03-26T14:07:30.601 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:07:30.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaf80a2880 msgr2=0x7fdaf80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb19600640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaf80a2880 0x7fdaf80a4d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55b652a228e0 tx=0x7fdb0c009620 comp rx=0 tx=0).stop 2024-03-26T14:07:30.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 msgr2=0x7fdb14199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.598+0000 7fdb19600640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 0x7fdb14199b10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400d950 tx=0x7fdb0400de10 comp rx=0 tx=0).stop 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 shutdown_connections 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaf80a2880 0x7fdaf80a4d40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb14104ab0 0x7fdb1419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb141073e0 0x7fdb141a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 --2- 172.21.15.137:0/1179073387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb14102180 0x7fdb14199b10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 >> 172.21.15.137:0/1179073387 conn(0x7fdb140fbb30 msgr2=0x7fdb14101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 shutdown_connections 2024-03-26T14:07:30.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:30.599+0000 7fdb19600640 1 -- 172.21.15.137:0/1179073387 wait complete. 2024-03-26T14:07:30.612 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:30 smithi137 ceph-mon[31539]: pgmap v612: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:30.612 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:30 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4200605491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:07:30.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:30 smithi179 ceph-mon[35472]: pgmap v612: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:30.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:30 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4200605491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:07:30.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:30 smithi159 ceph-mon[35501]: pgmap v612: 129 pgs: 129 active+clean; 781 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:30.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:30 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4200605491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:07:31.615 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:07:31.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:07:31.615 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:07:31.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 -- 172.21.15.137:0/1860297545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c101150 msgr2=0x7f919c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 --2- 172.21.15.137:0/1860297545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c101150 0x7f919c1015d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f9194009ca0 tx=0x7f919402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 -- 172.21.15.137:0/1860297545 shutdown_connections 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 --2- 172.21.15.137:0/1860297545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f919c1007b0 0x7f919c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 --2- 172.21.15.137:0/1860297545 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c101150 0x7f919c1015d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 --2- 172.21.15.137:0/1860297545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 0x7f919c100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 -- 172.21.15.137:0/1860297545 >> 172.21.15.137:0/1860297545 conn(0x7f919c0fba10 msgr2=0x7f919c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 -- 172.21.15.137:0/1860297545 shutdown_connections 2024-03-26T14:07:31.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.853+0000 7f91a1a00640 1 -- 172.21.15.137:0/1860297545 wait complete. 2024-03-26T14:07:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.854+0000 7f91a1a00640 1 Processor -- start 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.854+0000 7f91a1a00640 1 -- start start 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 0x7f919c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 0x7f919c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f919c101150 0x7f919c1a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c10f2a0 con 0x7f919c1007b0 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c10f120 con 0x7f919c101150 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c10f420 con 0x7f919c0ffe50 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f919b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 0x7f919c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f919b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 0x7f919c199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56134/0 (socket says 172.21.15.137:56134) 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f919b400640 1 -- 172.21.15.137:0/3192488233 learned_addr learned my addr 172.21.15.137:0/3192488233 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f919be00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 0x7f919c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.855+0000 7f91a1000640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f919c101150 0x7f919c1a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919b400640 1 -- 172.21.15.137:0/3192488233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 msgr2=0x7f919c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919b400640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 0x7f919c199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919b400640 1 -- 172.21.15.137:0/3192488233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f919c101150 msgr2=0x7f919c1a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919b400640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f919c101150 0x7f919c1a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919b400640 1 -- 172.21.15.137:0/3192488233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c1a1590 con 0x7f919c1007b0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919be00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 0x7f919c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f919b400640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 0x7f919c199eb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f918800d950 tx=0x7f918800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f9198c00640 1 -- 172.21.15.137:0/3192488233 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188017070 con 0x7f919c1007b0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f9198c00640 1 -- 172.21.15.137:0/3192488233 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9188009c30 con 0x7f919c1007b0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c1a1880 con 0x7f919c1007b0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f9198c00640 1 -- 172.21.15.137:0/3192488233 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800f620 con 0x7f919c1007b0 2024-03-26T14:07:31.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.856+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c1a1e20 con 0x7f919c1007b0 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 Processor -- start 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 -- start start 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a5b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573479b8eb0 con 0x557347a53ca0 2024-03-26T14:07:31.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557347961a60 con 0x557347a532b0 2024-03-26T14:07:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.861+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573479b4f50 con 0x557347a58e10 2024-03-26T14:07:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.862+0000 7f9198c00640 1 -- 172.21.15.137:0/3192488233 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f91880109a0 con 0x7f919c1007b0 2024-03-26T14:07:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.862+0000 7f919be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.862+0000 7f919be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56144/0 (socket says 172.21.15.137:56144) 2024-03-26T14:07:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.862+0000 7f919be00640 1 -- 172.21.15.137:0/2484765137 learned_addr learned my addr 172.21.15.137:0/2484765137 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:31.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.862+0000 7f91a1000640 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.862+0000 7f919b400640 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a5b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f919be00640 1 -- 172.21.15.137:0/2484765137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 msgr2=0x557347a5b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f919be00640 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a5b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f919be00640 1 -- 172.21.15.137:0/2484765137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 msgr2=0x557347a536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f919be00640 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f919be00640 1 -- 172.21.15.137:0/2484765137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573477957e0 con 0x557347a53ca0 2024-03-26T14:07:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f919be00640 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a587a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f918c00bd40 tx=0x7f918c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ffd1feae6aa78e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f9191600640 1 -- 172.21.15.137:0/2484765137 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c015070 con 0x557347a53ca0 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f9198c00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91800a2880 0x7f91800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f9191600640 1 -- 172.21.15.137:0/2484765137 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f918c00be30 con 0x557347a53ca0 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.863+0000 7f9191600640 1 -- 172.21.15.137:0/2484765137 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c011a00 con 0x557347a53ca0 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.864+0000 7f9198c00640 1 -- 172.21.15.137:0/3192488233 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f91880ceae0 con 0x7f919c1007b0 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.864+0000 7f91a1000640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91800a2880 0x7f91800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.864+0000 7f91a5bf9740 1 -- 172.21.15.137:0/2484765137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 msgr2=0x557347a587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.864+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a587a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f918c00bd40 tx=0x7f918c00f730 comp rx=0 tx=0).stop 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.864+0000 7f91a5bf9740 1 -- 172.21.15.137:0/2484765137 shutdown_connections 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a5b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a587a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/2484765137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 -- 172.21.15.137:0/2484765137 >> 172.21.15.137:0/2484765137 conn(0x557347a4e820 msgr2=0x557347a4ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a1000640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91800a2880 0x7f91800a4d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9194006010 tx=0x7f9194005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 -- 172.21.15.137:0/2484765137 shutdown_connections 2024-03-26T14:07:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 -- 172.21.15.137:0/2484765137 wait complete. 2024-03-26T14:07:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 -- start start 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.865+0000 7f91a5bf9740 1 Processor -- start 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a5d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557347a5bdc0 con 0x557347a53ca0 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557347974eb0 con 0x557347a532b0 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557347a5bf40 con 0x557347a58e10 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f91a5bf9740 10 client.0 _ll_register_callbacks cb 0x557347969150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f919b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a5d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f919b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a5d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56148/0 (socket says 172.21.15.137:56148) 2024-03-26T14:07:31.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.866+0000 7f919b400640 1 -- 172.21.15.137:0/341902615 learned_addr learned my addr 172.21.15.137:0/341902615 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919be00640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f91a1000640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919b400640 1 -- 172.21.15.137:0/341902615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 msgr2=0x557347a649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919b400640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919b400640 1 -- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 msgr2=0x557347a5d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919b400640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a5d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919b400640 1 -- 172.21.15.137:0/341902615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573477f5f70 con 0x557347a53ca0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f919b400640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a5d9b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9188097a40 tx=0x7f918809af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1050454fcaac4b83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.867+0000 7f9186000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f9186000640 1 -- 172.21.15.137:0/341902615 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188020040 con 0x557347a53ca0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f9186000640 1 -- 172.21.15.137:0/341902615 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f91880a3380 con 0x557347a53ca0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f918c00be30 con 0x557347a53ca0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f9186000640 1 -- 172.21.15.137:0/341902615 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918809b770 con 0x557347a53ca0 2024-03-26T14:07:31.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f91a5bf9740 10 client.16704 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55734768d170 con 0x557347a53ca0 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f9186000640 1 -- 172.21.15.137:0/341902615 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f918c00be30 con 0x557347a53ca0 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.868+0000 7f9186000640 1 -- 172.21.15.137:0/341902615 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f91880b9b70 con 0x557347a53ca0 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f91a5bf9740 10 client.16704 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f91a5bf9740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f91a5bf9740 20 client.16704 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a691c0 0x557347a6b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f91a5bf9740 4 client.16704 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573479c63f0 con 0x557347a691c0 2024-03-26T14:07:31.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.869+0000 7f919be00640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a691c0 0x557347a6b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.870+0000 7f9186000640 10 client.16704 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.870+0000 7f919be00640 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a691c0 0x557347a6b640 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496c075f0101105f server_cookie=bc5be019767128 in_seq=0 out_seq=0 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.871+0000 7f9186000640 1 -- 172.21.15.137:0/341902615 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5573479c63f0 con 0x557347a691c0 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.871+0000 7f9186000640 10 client.16704 handle_command_reply: tid=0 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.871+0000 7f91a5bf9740 1 client.16704 shutdown 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 msgr2=0x557347a5d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a5d9b0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9188097a40 tx=0x7f918809af40 comp rx=0 tx=0).stop 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 shutdown_connections 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a691c0 0x557347a6b640 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a58e10 0x557347a649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a53ca0 0x557347a5d9b0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/341902615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a5d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f9186000640 0 client.16704 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:31.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 >> 172.21.15.137:0/341902615 conn(0x557347a4e820 msgr2=0x557347a4da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:31.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 shutdown_connections 2024-03-26T14:07:31.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 1 -- 172.21.15.137:0/341902615 wait complete. 2024-03-26T14:07:31.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.872+0000 7f91a5bf9740 20 client.16704 trim_cache size 0 max 16384 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.891+0000 7f91a5bf9740 1 Processor -- start 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.891+0000 7f91a5bf9740 1 -- start start 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a6ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a532b0 0x557347a649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a53ca0 0x557347a70490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1ae800 con 0x557347a4e820 2024-03-26T14:07:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1ac300 con 0x557347a53ca0 2024-03-26T14:07:31.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1aa380 con 0x557347a532b0 2024-03-26T14:07:31.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a6ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a6ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56150/0 (socket says 172.21.15.137:56150) 2024-03-26T14:07:31.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 -- 172.21.15.137:0/3792351980 learned_addr learned my addr 172.21.15.137:0/3792351980 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:31.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a1000640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a53ca0 0x557347a70490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 -- 172.21.15.137:0/3792351980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a532b0 msgr2=0x557347a649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919b400640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a532b0 0x557347a649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a532b0 0x557347a649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 -- 172.21.15.137:0/3792351980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a53ca0 msgr2=0x557347a70490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a53ca0 0x557347a70490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919be00640 1 -- 172.21.15.137:0/3792351980 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557347a4f290 con 0x557347a4e820 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f91a1000640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a53ca0 0x557347a70490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.892+0000 7f919b400640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a532b0 0x557347a649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.893+0000 7f919be00640 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a6ff50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f918c00bc90 tx=0x7f918c00b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7307edf0cf0ed8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.893+0000 7f9190c00640 1 -- 172.21.15.137:0/3792351980 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c015040 con 0x557347a4e820 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.893+0000 7f9190c00640 1 -- 172.21.15.137:0/3792351980 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f918c02d070 con 0x557347a4e820 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.893+0000 7f9190c00640 1 -- 172.21.15.137:0/3792351980 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c005610 con 0x557347a4e820 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3792351980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 msgr2=0x557347a6ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a6ff50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f918c00bc90 tx=0x7f918c00b6f0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3792351980 shutdown_connections 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a53ca0 0x557347a70490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a532b0 0x557347a649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3792351980 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a6ff50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.894+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3792351980 >> 172.21.15.137:0/3792351980 conn(0x557347a526b0 msgr2=0x557347a4da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:31.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.895+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3792351980 shutdown_connections 2024-03-26T14:07:31.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.895+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3792351980 wait complete. 2024-03-26T14:07:31.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.895+0000 7f91a5bf9740 1 -- start start 2024-03-26T14:07:31.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.895+0000 7f91a5bf9740 1 Processor -- start 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a63ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a53ca0 0x557347ac4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1aa980 con 0x557347a4e820 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1abca0 con 0x557347a532b0 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c1aafe0 con 0x557347a53ca0 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a5bf9740 10 client.0 _ll_register_callbacks cb 0x7f918800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56164/0 (socket says 172.21.15.137:56164) 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 -- 172.21.15.137:0/3034853237 learned_addr learned my addr 172.21.15.137:0/3034853237 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919b400640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a63ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f91a1000640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a53ca0 0x557347ac4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 -- 172.21.15.137:0/3034853237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a53ca0 msgr2=0x557347ac4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a53ca0 0x557347ac4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 -- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 msgr2=0x557347a63ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a63ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919be00640 1 -- 172.21.15.137:0/3034853237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557347a55750 con 0x557347a4e820 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.896+0000 7f919b400640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a63ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f919be00640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a635a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f918c006010 tx=0x7f918c0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db8264472d0cf7f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f9186000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f9186000640 1 -- 172.21.15.137:0/3034853237 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c015040 con 0x557347a4e820 2024-03-26T14:07:31.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f9186000640 1 -- 172.21.15.137:0/3034853237 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f918c0055a0 con 0x557347a4e820 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f918c02d070 con 0x557347a4e820 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f9186000640 1 -- 172.21.15.137:0/3034853237 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f918c03e050 con 0x557347a4e820 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f91a5bf9740 10 client.16710 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.897+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557347ac5f00 con 0x557347a4e820 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f9186000640 1 -- 172.21.15.137:0/3034853237 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f918c02d070 con 0x557347a4e820 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f9186000640 1 -- 172.21.15.137:0/3034853237 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f918c004960 con 0x557347a4e820 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f91a5bf9740 10 client.16710 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f91a5bf9740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f91a5bf9740 20 client.16710 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a58e10 0x557347ac8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f91a5bf9740 4 client.16710 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:07:31.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5573479c0740 con 0x557347a58e10 2024-03-26T14:07:31.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.898+0000 7f919b400640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a58e10 0x557347ac8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:31.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.899+0000 7f9186000640 10 client.16710 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:31.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.899+0000 7f919b400640 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a58e10 0x557347ac8570 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7bf4033fb25f4c server_cookie=3db0fd77e6de6f44 in_seq=0 out_seq=0 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f9186000640 1 -- 172.21.15.137:0/3034853237 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5573479c0740 con 0x557347a58e10 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f9186000640 10 client.16710 handle_command_reply: tid=0 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 client.16710 shutdown 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 msgr2=0x557347a635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a635a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f918c006010 tx=0x7f918c0050d0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 shutdown_connections 2024-03-26T14:07:31.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557347a58e10 0x557347ac8570 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557347a532b0 0x557347a63ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557347a53ca0 0x557347ac4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f9186000640 0 client.16710 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 --2- 172.21.15.137:0/3034853237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557347a4e820 0x557347a635a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 >> 172.21.15.137:0/3034853237 conn(0x557347a526b0 msgr2=0x557347a4d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 shutdown_connections 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.902+0000 7f91a5bf9740 1 -- 172.21.15.137:0/3034853237 wait complete. 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a5bf9740 20 client.16710 trim_cache size 0 max 16384 2024-03-26T14:07:31.905 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:07:31.906 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:07:31.906 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "0799d288-1a3c-4b1a-bd9c-5b9d8c8ee1c4", 2024-03-26T14:07:31.906 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:07:31.906 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:07:31.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91800a2880 msgr2=0x7f91800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91800a2880 0x7f91800a4d40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9194006010 tx=0x7f9194005c30 comp rx=0 tx=0).stop 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 msgr2=0x7f919c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 0x7f919c199eb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f918800d950 tx=0x7f918800de10 comp rx=0 tx=0).stop 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 shutdown_connections 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f91800a2880 0x7f91800a4d40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f919c101150 0x7f919c1a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f919c1007b0 0x7f919c199eb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 --2- 172.21.15.137:0/3192488233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919c0ffe50 0x7f919c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.903+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 >> 172.21.15.137:0/3192488233 conn(0x7f919c0fba10 msgr2=0x7f919c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.904+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 shutdown_connections 2024-03-26T14:07:31.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:31.904+0000 7f91a1a00640 1 -- 172.21.15.137:0/3192488233 wait complete. 2024-03-26T14:07:31.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:07:31.921 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:07:32.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 -- 172.21.15.137:0/701373563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1072e0 msgr2=0x7fe71c1096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 --2- 172.21.15.137:0/701373563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1072e0 0x7fe71c1096d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe718009ca0 tx=0x7fe71802c100 comp rx=0 tx=0).stop 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 -- 172.21.15.137:0/701373563 shutdown_connections 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 --2- 172.21.15.137:0/701373563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe71c1049b0 0x7fe71c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 --2- 172.21.15.137:0/701373563 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1072e0 0x7fe71c1096d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 --2- 172.21.15.137:0/701373563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 0x7fe71c104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 -- 172.21.15.137:0/701373563 >> 172.21.15.137:0/701373563 conn(0x7fe71c0fba10 msgr2=0x7fe71c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 -- 172.21.15.137:0/701373563 shutdown_connections 2024-03-26T14:07:32.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.138+0000 7fe723200640 1 -- 172.21.15.137:0/701373563 wait complete. 2024-03-26T14:07:32.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 Processor -- start 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 -- start start 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 0x7fe71c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 0x7fe71c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe71c1072e0 0x7fe71c1a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c10d0c0 con 0x7fe71c1049b0 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c10cf40 con 0x7fe71c1072e0 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.139+0000 7fe723200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c10d240 con 0x7fe71c102080 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 0x7fe71c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 0x7fe71c199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56180/0 (socket says 172.21.15.137:56180) 2024-03-26T14:07:32.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 -- 172.21.15.137:0/1036679903 learned_addr learned my addr 172.21.15.137:0/1036679903 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721e00640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 0x7fe71c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe722800640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe71c1072e0 0x7fe71c1a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 -- 172.21.15.137:0/1036679903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 msgr2=0x7fe71c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 0x7fe71c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 -- 172.21.15.137:0/1036679903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe71c1072e0 msgr2=0x7fe71c1a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe71c1072e0 0x7fe71c1a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721400640 1 -- 172.21.15.137:0/1036679903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c1a1570 con 0x7fe71c1049b0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.140+0000 7fe721e00640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 0x7fe71c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.141+0000 7fe721400640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 0x7fe71c199e90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e9c0 tx=0x7fe70c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.141+0000 7fe716a00640 1 -- 172.21.15.137:0/1036679903 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00cd80 con 0x7fe71c1049b0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.141+0000 7fe716a00640 1 -- 172.21.15.137:0/1036679903 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe70c0054d0 con 0x7fe71c1049b0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.141+0000 7fe716a00640 1 -- 172.21.15.137:0/1036679903 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c010430 con 0x7fe71c1049b0 2024-03-26T14:07:32.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.141+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c1a1860 con 0x7fe71c1049b0 2024-03-26T14:07:32.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.141+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c1a1d60 con 0x7fe71c1049b0 2024-03-26T14:07:32.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.145+0000 7fe7275ad740 1 Processor -- start 2024-03-26T14:07:32.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 -- start start 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 0x55a0e066e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe716a00640 1 -- 172.21.15.137:0/1036679903 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe70c004fd0 con 0x7fe71c1049b0 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066ea40 0x55a0e0673540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e0676040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e05d3250 con 0x55a0e066e050 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e057ba30 con 0x55a0e0673bb0 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe7275ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e05cf2f0 con 0x55a0e066ea40 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe722800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 0x55a0e066e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe722800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 0x55a0e066e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56196/0 (socket says 172.21.15.137:56196) 2024-03-26T14:07:32.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.146+0000 7fe722800640 1 -- 172.21.15.137:0/289623712 learned_addr learned my addr 172.21.15.137:0/289623712 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe721400640 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e0676040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe721e00640 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066ea40 0x55a0e0673540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe722800640 1 -- 172.21.15.137:0/289623712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066ea40 msgr2=0x55a0e0673540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe722800640 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066ea40 0x55a0e0673540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe722800640 1 -- 172.21.15.137:0/289623712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 msgr2=0x55a0e0676040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe722800640 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e0676040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe722800640 1 -- 172.21.15.137:0/289623712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0e0566690 con 0x55a0e066e050 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.147+0000 7fe722800640 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 0x55a0e066e470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe718005e30 tx=0x7fe718005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901a19c182bbd5ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.148+0000 7fe716a00640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7040a2800 0x7fe7040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.148+0000 7fe70b000640 1 -- 172.21.15.137:0/289623712 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718043020 con 0x55a0e066e050 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.148+0000 7fe70b000640 1 -- 172.21.15.137:0/289623712 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe718032dc0 con 0x55a0e066e050 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.148+0000 7fe70b000640 1 -- 172.21.15.137:0/289623712 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718039040 con 0x55a0e066e050 2024-03-26T14:07:32.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.148+0000 7fe716a00640 1 -- 172.21.15.137:0/1036679903 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe70c0cf330 con 0x7fe71c1049b0 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.148+0000 7fe721e00640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7040a2800 0x7fe7040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 -- 172.21.15.137:0/289623712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 msgr2=0x55a0e066e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 0x55a0e066e470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe718005e30 tx=0x7fe718005f80 comp rx=0 tx=0).stop 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 -- 172.21.15.137:0/289623712 shutdown_connections 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e0676040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066ea40 0x55a0e0673540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 --2- 172.21.15.137:0/289623712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066e050 0x55a0e066e470 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 -- 172.21.15.137:0/289623712 >> 172.21.15.137:0/289623712 conn(0x55a0e0636200 msgr2=0x55a0e06699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 -- 172.21.15.137:0/289623712 shutdown_connections 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe7275ad740 1 -- 172.21.15.137:0/289623712 wait complete. 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.149+0000 7fe721e00640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7040a2800 0x7fe7040a4cc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55a0e066ee40 tx=0x7fe71000c2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 -- start start 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 Processor -- start 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e0678030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e0678570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e0676b00 con 0x55a0e066ea40 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e058ee80 con 0x55a0e0673bb0 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.150+0000 7fe7275ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e0676c80 con 0x55a0e066e050 2024-03-26T14:07:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe7275ad740 10 client.0 _ll_register_callbacks cb 0x55a0e05a1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e0678570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e0678570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56198/0 (socket says 172.21.15.137:56198) 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 -- 172.21.15.137:0/999602511 learned_addr learned my addr 172.21.15.137:0/999602511 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe722800640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721e00640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e0678030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 -- 172.21.15.137:0/999602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 msgr2=0x55a0e0678030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e0678030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 -- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 msgr2=0x55a0e067f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 -- 172.21.15.137:0/999602511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0e03b17d0 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe721400640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e0678570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe70c09b890 tx=0x7fe70c0d28e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803941eedd4c6982 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.151+0000 7fe6ff400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe6ff400640 1 -- 172.21.15.137:0/999602511 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c09b6d0 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe6ff400640 1 -- 172.21.15.137:0/999602511 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe70c0ccb70 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe718032dc0 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe6ff400640 1 -- 172.21.15.137:0/999602511 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70c0b4050 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe7275ad740 10 client.16722 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a0e02e73f0 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.152+0000 7fe6ff400640 1 -- 172.21.15.137:0/999602511 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe718032dc0 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe6ff400640 1 -- 172.21.15.137:0/999602511 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe70c0afce0 con 0x55a0e066ea40 2024-03-26T14:07:32.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe7275ad740 10 client.16722 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe7275ad740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe7275ad740 20 client.16722 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe7275ad740 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e0683e90 0x55a0e06862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe7275ad740 4 client.16722 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0e05e0790 con 0x55a0e0683e90 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.153+0000 7fe721e00640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e0683e90 0x55a0e06862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.154+0000 7fe721e00640 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e0683e90 0x55a0e06862f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1db89de310296e67 server_cookie=f5706bfd79c5bdac in_seq=0 out_seq=0 2024-03-26T14:07:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.154+0000 7fe6ff400640 10 client.16722 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.155+0000 7fe6ff400640 1 -- 172.21.15.137:0/999602511 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a0e05e0790 con 0x55a0e0683e90 2024-03-26T14:07:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.155+0000 7fe6ff400640 10 client.16722 handle_command_reply: tid=0 2024-03-26T14:07:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.155+0000 7fe7275ad740 1 client.16722 shutdown 2024-03-26T14:07:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 msgr2=0x55a0e0678570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e0678570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe70c09b890 tx=0x7fe70c0d28e0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 shutdown_connections 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e0683e90 0x55a0e06862f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e0678570 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 --2- 172.21.15.137:0/999602511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e0678030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 >> 172.21.15.137:0/999602511 conn(0x55a0e0636200 msgr2=0x55a0e0668bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe6ff400640 0 client.16722 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 shutdown_connections 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 1 -- 172.21.15.137:0/999602511 wait complete. 2024-03-26T14:07:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.156+0000 7fe7275ad740 20 client.16722 trim_cache size 0 max 16384 2024-03-26T14:07:32.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 Processor -- start 2024-03-26T14:07:32.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 -- start start 2024-03-26T14:07:32.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e066cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e066ea40 0x55a0e067f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 0x55a0e066d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1ae7b0 con 0x55a0e0673bb0 2024-03-26T14:07:32.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1ac2b0 con 0x55a0e066ea40 2024-03-26T14:07:32.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.177+0000 7fe7275ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1aa330 con 0x55a0e066e050 2024-03-26T14:07:32.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 0x55a0e066d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe721400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e066ea40 0x55a0e067f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 0x55a0e066d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56206/0 (socket says 172.21.15.137:56206) 2024-03-26T14:07:32.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 -- 172.21.15.137:0/2366273570 learned_addr learned my addr 172.21.15.137:0/2366273570 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe721e00640 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e066cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 -- 172.21.15.137:0/2366273570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 msgr2=0x55a0e066cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e066cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 -- 172.21.15.137:0/2366273570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e066ea40 msgr2=0x55a0e067f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e066ea40 0x55a0e067f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.178+0000 7fe722800640 1 -- 172.21.15.137:0/2366273570 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0e02a70f0 con 0x55a0e0673bb0 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.179+0000 7fe722800640 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 0x55a0e066d140 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe718004af0 tx=0x7fe718009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9481510d99ddba46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.179+0000 7fe70a600640 1 -- 172.21.15.137:0/2366273570 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71804c070 con 0x55a0e0673bb0 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.179+0000 7fe70a600640 1 -- 172.21.15.137:0/2366273570 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe718032e80 con 0x55a0e0673bb0 2024-03-26T14:07:32.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.179+0000 7fe70a600640 1 -- 172.21.15.137:0/2366273570 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718043070 con 0x55a0e0673bb0 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 -- 172.21.15.137:0/2366273570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 msgr2=0x55a0e066d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 0x55a0e066d140 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe718004af0 tx=0x7fe718009360 comp rx=0 tx=0).stop 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 -- 172.21.15.137:0/2366273570 shutdown_connections 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e066ea40 0x55a0e067f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e0673bb0 0x55a0e066d140 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 --2- 172.21.15.137:0/2366273570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e066cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 -- 172.21.15.137:0/2366273570 >> 172.21.15.137:0/2366273570 conn(0x55a0e0636200 msgr2=0x55a0e066fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 -- 172.21.15.137:0/2366273570 shutdown_connections 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 -- 172.21.15.137:0/2366273570 wait complete. 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.180+0000 7fe7275ad740 1 -- start start 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 Processor -- start 2024-03-26T14:07:32.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e067d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e067d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1aa930 con 0x55a0e066ea40 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1abc50 con 0x55a0e0673bb0 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1aaf90 con 0x55a0e066e050 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe7275ad740 10 client.0 _ll_register_callbacks cb 0x7fe70c0d2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe721400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e067d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe721400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e067d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56212/0 (socket says 172.21.15.137:56212) 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe721400640 1 -- 172.21.15.137:0/4138977692 learned_addr learned my addr 172.21.15.137:0/4138977692 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.181+0000 7fe722800640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721e00640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e067d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721400640 1 -- 172.21.15.137:0/4138977692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 msgr2=0x55a0e067d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721400640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e067d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721400640 1 -- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 msgr2=0x55a0e067e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721400640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721400640 1 -- 172.21.15.137:0/4138977692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0e067eda0 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe721400640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e067d810 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0097b0 tx=0x7fe70c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1f659bf015980a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe6ff400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe6ff400640 1 -- 172.21.15.137:0/4138977692 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0b8020 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe6ff400640 1 -- 172.21.15.137:0/4138977692 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe70c0a3e20 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.182+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe718032e80 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe6ff400640 1 -- 172.21.15.137:0/4138977692 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70c09b260 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 10 client.16728 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a0e06dfdb0 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe6ff400640 1 -- 172.21.15.137:0/4138977692 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe718032e80 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe6ff400640 1 -- 172.21.15.137:0/4138977692 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe70c0afce0 con 0x55a0e066ea40 2024-03-26T14:07:32.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 10 client.16728 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:32.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:32.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 20 client.16728 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:32.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e06e06a0 0x55a0e06e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 4 client.16728 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:07:32.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.183+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a0e05daae0 con 0x55a0e06e06a0 2024-03-26T14:07:32.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.184+0000 7fe721e00640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e06e06a0 0x55a0e06e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.184+0000 7fe721e00640 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e06e06a0 0x55a0e06e2ac0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700422bffefc06a5 server_cookie=a47715b2fa8994a2 in_seq=0 out_seq=0 2024-03-26T14:07:32.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.184+0000 7fe6ff400640 10 client.16728 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe6ff400640 1 -- 172.21.15.137:0/4138977692 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a0e05daae0 con 0x55a0e06e06a0 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe6ff400640 10 client.16728 handle_command_reply: tid=0 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 client.16728 shutdown 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 msgr2=0x55a0e067d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e067d810 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe70c0097b0 tx=0x7fe70c005880 comp rx=0 tx=0).stop 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 shutdown_connections 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a0e06e06a0 0x55a0e06e2ac0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0e0673bb0 0x55a0e067e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe6ff400640 0 client.16728 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a0e066ea40 0x55a0e067d810 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 --2- 172.21.15.137:0/4138977692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a0e066e050 0x55a0e067d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 >> 172.21.15.137:0/4138977692 conn(0x55a0e0636200 msgr2=0x55a0e066f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 shutdown_connections 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 1 -- 172.21.15.137:0/4138977692 wait complete. 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.185+0000 7fe7275ad740 20 client.16728 trim_cache size 0 max 16384 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:07:32.190 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7040a2800 msgr2=0x7fe7040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7040a2800 0x7fe7040a4cc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55a0e066ee40 tx=0x7fe71000c2d0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 msgr2=0x7fe71c199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 0x7fe71c199e90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e9c0 tx=0x7fe70c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 shutdown_connections 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7040a2800 0x7fe7040a4cc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe71c1072e0 0x7fe71c1a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe71c1049b0 0x7fe71c199e90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 --2- 172.21.15.137:0/1036679903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe71c102080 0x7fe71c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 >> 172.21.15.137:0/1036679903 conn(0x7fe71c0fba10 msgr2=0x7fe71c1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 shutdown_connections 2024-03-26T14:07:32.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.186+0000 7fe723200640 1 -- 172.21.15.137:0/1036679903 wait complete. 2024-03-26T14:07:32.205 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:07:32.205 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:07:32.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 -- 172.21.15.137:0/1851595528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c1038e0 msgr2=0x7f360c103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 --2- 172.21.15.137:0/1851595528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c1038e0 0x7f360c103d00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3600009900 tx=0x7f360002bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:32.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 -- 172.21.15.137:0/1851595528 shutdown_connections 2024-03-26T14:07:32.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 --2- 172.21.15.137:0/1851595528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f360c105490 0x7f360c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 --2- 172.21.15.137:0/1851595528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 0x7f360c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 --2- 172.21.15.137:0/1851595528 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c1038e0 0x7f360c103d00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 -- 172.21.15.137:0/1851595528 >> 172.21.15.137:0/1851595528 conn(0x7f360c0fee90 msgr2=0x7f360c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 -- 172.21.15.137:0/1851595528 shutdown_connections 2024-03-26T14:07:32.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.417+0000 7f3613600640 1 -- 172.21.15.137:0/1851595528 wait complete. 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.418+0000 7f3613600640 1 Processor -- start 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.418+0000 7f3613600640 1 -- start start 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.418+0000 7f3613600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f360c1038e0 0x7f360c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3613600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 0x7f360c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3613600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 0x7f360c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3613600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c10f430 con 0x7f360c105490 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3613600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c10f2b0 con 0x7f360c1038e0 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3613600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c10f5b0 con 0x7f360c104ad0 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 0x7f360c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 0x7f360c1a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56226/0 (socket says 172.21.15.137:56226) 2024-03-26T14:07:32.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 -- 172.21.15.137:0/2143429262 learned_addr learned my addr 172.21.15.137:0/2143429262 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3610c00640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f360c1038e0 0x7f360c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f360be00640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 0x7f360c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 -- 172.21.15.137:0/2143429262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 msgr2=0x7f360c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 0x7f360c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 -- 172.21.15.137:0/2143429262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f360c1038e0 msgr2=0x7f360c199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f360c1038e0 0x7f360c199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f3611600640 1 -- 172.21.15.137:0/2143429262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c1a1750 con 0x7f360c105490 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.419+0000 7f360be00640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 0x7f360c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:32.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.420+0000 7f3611600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 0x7f360c1a10e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f360400eb10 tx=0x7f360400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.420+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604004b20 con 0x7f360c105490 2024-03-26T14:07:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.420+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3604004cc0 con 0x7f360c105490 2024-03-26T14:07:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.420+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c1a1a40 con 0x7f360c105490 2024-03-26T14:07:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.420+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604010690 con 0x7f360c105490 2024-03-26T14:07:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.420+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c1a1f90 con 0x7f360c105490 2024-03-26T14:07:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.422+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c10ac70 con 0x7f360c105490 2024-03-26T14:07:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.425+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3604005000 con 0x7f360c105490 2024-03-26T14:07:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.426+0000 7f3609600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35f40a2880 0x7f35f40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.426+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f36040cf590 con 0x7f360c105490 2024-03-26T14:07:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.426+0000 7f3610c00640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35f40a2880 0x7f35f40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.427+0000 7f3610c00640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35f40a2880 0x7f35f40a4d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f360002c240 tx=0x7f360002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.429+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3604053740 con 0x7f360c105490 2024-03-26T14:07:32.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:32 smithi137 ceph-mon[31539]: pgmap v613: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:32.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:32 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:07:32.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:32 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:32.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:32 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:07:32.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:32 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:32.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:32 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:07:32.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:32 smithi179 ceph-mon[35472]: pgmap v613: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:32 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:07:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:32 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:32 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:07:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:32 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:32 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:07:32.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.696+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f360c02d070 con 0x7f360c105490 2024-03-26T14:07:32.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.698+0000 7f3609600640 1 -- 172.21.15.137:0/2143429262 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f360409c260 con 0x7f360c105490 2024-03-26T14:07:32.698 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:07:32.699 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:07:32.699 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:07:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.700+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35f40a2880 msgr2=0x7f35f40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.700+0000 7f3613600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35f40a2880 0x7f35f40a4d40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f360002c240 tx=0x7f360002c7e0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.700+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 msgr2=0x7f360c1a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 0x7f360c1a10e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f360400eb10 tx=0x7f360400c2d0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 shutdown_connections 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35f40a2880 0x7f35f40a4d40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f360c105490 0x7f360c1a10e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f360c104ad0 0x7f360c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 --2- 172.21.15.137:0/2143429262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f360c1038e0 0x7f360c199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 >> 172.21.15.137:0/2143429262 conn(0x7f360c0fee90 msgr2=0x7f360c107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 shutdown_connections 2024-03-26T14:07:32.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.701+0000 7f3613600640 1 -- 172.21.15.137:0/2143429262 wait complete. 2024-03-26T14:07:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:32 smithi159 ceph-mon[35501]: pgmap v613: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:32 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:07:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:32 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:32 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:07:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:32 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:32.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:32 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:07:32.719 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:07:32.719 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:07:32.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.946+0000 7fac1b200640 1 -- 172.21.15.137:0/240116985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac14104990 msgr2=0x7fac14104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.946+0000 7fac1b200640 1 --2- 172.21.15.137:0/240116985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac14104990 0x7fac14104e10 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fac04009900 tx=0x7fac0402bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:32.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 -- 172.21.15.137:0/240116985 shutdown_connections 2024-03-26T14:07:32.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 --2- 172.21.15.137:0/240116985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14105350 0x7fac1410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 --2- 172.21.15.137:0/240116985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac14104990 0x7fac14104e10 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 --2- 172.21.15.137:0/240116985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac141037a0 0x7fac14103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 -- 172.21.15.137:0/240116985 >> 172.21.15.137:0/240116985 conn(0x7fac140fed10 msgr2=0x7fac14101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 -- 172.21.15.137:0/240116985 shutdown_connections 2024-03-26T14:07:32.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.947+0000 7fac1b200640 1 -- 172.21.15.137:0/240116985 wait complete. 2024-03-26T14:07:32.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 Processor -- start 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 -- start start 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 0x7fac141999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14104990 0x7fac14199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac14105350 0x7fac141a0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1410f2d0 con 0x7fac141037a0 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1410f150 con 0x7fac14104990 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac1b200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1410f450 con 0x7fac14105350 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.948+0000 7fac19e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 0x7fac141999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 0x7fac141999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56238/0 (socket says 172.21.15.137:56238) 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 -- 172.21.15.137:0/2227796002 learned_addr learned my addr 172.21.15.137:0/2227796002 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19400640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14104990 0x7fac14199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac1a800640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac14105350 0x7fac141a0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 -- 172.21.15.137:0/2227796002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac14105350 msgr2=0x7fac141a0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac14105350 0x7fac141a0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 -- 172.21.15.137:0/2227796002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14104990 msgr2=0x7fac14199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14104990 0x7fac14199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 -- 172.21.15.137:0/2227796002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac141a15d0 con 0x7fac141037a0 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac1a800640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac14105350 0x7fac141a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19400640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14104990 0x7fac14199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:32.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.949+0000 7fac19e00640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 0x7fac141999b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fac0800d950 tx=0x7fac0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.950+0000 7fac0ea00640 1 -- 172.21.15.137:0/2227796002 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac08017070 con 0x7fac141037a0 2024-03-26T14:07:32.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.950+0000 7fac0ea00640 1 -- 172.21.15.137:0/2227796002 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac08009c30 con 0x7fac141037a0 2024-03-26T14:07:32.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.950+0000 7fac0ea00640 1 -- 172.21.15.137:0/2227796002 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800f620 con 0x7fac141037a0 2024-03-26T14:07:32.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.950+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac141a18c0 con 0x7fac141037a0 2024-03-26T14:07:32.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.950+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac141a1dc0 con 0x7fac141037a0 2024-03-26T14:07:32.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.954+0000 7fac0ea00640 1 -- 172.21.15.137:0/2227796002 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fac0801a070 con 0x7fac141037a0 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.954+0000 7fac1f46b740 1 Processor -- start 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.954+0000 7fac1f46b740 1 -- start start 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 0x55fa24be7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be8bc0 0x55fa24be9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24befe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1f46b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa24b4c2c0 con 0x55fa24be79f0 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1f46b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa24af4c90 con 0x55fa24be8bc0 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1f46b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa24b48360 con 0x55fa24be9580 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 0x55fa24be7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 0x55fa24be7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56252/0 (socket says 172.21.15.137:56252) 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac1a800640 1 -- 172.21.15.137:0/1837475039 learned_addr learned my addr 172.21.15.137:0/1837475039 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac19e00640 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be8bc0 0x55fa24be9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac0ea00640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabfc0a2800 0x7fabfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.955+0000 7fac19400640 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24befe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac0ea00640 1 -- 172.21.15.137:0/2227796002 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fac080ce950 con 0x7fac141037a0 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 -- 172.21.15.137:0/1837475039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 msgr2=0x55fa24befe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24befe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 -- 172.21.15.137:0/1837475039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be8bc0 msgr2=0x55fa24be9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be8bc0 0x55fa24be9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 -- 172.21.15.137:0/1837475039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa24b19080 con 0x55fa24be79f0 2024-03-26T14:07:32.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabfc0a2800 0x7fabfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac1a800640 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 0x55fa24be7e10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fac1000f390 tx=0x7fac1000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187a691eb72d9ab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.956+0000 7fac03000640 1 -- 172.21.15.137:0/1837475039 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1001a610 con 0x55fa24be79f0 2024-03-26T14:07:32.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.957+0000 7fac03000640 1 -- 172.21.15.137:0/1837475039 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac1001ac60 con 0x55fa24be79f0 2024-03-26T14:07:32.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.957+0000 7fac03000640 1 -- 172.21.15.137:0/1837475039 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000d9b0 con 0x55fa24be79f0 2024-03-26T14:07:32.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.957+0000 7fac1f46b740 1 -- 172.21.15.137:0/1837475039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 msgr2=0x55fa24be7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.957+0000 7fac1a800640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabfc0a2800 0x7fabfc0a4cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac10020000 tx=0x7fac1001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.957+0000 7fac1f46b740 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 0x55fa24be7e10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fac1000f390 tx=0x7fac1000f850 comp rx=0 tx=0).stop 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 -- 172.21.15.137:0/1837475039 shutdown_connections 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be8bc0 0x55fa24be9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24befe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 --2- 172.21.15.137:0/1837475039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be79f0 0x55fa24be7e10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 -- 172.21.15.137:0/1837475039 >> 172.21.15.137:0/1837475039 conn(0x55fa24be2de0 msgr2=0x55fa24be31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 -- 172.21.15.137:0/1837475039 shutdown_connections 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 -- 172.21.15.137:0/1837475039 wait complete. 2024-03-26T14:07:32.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 -- start start 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.958+0000 7fac1f46b740 1 Processor -- start 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bf91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa24bf0590 con 0x55fa24be8bc0 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa24b080d0 con 0x55fa24be79f0 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa24bf0710 con 0x55fa24be9580 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1f46b740 10 client.0 _ll_register_callbacks cb 0x55fa24b0ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac19400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac19400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56256/0 (socket says 172.21.15.137:56256) 2024-03-26T14:07:32.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac19400640 1 -- 172.21.15.137:0/2390838486 learned_addr learned my addr 172.21.15.137:0/2390838486 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac19e00640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.959+0000 7fac1a800640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bf91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19400640 1 -- 172.21.15.137:0/2390838486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 msgr2=0x55fa24bf91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19400640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bf91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19400640 1 -- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 msgr2=0x55fa24bf1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19400640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19400640 1 -- 172.21.15.137:0/2390838486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa248603f0 con 0x55fa24be8bc0 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19e00640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fac19400640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf21d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fac0402c170 tx=0x7fac040042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b989ffed3fe7d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fabf7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:32.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fabf7400640 1 -- 172.21.15.137:0/2390838486 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04005450 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.960+0000 7fabf7400640 1 -- 172.21.15.137:0/2390838486 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fac040046c0 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.961+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac1001ac60 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.961+0000 7fabf7400640 1 -- 172.21.15.137:0/2390838486 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac04051050 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.961+0000 7fac1f46b740 10 client.16746 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.961+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fa2492a7d0 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.961+0000 7fabf7400640 1 -- 172.21.15.137:0/2390838486 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac1001ac60 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.961+0000 7fabf7400640 1 -- 172.21.15.137:0/2390838486 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fac040037b0 con 0x55fa24be8bc0 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac1f46b740 10 client.16746 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac1f46b740 10 client.16746 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac1f46b740 20 client.16746 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24bfd6d0 0x55fa24bffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac1f46b740 4 client.16746 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:32.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa24b59800 con 0x55fa24bfd6d0 2024-03-26T14:07:32.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.962+0000 7fac19e00640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24bfd6d0 0x55fa24bffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.963+0000 7fac19e00640 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24bfd6d0 0x55fa24bffb70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5c7f5837e6f8a18 server_cookie=ae76292a653b65ee in_seq=0 out_seq=0 2024-03-26T14:07:32.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.963+0000 7fabf7400640 10 client.16746 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fabf7400640 1 -- 172.21.15.137:0/2390838486 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fa24b59800 con 0x55fa24bfd6d0 2024-03-26T14:07:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fabf7400640 10 client.16746 handle_command_reply: tid=0 2024-03-26T14:07:32.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 client.16746 shutdown 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 msgr2=0x55fa24bf21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf21d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fac0402c170 tx=0x7fac040042d0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 shutdown_connections 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24bfd6d0 0x55fa24bffb70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bf91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf21d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2390838486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 >> 172.21.15.137:0/2390838486 conn(0x55fa24be2de0 msgr2=0x55fa24be2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fabf7400640 0 client.16746 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:32.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 shutdown_connections 2024-03-26T14:07:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.965+0000 7fac1f46b740 1 -- 172.21.15.137:0/2390838486 wait complete. 2024-03-26T14:07:32.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.966+0000 7fac1f46b740 20 client.16746 trim_cache size 0 max 16384 2024-03-26T14:07:32.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 Processor -- start 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 -- start start 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bec330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141ae6d0 con 0x55fa24be8bc0 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141ac1d0 con 0x55fa24be79f0 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac1f46b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141aa250 con 0x55fa24be9580 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac19400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac19400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56266/0 (socket says 172.21.15.137:56266) 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.981+0000 7fac19400640 1 -- 172.21.15.137:0/2617045762 learned_addr learned my addr 172.21.15.137:0/2617045762 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19e00640 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac1a800640 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bec330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19400640 1 -- 172.21.15.137:0/2617045762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 msgr2=0x55fa24bec330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19400640 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bec330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19400640 1 -- 172.21.15.137:0/2617045762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 msgr2=0x55fa24bebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19400640 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19400640 1 -- 172.21.15.137:0/2617045762 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa24be3850 con 0x55fa24be8bc0 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac19400640 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf9160 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fac04003c00 tx=0x7fac0402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4783ef332f8b8db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.982+0000 7fac02600640 1 -- 172.21.15.137:0/2617045762 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04059020 con 0x55fa24be8bc0 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac02600640 1 -- 172.21.15.137:0/2617045762 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac04009520 con 0x55fa24be8bc0 2024-03-26T14:07:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac02600640 1 -- 172.21.15.137:0/2617045762 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04004920 con 0x55fa24be8bc0 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 -- 172.21.15.137:0/2617045762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 msgr2=0x55fa24bf9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf9160 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fac04003c00 tx=0x7fac0402bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 -- 172.21.15.137:0/2617045762 shutdown_connections 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be9580 0x55fa24bec330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be8bc0 0x55fa24bf9160 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2617045762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.983+0000 7fac1f46b740 1 -- 172.21.15.137:0/2617045762 >> 172.21.15.137:0/2617045762 conn(0x55fa24be2de0 msgr2=0x55fa24be68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.984+0000 7fac1f46b740 1 -- 172.21.15.137:0/2617045762 shutdown_connections 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.984+0000 7fac1f46b740 1 -- 172.21.15.137:0/2617045762 wait complete. 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.984+0000 7fac1f46b740 1 -- start start 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.984+0000 7fac1f46b740 1 Processor -- start 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be8bc0 0x55fa24bf6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 0x55fa24bf7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141aa850 con 0x55fa24be9580 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141abb70 con 0x55fa24be79f0 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac141aaeb0 con 0x55fa24be8bc0 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1f46b740 10 client.0 _ll_register_callbacks cb 0x7fac04009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 0x55fa24bf7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 0x55fa24bf7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56280/0 (socket says 172.21.15.137:56280) 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 -- 172.21.15.137:0/2422130444 learned_addr learned my addr 172.21.15.137:0/2422130444 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac19e00640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac19400640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be8bc0 0x55fa24bf6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 -- 172.21.15.137:0/2422130444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be8bc0 msgr2=0x55fa24bf6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be8bc0 0x55fa24bf6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 -- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 msgr2=0x55fa24bf6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.985+0000 7fac1a800640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fac1a800640 1 -- 172.21.15.137:0/2422130444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa24bf8440 con 0x55fa24be9580 2024-03-26T14:07:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fac19e00640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fac1a800640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 0x55fa24bf7e50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55fa24be8ac0 tx=0x7fac10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f1f44907f34672 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fabf7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fabf7400640 1 -- 172.21.15.137:0/2422130444 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000de50 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac04009520 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fabf7400640 1 -- 172.21.15.137:0/2422130444 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fac1000d3a0 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.986+0000 7fabf7400640 1 -- 172.21.15.137:0/2422130444 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1003e050 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 10 client.16752 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fa24c59100 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fabf7400640 1 -- 172.21.15.137:0/2422130444 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac04009520 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fabf7400640 1 -- 172.21.15.137:0/2422130444 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fac10030960 con 0x55fa24be9580 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 10 client.16752 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 10 client.16752 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:07:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 20 client.16752 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24c59820 0x55fa24c5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 4 client.16752 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:07:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.987+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa24b53b50 con 0x55fa24c59820 2024-03-26T14:07:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.988+0000 7fac19e00640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24c59820 0x55fa24c5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:32.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.988+0000 7fac19e00640 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24c59820 0x55fa24c5bc40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71648248b5ba7f03 server_cookie=e9e49dcc98d904aa in_seq=0 out_seq=0 2024-03-26T14:07:32.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.988+0000 7fabf7400640 10 client.16752 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:32.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fabf7400640 1 -- 172.21.15.137:0/2422130444 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa24b53b50 con 0x55fa24c59820 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fabf7400640 10 client.16752 handle_command_reply: tid=0 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 client.16752 shutdown 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 msgr2=0x55fa24bf7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 0x55fa24bf7e50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x55fa24be8ac0 tx=0x7fac10004a40 comp rx=0 tx=0).stop 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 shutdown_connections 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55fa24c59820 0x55fa24c5bc40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fa24be9580 0x55fa24bf7e50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa24be8bc0 0x55fa24bf6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 --2- 172.21.15.137:0/2422130444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fa24be79f0 0x55fa24bf6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 >> 172.21.15.137:0/2422130444 conn(0x55fa24be2de0 msgr2=0x55fa24be18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fabf7400640 0 client.16752 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:07:32.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 shutdown_connections 2024-03-26T14:07:32.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 1 -- 172.21.15.137:0/2422130444 wait complete. 2024-03-26T14:07:32.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.989+0000 7fac1f46b740 20 client.16752 trim_cache size 0 max 16384 2024-03-26T14:07:32.995 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabfc0a2800 msgr2=0x7fabfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabfc0a2800 0x7fabfc0a4cc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac10020000 tx=0x7fac1001d000 comp rx=0 tx=0).stop 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 msgr2=0x7fac141999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 0x7fac141999b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fac0800d950 tx=0x7fac0800de10 comp rx=0 tx=0).stop 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 shutdown_connections 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fabfc0a2800 0x7fabfc0a4cc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac14104990 0x7fac14199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac14105350 0x7fac141a0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 --2- 172.21.15.137:0/2227796002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac141037a0 0x7fac141999b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 >> 172.21.15.137:0/2227796002 conn(0x7fac140fed10 msgr2=0x7fac14107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 shutdown_connections 2024-03-26T14:07:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:32.990+0000 7fac1b200640 1 -- 172.21.15.137:0/2227796002 wait complete. 2024-03-26T14:07:33.011 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:07:33.012 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:07:33.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 -- 172.21.15.137:0/1996542389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 msgr2=0x7fd5b81019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 --2- 172.21.15.137:0/1996542389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81019c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009900 tx=0x7fd5ac02bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:33.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 -- 172.21.15.137:0/1996542389 shutdown_connections 2024-03-26T14:07:33.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 --2- 172.21.15.137:0/1996542389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b8102790 0x7fd5b8102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 --2- 172.21.15.137:0/1996542389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5b8103150 0x7fd5b81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 --2- 172.21.15.137:0/1996542389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81019c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 -- 172.21.15.137:0/1996542389 >> 172.21.15.137:0/1996542389 conn(0x7fd5b80fcb10 msgr2=0x7fd5b80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 -- 172.21.15.137:0/1996542389 shutdown_connections 2024-03-26T14:07:33.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.227+0000 7fd5be000640 1 -- 172.21.15.137:0/1996542389 wait complete. 2024-03-26T14:07:33.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.228+0000 7fd5be000640 1 Processor -- start 2024-03-26T14:07:33.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.228+0000 7fd5be000640 1 -- start start 2024-03-26T14:07:33.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5be000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5be000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5b8102790 0x7fd5b8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5be000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b8103150 0x7fd5b81a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5be000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b810d0c0 con 0x7fd5b81015a0 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5be000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b810cf40 con 0x7fd5b8103150 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5be000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b810d240 con 0x7fd5b8102790 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5bcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5bcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56310/0 (socket says 172.21.15.137:56310) 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5bcc00640 1 -- 172.21.15.137:0/1155156918 learned_addr learned my addr 172.21.15.137:0/1155156918 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5b8102790 0x7fd5b8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.229+0000 7fd5bd600640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b8103150 0x7fd5b81a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5bcc00640 1 -- 172.21.15.137:0/1155156918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5b8102790 msgr2=0x7fd5b8199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5b8102790 0x7fd5b8199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5bcc00640 1 -- 172.21.15.137:0/1155156918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b8103150 msgr2=0x7fd5b81a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b8103150 0x7fd5b81a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5bcc00640 1 -- 172.21.15.137:0/1155156918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b81a15f0 con 0x7fd5b81015a0 2024-03-26T14:07:33.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81999d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5b5600640 1 -- 172.21.15.137:0/1155156918 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac043020 con 0x7fd5b81015a0 2024-03-26T14:07:33.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5b5600640 1 -- 172.21.15.137:0/1155156918 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5ac004aa0 con 0x7fd5b81015a0 2024-03-26T14:07:33.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5b5600640 1 -- 172.21.15.137:0/1155156918 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac003d00 con 0x7fd5b81015a0 2024-03-26T14:07:33.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b81a1880 con 0x7fd5b81015a0 2024-03-26T14:07:33.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.230+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b81a1d80 con 0x7fd5b81015a0 2024-03-26T14:07:33.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 Processor -- start 2024-03-26T14:07:33.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 -- start start 2024-03-26T14:07:33.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5b5600640 1 -- 172.21.15.137:0/1155156918 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd5ac046020 con 0x7fd5b81015a0 2024-03-26T14:07:33.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e901e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e903040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e909e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e8662c0 con 0x55737e902bc0 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e80ec90 con 0x55737e903580 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.234+0000 7fd5c2408740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e862360 con 0x55737e9019f0 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e903040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e903040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56312/0 (socket says 172.21.15.137:56312) 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 -- 172.21.15.137:0/3903861560 learned_addr learned my addr 172.21.15.137:0/3903861560 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bd600640 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e901e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e909e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 -- 172.21.15.137:0/3903861560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 msgr2=0x55737e901e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e901e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 -- 172.21.15.137:0/3903861560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 msgr2=0x55737e909e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e909e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.235+0000 7fd5bcc00640 1 -- 172.21.15.137:0/3903861560 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737e833080 con 0x55737e902bc0 2024-03-26T14:07:33.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e903040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0f9400 tx=0x7fd5ac0f94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4597e908dae579 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5b5600640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd5a00a2880 0x7fd5a00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5b5600640 1 -- 172.21.15.137:0/1155156918 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd5ac0f1430 con 0x7fd5b81015a0 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5bd600640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd5a00a2880 0x7fd5a00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5a9c00640 1 -- 172.21.15.137:0/3903861560 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0b9ca0 con 0x55737e902bc0 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5a9c00640 1 -- 172.21.15.137:0/3903861560 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5ac0fe030 con 0x55737e902bc0 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.236+0000 7fd5a9c00640 1 -- 172.21.15.137:0/3903861560 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0bd360 con 0x55737e902bc0 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 -- 172.21.15.137:0/3903861560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 msgr2=0x55737e903040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e903040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac0f9400 tx=0x7fd5ac0f94f0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5bd600640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd5a00a2880 0x7fd5a00a4d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55737e902ac0 tx=0x7fd5b0009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 -- 172.21.15.137:0/3903861560 shutdown_connections 2024-03-26T14:07:33.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e909e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e903040 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3903861560 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e901e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 -- 172.21.15.137:0/3903861560 >> 172.21.15.137:0/3903861560 conn(0x55737e8fcde0 msgr2=0x55737e8fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 -- 172.21.15.137:0/3903861560 shutdown_connections 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.237+0000 7fd5c2408740 1 -- 172.21.15.137:0/3903861560 wait complete. 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.238+0000 7fd5c2408740 1 -- start start 2024-03-26T14:07:33.239 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.238+0000 7fd5c2408740 1 Processor -- start 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.238+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e90b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.238+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e902bc0 0x55737e90bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 0x55737e912f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5c2408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e90a350 con 0x55737e903580 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5c2408740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e8220d0 con 0x55737e902bc0 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5c2408740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737e90a4d0 con 0x55737e9019f0 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5c2408740 10 client.0 _ll_register_callbacks cb 0x55737e826e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:33.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 0x55737e912f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 0x55737e912f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56326/0 (socket says 172.21.15.137:56326) 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 -- 172.21.15.137:0/3932881447 learned_addr learned my addr 172.21.15.137:0/3932881447 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e902bc0 0x55737e90bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e90b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 -- 172.21.15.137:0/3932881447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 msgr2=0x55737e90b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e90b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 -- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e902bc0 msgr2=0x55737e90bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e902bc0 0x55737e90bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.239+0000 7fd5bd600640 1 -- 172.21.15.137:0/3932881447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55737e57a3f0 con 0x55737e903580 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd5bd600640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 0x55737e912f50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0012780 tx=0x7fd5b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541f9f12ff8c1161 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd59e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd59e000640 1 -- 172.21.15.137:0/3932881447 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0004d20 con 0x55737e903580 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd59e000640 1 -- 172.21.15.137:0/3932881447 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd5b000e8d0 con 0x55737e903580 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5ac0fe030 con 0x55737e903580 2024-03-26T14:07:33.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd59e000640 1 -- 172.21.15.137:0/3932881447 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b002f050 con 0x55737e903580 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd5c2408740 10 client.16764 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.240+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55737e6447d0 con 0x55737e903580 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd59e000640 1 -- 172.21.15.137:0/3932881447 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5ac0fe030 con 0x55737e903580 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd59e000640 1 -- 172.21.15.137:0/3932881447 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd5b000f040 con 0x55737e903580 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd5c2408740 10 client.16764 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd5c2408740 10 client.16764 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd5c2408740 20 client.16764 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9175a0 0x55737e919990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd5c2408740 4 client.16764 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:33.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.241+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55737e873800 con 0x55737e9175a0 2024-03-26T14:07:33.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.242+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9175a0 0x55737e919990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.243+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9175a0 0x55737e919990 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8a08f6c67d1bb9e server_cookie=8026da9f6d1cddf4 in_seq=0 out_seq=0 2024-03-26T14:07:33.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.243+0000 7fd59e000640 10 client.16764 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:33.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd59e000640 1 -- 172.21.15.137:0/3932881447 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55737e873800 con 0x55737e9175a0 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd59e000640 10 client.16764 handle_command_reply: tid=0 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 client.16764 shutdown 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 msgr2=0x55737e912f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 0x55737e912f50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0012780 tx=0x7fd5b0004a40 comp rx=0 tx=0).stop 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 shutdown_connections 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9175a0 0x55737e919990 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e902bc0 0x55737e90bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e903580 0x55737e912f50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3932881447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e90b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd59e000640 0 client.16764 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.245+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 >> 172.21.15.137:0/3932881447 conn(0x55737e8fcde0 msgr2=0x55737e8fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.246+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 shutdown_connections 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.246+0000 7fd5c2408740 1 -- 172.21.15.137:0/3932881447 wait complete. 2024-03-26T14:07:33.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.246+0000 7fd5c2408740 20 client.16764 trim_cache size 0 max 16384 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.264+0000 7fd5c2408740 1 Processor -- start 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.264+0000 7fd5c2408740 1 -- start start 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e9019f0 0x55737e900750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e912f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e903580 0x55737e91e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5c2408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81ae750 con 0x55737e902bc0 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5c2408740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81ac250 con 0x55737e9019f0 2024-03-26T14:07:33.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5c2408740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81aa2d0 con 0x55737e903580 2024-03-26T14:07:33.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e912f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e912f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56334/0 (socket says 172.21.15.137:56334) 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5b7e00640 1 -- 172.21.15.137:0/4016784239 learned_addr learned my addr 172.21.15.137:0/4016784239 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e9019f0 0x55737e900750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.265+0000 7fd5bd600640 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e903580 0x55737e91e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5b7e00640 1 -- 172.21.15.137:0/4016784239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e903580 msgr2=0x55737e91e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e903580 0x55737e91e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5b7e00640 1 -- 172.21.15.137:0/4016784239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e9019f0 msgr2=0x55737e900750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e9019f0 0x55737e900750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5b7e00640 1 -- 172.21.15.137:0/4016784239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737e902640 con 0x55737e902bc0 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5bd600640 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e903580 0x55737e91e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e912f10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400bc30 tx=0x7fd5a4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f646723cbcd60b63 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5a9200640 1 -- 172.21.15.137:0/4016784239 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a400bd20 con 0x55737e902bc0 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5a9200640 1 -- 172.21.15.137:0/4016784239 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5a40035f0 con 0x55737e902bc0 2024-03-26T14:07:33.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.266+0000 7fd5a9200640 1 -- 172.21.15.137:0/4016784239 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a401c680 con 0x55737e902bc0 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 -- 172.21.15.137:0/4016784239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 msgr2=0x55737e912f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e912f10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400bc30 tx=0x7fd5a4012820 comp rx=0 tx=0).stop 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 -- 172.21.15.137:0/4016784239 shutdown_connections 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e903580 0x55737e91e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e912f10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 --2- 172.21.15.137:0/4016784239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e9019f0 0x55737e900750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 -- 172.21.15.137:0/4016784239 >> 172.21.15.137:0/4016784239 conn(0x55737e8fcde0 msgr2=0x55737e900040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 -- 172.21.15.137:0/4016784239 shutdown_connections 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.267+0000 7fd5c2408740 1 -- 172.21.15.137:0/4016784239 wait complete. 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 -- start start 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 Processor -- start 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e912ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e9112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e9117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81aa8d0 con 0x55737e902bc0 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81abbf0 con 0x55737e903580 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.268+0000 7fd5c2408740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81aaf30 con 0x55737e9019f0 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5c2408740 10 client.0 _ll_register_callbacks cb 0x7fd5b0009750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e9112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e9112b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56344/0 (socket says 172.21.15.137:56344) 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 -- 172.21.15.137:0/3883271265 learned_addr learned my addr 172.21.15.137:0/3883271265 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5bd600640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e9117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e912ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 -- 172.21.15.137:0/3883271265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 msgr2=0x55737e912ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e912ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 -- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 msgr2=0x55737e9117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e9117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.269+0000 7fd5b7e00640 1 -- 172.21.15.137:0/3883271265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55737e911ee0 con 0x55737e902bc0 2024-03-26T14:07:33.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd5b7e00640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e9112b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40039d0 tx=0x7fd5a4003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4478c1b397a6cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd59e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:33.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd59e000640 1 -- 172.21.15.137:0/3883271265 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a402b060 con 0x55737e902bc0 2024-03-26T14:07:33.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd59e000640 1 -- 172.21.15.137:0/3883271265 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd5a4018040 con 0x55737e902bc0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5a40035f0 con 0x55737e902bc0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd59e000640 1 -- 172.21.15.137:0/3883271265 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a4032050 con 0x55737e902bc0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd5c2408740 10 client.16770 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.270+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55737e973d10 con 0x55737e902bc0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd59e000640 1 -- 172.21.15.137:0/3883271265 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5a40035f0 con 0x55737e902bc0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd59e000640 1 -- 172.21.15.137:0/3883271265 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd5a401c9f0 con 0x55737e902bc0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd5c2408740 10 client.16770 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd5c2408740 10 client.16770 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd5c2408740 20 client.16770 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9745f0 0x55737e976a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd5c2408740 4 client.16770 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.271+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55737e86db50 con 0x55737e9745f0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.272+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9745f0 0x55737e976a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5bcc00640 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9745f0 0x55737e976a10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abe6ae78bbfaa17 server_cookie=d6d30a4c97498c75 in_seq=0 out_seq=0 2024-03-26T14:07:33.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd59e000640 10 client.16770 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd59e000640 1 -- 172.21.15.137:0/3883271265 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55737e86db50 con 0x55737e9745f0 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd59e000640 10 client.16770 handle_command_reply: tid=0 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 client.16770 shutdown 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 msgr2=0x55737e9112b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e9112b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40039d0 tx=0x7fd5a4003a00 comp rx=0 tx=0).stop 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 shutdown_connections 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55737e9745f0 0x55737e976a10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55737e903580 0x55737e9117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55737e902bc0 0x55737e9112b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 --2- 172.21.15.137:0/3883271265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55737e9019f0 0x55737e912ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd59e000640 0 client.16770 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 >> 172.21.15.137:0/3883271265 conn(0x55737e8fcde0 msgr2=0x55737e9008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.273+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 shutdown_connections 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5c2408740 1 -- 172.21.15.137:0/3883271265 wait complete. 2024-03-26T14:07:33.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5c2408740 20 client.16770 trim_cache size 0 max 16384 2024-03-26T14:07:33.276 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:07:33.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd5a00a2880 msgr2=0x7fd5a00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd5a00a2880 0x7fd5a00a4d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x55737e902ac0 tx=0x7fd5b0009580 comp rx=0 tx=0).stop 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 msgr2=0x7fd5b81999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81999d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac000c00 tx=0x7fd5ac004270 comp rx=0 tx=0).stop 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 shutdown_connections 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd5a00a2880 0x7fd5a00a4d40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b8103150 0x7fd5b81a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5b8102790 0x7fd5b8199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 --2- 172.21.15.137:0/1155156918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5b81015a0 0x7fd5b81999d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.274+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 >> 172.21.15.137:0/1155156918 conn(0x7fd5b80fcb10 msgr2=0x7fd5b80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.275+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 shutdown_connections 2024-03-26T14:07:33.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.275+0000 7fd5be000640 1 -- 172.21.15.137:0/1155156918 wait complete. 2024-03-26T14:07:33.293 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:07:33.293 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:07:33.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 -- 172.21.15.137:0/4012508483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0072ce0 msgr2=0x7f56b0073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 --2- 172.21.15.137:0/4012508483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0072ce0 0x7f56b0073160 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f569c009900 tx=0x7f569c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:07:33.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 -- 172.21.15.137:0/4012508483 shutdown_connections 2024-03-26T14:07:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 --2- 172.21.15.137:0/4012508483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56b010fa70 0x7f56b0111ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 --2- 172.21.15.137:0/4012508483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0072ce0 0x7f56b0073160 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 --2- 172.21.15.137:0/4012508483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56b0109340 0x7f56b0072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 -- 172.21.15.137:0/4012508483 >> 172.21.15.137:0/4012508483 conn(0x7f56b006e580 msgr2=0x7f56b00709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 -- 172.21.15.137:0/4012508483 shutdown_connections 2024-03-26T14:07:33.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.500+0000 7f56afe00640 1 -- 172.21.15.137:0/4012508483 wait complete. 2024-03-26T14:07:33.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.501+0000 7f56afe00640 1 Processor -- start 2024-03-26T14:07:33.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.501+0000 7f56afe00640 1 -- start start 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56afe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56b0072ce0 0x7f56b01a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 0x7f56b01a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56b010fa70 0x7f56b01a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56afe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0115900 con 0x7f56b0109340 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56afe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0115780 con 0x7f56b010fa70 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56afe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0115a80 con 0x7f56b0072ce0 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 0x7f56b01a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 0x7f56b01a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56360/0 (socket says 172.21.15.137:56360) 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56ae000640 1 -- 172.21.15.137:0/38297913 learned_addr learned my addr 172.21.15.137:0/38297913 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56aea00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56b0072ce0 0x7f56b01a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.502+0000 7f56af400640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56b010fa70 0x7f56b01a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56ae000640 1 -- 172.21.15.137:0/38297913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56b0072ce0 msgr2=0x7f56b01a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56ae000640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56b0072ce0 0x7f56b01a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56ae000640 1 -- 172.21.15.137:0/38297913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56b010fa70 msgr2=0x7f56b01a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56ae000640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56b010fa70 0x7f56b01a97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56ae000640 1 -- 172.21.15.137:0/38297913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b01a9e10 con 0x7f56b0109340 2024-03-26T14:07:33.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56ae000640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 0x7f56b01a2730 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56a7400640 1 -- 172.21.15.137:0/38297913 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c043020 con 0x7f56b0109340 2024-03-26T14:07:33.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56a7400640 1 -- 172.21.15.137:0/38297913 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569c004aa0 con 0x7f56b0109340 2024-03-26T14:07:33.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56a7400640 1 -- 172.21.15.137:0/38297913 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c003d70 con 0x7f56b0109340 2024-03-26T14:07:33.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.503+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01aa0a0 con 0x7f56b0109340 2024-03-26T14:07:33.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.504+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b01aa640 con 0x7f56b0109340 2024-03-26T14:07:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.508+0000 7f56b8644740 1 Processor -- start 2024-03-26T14:07:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.508+0000 7f56b8644740 1 -- start start 2024-03-26T14:07:33.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.508+0000 7f56a7400640 1 -- 172.21.15.137:0/38297913 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f569c046020 con 0x7f56b0109340 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.508+0000 7f56b8644740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfc4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.508+0000 7f56b8644740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfc6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.508+0000 7f56b8644740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfcce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56b8644740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930cf292c0 con 0x55930cfc6580 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56b8644740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930ced1c90 con 0x55930cfc5bc0 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56b8644740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930cf25360 con 0x55930cfc49f0 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfcce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfc6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfcce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56376/0 (socket says 172.21.15.137:56376) 2024-03-26T14:07:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56ae000640 1 -- 172.21.15.137:0/564989936 learned_addr learned my addr 172.21.15.137:0/564989936 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.509+0000 7f56af400640 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfc4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56ae000640 1 -- 172.21.15.137:0/564989936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 msgr2=0x55930cfc4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56ae000640 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfc4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56ae000640 1 -- 172.21.15.137:0/564989936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 msgr2=0x55930cfc6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56ae000640 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfc6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56ae000640 1 -- 172.21.15.137:0/564989936 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55930cef6080 con 0x55930cfc6580 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56a7400640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56940a2880 0x7f56940a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56ae000640 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfcce10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f569c0f9190 tx=0x7f569c0f9280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d40f46531d902b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56a7400640 1 -- 172.21.15.137:0/38297913 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f569c0f1110 con 0x7f56b0109340 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.510+0000 7f56af400640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56940a2880 0x7f56940a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.511+0000 7f569be00640 1 -- 172.21.15.137:0/564989936 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0b9aa0 con 0x55930cfc6580 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.511+0000 7f569be00640 1 -- 172.21.15.137:0/564989936 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569c004c50 con 0x55930cfc6580 2024-03-26T14:07:33.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.511+0000 7f569be00640 1 -- 172.21.15.137:0/564989936 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0f4c80 con 0x55930cfc6580 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 -- 172.21.15.137:0/564989936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 msgr2=0x55930cfcce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56af400640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56940a2880 0x7f56940a4d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55930cfc5ac0 tx=0x7f56a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfcce10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f569c0f9190 tx=0x7f569c0f9280 comp rx=0 tx=0).stop 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 -- 172.21.15.137:0/564989936 shutdown_connections 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfc6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfcce10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 --2- 172.21.15.137:0/564989936 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfc4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 -- 172.21.15.137:0/564989936 >> 172.21.15.137:0/564989936 conn(0x55930cfbfde0 msgr2=0x55930cfc01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 -- 172.21.15.137:0/564989936 shutdown_connections 2024-03-26T14:07:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 -- 172.21.15.137:0/564989936 wait complete. 2024-03-26T14:07:33.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.512+0000 7f56b8644740 1 -- start start 2024-03-26T14:07:33.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 Processor -- start 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfcee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930cf292c0 con 0x55930cfc6580 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930cee50d0 con 0x55930cfc5bc0 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930cfcd350 con 0x55930cfc49f0 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56b8644740 10 client.0 _ll_register_callbacks cb 0x55930cee9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56382/0 (socket says 172.21.15.137:56382) 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.513+0000 7f56af400640 1 -- 172.21.15.137:0/2398783384 learned_addr learned my addr 172.21.15.137:0/2398783384 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56ae000640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfcee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56aea00640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56af400640 1 -- 172.21.15.137:0/2398783384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 msgr2=0x55930cfce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56af400640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56af400640 1 -- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 msgr2=0x55930cfcee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56af400640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfcee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56af400640 1 -- 172.21.15.137:0/2398783384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55930cc3d3f0 con 0x55930cfc6580 2024-03-26T14:07:33.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f56af400640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd5e60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f56a801f5a0 tx=0x7f56a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b75ec76a823170 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.514+0000 7f5698c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:33.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f5698c00640 1 -- 172.21.15.137:0/2398783384 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a801c640 con 0x55930cfc6580 2024-03-26T14:07:33.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f5698c00640 1 -- 172.21.15.137:0/2398783384 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f56a8017030 con 0x55930cfc6580 2024-03-26T14:07:33.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f569c004c50 con 0x55930cfc6580 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f5698c00640 1 -- 172.21.15.137:0/2398783384 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56a801c7e0 con 0x55930cfc6580 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f56b8644740 10 client.16782 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55930cd077d0 con 0x55930cfc6580 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.515+0000 7f5698c00640 1 -- 172.21.15.137:0/2398783384 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569c004c50 con 0x55930cfc6580 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f5698c00640 1 -- 172.21.15.137:0/2398783384 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f56a8028b50 con 0x55930cfc6580 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56b8644740 10 client.16782 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56b8644740 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:33.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56b8644740 20 client.16782 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:33.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56b8644740 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930cfda4e0 0x55930cfdc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56b8644740 4 client.16782 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:33.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55930cf36800 con 0x55930cfda4e0 2024-03-26T14:07:33.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.516+0000 7f56aea00640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930cfda4e0 0x55930cfdc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.517+0000 7f56aea00640 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930cfda4e0 0x55930cfdc920 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6ae0a7b1db4864 server_cookie=c0a4fcd11d135d82 in_seq=0 out_seq=0 2024-03-26T14:07:33.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.517+0000 7f5698c00640 10 client.16782 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.518+0000 7f5698c00640 1 -- 172.21.15.137:0/2398783384 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55930cf36800 con 0x55930cfda4e0 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.518+0000 7f5698c00640 10 client.16782 handle_command_reply: tid=0 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 client.16782 shutdown 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 msgr2=0x55930cfd5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd5e60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f56a801f5a0 tx=0x7f56a8004ab0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 shutdown_connections 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930cfda4e0 0x55930cfdc920 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfcee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd5e60 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 --2- 172.21.15.137:0/2398783384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f5698c00640 0 client.16782 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:33.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 >> 172.21.15.137:0/2398783384 conn(0x55930cfbfde0 msgr2=0x55930cfbf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 shutdown_connections 2024-03-26T14:07:33.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 1 -- 172.21.15.137:0/2398783384 wait complete. 2024-03-26T14:07:33.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.519+0000 7f56b8644740 20 client.16782 trim_cache size 0 max 16384 2024-03-26T14:07:33.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 Processor -- start 2024-03-26T14:07:33.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 -- start start 2024-03-26T14:07:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfbaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 0x55930cfd5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc6580 0x55930cfc8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01b7040 con 0x55930cfc5bc0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01b4b40 con 0x55930cfc6580 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56b8644740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01b2bc0 con 0x55930cfc49f0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.537+0000 7f56ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 0x55930cfd5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 0x55930cfd5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56384/0 (socket says 172.21.15.137:56384) 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 -- 172.21.15.137:0/3389502756 learned_addr learned my addr 172.21.15.137:0/3389502756 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56af400640 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc6580 0x55930cfc8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56aea00640 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfbaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 -- 172.21.15.137:0/3389502756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 msgr2=0x55930cfbaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfbaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 -- 172.21.15.137:0/3389502756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc6580 msgr2=0x55930cfc8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc6580 0x55930cfc8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 -- 172.21.15.137:0/3389502756 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55930cfcd490 con 0x55930cfc5bc0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56af400640 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc6580 0x55930cfc8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.538+0000 7f56ae000640 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 0x55930cfd5e20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f569c0bd500 tx=0x7f569c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1690d43aa2ab6e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.539+0000 7f569b400640 1 -- 172.21.15.137:0/3389502756 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0044e0 con 0x55930cfc5bc0 2024-03-26T14:07:33.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.539+0000 7f569b400640 1 -- 172.21.15.137:0/3389502756 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569c004c50 con 0x55930cfc5bc0 2024-03-26T14:07:33.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.539+0000 7f569b400640 1 -- 172.21.15.137:0/3389502756 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c0bd160 con 0x55930cfc5bc0 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.539+0000 7f56b8644740 1 -- 172.21.15.137:0/3389502756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 msgr2=0x55930cfd5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.539+0000 7f56b8644740 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 0x55930cfd5e20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f569c0bd500 tx=0x7f569c009560 comp rx=0 tx=0).stop 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 -- 172.21.15.137:0/3389502756 shutdown_connections 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc6580 0x55930cfc8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc5bc0 0x55930cfd5e20 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 --2- 172.21.15.137:0/3389502756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfbaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 -- 172.21.15.137:0/3389502756 >> 172.21.15.137:0/3389502756 conn(0x55930cfbfde0 msgr2=0x55930cfc38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 -- 172.21.15.137:0/3389502756 shutdown_connections 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 -- 172.21.15.137:0/3389502756 wait complete. 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 -- start start 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.540+0000 7f56b8644740 1 Processor -- start 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfd57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfd5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01b31c0 con 0x55930cfc6580 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01b44e0 con 0x55930cfc5bc0 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b01b3820 con 0x55930cfc49f0 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56b8644740 10 client.0 _ll_register_callbacks cb 0x7f56a8003f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56392/0 (socket says 172.21.15.137:56392) 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56af400640 1 -- 172.21.15.137:0/966063952 learned_addr learned my addr 172.21.15.137:0/966063952 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:33.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56ae000640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfd5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.541+0000 7f56aea00640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfd57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56af400640 1 -- 172.21.15.137:0/966063952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 msgr2=0x55930cfd57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56af400640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfd57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56af400640 1 -- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 msgr2=0x55930cfd5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:33.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56af400640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfd5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56af400640 1 -- 172.21.15.137:0/966063952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55930cfd2aa0 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56af400640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd2270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f56a801feb0 tx=0x7f56a801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67568239a3a6c35d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f5698c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f5698c00640 1 -- 172.21.15.137:0/966063952 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a803a020 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f5698c00640 1 -- 172.21.15.137:0/966063952 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f56a801c980 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.542+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f569c004c50 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f5698c00640 1 -- 172.21.15.137:0/966063952 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56a8045050 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 10 client.16788 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55930d036ae0 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f5698c00640 1 -- 172.21.15.137:0/966063952 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569c004c50 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f5698c00640 1 -- 172.21.15.137:0/966063952 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f56a8028540 con 0x55930cfc6580 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 10 client.16788 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 20 client.16788 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:33.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930d037440 0x55930d039860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:33.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 4 client.16788 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:07:33.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.543+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55930cf30b50 con 0x55930d037440 2024-03-26T14:07:33.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.544+0000 7f56aea00640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930d037440 0x55930d039860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.545+0000 7f56aea00640 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930d037440 0x55930d039860 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d3783d8da848759 server_cookie=7444470a0c186665 in_seq=0 out_seq=0 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.545+0000 7f5698c00640 10 client.16788 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.545+0000 7f5698c00640 1 -- 172.21.15.137:0/966063952 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55930cf30b50 con 0x55930d037440 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.545+0000 7f5698c00640 10 client.16788 handle_command_reply: tid=0 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.545+0000 7f56b8644740 1 client.16788 shutdown 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 msgr2=0x55930cfd2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd2270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f56a801feb0 tx=0x7f56a801c730 comp rx=0 tx=0).stop 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 shutdown_connections 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55930d037440 0x55930d039860 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55930cfc5bc0 0x55930cfd5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55930cfc6580 0x55930cfd2270 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 --2- 172.21.15.137:0/966063952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55930cfc49f0 0x55930cfd57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f5698c00640 0 client.16788 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:33.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 >> 172.21.15.137:0/966063952 conn(0x55930cfbfde0 msgr2=0x55930cfbe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 shutdown_connections 2024-03-26T14:07:33.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 1 -- 172.21.15.137:0/966063952 wait complete. 2024-03-26T14:07:33.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.546+0000 7f56b8644740 20 client.16788 trim_cache size 0 max 16384 2024-03-26T14:07:33.548 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:07:33.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56940a2880 msgr2=0x7f56940a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56940a2880 0x7f56940a4d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x55930cfc5ac0 tx=0x7f56a8009500 comp rx=0 tx=0).stop 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 msgr2=0x7f56b01a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 0x7f56b01a2730 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f569c000c00 tx=0x7f569c0042f0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 shutdown_connections 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56940a2880 0x7f56940a4d40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56b010fa70 0x7f56b01a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56b0109340 0x7f56b01a2730 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 --2- 172.21.15.137:0/38297913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56b0072ce0 0x7f56b01a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 >> 172.21.15.137:0/38297913 conn(0x7f56b006e580 msgr2=0x7f56b01097b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 shutdown_connections 2024-03-26T14:07:33.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:33.547+0000 7f56afe00640 1 -- 172.21.15.137:0/38297913 wait complete. 2024-03-26T14:07:33.561 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:33 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2143429262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:07:33.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:33 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2143429262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:07:33.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:33 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2143429262' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:07:34.564 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:07:34.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:07:34.565 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:07:34.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:34 smithi179 ceph-mon[35472]: pgmap v614: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:34.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:34 smithi159 ceph-mon[35501]: pgmap v614: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:34.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:34 smithi137 ceph-mon[31539]: pgmap v614: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 -- 172.21.15.137:0/3037498818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e8102060 msgr2=0x7f26e8104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 --2- 172.21.15.137:0/3037498818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e8102060 0x7f26e8104450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009900 tx=0x7f26d802bc40 comp rx=0 tx=0).stop 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 -- 172.21.15.137:0/3037498818 shutdown_connections 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 --2- 172.21.15.137:0/3037498818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e81072c0 0x7f26e81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 --2- 172.21.15.137:0/3037498818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e8104990 0x7f26e8106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 --2- 172.21.15.137:0/3037498818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e8102060 0x7f26e8104450 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 -- 172.21.15.137:0/3037498818 >> 172.21.15.137:0/3037498818 conn(0x7f26e80fba10 msgr2=0x7f26e80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 -- 172.21.15.137:0/3037498818 shutdown_connections 2024-03-26T14:07:34.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.798+0000 7f26e7200640 1 -- 172.21.15.137:0/3037498818 wait complete. 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 Processor -- start 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 -- start start 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e8102060 0x7f26e81976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e8104990 0x7f26e8197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 0x7f26e819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e810d110 con 0x7f26e81072c0 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e810cf90 con 0x7f26e8102060 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.799+0000 7f26e7200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e810d290 con 0x7f26e8104990 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 0x7f26e819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 0x7f26e819ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56418/0 (socket says 172.21.15.137:56418) 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 -- 172.21.15.137:0/2155282015 learned_addr learned my addr 172.21.15.137:0/2155282015 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e5e00640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e8102060 0x7f26e81976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e5400640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e8104990 0x7f26e8197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 -- 172.21.15.137:0/2155282015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e8104990 msgr2=0x7f26e8197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e8104990 0x7f26e8197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 -- 172.21.15.137:0/2155282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e8102060 msgr2=0x7f26e81976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e8102060 0x7f26e81976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 -- 172.21.15.137:0/2155282015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e819f2e0 con 0x7f26e81072c0 2024-03-26T14:07:34.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e5e00640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e8102060 0x7f26e81976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:07:34.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.800+0000 7f26e6800640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 0x7f26e819ec70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f26e000cce0 tx=0x7f26e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:34.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.801+0000 7f26dea00640 1 -- 172.21.15.137:0/2155282015 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000f450 con 0x7f26e81072c0 2024-03-26T14:07:34.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.801+0000 7f26dea00640 1 -- 172.21.15.137:0/2155282015 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26e000faa0 con 0x7f26e81072c0 2024-03-26T14:07:34.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.801+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e819f5d0 con 0x7f26e81072c0 2024-03-26T14:07:34.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.801+0000 7f26dea00640 1 -- 172.21.15.137:0/2155282015 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0018b70 con 0x7f26e81072c0 2024-03-26T14:07:34.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.801+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e819fad0 con 0x7f26e81072c0 2024-03-26T14:07:34.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 Processor -- start 2024-03-26T14:07:34.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26dea00640 1 -- 172.21.15.137:0/2155282015 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f26e0018d10 con 0x7f26e81072c0 2024-03-26T14:07:34.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 -- start start 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 0x5626148e2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148e3220 0x5626148e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261483eeb0 con 0x5626148de0b0 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626147e7a60 con 0x5626148e3220 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.805+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261483af50 con 0x5626148dd6c0 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 0x5626148e2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 0x5626148e2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56420/0 (socket says 172.21.15.137:56420) 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 -- 172.21.15.137:0/2778427178 learned_addr learned my addr 172.21.15.137:0/2778427178 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5400640 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5400640 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148e3220 0x5626148e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 -- 172.21.15.137:0/2778427178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 msgr2=0x5626148ddae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148ddae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 -- 172.21.15.137:0/2778427178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148e3220 msgr2=0x5626148e56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148e3220 0x5626148e56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26e5e00640 1 -- 172.21.15.137:0/2778427178 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261461b7e0 con 0x5626148de0b0 2024-03-26T14:07:34.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.806+0000 7f26dea00640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.807+0000 7f26dea00640 1 -- 172.21.15.137:0/2155282015 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f26e00cf350 con 0x7f26e81072c0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.807+0000 7f26e5e00640 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 0x5626148e2bb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f26d8035000 tx=0x7f26d802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2dc3c6b669844d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.807+0000 7f26e5400640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.807+0000 7f26d3000640 1 -- 172.21.15.137:0/2778427178 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8044020 con 0x5626148de0b0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.807+0000 7f26d3000640 1 -- 172.21.15.137:0/2778427178 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26d8005040 con 0x5626148de0b0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.807+0000 7f26d3000640 1 -- 172.21.15.137:0/2778427178 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80049c0 con 0x5626148de0b0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 -- 172.21.15.137:0/2778427178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 msgr2=0x5626148e2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 0x5626148e2bb0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f26d8035000 tx=0x7f26d802c970 comp rx=0 tx=0).stop 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26e5400640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26d4005f80 tx=0x7f26d4005f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 -- 172.21.15.137:0/2778427178 shutdown_connections 2024-03-26T14:07:34.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148e3220 0x5626148e56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148de0b0 0x5626148e2bb0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/2778427178 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148ddae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 -- 172.21.15.137:0/2778427178 >> 172.21.15.137:0/2778427178 conn(0x5626148d4820 msgr2=0x5626148d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 -- 172.21.15.137:0/2778427178 shutdown_connections 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.808+0000 7f26ef5fc740 1 -- 172.21.15.137:0/2778427178 wait complete. 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 -- start start 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 Processor -- start 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 0x5626148eed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626148e60e0 con 0x5626148e3220 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626147faeb0 con 0x5626148de0b0 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.809+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626148e6260 con 0x5626148dd6c0 2024-03-26T14:07:34.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26ef5fc740 10 client.0 _ll_register_callbacks cb 0x5626147ef150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 0x5626148eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 0x5626148eed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56430/0 (socket says 172.21.15.137:56430) 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 -- 172.21.15.137:0/1490454817 learned_addr learned my addr 172.21.15.137:0/1490454817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e5400640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e5e00640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 -- 172.21.15.137:0/1490454817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 msgr2=0x5626148e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 -- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 msgr2=0x5626148e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 -- 172.21.15.137:0/1490454817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56261467bf70 con 0x5626148e3220 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26e6800640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 0x5626148eed60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f26e0097d80 tx=0x7f26e009b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca3147d9aa28094 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.810+0000 7f26c7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26c7400640 1 -- 172.21.15.137:0/1490454817 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e00b4060 con 0x5626148e3220 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26c7400640 1 -- 172.21.15.137:0/1490454817 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f26e009bb80 con 0x5626148e3220 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26d8005040 con 0x5626148e3220 2024-03-26T14:07:34.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26c7400640 1 -- 172.21.15.137:0/1490454817 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26e009b920 con 0x5626148e3220 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26ef5fc740 10 client.16800 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562614513170 con 0x5626148e3220 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26c7400640 1 -- 172.21.15.137:0/1490454817 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26d8005040 con 0x5626148e3220 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.811+0000 7f26c7400640 1 -- 172.21.15.137:0/1490454817 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f26e0086150 con 0x5626148e3220 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26ef5fc740 10 client.16800 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26ef5fc740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26ef5fc740 20 client.16800 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148f3620 0x5626148f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26ef5fc740 4 client.16800 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56261484c3f0 con 0x5626148f3620 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.812+0000 7f26e5e00640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148f3620 0x5626148f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.813+0000 7f26e5e00640 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148f3620 0x5626148f5ae0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0fccf49e760dfd server_cookie=ff662b791bcb95bc in_seq=0 out_seq=0 2024-03-26T14:07:34.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.813+0000 7f26c7400640 10 client.16800 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26c7400640 1 -- 172.21.15.137:0/1490454817 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56261484c3f0 con 0x5626148f3620 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26c7400640 10 client.16800 handle_command_reply: tid=0 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 client.16800 shutdown 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 msgr2=0x5626148eed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 0x5626148eed60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f26e0097d80 tx=0x7f26e009b340 comp rx=0 tx=0).stop 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 shutdown_connections 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148f3620 0x5626148f5ae0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148e3220 0x5626148eed60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/1490454817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148dd6c0 0x5626148e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26c7400640 0 client.16800 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 >> 172.21.15.137:0/1490454817 conn(0x5626148d4820 msgr2=0x5626148d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 shutdown_connections 2024-03-26T14:07:34.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 1 -- 172.21.15.137:0/1490454817 wait complete. 2024-03-26T14:07:34.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.815+0000 7f26ef5fc740 20 client.16800 trim_cache size 0 max 16384 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 Processor -- start 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 -- start start 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148d4820 0x5626148d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 0x5626148eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81aa190 con 0x5626148dd6c0 2024-03-26T14:07:34.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81a7c90 con 0x5626148de0b0 2024-03-26T14:07:34.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81a5d10 con 0x5626148d4820 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.833+0000 7f26e5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 0x5626148eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 0x5626148eed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56434/0 (socket says 172.21.15.137:56434) 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 -- 172.21.15.137:0/4042303941 learned_addr learned my addr 172.21.15.137:0/4042303941 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e6800640 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5e00640 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148d4820 0x5626148d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 -- 172.21.15.137:0/4042303941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148d4820 msgr2=0x5626148d3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148d4820 0x5626148d3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 -- 172.21.15.137:0/4042303941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 msgr2=0x5626148fa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148fa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 -- 172.21.15.137:0/4042303941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626148f33e0 con 0x5626148dd6c0 2024-03-26T14:07:34.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.834+0000 7f26e5400640 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 0x5626148eed20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f26e8198bd0 tx=0x7f26d4013050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1d50e6f4e16161 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:34.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.835+0000 7f26d2600640 1 -- 172.21.15.137:0/4042303941 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d401f980 con 0x5626148dd6c0 2024-03-26T14:07:34.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.835+0000 7f26d2600640 1 -- 172.21.15.137:0/4042303941 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26d402b480 con 0x5626148dd6c0 2024-03-26T14:07:34.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.835+0000 7f26d2600640 1 -- 172.21.15.137:0/4042303941 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4020680 con 0x5626148dd6c0 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.835+0000 7f26ef5fc740 1 -- 172.21.15.137:0/4042303941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 msgr2=0x5626148eed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 0x5626148eed20 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f26e8198bd0 tx=0x7f26d4013050 comp rx=0 tx=0).stop 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 -- 172.21.15.137:0/4042303941 shutdown_connections 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148de0b0 0x5626148fa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148dd6c0 0x5626148eed20 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/4042303941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148d4820 0x5626148d3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 -- 172.21.15.137:0/4042303941 >> 172.21.15.137:0/4042303941 conn(0x5626148e69f0 msgr2=0x5626148d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:34.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 -- 172.21.15.137:0/4042303941 shutdown_connections 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 -- 172.21.15.137:0/4042303941 wait complete. 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 -- start start 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.836+0000 7f26ef5fc740 1 Processor -- start 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 0x5626148ed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148dd6c0 0x5626148eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148de0b0 0x56261494ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81a6310 con 0x5626148d4820 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81a7630 con 0x5626148dd6c0 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81a6970 con 0x5626148de0b0 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26ef5fc740 10 client.0 _ll_register_callbacks cb 0x7f26e00077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 0x5626148ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 0x5626148ed890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56444/0 (socket says 172.21.15.137:56444) 2024-03-26T14:07:34.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 -- 172.21.15.137:0/353811643 learned_addr learned my addr 172.21.15.137:0/353811643 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5400640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148dd6c0 0x5626148eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 -- 172.21.15.137:0/353811643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148de0b0 msgr2=0x56261494ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148de0b0 0x56261494ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 -- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148dd6c0 msgr2=0x5626148eddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148dd6c0 0x5626148eddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5e00640 1 -- 172.21.15.137:0/353811643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56261494f4b0 con 0x5626148d4820 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.837+0000 7f26e5400640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148dd6c0 0x5626148eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26e5e00640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 0x5626148ed890 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f26d802c180 tx=0x7f26d802c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6802997365702a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26c7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26c7400640 1 -- 172.21.15.137:0/353811643 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8052070 con 0x5626148d4820 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26c7400640 1 -- 172.21.15.137:0/353811643 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f26d8004040 con 0x5626148d4820 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26d402b480 con 0x5626148d4820 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26c7400640 1 -- 172.21.15.137:0/353811643 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d805d050 con 0x5626148d4820 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26ef5fc740 10 client.16806 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:34.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.838+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562614950670 con 0x5626148d4820 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26c7400640 1 -- 172.21.15.137:0/353811643 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26d402b480 con 0x5626148d4820 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26c7400640 1 -- 172.21.15.137:0/353811643 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f26d80046c0 con 0x5626148d4820 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26ef5fc740 10 client.16806 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26ef5fc740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26ef5fc740 20 client.16806 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148e3220 0x562614952c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26ef5fc740 4 client.16806 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.839+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562614846740 con 0x5626148e3220 2024-03-26T14:07:34.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.840+0000 7f26e5400640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148e3220 0x562614952c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.841+0000 7f26c7400640 10 client.16806 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:34.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.841+0000 7f26e5400640 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148e3220 0x562614952c50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6beccd4f631191 server_cookie=1b98503b6b1601a8 in_seq=0 out_seq=0 2024-03-26T14:07:34.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26c7400640 1 -- 172.21.15.137:0/353811643 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562614846740 con 0x5626148e3220 2024-03-26T14:07:34.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26c7400640 10 client.16806 handle_command_reply: tid=0 2024-03-26T14:07:34.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 client.16806 shutdown 2024-03-26T14:07:34.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 msgr2=0x5626148ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 0x5626148ed890 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f26d802c180 tx=0x7f26d802c810 comp rx=0 tx=0).stop 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 shutdown_connections 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5626148e3220 0x562614952c50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626148dd6c0 0x5626148eddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5626148de0b0 0x56261494ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 --2- 172.21.15.137:0/353811643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5626148d4820 0x5626148ed890 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 >> 172.21.15.137:0/353811643 conn(0x5626148e69f0 msgr2=0x5626148d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26c7400640 0 client.16806 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 shutdown_connections 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 1 -- 172.21.15.137:0/353811643 wait complete. 2024-03-26T14:07:34.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.844+0000 7f26ef5fc740 20 client.16806 trim_cache size 0 max 16384 2024-03-26T14:07:34.846 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:07:34.846 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:07:34.846 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "b2c4a0c2-c5f7-4cd5-9797-abce805a0691", 2024-03-26T14:07:34.846 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:07:34.846 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f26cc0a2800 msgr2=0x7f26cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f26d4005f80 tx=0x7f26d4005f10 comp rx=0 tx=0).stop 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 msgr2=0x7f26e819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 0x7f26e819ec70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f26e000cce0 tx=0x7f26e000a620 comp rx=0 tx=0).stop 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 shutdown_connections 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f26cc0a2800 0x7f26cc0a4cc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26e81072c0 0x7f26e819ec70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26e8104990 0x7f26e8197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 --2- 172.21.15.137:0/2155282015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26e8102060 0x7f26e81976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 >> 172.21.15.137:0/2155282015 conn(0x7f26e80fba10 msgr2=0x7f26e8102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 shutdown_connections 2024-03-26T14:07:34.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:34.845+0000 7f26e7200640 1 -- 172.21.15.137:0/2155282015 wait complete. 2024-03-26T14:07:34.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:07:34.863 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 -- 172.21.15.137:0/3837437079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec107320 msgr2=0x7f93ec109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 --2- 172.21.15.137:0/3837437079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec107320 0x7f93ec109710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009ca0 tx=0x7f93e402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 -- 172.21.15.137:0/3837437079 shutdown_connections 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 --2- 172.21.15.137:0/3837437079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93ec1049f0 0x7f93ec106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 --2- 172.21.15.137:0/3837437079 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec107320 0x7f93ec109710 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 --2- 172.21.15.137:0/3837437079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec1020c0 0x7f93ec1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 -- 172.21.15.137:0/3837437079 >> 172.21.15.137:0/3837437079 conn(0x7f93ec0fba10 msgr2=0x7f93ec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.081+0000 7f93f2400640 1 -- 172.21.15.137:0/3837437079 shutdown_connections 2024-03-26T14:07:35.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.082+0000 7f93f2400640 1 -- 172.21.15.137:0/3837437079 wait complete. 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 Processor -- start 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 -- start start 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93ec1020c0 0x7f93ec199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 0x7f93ec199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec107320 0x7f93ec1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec10d0d0 con 0x7f93ec1049f0 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec10cf50 con 0x7f93ec1020c0 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.083+0000 7f93f2400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec10d250 con 0x7f93ec107320 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 0x7f93ec199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 0x7f93ec199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56468/0 (socket says 172.21.15.137:56468) 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 -- 172.21.15.137:0/946213965 learned_addr learned my addr 172.21.15.137:0/946213965 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93f1000640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93ec1020c0 0x7f93ec199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93f1a00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec107320 0x7f93ec1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 -- 172.21.15.137:0/946213965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec107320 msgr2=0x7f93ec1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec107320 0x7f93ec1a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 -- 172.21.15.137:0/946213965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93ec1020c0 msgr2=0x7f93ec199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93ec1020c0 0x7f93ec199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 -- 172.21.15.137:0/946213965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec1a15a0 con 0x7f93ec1049f0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93f1a00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec107320 0x7f93ec1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.084+0000 7f93ebe00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 0x7f93ec199ec0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f93d800e9c0 tx=0x7f93d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.085+0000 7f93e9600640 1 -- 172.21.15.137:0/946213965 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d800cd80 con 0x7f93ec1049f0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.085+0000 7f93e9600640 1 -- 172.21.15.137:0/946213965 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93d80054d0 con 0x7f93ec1049f0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.085+0000 7f93e9600640 1 -- 172.21.15.137:0/946213965 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8010430 con 0x7f93ec1049f0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.085+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec1a1890 con 0x7f93ec1049f0 2024-03-26T14:07:35.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.085+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec1a1d90 con 0x7f93ec1049f0 2024-03-26T14:07:35.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.088+0000 7f93f67e2740 1 Processor -- start 2024-03-26T14:07:35.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.088+0000 7f93f67e2740 1 -- start start 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede898470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 0x558ede89d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede89dbb0 0x558ede8a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f67e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ede7fd250 con 0x558ede89dbb0 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f67e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ede7a5a30 con 0x558ede898050 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f67e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ede7f92f0 con 0x558ede898a40 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93e9600640 1 -- 172.21.15.137:0/946213965 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f93d8004fd0 con 0x7f93ec1049f0 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f1000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 0x558ede89d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede898470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f1000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 0x558ede89d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:34306/0 (socket says 172.21.15.137:34306) 2024-03-26T14:07:35.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.089+0000 7f93f1000640 1 -- 172.21.15.137:0/490298063 learned_addr learned my addr 172.21.15.137:0/490298063 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:35.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.090+0000 7f93ebe00640 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede89dbb0 0x558ede8a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.090+0000 7f93f1000640 1 -- 172.21.15.137:0/490298063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 msgr2=0x558ede898470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.090+0000 7f93f1000640 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede898470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.090+0000 7f93f1000640 1 -- 172.21.15.137:0/490298063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede89dbb0 msgr2=0x558ede8a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.090+0000 7f93f1000640 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede89dbb0 0x558ede8a0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.090+0000 7f93f1000640 1 -- 172.21.15.137:0/490298063 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ede790690 con 0x558ede898a40 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93f1000640 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 0x558ede89d540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009c90 tx=0x7f93e00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95b227dd7018fce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93de000640 1 -- 172.21.15.137:0/490298063 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0012450 con 0x558ede898a40 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93de000640 1 -- 172.21.15.137:0/490298063 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93e0012aa0 con 0x558ede898a40 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93de000640 1 -- 172.21.15.137:0/490298063 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e000fb50 con 0x558ede898a40 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93e9600640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93d40a2880 0x7f93d40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93e9600640 1 -- 172.21.15.137:0/946213965 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f93d80cf310 con 0x7f93ec1049f0 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.091+0000 7f93f1a00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93d40a2880 0x7f93d40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 -- 172.21.15.137:0/490298063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 msgr2=0x558ede89d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 0x558ede89d540 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009c90 tx=0x7f93e00117b0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 -- 172.21.15.137:0/490298063 shutdown_connections 2024-03-26T14:07:35.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede89dbb0 0x558ede8a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898a40 0x558ede89d540 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 --2- 172.21.15.137:0/490298063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede898470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 -- 172.21.15.137:0/490298063 >> 172.21.15.137:0/490298063 conn(0x558ede860200 msgr2=0x558ede8939c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f1a00640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93d40a2880 0x7f93d40a4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x558ede8992a0 tx=0x7f93e4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 -- 172.21.15.137:0/490298063 shutdown_connections 2024-03-26T14:07:35.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.092+0000 7f93f67e2740 1 -- 172.21.15.137:0/490298063 wait complete. 2024-03-26T14:07:35.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.093+0000 7f93f67e2740 1 -- start start 2024-03-26T14:07:35.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.093+0000 7f93f67e2740 1 Processor -- start 2024-03-26T14:07:35.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede8a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede8a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ede8a0b60 con 0x558ede898a40 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ede7b8e80 con 0x558ede898050 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ede8a0ce0 con 0x558ede89dbb0 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f67e2740 10 client.0 _ll_register_callbacks cb 0x558ede7cb130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56496/0 (socket says 172.21.15.137:56496) 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 -- 172.21.15.137:0/662847628 learned_addr learned my addr 172.21.15.137:0/662847628 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f1000640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede8a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93f1a00640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede8a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 -- 172.21.15.137:0/662847628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 msgr2=0x558ede8a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede8a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 -- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 msgr2=0x558ede8a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede8a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.094+0000 7f93ebe00640 1 -- 172.21.15.137:0/662847628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ede5db7d0 con 0x558ede898a40 2024-03-26T14:07:35.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93ebe00640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a2530 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f93d80982b0 tx=0x7f93d8004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a45ec61602fa92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93d2a00640 1 -- 172.21.15.137:0/662847628 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d809b6b0 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93d2a00640 1 -- 172.21.15.137:0/662847628 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f93d80860c0 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e0012aa0 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93d2a00640 1 -- 172.21.15.137:0/662847628 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d80cf170 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93f67e2740 10 client.16818 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.095+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558ede5113f0 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93d2a00640 1 -- 172.21.15.137:0/662847628 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93e0012aa0 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93d2a00640 1 -- 172.21.15.137:0/662847628 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f93d80a3bf0 con 0x558ede898a40 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93f67e2740 10 client.16818 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93f67e2740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93f67e2740 20 client.16818 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93f67e2740 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede8add50 0x558ede8b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93f67e2740 4 client.16818 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:07:35.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.096+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ede80a790 con 0x558ede8add50 2024-03-26T14:07:35.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.097+0000 7f93f1000640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede8add50 0x558ede8b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.097+0000 7f93f1000640 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede8add50 0x558ede8b01a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cdeb8b1d0e5df50 server_cookie=69c2280bec9f6c32 in_seq=0 out_seq=0 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.097+0000 7f93d2a00640 10 client.16818 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.099+0000 7f93d2a00640 1 -- 172.21.15.137:0/662847628 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x558ede80a790 con 0x558ede8add50 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.099+0000 7f93d2a00640 10 client.16818 handle_command_reply: tid=0 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.099+0000 7f93f67e2740 1 client.16818 shutdown 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.099+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 msgr2=0x558ede8a2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.099+0000 7f93f67e2740 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a2530 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f93d80982b0 tx=0x7f93d8004950 comp rx=0 tx=0).stop 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 shutdown_connections 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede8add50 0x558ede8b01a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede8a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a2530 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 --2- 172.21.15.137:0/662847628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede8a1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 >> 172.21.15.137:0/662847628 conn(0x558ede860200 msgr2=0x558ede892bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93d2a00640 0 client.16818 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 shutdown_connections 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 1 -- 172.21.15.137:0/662847628 wait complete. 2024-03-26T14:07:35.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.100+0000 7f93f67e2740 20 client.16818 trim_cache size 0 max 16384 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.117+0000 7f93f67e2740 1 Processor -- start 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.117+0000 7f93f67e2740 1 -- start start 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede896c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede897140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f67e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1ae770 con 0x558ede898a40 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f67e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1ac270 con 0x558ede898050 2024-03-26T14:07:35.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f67e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1aa2f0 con 0x558ede89dbb0 2024-03-26T14:07:35.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56508/0 (socket says 172.21.15.137:56508) 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93ebe00640 1 -- 172.21.15.137:0/3743336552 learned_addr learned my addr 172.21.15.137:0/3743336552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f1000640 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede896c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93f1a00640 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede897140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.118+0000 7f93ebe00640 1 -- 172.21.15.137:0/3743336552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 msgr2=0x558ede897140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93ebe00640 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede897140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93ebe00640 1 -- 172.21.15.137:0/3743336552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 msgr2=0x558ede896c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93ebe00640 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede896c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93ebe00640 1 -- 172.21.15.137:0/3743336552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ede4d10f0 con 0x558ede898a40 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93ebe00640 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a9530 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f93d80d2930 tx=0x7f93d8099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6018e34f83850e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93dd600640 1 -- 172.21.15.137:0/3743336552 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d80d6070 con 0x558ede898a40 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93dd600640 1 -- 172.21.15.137:0/3743336552 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93d809b8e0 con 0x558ede898a40 2024-03-26T14:07:35.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.119+0000 7f93dd600640 1 -- 172.21.15.137:0/3743336552 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d809b240 con 0x558ede898a40 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 -- 172.21.15.137:0/3743336552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 msgr2=0x558ede8a9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a9530 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f93d80d2930 tx=0x7f93d8099df0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 -- 172.21.15.137:0/3743336552 shutdown_connections 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede89dbb0 0x558ede897140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a9530 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 --2- 172.21.15.137:0/3743336552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede898050 0x558ede896c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 -- 172.21.15.137:0/3743336552 >> 172.21.15.137:0/3743336552 conn(0x558ede860200 msgr2=0x558ede899d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 -- 172.21.15.137:0/3743336552 shutdown_connections 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.120+0000 7f93f67e2740 1 -- 172.21.15.137:0/3743336552 wait complete. 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 -- start start 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 Processor -- start 2024-03-26T14:07:35.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898050 0x558ede8a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede89dbb0 0x558ede8a8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1aa8f0 con 0x558ede898a40 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1abc10 con 0x558ede89dbb0 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1aaf50 con 0x558ede898050 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.121+0000 7f93f67e2740 10 client.0 _ll_register_callbacks cb 0x7f93d800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:56522/0 (socket says 172.21.15.137:56522) 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 -- 172.21.15.137:0/4111858295 learned_addr learned my addr 172.21.15.137:0/4111858295 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93f1a00640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede89dbb0 0x558ede8a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93f1000640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898050 0x558ede8a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 -- 172.21.15.137:0/4111858295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898050 msgr2=0x558ede8a6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898050 0x558ede8a6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 -- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede89dbb0 msgr2=0x558ede8a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede89dbb0 0x558ede8a8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 -- 172.21.15.137:0/4111858295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ede8a8a60 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93ebe00640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a7460 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000c00 tx=0x7f93d8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fc0331f07d334f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.122+0000 7f93d2a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93d2a00640 1 -- 172.21.15.137:0/4111858295 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d80d6070 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93d2a00640 1 -- 172.21.15.137:0/4111858295 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f93d809ad30 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ede804ae0 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93d2a00640 1 -- 172.21.15.137:0/4111858295 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d809b420 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93f67e2740 10 client.16824 fetch_fsmap learned FSMap version 24 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558ede909860 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93d2a00640 1 -- 172.21.15.137:0/4111858295 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x558ede804ae0 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93d2a00640 1 -- 172.21.15.137:0/4111858295 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f93d809a7a0 con 0x558ede898a40 2024-03-26T14:07:35.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93f67e2740 10 client.16824 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:07:35.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93f67e2740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:07:35.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.123+0000 7f93f67e2740 20 client.16824 populate_metadata read hostname 'smithi137' 2024-03-26T14:07:35.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.124+0000 7f93f67e2740 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede909f30 0x558ede90c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:07:35.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.124+0000 7f93f67e2740 4 client.16824 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:07:35.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.124+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ede7823a0 con 0x558ede909f30 2024-03-26T14:07:35.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.124+0000 7f93f1000640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede909f30 0x558ede90c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f1000640 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede909f30 0x558ede90c350 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bbbceed3f2ece42 server_cookie=524c08b5b270bffa in_seq=0 out_seq=0 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93d2a00640 10 client.16824 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93d2a00640 1 -- 172.21.15.137:0/4111858295 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558ede7823a0 con 0x558ede909f30 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93d2a00640 10 client.16824 handle_command_reply: tid=0 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 client.16824 shutdown 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 msgr2=0x558ede8a7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a7460 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f93d8000c00 tx=0x7f93d8005740 comp rx=0 tx=0).stop 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 shutdown_connections 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x558ede909f30 0x558ede90c350 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ede89dbb0 0x558ede8a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x558ede898a40 0x558ede8a7460 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93d2a00640 0 client.16824 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 --2- 172.21.15.137:0/4111858295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ede898050 0x558ede8a6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 >> 172.21.15.137:0/4111858295 conn(0x558ede860200 msgr2=0x558ede892330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.125+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 shutdown_connections 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f67e2740 1 -- 172.21.15.137:0/4111858295 wait complete. 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f67e2740 20 client.16824 trim_cache size 0 max 16384 2024-03-26T14:07:35.129 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: "b2c4a0c2-c5f7-4cd5-9797-abce805a0691": { 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "b2c4a0c2-c5f7-4cd5-9797-abce805a0691", 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:07:35.130 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93d40a2880 msgr2=0x7f93d40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93d40a2880 0x7f93d40a4d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x558ede8992a0 tx=0x7f93e4005d80 comp rx=0 tx=0).stop 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 msgr2=0x7f93ec199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 0x7f93ec199ec0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f93d800e9c0 tx=0x7f93d800ee80 comp rx=0 tx=0).stop 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 shutdown_connections 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93d40a2880 0x7f93d40a4d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93ec107320 0x7f93ec1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93ec1049f0 0x7f93ec199ec0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 --2- 172.21.15.137:0/946213965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93ec1020c0 0x7f93ec199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:07:35.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.126+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 >> 172.21.15.137:0/946213965 conn(0x7f93ec0fba10 msgr2=0x7f93ec101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:07:35.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.127+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 shutdown_connections 2024-03-26T14:07:35.132 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:07:35.127+0000 7f93f2400640 1 -- 172.21.15.137:0/946213965 wait complete. 2024-03-26T14:07:35.147 INFO:tasks.cephfs.filesystem:scrub status for tag:b2c4a0c2-c5f7-4cd5-9797-abce805a0691 - {'path': '/', 'tag': 'b2c4a0c2-c5f7-4cd5-9797-abce805a0691', 'options': 'recursive,force'} 2024-03-26T14:07:35.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:35 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:07:35.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:35 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:35.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:35 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:07:35.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:35 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:35 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:07:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:35 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:35 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:07:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:35 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:35.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:35 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:07:35.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:35 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:35.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:35 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:07:35.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:35 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:07:36.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:36 smithi159 ceph-mon[35501]: pgmap v615: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:07:36.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:36 smithi137 ceph-mon[31539]: pgmap v615: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:07:36.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:36 smithi179 ceph-mon[35472]: pgmap v615: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:07:37.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:37 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:07:37.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:37 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:07:37.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:37 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:07:38.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:38 smithi159 ceph-mon[35501]: pgmap v616: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-26T14:07:38.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:38 smithi137 ceph-mon[31539]: pgmap v616: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-26T14:07:38.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:38 smithi179 ceph-mon[35472]: pgmap v616: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-26T14:07:40.726 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:40 smithi179 ceph-mon[35472]: pgmap v617: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:07:40.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:40 smithi137 ceph-mon[31539]: pgmap v617: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:07:40.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:40 smithi159 ceph-mon[35501]: pgmap v617: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:07:42.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:42 smithi137 ceph-mon[31539]: pgmap v618: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:42.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:42 smithi179 ceph-mon[35472]: pgmap v618: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:42.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:42 smithi159 ceph-mon[35501]: pgmap v618: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:07:44.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:44 smithi137 ceph-mon[31539]: pgmap v619: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:07:44.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:44 smithi179 ceph-mon[35472]: pgmap v619: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:07:44.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:44 smithi159 ceph-mon[35501]: pgmap v619: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:07:46.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:46 smithi137 ceph-mon[31539]: pgmap v620: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:07:46.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:46 smithi179 ceph-mon[35472]: pgmap v620: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:07:46.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:46 smithi159 ceph-mon[35501]: pgmap v620: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:07:48.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:48 smithi137 ceph-mon[31539]: pgmap v621: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-26T14:07:48.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:48 smithi179 ceph-mon[35472]: pgmap v621: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-26T14:07:48.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:48 smithi159 ceph-mon[35501]: pgmap v621: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-03-26T14:07:50.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:50 smithi137 ceph-mon[31539]: pgmap v622: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:50.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:50 smithi179 ceph-mon[35472]: pgmap v622: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:50.951 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:50 smithi159 ceph-mon[35501]: pgmap v622: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:52.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:52 smithi137 ceph-mon[31539]: pgmap v623: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:07:52.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:52 smithi179 ceph-mon[35472]: pgmap v623: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:07:52.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:52 smithi159 ceph-mon[35501]: pgmap v623: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:07:54.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:54 smithi137 ceph-mon[31539]: pgmap v624: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:54.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:54 smithi179 ceph-mon[35472]: pgmap v624: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:54.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:54 smithi159 ceph-mon[35501]: pgmap v624: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:07:56.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:56 smithi137 ceph-mon[31539]: pgmap v625: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:56.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:56 smithi179 ceph-mon[35472]: pgmap v625: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:56.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:56 smithi159 ceph-mon[35501]: pgmap v625: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:07:58.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:07:58 smithi137 ceph-mon[31539]: pgmap v626: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:58.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:07:58 smithi179 ceph-mon[35472]: pgmap v626: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:07:58.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:07:58 smithi159 ceph-mon[35501]: pgmap v626: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:00.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:00 smithi137 ceph-mon[31539]: pgmap v627: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:00.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:00 smithi179 ceph-mon[35472]: pgmap v627: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:00.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:00 smithi159 ceph-mon[35501]: pgmap v627: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:02.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:02 smithi137 ceph-mon[31539]: pgmap v628: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:02.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:02 smithi179 ceph-mon[35472]: pgmap v628: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:02.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:02 smithi159 ceph-mon[35501]: pgmap v628: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:04.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:04 smithi137 ceph-mon[31539]: pgmap v629: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:04.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:04 smithi179 ceph-mon[35472]: pgmap v629: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:04.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:04 smithi159 ceph-mon[35501]: pgmap v629: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:05.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:08:05.148 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.366+0000 7f3862200640 1 -- 172.21.15.137:0/470455207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c102810 msgr2=0x7f385c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.366+0000 7f3862200640 1 --2- 172.21.15.137:0/470455207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c102810 0x7f385c102c90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f3848009930 tx=0x7f384802bc20 comp rx=0 tx=0).stop 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.366+0000 7f3862200640 1 -- 172.21.15.137:0/470455207 shutdown_connections 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.367+0000 7f3862200640 1 --2- 172.21.15.137:0/470455207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c1031d0 0x7f385c109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.367+0000 7f3862200640 1 --2- 172.21.15.137:0/470455207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c102810 0x7f385c102c90 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.367+0000 7f3862200640 1 --2- 172.21.15.137:0/470455207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c0ff840 0x7f385c0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.367+0000 7f3862200640 1 -- 172.21.15.137:0/470455207 >> 172.21.15.137:0/470455207 conn(0x7f385c0fb190 msgr2=0x7f385c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.367+0000 7f3862200640 1 -- 172.21.15.137:0/470455207 shutdown_connections 2024-03-26T14:08:05.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.367+0000 7f3862200640 1 -- 172.21.15.137:0/470455207 wait complete. 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 Processor -- start 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 -- start start 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 0x7f385c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c102810 0x7f385c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c1031d0 0x7f385c1a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c10d0f0 con 0x7f385c0ff840 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c10cf70 con 0x7f385c1031d0 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3862200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c10d270 con 0x7f385c102810 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3860e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 0x7f385c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3860e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 0x7f385c199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60416/0 (socket says 172.21.15.137:60416) 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.368+0000 7f3860e00640 1 -- 172.21.15.137:0/3715005245 learned_addr learned my addr 172.21.15.137:0/3715005245 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3861800640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c1031d0 0x7f385c1a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f385be00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c102810 0x7f385c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3860e00640 1 -- 172.21.15.137:0/3715005245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c102810 msgr2=0x7f385c199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3860e00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c102810 0x7f385c199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3860e00640 1 -- 172.21.15.137:0/3715005245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c1031d0 msgr2=0x7f385c1a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3860e00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c1031d0 0x7f385c1a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3860e00640 1 -- 172.21.15.137:0/3715005245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c1a1590 con 0x7f385c0ff840 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3861800640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c1031d0 0x7f385c1a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f385be00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c102810 0x7f385c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3860e00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 0x7f385c199970 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f385000d770 tx=0x7f385000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.369+0000 7f3859600640 1 -- 172.21.15.137:0/3715005245 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850017070 con 0x7f385c0ff840 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.370+0000 7f3859600640 1 -- 172.21.15.137:0/3715005245 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3850009c30 con 0x7f385c0ff840 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.370+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c1a1880 con 0x7f385c0ff840 2024-03-26T14:08:05.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.370+0000 7f3859600640 1 -- 172.21.15.137:0/3715005245 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850003d00 con 0x7f385c0ff840 2024-03-26T14:08:05.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.370+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c1a1e20 con 0x7f385c0ff840 2024-03-26T14:08:05.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.374+0000 7f38665cf740 1 Processor -- start 2024-03-26T14:08:05.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.374+0000 7f38665cf740 1 -- start start 2024-03-26T14:08:05.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.374+0000 7f38665cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 0x55a961648470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.374+0000 7f38665cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648a40 0x55a96164d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3859600640 1 -- 172.21.15.137:0/3715005245 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3850003ea0 con 0x7f385c0ff840 2024-03-26T14:08:05.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f38665cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a96164dbb0 0x55a961650040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f38665cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9615ad250 con 0x55a961648050 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f38665cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961555a30 con 0x55a961648a40 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f38665cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9615a92f0 con 0x55a96164dbb0 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 0x55a961648470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 0x55a961648470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60426/0 (socket says 172.21.15.137:60426) 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 -- 172.21.15.137:0/2290526786 learned_addr learned my addr 172.21.15.137:0/2290526786 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3860e00640 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648a40 0x55a96164d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f385be00640 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a96164dbb0 0x55a961650040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 -- 172.21.15.137:0/2290526786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a96164dbb0 msgr2=0x55a961650040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a96164dbb0 0x55a961650040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 -- 172.21.15.137:0/2290526786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648a40 msgr2=0x55a96164d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648a40 0x55a96164d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.375+0000 7f3861800640 1 -- 172.21.15.137:0/2290526786 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a961540690 con 0x55a961648050 2024-03-26T14:08:05.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.376+0000 7f3861800640 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 0x55a961648470 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3854008e60 tx=0x7f385400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d95a0a67e8c845 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.376+0000 7f384e000640 1 -- 172.21.15.137:0/2290526786 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400f830 con 0x55a961648050 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.376+0000 7f384e000640 1 -- 172.21.15.137:0/2290526786 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3854018480 con 0x55a961648050 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.376+0000 7f384e000640 1 -- 172.21.15.137:0/2290526786 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e610 con 0x55a961648050 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f3859600640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38440a2880 0x7f38440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f3859600640 1 -- 172.21.15.137:0/3715005245 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f38500cebb0 con 0x7f385c0ff840 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f3860e00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38440a2880 0x7f38440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 -- 172.21.15.137:0/2290526786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 msgr2=0x55a961648470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 0x55a961648470 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3854008e60 tx=0x7f385400d710 comp rx=0 tx=0).stop 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 -- 172.21.15.137:0/2290526786 shutdown_connections 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648a40 0x55a96164d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a96164dbb0 0x55a961650040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 --2- 172.21.15.137:0/2290526786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648050 0x55a961648470 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 -- 172.21.15.137:0/2290526786 >> 172.21.15.137:0/2290526786 conn(0x55a961610200 msgr2=0x55a9616439c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 -- 172.21.15.137:0/2290526786 shutdown_connections 2024-03-26T14:08:05.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.377+0000 7f38665cf740 1 -- 172.21.15.137:0/2290526786 wait complete. 2024-03-26T14:08:05.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.378+0000 7f3860e00640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38440a2880 0x7f38440a4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3850097b50 tx=0x7f3850005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.378+0000 7f38665cf740 1 -- start start 2024-03-26T14:08:05.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.378+0000 7f38665cf740 1 Processor -- start 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648050 0x55a961652030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648a40 0x55a961652570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 0x55a9616595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961650b60 con 0x55a96164dbb0 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961568e80 con 0x55a961648050 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961650ce0 con 0x55a961648a40 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f38665cf740 10 client.0 _ll_register_callbacks cb 0x55a96157b130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f3861800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 0x55a9616595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f3861800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 0x55a9616595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60438/0 (socket says 172.21.15.137:60438) 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f3861800640 1 -- 172.21.15.137:0/853610169 learned_addr learned my addr 172.21.15.137:0/853610169 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f3861800640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648050 0x55a961652030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f385be00640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648a40 0x55a961652570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f3861800640 1 -- 172.21.15.137:0/853610169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648a40 msgr2=0x55a961652570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.379+0000 7f3861800640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648a40 0x55a961652570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3861800640 1 -- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648050 msgr2=0x55a961652030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3861800640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648050 0x55a961652030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3861800640 1 -- 172.21.15.137:0/853610169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a96138b7d0 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3861800640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 0x55a9616595b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3854008fd0 tx=0x7f3854018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48514d30020130a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3842a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3842a00640 1 -- 172.21.15.137:0/853610169 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385401c070 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3842a00640 1 -- 172.21.15.137:0/853610169 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3854018960 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3854018480 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f3842a00640 1 -- 172.21.15.137:0/853610169 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f385402f050 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f38665cf740 10 client.16836 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.380+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a9612c13f0 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f3842a00640 1 -- 172.21.15.137:0/853610169 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3854018480 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f3842a00640 1 -- 172.21.15.137:0/853610169 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f38540263b0 con 0x55a96164dbb0 2024-03-26T14:08:05.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f38665cf740 10 client.16836 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f38665cf740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f38665cf740 20 client.16836 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f38665cf740 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a96165dbe0 0x55a96165fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f38665cf740 4 client.16836 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.381+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9615ba790 con 0x55a96165dbe0 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.382+0000 7f385be00640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a96165dbe0 0x55a96165fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.383+0000 7f385be00640 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a96165dbe0 0x55a96165fff0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c506325b9f92bb89 server_cookie=9971c70793bec79b in_seq=0 out_seq=0 2024-03-26T14:08:05.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.383+0000 7f3842a00640 10 client.16836 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.384+0000 7f3842a00640 1 -- 172.21.15.137:0/853610169 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a9615ba790 con 0x55a96165dbe0 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.384+0000 7f3842a00640 10 client.16836 handle_command_reply: tid=0 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.384+0000 7f38665cf740 1 client.16836 shutdown 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 msgr2=0x55a9616595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 0x55a9616595b0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3854008fd0 tx=0x7f3854018f90 comp rx=0 tx=0).stop 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 shutdown_connections 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a96165dbe0 0x55a96165fff0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a96164dbb0 0x55a9616595b0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648a40 0x55a961652570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f3842a00640 0 client.16836 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 --2- 172.21.15.137:0/853610169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a961648050 0x55a961652030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 >> 172.21.15.137:0/853610169 conn(0x55a961610200 msgr2=0x55a961642bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 shutdown_connections 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 1 -- 172.21.15.137:0/853610169 wait complete. 2024-03-26T14:08:05.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.385+0000 7f38665cf740 20 client.16836 trim_cache size 0 max 16384 2024-03-26T14:08:05.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.407+0000 7f38665cf740 1 Processor -- start 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.407+0000 7f38665cf740 1 -- start start 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f38665cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961647310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f38665cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961647850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f38665cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961659570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f38665cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1ae7e0 con 0x55a961648a40 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f38665cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1ac2e0 con 0x55a96164dbb0 2024-03-26T14:08:05.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f38665cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1aa360 con 0x55a961648050 2024-03-26T14:08:05.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961647850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961647850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60440/0 (socket says 172.21.15.137:60440) 2024-03-26T14:08:05.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 -- 172.21.15.137:0/44230532 learned_addr learned my addr 172.21.15.137:0/44230532 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f3861800640 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961659570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 -- 172.21.15.137:0/44230532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 msgr2=0x55a961647310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961647310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 -- 172.21.15.137:0/44230532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 msgr2=0x55a961659570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961659570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f385be00640 1 -- 172.21.15.137:0/44230532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9612810f0 con 0x55a961648a40 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.408+0000 7f3861800640 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961659570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.409+0000 7f385be00640 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961647850 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f385c19ae80 tx=0x7f3848005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db24706f7e150001 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.409+0000 7f384d600640 1 -- 172.21.15.137:0/44230532 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384802cc50 con 0x55a961648a40 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.409+0000 7f384d600640 1 -- 172.21.15.137:0/44230532 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3848005040 con 0x55a961648a40 2024-03-26T14:08:05.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.409+0000 7f384d600640 1 -- 172.21.15.137:0/44230532 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848004b50 con 0x55a961648a40 2024-03-26T14:08:05.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 -- 172.21.15.137:0/44230532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 msgr2=0x55a961647850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961647850 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f385c19ae80 tx=0x7f3848005b50 comp rx=0 tx=0).stop 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 -- 172.21.15.137:0/44230532 shutdown_connections 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961659570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961647850 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 --2- 172.21.15.137:0/44230532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961647310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 -- 172.21.15.137:0/44230532 >> 172.21.15.137:0/44230532 conn(0x55a961610200 msgr2=0x55a961646c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 -- 172.21.15.137:0/44230532 shutdown_connections 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 -- 172.21.15.137:0/44230532 wait complete. 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.410+0000 7f38665cf740 1 -- start start 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 Processor -- start 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961656f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961657460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961658470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1aa960 con 0x55a961648a40 2024-03-26T14:08:05.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1abc80 con 0x55a96164dbb0 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1aafc0 con 0x55a961648050 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f38665cf740 10 client.0 _ll_register_callbacks cb 0x7f3854008cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f385be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961657460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f385be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961657460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60452/0 (socket says 172.21.15.137:60452) 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f385be00640 1 -- 172.21.15.137:0/3922732157 learned_addr learned my addr 172.21.15.137:0/3922732157 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.411+0000 7f3861800640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961658470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f3861800640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961656f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f385be00640 1 -- 172.21.15.137:0/3922732157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 msgr2=0x55a961656f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f385be00640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961656f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f385be00640 1 -- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 msgr2=0x55a961658470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f385be00640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961658470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f385be00640 1 -- 172.21.15.137:0/3922732157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a961658a60 con 0x55a961648a40 2024-03-26T14:08:05.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f3861800640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961658470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f385be00640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961657460 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3848036040 tx=0x7f384802cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aae53a3ab31d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f3842a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f3842a00640 1 -- 172.21.15.137:0/3922732157 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480053f0 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f3842a00640 1 -- 172.21.15.137:0/3922732157 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3848005590 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.412+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3848005040 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f3842a00640 1 -- 172.21.15.137:0/3922732157 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f384805f050 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 10 client.16842 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a9616b96f0 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f3842a00640 1 -- 172.21.15.137:0/3922732157 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3848005040 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f3842a00640 1 -- 172.21.15.137:0/3922732157 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f38480035f0 con 0x55a961648a40 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 10 client.16842 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 20 client.16842 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a9616ba070 0x55a9616bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 4 client.16842 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:08:05.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.413+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a9615b4ae0 con 0x55a9616ba070 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.414+0000 7f3861800640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a9616ba070 0x55a9616bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.415+0000 7f3861800640 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a9616ba070 0x55a9616bc490 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22ac725a4492968 server_cookie=9ffba176825f010c in_seq=0 out_seq=0 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.415+0000 7f3842a00640 10 client.16842 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.415+0000 7f3842a00640 1 -- 172.21.15.137:0/3922732157 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a9615b4ae0 con 0x55a9616ba070 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.415+0000 7f3842a00640 10 client.16842 handle_command_reply: tid=0 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.415+0000 7f38665cf740 1 client.16842 shutdown 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 msgr2=0x55a961657460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961657460 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3848036040 tx=0x7f384802cbb0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 shutdown_connections 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a9616ba070 0x55a9616bc490 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a96164dbb0 0x55a961658470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a961648a40 0x55a961657460 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 --2- 172.21.15.137:0/3922732157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a961648050 0x55a961656f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f3842a00640 0 client.16842 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 >> 172.21.15.137:0/3922732157 conn(0x55a961610200 msgr2=0x55a961647490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 shutdown_connections 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 1 -- 172.21.15.137:0/3922732157 wait complete. 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.416+0000 7f38665cf740 20 client.16842 trim_cache size 0 max 16384 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:08:05.418 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38440a2880 msgr2=0x7f38440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38440a2880 0x7f38440a4d40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3850097b50 tx=0x7f3850005f30 comp rx=0 tx=0).stop 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 msgr2=0x7f385c199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 0x7f385c199970 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f385000d770 tx=0x7f385000dc30 comp rx=0 tx=0).stop 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 shutdown_connections 2024-03-26T14:08:05.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f38440a2880 0x7f38440a4d40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f385c1031d0 0x7f385c1a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f385c102810 0x7f385c199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 --2- 172.21.15.137:0/3715005245 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f385c0ff840 0x7f385c199970 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 >> 172.21.15.137:0/3715005245 conn(0x7f385c0fb190 msgr2=0x7f385c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 shutdown_connections 2024-03-26T14:08:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.417+0000 7f3862200640 1 -- 172.21.15.137:0/3715005245 wait complete. 2024-03-26T14:08:05.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:08:05.437 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:08:05.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 -- 172.21.15.137:0/520382089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b90101f70 msgr2=0x7f8b90104360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 --2- 172.21.15.137:0/520382089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b90101f70 0x7f8b90104360 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009900 tx=0x7f8b8002bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 -- 172.21.15.137:0/520382089 shutdown_connections 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 --2- 172.21.15.137:0/520382089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b901071d0 0x7f8b90109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 --2- 172.21.15.137:0/520382089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b901048a0 0x7f8b90106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 --2- 172.21.15.137:0/520382089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b90101f70 0x7f8b90104360 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 -- 172.21.15.137:0/520382089 >> 172.21.15.137:0/520382089 conn(0x7f8b900fb980 msgr2=0x7f8b900fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 -- 172.21.15.137:0/520382089 shutdown_connections 2024-03-26T14:08:05.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.640+0000 7f8b95000640 1 -- 172.21.15.137:0/520382089 wait complete. 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.641+0000 7f8b95000640 1 Processor -- start 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.641+0000 7f8b95000640 1 -- start start 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b95000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b90101f70 0x7f8b90199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b95000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b901048a0 0x7f8b90199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b95000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 0x7f8b901a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b95000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9010d030 con 0x7f8b901071d0 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b95000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9010ceb0 con 0x7f8b901048a0 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b95000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9010d1b0 con 0x7f8b90101f70 2024-03-26T14:08:05.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b8fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 0x7f8b901a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b8fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 0x7f8b901a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60470/0 (socket says 172.21.15.137:60470) 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b8fe00640 1 -- 172.21.15.137:0/1910764602 learned_addr learned my addr 172.21.15.137:0/1910764602 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b8ea00640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b901048a0 0x7f8b90199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.642+0000 7f8b8f400640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b90101f70 0x7f8b90199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b8fe00640 1 -- 172.21.15.137:0/1910764602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b90101f70 msgr2=0x7f8b90199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b8fe00640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b90101f70 0x7f8b90199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b8fe00640 1 -- 172.21.15.137:0/1910764602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b901048a0 msgr2=0x7f8b90199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b8fe00640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b901048a0 0x7f8b90199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b8fe00640 1 -- 172.21.15.137:0/1910764602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b901a1550 con 0x7f8b901071d0 2024-03-26T14:08:05.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b8fe00640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 0x7f8b901a0ee0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800eb10 tx=0x7f8b8800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88004b20 con 0x7f8b901071d0 2024-03-26T14:08:05.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8b88004cc0 con 0x7f8b901071d0 2024-03-26T14:08:05.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b901a1840 con 0x7f8b901071d0 2024-03-26T14:08:05.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.643+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88010690 con 0x7f8b901071d0 2024-03-26T14:08:05.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.644+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901a1d40 con 0x7f8b901071d0 2024-03-26T14:08:05.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.645+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b90108d20 con 0x7f8b901071d0 2024-03-26T14:08:05.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.647+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8b88005020 con 0x7f8b901071d0 2024-03-26T14:08:05.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.649+0000 7f8b87e00640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8b740a2800 0x7f8b740a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:05.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.649+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8b880cf470 con 0x7f8b901071d0 2024-03-26T14:08:05.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.649+0000 7f8b8f400640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8b740a2800 0x7f8b740a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:05.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.650+0000 7f8b8f400640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8b740a2800 0x7f8b740a4cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000c00 tx=0x7f8b80005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:05.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.654+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8b8809c140 con 0x7f8b901071d0 2024-03-26T14:08:05.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.930+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b900622a0 con 0x7f8b901071d0 2024-03-26T14:08:05.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.932+0000 7f8b87e00640 1 -- 172.21.15.137:0/1910764602 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f8b880afe20 con 0x7f8b901071d0 2024-03-26T14:08:05.931 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:08:05.931 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:08:05.932 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8b740a2800 msgr2=0x7f8b740a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8b740a2800 0x7f8b740a4cc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000c00 tx=0x7f8b80005be0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 msgr2=0x7f8b901a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 0x7f8b901a0ee0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800eb10 tx=0x7f8b8800c2d0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 shutdown_connections 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8b740a2800 0x7f8b740a4cc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8b901048a0 0x7f8b90199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8b901071d0 0x7f8b901a0ee0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 --2- 172.21.15.137:0/1910764602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b90101f70 0x7f8b90199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 >> 172.21.15.137:0/1910764602 conn(0x7f8b900fb980 msgr2=0x7f8b901017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 shutdown_connections 2024-03-26T14:08:05.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:05.935+0000 7f8b95000640 1 -- 172.21.15.137:0/1910764602 wait complete. 2024-03-26T14:08:05.954 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:08:05.954 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.176+0000 7fc28c400640 1 -- 172.21.15.137:0/4142003862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2841028a0 msgr2=0x7fc284102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.176+0000 7fc28c400640 1 --2- 172.21.15.137:0/4142003862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2841028a0 0x7fc284102d20 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc270009900 tx=0x7fc27002bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.176+0000 7fc28c400640 1 -- 172.21.15.137:0/4142003862 shutdown_connections 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.176+0000 7fc28c400640 1 --2- 172.21.15.137:0/4142003862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc284103260 0x7fc284109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.176+0000 7fc28c400640 1 --2- 172.21.15.137:0/4142003862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc2841028a0 0x7fc284102d20 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.176+0000 7fc28c400640 1 --2- 172.21.15.137:0/4142003862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2841016b0 0x7fc284101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.177+0000 7fc28c400640 1 -- 172.21.15.137:0/4142003862 >> 172.21.15.137:0/4142003862 conn(0x7fc2840fcc40 msgr2=0x7fc2840ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.177+0000 7fc28c400640 1 -- 172.21.15.137:0/4142003862 shutdown_connections 2024-03-26T14:08:06.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.177+0000 7fc28c400640 1 -- 172.21.15.137:0/4142003862 wait complete. 2024-03-26T14:08:06.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 Processor -- start 2024-03-26T14:08:06.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 -- start start 2024-03-26T14:08:06.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2841016b0 0x7fc2841978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2841028a0 0x7fc284197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 0x7fc28419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410d1c0 con 0x7fc284103260 2024-03-26T14:08:06.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410d040 con 0x7fc2841028a0 2024-03-26T14:08:06.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.178+0000 7fc28c400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28410d340 con 0x7fc2841016b0 2024-03-26T14:08:06.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 0x7fc28419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 0x7fc28419eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60478/0 (socket says 172.21.15.137:60478) 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 -- 172.21.15.137:0/631350523 learned_addr learned my addr 172.21.15.137:0/631350523 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc289000640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2841028a0 0x7fc284197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc289a00640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2841016b0 0x7fc2841978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 -- 172.21.15.137:0/631350523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2841016b0 msgr2=0x7fc2841978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2841016b0 0x7fc2841978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 -- 172.21.15.137:0/631350523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2841028a0 msgr2=0x7fc284197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2841028a0 0x7fc284197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc28a400640 1 -- 172.21.15.137:0/631350523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28419f510 con 0x7fc284103260 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.179+0000 7fc289000640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2841028a0 0x7fc284197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.180+0000 7fc28a400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 0x7fc28419eea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00eaa0 tx=0x7fc27c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.180+0000 7fc282000640 1 -- 172.21.15.137:0/631350523 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00cdb0 con 0x7fc284103260 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.180+0000 7fc282000640 1 -- 172.21.15.137:0/631350523 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc27c005020 con 0x7fc284103260 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.180+0000 7fc282000640 1 -- 172.21.15.137:0/631350523 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0116d0 con 0x7fc284103260 2024-03-26T14:08:06.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.180+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28419f800 con 0x7fc284103260 2024-03-26T14:08:06.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.180+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28419fd50 con 0x7fc284103260 2024-03-26T14:08:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.183+0000 7fc282000640 1 -- 172.21.15.137:0/631350523 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc27c0051d0 con 0x7fc284103260 2024-03-26T14:08:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.183+0000 7fc28f16a740 1 Processor -- start 2024-03-26T14:08:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 -- start start 2024-03-26T14:08:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c3394a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3394b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c33952040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c338aa2c0 con 0x563c3394adf0 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33852c90 con 0x563c33949c20 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc28f16a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c338a6360 con 0x563c3394b7b0 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc289a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3394b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc289a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3394b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60480/0 (socket says 172.21.15.137:60480) 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc289a00640 1 -- 172.21.15.137:0/3974367393 learned_addr learned my addr 172.21.15.137:0/3974367393 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.184+0000 7fc289000640 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c3394a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289000640 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c33952040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289a00640 1 -- 172.21.15.137:0/3974367393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 msgr2=0x563c33952040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289a00640 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c33952040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289a00640 1 -- 172.21.15.137:0/3974367393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 msgr2=0x563c3394a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289a00640 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c3394a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289a00640 1 -- 172.21.15.137:0/3974367393 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c33877080 con 0x563c3394adf0 2024-03-26T14:08:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289000640 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c33952040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289a00640 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3394b270 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc27800bc90 tx=0x7fc27800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c603beba27b1ab5e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc282000640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc26c0a2880 0x7fc26c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc282000640 1 -- 172.21.15.137:0/631350523 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc27c0cf0c0 con 0x7fc284103260 2024-03-26T14:08:06.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc276a00640 1 -- 172.21.15.137:0/3974367393 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278015070 con 0x563c3394adf0 2024-03-26T14:08:06.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc289000640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc26c0a2880 0x7fc26c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.185+0000 7fc276a00640 1 -- 172.21.15.137:0/3974367393 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2780035f0 con 0x563c3394adf0 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc276a00640 1 -- 172.21.15.137:0/3974367393 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780116a0 con 0x563c3394adf0 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 -- 172.21.15.137:0/3974367393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 msgr2=0x563c3394b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3394b270 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc27800bc90 tx=0x7fc27800f730 comp rx=0 tx=0).stop 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc289000640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc26c0a2880 0x7fc26c0a4d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc284198e00 tx=0x7fc2700058b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 -- 172.21.15.137:0/3974367393 shutdown_connections 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c33952040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3394b270 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3974367393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c3394a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 -- 172.21.15.137:0/3974367393 >> 172.21.15.137:0/3974367393 conn(0x563c33945010 msgr2=0x563c33945410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.186+0000 7fc28f16a740 1 -- 172.21.15.137:0/3974367393 shutdown_connections 2024-03-26T14:08:06.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.187+0000 7fc28f16a740 1 -- 172.21.15.137:0/3974367393 wait complete. 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.187+0000 7fc28f16a740 1 -- start start 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.187+0000 7fc28f16a740 1 Processor -- start 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 0x563c33953e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394adf0 0x563c339543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c3395b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c339527c0 con 0x563c33949c20 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c338660d0 con 0x563c3394adf0 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c33952940 con 0x563c3394b7b0 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28f16a740 10 client.0 _ll_register_callbacks cb 0x563c3386ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 0x563c33953e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 0x563c33953e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60492/0 (socket says 172.21.15.137:60492) 2024-03-26T14:08:06.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 -- 172.21.15.137:0/3096532661 learned_addr learned my addr 172.21.15.137:0/3096532661 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289000640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394adf0 0x563c339543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc28a400640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c3395b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 -- 172.21.15.137:0/3096532661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 msgr2=0x563c3395b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c3395b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 -- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394adf0 msgr2=0x563c339543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394adf0 0x563c339543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.188+0000 7fc289a00640 1 -- 172.21.15.137:0/3096532661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c335be3f0 con 0x563c33949c20 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc289a00640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 0x563c33953e90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc27800ba00 tx=0x7fc27800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab31178d9b40f431 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc26b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc26b400640 1 -- 172.21.15.137:0/3096532661 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278015030 con 0x563c33949c20 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc26b400640 1 -- 172.21.15.137:0/3096532661 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc278004000 con 0x563c33949c20 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2780035f0 con 0x563c33949c20 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc26b400640 1 -- 172.21.15.137:0/3096532661 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc278003bf0 con 0x563c33949c20 2024-03-26T14:08:06.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc28f16a740 10 client.16860 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563c336887d0 con 0x563c33949c20 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.189+0000 7fc26b400640 1 -- 172.21.15.137:0/3096532661 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2780035f0 con 0x563c33949c20 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc26b400640 1 -- 172.21.15.137:0/3096532661 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc2780048a0 con 0x563c33949c20 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc28f16a740 10 client.16860 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc28f16a740 10 client.16860 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc28f16a740 20 client.16860 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c3395f510 0x563c33961940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc28f16a740 4 client.16860 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c338b7800 con 0x563c3395f510 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.190+0000 7fc289000640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c3395f510 0x563c33961940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.191+0000 7fc289000640 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c3395f510 0x563c33961940 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bde4978d5da40760 server_cookie=deabf60b5ca076b8 in_seq=0 out_seq=0 2024-03-26T14:08:06.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.191+0000 7fc26b400640 10 client.16860 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc26b400640 1 -- 172.21.15.137:0/3096532661 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563c338b7800 con 0x563c3395f510 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc26b400640 10 client.16860 handle_command_reply: tid=0 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 client.16860 shutdown 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 msgr2=0x563c33953e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 0x563c33953e90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc27800ba00 tx=0x7fc27800bea0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 shutdown_connections 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c3395f510 0x563c33961940 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394adf0 0x563c339543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394b7b0 0x563c3395b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 --2- 172.21.15.137:0/3096532661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c33949c20 0x563c33953e90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 >> 172.21.15.137:0/3096532661 conn(0x563c33945010 msgr2=0x563c3393feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc26b400640 0 client.16860 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:06.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 shutdown_connections 2024-03-26T14:08:06.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 1 -- 172.21.15.137:0/3096532661 wait complete. 2024-03-26T14:08:06.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.193+0000 7fc28f16a740 20 client.16860 trim_cache size 0 max 16384 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.212+0000 7fc28f16a740 1 Processor -- start 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 -- start start 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c33949c20 0x563c33948d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3395b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394b7b0 0x563c33966200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841aa570 con 0x563c3394adf0 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a8070 con 0x563c3394b7b0 2024-03-26T14:08:06.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc28f16a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a60f0 con 0x563c33949c20 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc289000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3395b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc289000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3395b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60506/0 (socket says 172.21.15.137:60506) 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.213+0000 7fc289000640 1 -- 172.21.15.137:0/4107413482 learned_addr learned my addr 172.21.15.137:0/4107413482 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc28a400640 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394b7b0 0x563c33966200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289a00640 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c33949c20 0x563c33948d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289000640 1 -- 172.21.15.137:0/4107413482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c33949c20 msgr2=0x563c33948d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289000640 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c33949c20 0x563c33948d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289000640 1 -- 172.21.15.137:0/4107413482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394b7b0 msgr2=0x563c33966200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289000640 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394b7b0 0x563c33966200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289000640 1 -- 172.21.15.137:0/4107413482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c3394a610 con 0x563c3394adf0 2024-03-26T14:08:06.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289a00640 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c33949c20 0x563c33948d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:06.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc289000640 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3395b2c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc27002c170 tx=0x7fc270044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a57d1846dc2703 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc276000640 1 -- 172.21.15.137:0/4107413482 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27005b070 con 0x563c3394adf0 2024-03-26T14:08:06.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc276000640 1 -- 172.21.15.137:0/4107413482 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc270047040 con 0x563c3394adf0 2024-03-26T14:08:06.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.214+0000 7fc276000640 1 -- 172.21.15.137:0/4107413482 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc270003850 con 0x563c3394adf0 2024-03-26T14:08:06.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 -- 172.21.15.137:0/4107413482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 msgr2=0x563c3395b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3395b2c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc27002c170 tx=0x7fc270044000 comp rx=0 tx=0).stop 2024-03-26T14:08:06.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 -- 172.21.15.137:0/4107413482 shutdown_connections 2024-03-26T14:08:06.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c3394b7b0 0x563c33966200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394adf0 0x563c3395b2c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 --2- 172.21.15.137:0/4107413482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c33949c20 0x563c33948d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.215+0000 7fc28f16a740 1 -- 172.21.15.137:0/4107413482 >> 172.21.15.137:0/4107413482 conn(0x563c33945010 msgr2=0x563c3393feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.216+0000 7fc28f16a740 1 -- 172.21.15.137:0/4107413482 shutdown_connections 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.216+0000 7fc28f16a740 1 -- 172.21.15.137:0/4107413482 wait complete. 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.216+0000 7fc28f16a740 1 -- start start 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.216+0000 7fc28f16a740 1 Processor -- start 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c33956af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394adf0 0x563c33957030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 0x563c33959f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a66f0 con 0x563c3394b7b0 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a7a10 con 0x563c33949c20 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a6d50 con 0x563c3394adf0 2024-03-26T14:08:06.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28f16a740 10 client.0 _ll_register_callbacks cb 0x7fc27800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 0x563c33959f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 0x563c33959f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60520/0 (socket says 172.21.15.137:60520) 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 -- 172.21.15.137:0/832299055 learned_addr learned my addr 172.21.15.137:0/832299055 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc289a00640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c33956af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc289000640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394adf0 0x563c33957030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 -- 172.21.15.137:0/832299055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394adf0 msgr2=0x563c33957030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394adf0 0x563c33957030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 -- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 msgr2=0x563c33956af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c33956af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc28a400640 1 -- 172.21.15.137:0/832299055 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c3395a580 con 0x563c3394b7b0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.217+0000 7fc289000640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394adf0 0x563c33957030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc28a400640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 0x563c33959f90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc27c097ab0 tx=0x7fc27c09be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5eb0c033d19b93f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc26b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc26b400640 1 -- 172.21.15.137:0/832299055 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0b3020 con 0x563c3394b7b0 2024-03-26T14:08:06.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc26b400640 1 -- 172.21.15.137:0/832299055 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc27c0a3960 con 0x563c3394b7b0 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc270047040 con 0x563c3394b7b0 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc26b400640 1 -- 172.21.15.137:0/832299055 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27c01f040 con 0x563c3394b7b0 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc28f16a740 10 client.16866 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.218+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563c339bae40 con 0x563c3394b7b0 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc26b400640 1 -- 172.21.15.137:0/832299055 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc270047040 con 0x563c3394b7b0 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc26b400640 1 -- 172.21.15.137:0/832299055 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc27c00f040 con 0x563c3394b7b0 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc28f16a740 10 client.16866 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc28f16a740 10 client.16866 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc28f16a740 20 client.16866 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc28f16a740 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c339bb840 0x563c339bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc28f16a740 4 client.16866 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:08:06.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563c338b1b50 con 0x563c339bb840 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.219+0000 7fc289a00640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c339bb840 0x563c339bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.220+0000 7fc289a00640 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c339bb840 0x563c339bdc60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=295138bb0182c4bc server_cookie=c722a90e4c8a64cc in_seq=0 out_seq=0 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.220+0000 7fc26b400640 10 client.16866 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc26b400640 1 -- 172.21.15.137:0/832299055 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563c338b1b50 con 0x563c339bb840 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc26b400640 10 client.16866 handle_command_reply: tid=0 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 client.16866 shutdown 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 msgr2=0x563c33959f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 0x563c33959f90 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc27c097ab0 tx=0x7fc27c09be10 comp rx=0 tx=0).stop 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 shutdown_connections 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x563c339bb840 0x563c339bdc60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563c3394b7b0 0x563c33959f90 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563c3394adf0 0x563c33957030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 --2- 172.21.15.137:0/832299055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c33949c20 0x563c33956af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 >> 172.21.15.137:0/832299055 conn(0x563c33945010 msgr2=0x563c33966200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc26b400640 0 client.16866 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 shutdown_connections 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 1 -- 172.21.15.137:0/832299055 wait complete. 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.221+0000 7fc28f16a740 20 client.16866 trim_cache size 0 max 16384 2024-03-26T14:08:06.224 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc26c0a2880 msgr2=0x7fc26c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc26c0a2880 0x7fc26c0a4d40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc284198e00 tx=0x7fc2700058b0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 msgr2=0x7fc28419eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 0x7fc28419eea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00eaa0 tx=0x7fc27c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 shutdown_connections 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc26c0a2880 0x7fc26c0a4d40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2841028a0 0x7fc284197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc284103260 0x7fc28419eea0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 --2- 172.21.15.137:0/631350523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2841016b0 0x7fc2841978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 >> 172.21.15.137:0/631350523 conn(0x7fc2840fcc40 msgr2=0x7fc284105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 shutdown_connections 2024-03-26T14:08:06.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.222+0000 7fc28c400640 1 -- 172.21.15.137:0/631350523 wait complete. 2024-03-26T14:08:06.242 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:08:06.242 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 -- 172.21.15.137:0/818701117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a24105350 msgr2=0x7f9a2410bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/818701117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a24105350 0x7f9a2410bbe0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009ca0 tx=0x7f9a1c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 -- 172.21.15.137:0/818701117 shutdown_connections 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/818701117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 0x7f9a24104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/818701117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a24105350 0x7f9a2410bbe0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/818701117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a241037a0 0x7f9a24103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 -- 172.21.15.137:0/818701117 >> 172.21.15.137:0/818701117 conn(0x7f9a240fed10 msgr2=0x7f9a24101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 -- 172.21.15.137:0/818701117 shutdown_connections 2024-03-26T14:08:06.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.462+0000 7f9a2aa00640 1 -- 172.21.15.137:0/818701117 wait complete. 2024-03-26T14:08:06.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.463+0000 7f9a2aa00640 1 Processor -- start 2024-03-26T14:08:06.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 -- start start 2024-03-26T14:08:06.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 0x7f9a24199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 0x7f9a24199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a24105350 0x7f9a241a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2410f2d0 con 0x7f9a241037a0 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2410f150 con 0x7f9a24104990 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a2aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2410f450 con 0x7f9a24105350 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a29600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 0x7f9a24199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a29600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 0x7f9a24199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60544/0 (socket says 172.21.15.137:60544) 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a29600640 1 -- 172.21.15.137:0/1007920190 learned_addr learned my addr 172.21.15.137:0/1007920190 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.464+0000 7f9a28c00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 0x7f9a24199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a24105350 0x7f9a241a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a29600640 1 -- 172.21.15.137:0/1007920190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a24105350 msgr2=0x7f9a241a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a29600640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a24105350 0x7f9a241a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a29600640 1 -- 172.21.15.137:0/1007920190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 msgr2=0x7f9a24199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a29600640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 0x7f9a24199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a29600640 1 -- 172.21.15.137:0/1007920190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a241a1560 con 0x7f9a241037a0 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a24105350 0x7f9a241a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a28c00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 0x7f9a24199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a29600640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 0x7f9a24199940 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800e9c0 tx=0x7f9a1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a22000640 1 -- 172.21.15.137:0/1007920190 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800cd80 con 0x7f9a241037a0 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a241a1850 con 0x7f9a241037a0 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.465+0000 7f9a22000640 1 -- 172.21.15.137:0/1007920190 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a180035f0 con 0x7f9a241037a0 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.466+0000 7f9a22000640 1 -- 172.21.15.137:0/1007920190 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18010630 con 0x7f9a241037a0 2024-03-26T14:08:06.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.466+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a241a1d50 con 0x7f9a241037a0 2024-03-26T14:08:06.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 Processor -- start 2024-03-26T14:08:06.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 -- start start 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b2fae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b2fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b302e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307b25f2c0 con 0x56307b2fc580 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307b207c90 con 0x56307b2fa9f0 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.470+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307b25b360 con 0x56307b2fbbc0 2024-03-26T14:08:06.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a22000640 1 -- 172.21.15.137:0/1007920190 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9a180037a0 con 0x7f9a241037a0 2024-03-26T14:08:06.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b302e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b302e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60558/0 (socket says 172.21.15.137:60558) 2024-03-26T14:08:06.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 -- 172.21.15.137:0/2460508015 learned_addr learned my addr 172.21.15.137:0/2460508015 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a2a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b2fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 -- 172.21.15.137:0/2460508015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 msgr2=0x56307b2fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a29600640 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b2fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b2fc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 -- 172.21.15.137:0/2460508015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 msgr2=0x56307b2fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b2fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a28c00640 1 -- 172.21.15.137:0/2460508015 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56307b22c080 con 0x56307b2fc580 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.471+0000 7f9a2a000640 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b2fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.472+0000 7f9a28c00640 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b302e10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009cf0 tx=0x7f9a100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2874710b4934e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.472+0000 7f9a16600640 1 -- 172.21.15.137:0/2460508015 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10012450 con 0x56307b2fc580 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.472+0000 7f9a16600640 1 -- 172.21.15.137:0/2460508015 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a10012aa0 con 0x56307b2fc580 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.472+0000 7f9a16600640 1 -- 172.21.15.137:0/2460508015 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000fb80 con 0x56307b2fc580 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.472+0000 7f9a22000640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2800 0x7f9a0c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.472+0000 7f9a22000640 1 -- 172.21.15.137:0/1007920190 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9a180cf430 con 0x7f9a241037a0 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2800 0x7f9a0c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 -- 172.21.15.137:0/2460508015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 msgr2=0x56307b302e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b302e10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009cf0 tx=0x7f9a100117b0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 -- 172.21.15.137:0/2460508015 shutdown_connections 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b302e10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b2fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/2460508015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b2fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 -- 172.21.15.137:0/2460508015 >> 172.21.15.137:0/2460508015 conn(0x56307b2f5de0 msgr2=0x56307b2f61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 -- 172.21.15.137:0/2460508015 shutdown_connections 2024-03-26T14:08:06.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2800 0x7f9a0c0a4cc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x56307b2fbac0 tx=0x7f9a1c005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.473+0000 7f9a2edd6740 1 -- 172.21.15.137:0/2460508015 wait complete. 2024-03-26T14:08:06.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.474+0000 7f9a2edd6740 1 -- start start 2024-03-26T14:08:06.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.474+0000 7f9a2edd6740 1 Processor -- start 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fa9f0 0x56307b304c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 0x56307b305170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fc580 0x56307b30c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307b303590 con 0x56307b2fbbc0 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307b21b0d0 con 0x56307b2fc580 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56307b303710 con 0x56307b2fa9f0 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2edd6740 10 client.0 _ll_register_callbacks cb 0x56307b21fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 0x56307b305170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 0x56307b305170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60560/0 (socket says 172.21.15.137:60560) 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 -- 172.21.15.137:0/1397971647 learned_addr learned my addr 172.21.15.137:0/1397971647 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fc580 0x56307b30c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a29600640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fa9f0 0x56307b304c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 -- 172.21.15.137:0/1397971647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fa9f0 msgr2=0x56307b304c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fa9f0 0x56307b304c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 -- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fc580 msgr2=0x56307b30c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fc580 0x56307b30c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a28c00640 1 -- 172.21.15.137:0/1397971647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56307af733f0 con 0x56307b2fbbc0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.475+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fc580 0x56307b30c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.476+0000 7f9a28c00640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 0x56307b305170 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9a1001c040 tx=0x7f9a10005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb55654b5987b7ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.476+0000 7f9a0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.476+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1397971647 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10029030 con 0x56307b2fbbc0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.476+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1397971647 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9a10020040 con 0x56307b2fbbc0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.476+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a10012aa0 con 0x56307b2fbbc0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1397971647 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a10033050 con 0x56307b2fbbc0 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 10 client.16878 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:06.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56307b03d7d0 con 0x56307b2fbbc0 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1397971647 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a10012aa0 con 0x56307b2fbbc0 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1397971647 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9a1000fa20 con 0x56307b2fbbc0 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 10 client.16878 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 10 client.16878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 20 client.16878 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b310420 0x56307b312810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 4 client.16878 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.477+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56307b26c800 con 0x56307b310420 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.478+0000 7f9a29600640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b310420 0x56307b312810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.479+0000 7f9a0aa00640 10 client.16878 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:06.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.479+0000 7f9a29600640 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b310420 0x56307b312810 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca1d132e4d3b751a server_cookie=b6678c9eb6e6cb41 in_seq=0 out_seq=0 2024-03-26T14:08:06.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.480+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1397971647 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56307b26c800 con 0x56307b310420 2024-03-26T14:08:06.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.480+0000 7f9a0aa00640 10 client.16878 handle_command_reply: tid=0 2024-03-26T14:08:06.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 client.16878 shutdown 2024-03-26T14:08:06.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 msgr2=0x56307b305170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 0x56307b305170 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9a1001c040 tx=0x7f9a10005740 comp rx=0 tx=0).stop 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 shutdown_connections 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b310420 0x56307b312810 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fc580 0x56307b30c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fbbc0 0x56307b305170 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1397971647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fa9f0 0x56307b304c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 >> 172.21.15.137:0/1397971647 conn(0x56307b2f5de0 msgr2=0x56307b2f5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a0aa00640 0 client.16878 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 shutdown_connections 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1397971647 wait complete. 2024-03-26T14:08:06.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.481+0000 7f9a2edd6740 20 client.16878 trim_cache size 0 max 16384 2024-03-26T14:08:06.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 Processor -- start 2024-03-26T14:08:06.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 -- start start 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fa9f0 0x56307b312430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 0x56307b30c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fc580 0x56307b2f9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241ae770 con 0x56307b2fa9f0 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241ac270 con 0x56307b2fbbc0 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.498+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241aa2f0 con 0x56307b2fc580 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a28c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 0x56307b30c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a28c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 0x56307b30c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:59356/0 (socket says 172.21.15.137:59356) 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a29600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fa9f0 0x56307b312430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a28c00640 1 -- 172.21.15.137:0/514849781 learned_addr learned my addr 172.21.15.137:0/514849781 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a2a000640 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fc580 0x56307b2f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a29600640 1 -- 172.21.15.137:0/514849781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fc580 msgr2=0x56307b2f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a29600640 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fc580 0x56307b2f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a29600640 1 -- 172.21.15.137:0/514849781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 msgr2=0x56307b30c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a29600640 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 0x56307b30c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a29600640 1 -- 172.21.15.137:0/514849781 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56307b2f6d50 con 0x56307b2fa9f0 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.499+0000 7f9a28c00640 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 0x56307b30c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.500+0000 7f9a29600640 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fa9f0 0x56307b312430 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800cfd0 tx=0x7f9a18086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c839cda9750cdcd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.500+0000 7f9a15c00640 1 -- 172.21.15.137:0/514849781 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1809fb50 con 0x56307b2fa9f0 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.500+0000 7f9a15c00640 1 -- 172.21.15.137:0/514849781 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a180a41a0 con 0x56307b2fa9f0 2024-03-26T14:08:06.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.500+0000 7f9a15c00640 1 -- 172.21.15.137:0/514849781 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a18097750 con 0x56307b2fa9f0 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 -- 172.21.15.137:0/514849781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fa9f0 msgr2=0x56307b312430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fa9f0 0x56307b312430 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800cfd0 tx=0x7f9a18086040 comp rx=0 tx=0).stop 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 -- 172.21.15.137:0/514849781 shutdown_connections 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fbbc0 0x56307b30c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fc580 0x56307b2f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/514849781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fa9f0 0x56307b312430 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 -- 172.21.15.137:0/514849781 >> 172.21.15.137:0/514849781 conn(0x56307b2f5de0 msgr2=0x56307b2f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 -- 172.21.15.137:0/514849781 shutdown_connections 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 -- 172.21.15.137:0/514849781 wait complete. 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 -- start start 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.501+0000 7f9a2edd6740 1 Processor -- start 2024-03-26T14:08:06.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b30a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b30a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b30b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241aa8f0 con 0x56307b2fc580 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241abc10 con 0x56307b2fa9f0 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241aaf50 con 0x56307b2fbbc0 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2edd6740 10 client.0 _ll_register_callbacks cb 0x7f9a1c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b30b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b30b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60590/0 (socket says 172.21.15.137:60590) 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a2a000640 1 -- 172.21.15.137:0/1930463195 learned_addr learned my addr 172.21.15.137:0/1930463195 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.502+0000 7f9a29600640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b30a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a28c00640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b30a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2a000640 1 -- 172.21.15.137:0/1930463195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 msgr2=0x56307b30a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b30a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2a000640 1 -- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 msgr2=0x56307b30a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b30a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2a000640 1 -- 172.21.15.137:0/1930463195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56307b30bc10 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a29600640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b30a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2a000640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b30b620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009a60 tx=0x7f9a1c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f1d0e5bfcf2f65 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1930463195 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c052020 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1930463195 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9a1c004040 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.503+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a180a41a0 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1930463195 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1c048040 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 10 client.16884 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56307b36c510 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1930463195 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a180a41a0 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1930463195 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9a1c04e930 con 0x56307b2fc580 2024-03-26T14:08:06.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 10 client.16884 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:06.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 10 client.16884 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:08:06.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 20 client.16884 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:06.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b36cf40 0x56307b36f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 4 client.16884 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:08:06.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.504+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56307b266b50 con 0x56307b36cf40 2024-03-26T14:08:06.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.505+0000 7f9a29600640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b36cf40 0x56307b36f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a29600640 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b36cf40 0x56307b36f360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a0d6ea0e77e78437 server_cookie=229c2200e688bc71 in_seq=0 out_seq=0 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a0aa00640 10 client.16884 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a0aa00640 1 -- 172.21.15.137:0/1930463195 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56307b266b50 con 0x56307b36cf40 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a0aa00640 10 client.16884 handle_command_reply: tid=0 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 client.16884 shutdown 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 msgr2=0x56307b30b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b30b620 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009a60 tx=0x7f9a1c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 shutdown_connections 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56307b36cf40 0x56307b36f360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56307b2fc580 0x56307b30b620 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56307b2fbbc0 0x56307b30a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 --2- 172.21.15.137:0/1930463195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56307b2fa9f0 0x56307b30a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 >> 172.21.15.137:0/1930463195 conn(0x56307b2f5de0 msgr2=0x56307b2f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a0aa00640 0 client.16884 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.506+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 shutdown_connections 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2edd6740 1 -- 172.21.15.137:0/1930463195 wait complete. 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2edd6740 20 client.16884 trim_cache size 0 max 16384 2024-03-26T14:08:06.509 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:08:06.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2800 msgr2=0x7f9a0c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2800 0x7f9a0c0a4cc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x56307b2fbac0 tx=0x7f9a1c005d80 comp rx=0 tx=0).stop 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 msgr2=0x7f9a24199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 0x7f9a24199940 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9a1800e9c0 tx=0x7f9a1800ee80 comp rx=0 tx=0).stop 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 shutdown_connections 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9a0c0a2800 0x7f9a0c0a4cc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9a24104990 0x7f9a24199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a24105350 0x7f9a241a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 --2- 172.21.15.137:0/1007920190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9a241037a0 0x7f9a24199940 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.507+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 >> 172.21.15.137:0/1007920190 conn(0x7f9a240fed10 msgr2=0x7f9a24107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.508+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 shutdown_connections 2024-03-26T14:08:06.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.508+0000 7f9a2aa00640 1 -- 172.21.15.137:0/1007920190 wait complete. 2024-03-26T14:08:06.525 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:08:06.525 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:08:06.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2327482364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 msgr2=0x7f2ff4102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2327482364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4102c10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009900 tx=0x7f2fe002bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2327482364 shutdown_connections 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2327482364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ff4103150 0x7f2ff41099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2327482364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4102c10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2327482364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ff41015a0 0x7f2ff41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2327482364 >> 172.21.15.137:0/2327482364 conn(0x7f2ff40fcb10 msgr2=0x7f2ff40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2327482364 shutdown_connections 2024-03-26T14:08:06.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.734+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2327482364 wait complete. 2024-03-26T14:08:06.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.735+0000 7f2ff3e00640 1 Processor -- start 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.735+0000 7f2ff3e00640 1 -- start start 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.735+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ff41015a0 0x7f2ff41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ff4103150 0x7f2ff419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff410d120 con 0x7f2ff4102790 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff410cfa0 con 0x7f2ff4103150 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff3e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff410d2a0 con 0x7f2ff41015a0 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60600/0 (socket says 172.21.15.137:60600) 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 -- 172.21.15.137:0/2093803215 learned_addr learned my addr 172.21.15.137:0/2093803215 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ff41015a0 0x7f2ff41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff3400640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ff4103150 0x7f2ff419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 -- 172.21.15.137:0/2093803215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ff41015a0 msgr2=0x7f2ff41977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ff41015a0 0x7f2ff41977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 -- 172.21.15.137:0/2093803215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ff4103150 msgr2=0x7f2ff419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ff4103150 0x7f2ff419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.736+0000 7f2ff2000640 1 -- 172.21.15.137:0/2093803215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff419f3e0 con 0x7f2ff4102790 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.737+0000 7f2ff2000640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4197d00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0000c00 tx=0x7f2fe00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.737+0000 7f2feb400640 1 -- 172.21.15.137:0/2093803215 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0043020 con 0x7f2ff4102790 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.737+0000 7f2feb400640 1 -- 172.21.15.137:0/2093803215 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fe0004aa0 con 0x7f2ff4102790 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.737+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff419f670 con 0x7f2ff4102790 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.737+0000 7f2feb400640 1 -- 172.21.15.137:0/2093803215 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003d00 con 0x7f2ff4102790 2024-03-26T14:08:06.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.737+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff419fc10 con 0x7f2ff4102790 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 Processor -- start 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 -- start start 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 0x55fec2ddbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ddd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddd580 0x55fec2de3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec2d402c0 con 0x55fec2ddb9f0 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec2ce8c90 con 0x55fec2ddd580 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.741+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec2d3c360 con 0x55fec2ddcbc0 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2feb400640 1 -- 172.21.15.137:0/2093803215 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2fe0046020 con 0x7f2ff4102790 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 0x55fec2ddbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 0x55fec2ddbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60604/0 (socket says 172.21.15.137:60604) 2024-03-26T14:08:06.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 -- 172.21.15.137:0/768488141 learned_addr learned my addr 172.21.15.137:0/768488141 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ddd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 -- 172.21.15.137:0/768488141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 msgr2=0x55fec2ddd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ddd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff2000640 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddd580 0x55fec2de3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 -- 172.21.15.137:0/768488141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddd580 msgr2=0x55fec2de3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddd580 0x55fec2de3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.742+0000 7f2ff3400640 1 -- 172.21.15.137:0/768488141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fec2d0d080 con 0x55fec2ddb9f0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.743+0000 7f2ff3400640 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 0x55fec2ddbe10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009ee0 tx=0x7f2fec010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448e26f30f06e829 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.743+0000 7f2fdfe00640 1 -- 172.21.15.137:0/768488141 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec016070 con 0x55fec2ddb9f0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.743+0000 7f2fdfe00640 1 -- 172.21.15.137:0/768488141 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fec004d20 con 0x55fec2ddb9f0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.743+0000 7f2fdfe00640 1 -- 172.21.15.137:0/768488141 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec012780 con 0x55fec2ddb9f0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.743+0000 7f2feb400640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2fd80a2880 0x7f2fd80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.743+0000 7f2feb400640 1 -- 172.21.15.137:0/2093803215 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2fe00f11f0 con 0x7f2ff4102790 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2fd80a2880 0x7f2fd80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/768488141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 msgr2=0x55fec2ddbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 0x55fec2ddbe10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009ee0 tx=0x7f2fec010740 comp rx=0 tx=0).stop 2024-03-26T14:08:06.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/768488141 shutdown_connections 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddd580 0x55fec2de3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ddd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/768488141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddb9f0 0x55fec2ddbe10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/768488141 >> 172.21.15.137:0/768488141 conn(0x55fec2dd6de0 msgr2=0x55fec2dd71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/768488141 shutdown_connections 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/768488141 wait complete. 2024-03-26T14:08:06.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.744+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2fd80a2880 0x7f2fd80a4d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ff41025f0 tx=0x7f2fe4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.745+0000 7f2ffc9eb740 1 -- start start 2024-03-26T14:08:06.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.745+0000 7f2ffc9eb740 1 Processor -- start 2024-03-26T14:08:06.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2de5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2de61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2ded210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec2de4590 con 0x55fec2ddd580 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec2cfc0d0 con 0x55fec2ddb9f0 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fec2de4710 con 0x55fec2ddcbc0 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ffc9eb740 10 client.0 _ll_register_callbacks cb 0x55fec2d00e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2ded210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2ded210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60610/0 (socket says 172.21.15.137:60610) 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 -- 172.21.15.137:0/3351143809 learned_addr learned my addr 172.21.15.137:0/3351143809 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2de5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff2000640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2de61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 -- 172.21.15.137:0/3351143809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 msgr2=0x55fec2de61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2de61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 -- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 msgr2=0x55fec2de5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2de5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.746+0000 7f2ff3400640 1 -- 172.21.15.137:0/3351143809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fec2a543f0 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.747+0000 7f2ff3400640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2ded210 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f2fec004c70 tx=0x7f2fec004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fbd5bfe72f7a2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.747+0000 7f2fdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.747+0000 7f2fdcc00640 1 -- 172.21.15.137:0/3351143809 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec016040 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.747+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fec004d20 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.747+0000 7f2fdcc00640 1 -- 172.21.15.137:0/3351143809 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2fec0052e0 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2fdcc00640 1 -- 172.21.15.137:0/3351143809 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fec030050 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2ffc9eb740 10 client.16896 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fec2b1e7d0 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2fdcc00640 1 -- 172.21.15.137:0/3351143809 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fec004d20 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2fdcc00640 1 -- 172.21.15.137:0/3351143809 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2fec02b030 con 0x55fec2ddd580 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2ffc9eb740 10 client.16896 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2ffc9eb740 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.748+0000 7f2ffc9eb740 20 client.16896 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:06.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.749+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2df1370 0x55fec2df3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.749+0000 7f2ffc9eb740 4 client.16896 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:06.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.749+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fec2d4d800 con 0x55fec2df1370 2024-03-26T14:08:06.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.749+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2df1370 0x55fec2df3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.750+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2df1370 0x55fec2df3760 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53342e2bc0b83f5 server_cookie=443df9baf0b62da7 in_seq=0 out_seq=0 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.750+0000 7f2fdcc00640 10 client.16896 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2fdcc00640 1 -- 172.21.15.137:0/3351143809 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fec2d4d800 con 0x55fec2df1370 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2fdcc00640 10 client.16896 handle_command_reply: tid=0 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 client.16896 shutdown 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 msgr2=0x55fec2ded210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2ded210 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f2fec004c70 tx=0x7f2fec004ca0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 shutdown_connections 2024-03-26T14:08:06.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2df1370 0x55fec2df3760 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2ded210 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2de61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/3351143809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2de5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 >> 172.21.15.137:0/3351143809 conn(0x55fec2dd6de0 msgr2=0x55fec2dd6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2fdcc00640 0 client.16896 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 shutdown_connections 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/3351143809 wait complete. 2024-03-26T14:08:06.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.752+0000 7f2ffc9eb740 20 client.16896 trim_cache size 0 max 16384 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.766+0000 7f2ffc9eb740 1 Processor -- start 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.766+0000 7f2ffc9eb740 1 -- start start 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dd5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ded1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2df7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41ac590 con 0x55fec2ddd580 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41aa090 con 0x55fec2ddb9f0 2024-03-26T14:08:06.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41a8110 con 0x55fec2ddcbc0 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2df7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2df7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60620/0 (socket says 172.21.15.137:60620) 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ff3400640 1 -- 172.21.15.137:0/2597762329 learned_addr learned my addr 172.21.15.137:0/2597762329 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dd5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.767+0000 7f2ff2000640 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ded1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2ff3400640 1 -- 172.21.15.137:0/2597762329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 msgr2=0x55fec2ded1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2ff3400640 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ded1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2ff3400640 1 -- 172.21.15.137:0/2597762329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 msgr2=0x55fec2dd5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2ff3400640 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dd5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2ff3400640 1 -- 172.21.15.137:0/2597762329 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fec2dd6b50 con 0x55fec2ddd580 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2ff3400640 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2df7f60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005b40 tx=0x7f2fec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668cba33a7f7ded2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2fdf400640 1 -- 172.21.15.137:0/2597762329 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec039070 con 0x55fec2ddd580 2024-03-26T14:08:06.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2fdf400640 1 -- 172.21.15.137:0/2597762329 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fec009560 con 0x55fec2ddd580 2024-03-26T14:08:06.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.768+0000 7f2fdf400640 1 -- 172.21.15.137:0/2597762329 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0056b0 con 0x55fec2ddd580 2024-03-26T14:08:06.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/2597762329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 msgr2=0x55fec2df7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2df7f60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005b40 tx=0x7f2fec005f90 comp rx=0 tx=0).stop 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/2597762329 shutdown_connections 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddd580 0x55fec2df7f60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddcbc0 0x55fec2ded1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/2597762329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dd5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/2597762329 >> 172.21.15.137:0/2597762329 conn(0x55fec2dd6de0 msgr2=0x55fec2dd58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/2597762329 shutdown_connections 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.769+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/2597762329 wait complete. 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 -- start start 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 Processor -- start 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dece30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 0x55fec2deb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddd580 0x55fec2deb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41a8710 con 0x55fec2ddcbc0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41a9a30 con 0x55fec2ddb9f0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.770+0000 7f2ffc9eb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41a8d70 con 0x55fec2ddd580 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ffc9eb740 10 client.0 _ll_register_callbacks cb 0x7f2fec009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 0x55fec2deb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 0x55fec2deb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60636/0 (socket says 172.21.15.137:60636) 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 -- 172.21.15.137:0/572871045 learned_addr learned my addr 172.21.15.137:0/572871045 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff3400640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddd580 0x55fec2deb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 -- 172.21.15.137:0/572871045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddd580 msgr2=0x55fec2deb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddd580 0x55fec2deb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 -- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 msgr2=0x55fec2dece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 -- 172.21.15.137:0/572871045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fec2debec0 con 0x55fec2ddcbc0 2024-03-26T14:08:06.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2ff2000640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 0x55fec2deb440 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00ba270 tx=0x7f2fe0041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8746a04e3efa32 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.771+0000 7f2fdcc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2fdcc00640 1 -- 172.21.15.137:0/572871045 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00d6030 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2fdcc00640 1 -- 172.21.15.137:0/572871045 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2fe00ee9f0 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fec009560 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2fdcc00640 1 -- 172.21.15.137:0/572871045 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe00bdfa0 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2ffc9eb740 10 client.16902 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fec2e4d9a0 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2fdcc00640 1 -- 172.21.15.137:0/572871045 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2fec009560 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2fdcc00640 1 -- 172.21.15.137:0/572871045 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2fe00bdb00 con 0x55fec2ddcbc0 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2ffc9eb740 10 client.16902 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.772+0000 7f2ffc9eb740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.773+0000 7f2ffc9eb740 20 client.16902 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.773+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2e4e300 0x55fec2e50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.773+0000 7f2ffc9eb740 4 client.16902 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:08:06.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.773+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fec2de5a80 con 0x55fec2e4e300 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.773+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2e4e300 0x55fec2e50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.774+0000 7f2ff2a00640 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2e4e300 0x55fec2e50720 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d770d3f7033bfeb server_cookie=95ff1540639e8b8c in_seq=0 out_seq=0 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.774+0000 7f2fdcc00640 10 client.16902 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2fdcc00640 1 -- 172.21.15.137:0/572871045 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fec2de5a80 con 0x55fec2e4e300 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2fdcc00640 10 client.16902 handle_command_reply: tid=0 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 client.16902 shutdown 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 msgr2=0x55fec2deb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 0x55fec2deb440 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00ba270 tx=0x7f2fe0041040 comp rx=0 tx=0).stop 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 shutdown_connections 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55fec2e4e300 0x55fec2e50720 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fec2ddd580 0x55fec2deb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2fdcc00640 0 client.16902 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fec2ddcbc0 0x55fec2deb440 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 --2- 172.21.15.137:0/572871045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fec2ddb9f0 0x55fec2dece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 >> 172.21.15.137:0/572871045 conn(0x55fec2dd6de0 msgr2=0x55fec2dd6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 shutdown_connections 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 1 -- 172.21.15.137:0/572871045 wait complete. 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.775+0000 7f2ffc9eb740 20 client.16902 trim_cache size 0 max 16384 2024-03-26T14:08:06.777 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2fd80a2880 msgr2=0x7f2fd80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2fd80a2880 0x7f2fd80a4d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2ff41025f0 tx=0x7f2fe4005f90 comp rx=0 tx=0).stop 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 msgr2=0x7f2ff4197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4197d00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0000c00 tx=0x7f2fe00042f0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 shutdown_connections 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2fd80a2880 0x7f2fd80a4d40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ff4103150 0x7f2ff419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ff4102790 0x7f2ff4197d00 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 --2- 172.21.15.137:0/2093803215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ff41015a0 0x7f2ff41977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 >> 172.21.15.137:0/2093803215 conn(0x7f2ff40fcb10 msgr2=0x7f2ff4105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:06.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 shutdown_connections 2024-03-26T14:08:06.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:06.776+0000 7f2ff3e00640 1 -- 172.21.15.137:0/2093803215 wait complete. 2024-03-26T14:08:06.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:06 smithi137 ceph-mon[31539]: pgmap v630: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:06.784 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:06 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1910764602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:08:06.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:06 smithi179 ceph-mon[35472]: pgmap v630: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:06.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:06 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1910764602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:08:06.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:06 smithi159 ceph-mon[35501]: pgmap v630: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:06.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1910764602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:08:07.798 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:08:07.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:08:07.798 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:08:08.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 -- 172.21.15.137:0/4233730347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0102030 msgr2=0x7fcbb01088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 --2- 172.21.15.137:0/4233730347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0102030 0x7fcbb01088c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ca0 tx=0x7fcba802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 -- 172.21.15.137:0/4233730347 shutdown_connections 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 --2- 172.21.15.137:0/4233730347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0101670 0x7fcbb0101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 --2- 172.21.15.137:0/4233730347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0102030 0x7fcbb01088c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 --2- 172.21.15.137:0/4233730347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0100480 0x7fcbb01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 -- 172.21.15.137:0/4233730347 >> 172.21.15.137:0/4233730347 conn(0x7fcbb00fba10 msgr2=0x7fcbb00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 -- 172.21.15.137:0/4233730347 shutdown_connections 2024-03-26T14:08:08.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.039+0000 7fcbb5400640 1 -- 172.21.15.137:0/4233730347 wait complete. 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.040+0000 7fcbb5400640 1 Processor -- start 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.040+0000 7fcbb5400640 1 -- start start 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb5400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0100480 0x7fcbb0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 0x7fcbb0199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0102030 0x7fcbb01a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb5400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb010bf90 con 0x7fcbb0101670 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb010be10 con 0x7fcbb0102030 2024-03-26T14:08:08.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb5400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb010c110 con 0x7fcbb0100480 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 0x7fcbb0199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbaf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 0x7fcbb0199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39468/0 (socket says 172.21.15.137:39468) 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbaf400640 1 -- 172.21.15.137:0/1856630525 learned_addr learned my addr 172.21.15.137:0/1856630525 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbafe00640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0100480 0x7fcbb0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.041+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0102030 0x7fcbb01a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbaf400640 1 -- 172.21.15.137:0/1856630525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0100480 msgr2=0x7fcbb0199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbaf400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0100480 0x7fcbb0199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbaf400640 1 -- 172.21.15.137:0/1856630525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0102030 msgr2=0x7fcbb01a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbaf400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0102030 0x7fcbb01a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbaf400640 1 -- 172.21.15.137:0/1856630525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb01a1640 con 0x7fcbb0101670 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0102030 0x7fcbb01a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:08.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbaf400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 0x7fcbb0199f60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00e9f0 tx=0x7fcb9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbacc00640 1 -- 172.21.15.137:0/1856630525 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00cd80 con 0x7fcbb0101670 2024-03-26T14:08:08.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbacc00640 1 -- 172.21.15.137:0/1856630525 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb9c0108e0 con 0x7fcbb0101670 2024-03-26T14:08:08.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbacc00640 1 -- 172.21.15.137:0/1856630525 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c018920 con 0x7fcbb0101670 2024-03-26T14:08:08.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.042+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb01a1930 con 0x7fcbb0101670 2024-03-26T14:08:08.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.043+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb01a1e80 con 0x7fcbb0101670 2024-03-26T14:08:08.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.046+0000 7fcbb962f740 1 Processor -- start 2024-03-26T14:08:08.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.046+0000 7fcbb962f740 1 -- start start 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbacc00640 1 -- 172.21.15.137:0/1856630525 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcb9c010a90 con 0x7fcbb0101670 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb4c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb962f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cddaa9eb0 con 0x561cddb442b0 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb962f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdda52a60 con 0x561cddb49e10 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb962f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cddaa5f50 con 0x561cddb44ca0 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39480/0 (socket says 172.21.15.137:39480) 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbb4a00640 1 -- 172.21.15.137:0/711694488 learned_addr learned my addr 172.21.15.137:0/711694488 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbaf400640 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb4c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.047+0000 7fcbafe00640 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbb4a00640 1 -- 172.21.15.137:0/711694488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 msgr2=0x561cddb497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbb4a00640 1 -- 172.21.15.137:0/711694488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 msgr2=0x561cddb4c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb4c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbb4a00640 1 -- 172.21.15.137:0/711694488 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cdd8867e0 con 0x561cddb442b0 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb446d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcba8037040 tx=0x7fcba8005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386c4363b5c86415 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcbacc00640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb940a2800 0x7fcb940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.048+0000 7fcba5200640 1 -- 172.21.15.137:0/711694488 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8039050 con 0x561cddb442b0 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.049+0000 7fcba5200640 1 -- 172.21.15.137:0/711694488 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcba8045070 con 0x561cddb442b0 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.049+0000 7fcba5200640 1 -- 172.21.15.137:0/711694488 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80042a0 con 0x561cddb442b0 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.049+0000 7fcbacc00640 1 -- 172.21.15.137:0/1856630525 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcb9c0cf2c0 con 0x7fcbb0101670 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.049+0000 7fcbafe00640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb940a2800 0x7fcb940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.049+0000 7fcbb962f740 1 -- 172.21.15.137:0/711694488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 msgr2=0x561cddb446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.049+0000 7fcbb962f740 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb446d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcba8037040 tx=0x7fcba8005f80 comp rx=0 tx=0).stop 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 -- 172.21.15.137:0/711694488 shutdown_connections 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb4c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 --2- 172.21.15.137:0/711694488 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb446d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 -- 172.21.15.137:0/711694488 >> 172.21.15.137:0/711694488 conn(0x561cddb3f820 msgr2=0x561cddb3fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbafe00640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb940a2800 0x7fcb940a4cc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005fd0 tx=0x7fcba0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 -- 172.21.15.137:0/711694488 shutdown_connections 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 -- 172.21.15.137:0/711694488 wait complete. 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.050+0000 7fcbb962f740 1 -- start start 2024-03-26T14:08:08.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 Processor -- start 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb4e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cddb4cd60 con 0x561cddb442b0 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cdda65eb0 con 0x561cddb49e10 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cddb4cee0 con 0x561cddb44ca0 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbb962f740 10 client.0 _ll_register_callbacks cb 0x561cdda5a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb4e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39492/0 (socket says 172.21.15.137:39492) 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.051+0000 7fcbafe00640 1 -- 172.21.15.137:0/2550552907 learned_addr learned my addr 172.21.15.137:0/2550552907 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbaf400640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb4e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbafe00640 1 -- 172.21.15.137:0/2550552907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 msgr2=0x561cddb4e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbafe00640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb4e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbafe00640 1 -- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 msgr2=0x561cddb559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbafe00640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbafe00640 1 -- 172.21.15.137:0/2550552907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cdd8e6f70 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcbafe00640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb4e430 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcbb01014d0 tx=0x7fcba00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1108cd7eedb3b6d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcb99600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.052+0000 7fcb99600640 1 -- 172.21.15.137:0/2550552907 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0005220 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcb99600640 1 -- 172.21.15.137:0/2550552907 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcba00053c0 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba8045070 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcb99600640 1 -- 172.21.15.137:0/2550552907 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba002f050 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcbb962f740 10 client.16914 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561cdd77e170 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcb99600640 1 -- 172.21.15.137:0/2550552907 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcba8045070 con 0x561cddb442b0 2024-03-26T14:08:08.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcb99600640 1 -- 172.21.15.137:0/2550552907 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcba00049d0 con 0x561cddb442b0 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcbb962f740 10 client.16914 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.053+0000 7fcbb962f740 10 client.16914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.054+0000 7fcbb962f740 20 client.16914 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.054+0000 7fcbb962f740 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddb5a320 0x561cddb5c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.054+0000 7fcbb962f740 4 client.16914 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.054+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cddab73f0 con 0x561cddb5a320 2024-03-26T14:08:08.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.054+0000 7fcbaf400640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddb5a320 0x561cddb5c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.055+0000 7fcbaf400640 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddb5a320 0x561cddb5c7c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972a21adc16c2ce6 server_cookie=ac7543cfdb9bc837 in_seq=0 out_seq=0 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.055+0000 7fcb99600640 10 client.16914 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcb99600640 1 -- 172.21.15.137:0/2550552907 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561cddab73f0 con 0x561cddb5a320 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcb99600640 10 client.16914 handle_command_reply: tid=0 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 client.16914 shutdown 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 msgr2=0x561cddb4e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb4e430 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcbb01014d0 tx=0x7fcba00046c0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 shutdown_connections 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddb5a320 0x561cddb5c7c0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb49e10 0x561cddb559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb4e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 --2- 172.21.15.137:0/2550552907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb442b0 0x561cddb4e430 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcb99600640 0 client.16914 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 >> 172.21.15.137:0/2550552907 conn(0x561cddb3f820 msgr2=0x561cddb3ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 shutdown_connections 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.057+0000 7fcbb962f740 1 -- 172.21.15.137:0/2550552907 wait complete. 2024-03-26T14:08:08.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.058+0000 7fcbb962f740 20 client.16914 trim_cache size 0 max 16384 2024-03-26T14:08:08.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 Processor -- start 2024-03-26T14:08:08.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 -- start start 2024-03-26T14:08:08.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb442b0 0x561cddb3e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb55970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb61080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01ae7f0 con 0x561cddb49e10 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01ac2f0 con 0x561cddb442b0 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb962f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01aa370 con 0x561cddb44ca0 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.079+0000 7fcbb4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb61080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb61080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39508/0 (socket says 172.21.15.137:39508) 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 -- 172.21.15.137:0/3074717406 learned_addr learned my addr 172.21.15.137:0/3074717406 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbafe00640 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb442b0 0x561cddb3e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbaf400640 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb55970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 -- 172.21.15.137:0/3074717406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 msgr2=0x561cddb55970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb55970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 -- 172.21.15.137:0/3074717406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb442b0 msgr2=0x561cddb3e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb442b0 0x561cddb3e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 -- 172.21.15.137:0/3074717406 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cddb5a0e0 con 0x561cddb49e10 2024-03-26T14:08:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbafe00640 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb442b0 0x561cddb3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:08.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.080+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb61080 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fcba8004af0 tx=0x7fcba8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b15d2469f15e89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcb9be00640 1 -- 172.21.15.137:0/3074717406 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba802c770 con 0x561cddb49e10 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcb9be00640 1 -- 172.21.15.137:0/3074717406 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcba8045040 con 0x561cddb49e10 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcb9be00640 1 -- 172.21.15.137:0/3074717406 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004230 con 0x561cddb49e10 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 -- 172.21.15.137:0/3074717406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 msgr2=0x561cddb61080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb61080 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fcba8004af0 tx=0x7fcba8000f40 comp rx=0 tx=0).stop 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 -- 172.21.15.137:0/3074717406 shutdown_connections 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb61080 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb44ca0 0x561cddb55970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 --2- 172.21.15.137:0/3074717406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb442b0 0x561cddb3e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.081+0000 7fcbb962f740 1 -- 172.21.15.137:0/3074717406 >> 172.21.15.137:0/3074717406 conn(0x561cddb3f820 msgr2=0x561cddb3e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.082+0000 7fcbb962f740 1 -- 172.21.15.137:0/3074717406 shutdown_connections 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.082+0000 7fcbb962f740 1 -- 172.21.15.137:0/3074717406 wait complete. 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.082+0000 7fcbb962f740 1 -- start start 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.082+0000 7fcbb962f740 1 Processor -- start 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.082+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb442b0 0x561cddb54180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.082+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb44ca0 0x561cddb546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb962f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb54c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb962f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01aa970 con 0x561cddb49e10 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb962f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01abc90 con 0x561cddb44ca0 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb962f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01aafd0 con 0x561cddb442b0 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb962f740 10 client.0 _ll_register_callbacks cb 0x7fcba000bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb54c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb54c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39514/0 (socket says 172.21.15.137:39514) 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 -- 172.21.15.137:0/4144792767 learned_addr learned my addr 172.21.15.137:0/4144792767 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbafe00640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb442b0 0x561cddb54180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbaf400640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb44ca0 0x561cddb546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 -- 172.21.15.137:0/4144792767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb442b0 msgr2=0x561cddb54180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb442b0 0x561cddb54180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 -- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb44ca0 msgr2=0x561cddb546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb44ca0 0x561cddb546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.083+0000 7fcbb4a00640 1 -- 172.21.15.137:0/4144792767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cddbb5c50 con 0x561cddb49e10 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcbb4a00640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb54c00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcba802c630 tx=0x7fcba8032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d458508a92d8649 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcb99600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcb99600640 1 -- 172.21.15.137:0/4144792767 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8039050 con 0x561cddb49e10 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcba8045040 con 0x561cddb49e10 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcb99600640 1 -- 172.21.15.137:0/4144792767 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcba8003ef0 con 0x561cddb49e10 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcb99600640 1 -- 172.21.15.137:0/4144792767 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba802c770 con 0x561cddb49e10 2024-03-26T14:08:08.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcbb962f740 10 client.16920 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.084+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561cddbb6f30 con 0x561cddb49e10 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcb99600640 1 -- 172.21.15.137:0/4144792767 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcba8045040 con 0x561cddb49e10 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcb99600640 1 -- 172.21.15.137:0/4144792767 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcba8053780 con 0x561cddb49e10 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcbb962f740 10 client.16920 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcbb962f740 10 client.16920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcbb962f740 20 client.16920 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcbb962f740 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddbb7420 0x561cddbb9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcbb962f740 4 client.16920 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:08:08.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.085+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561cddab1740 con 0x561cddbb7420 2024-03-26T14:08:08.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.086+0000 7fcbafe00640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddbb7420 0x561cddbb9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.086+0000 7fcbafe00640 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddbb7420 0x561cddbb9810 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d8b66c8dfe4f10f server_cookie=4e7e45d14e3bba2c in_seq=0 out_seq=0 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.086+0000 7fcb99600640 10 client.16920 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcb99600640 1 -- 172.21.15.137:0/4144792767 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561cddab1740 con 0x561cddbb7420 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcb99600640 10 client.16920 handle_command_reply: tid=0 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 client.16920 shutdown 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 msgr2=0x561cddb54c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb54c00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fcba802c630 tx=0x7fcba8032e60 comp rx=0 tx=0).stop 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 shutdown_connections 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561cddbb7420 0x561cddbb9810 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561cddb44ca0 0x561cddb546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561cddb49e10 0x561cddb54c00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 --2- 172.21.15.137:0/4144792767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561cddb442b0 0x561cddb54180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcb99600640 0 client.16920 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 >> 172.21.15.137:0/4144792767 conn(0x561cddb3f820 msgr2=0x561cddb3ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 shutdown_connections 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.089+0000 7fcbb962f740 1 -- 172.21.15.137:0/4144792767 wait complete. 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb962f740 20 client.16920 trim_cache size 0 max 16384 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "3288ae6a-45b4-4f73-a987-6b9d73b9daad", 2024-03-26T14:08:08.090 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb940a2800 msgr2=0x7fcb940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb940a2800 0x7fcb940a4cc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcba0005fd0 tx=0x7fcba0005c50 comp rx=0 tx=0).stop 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 msgr2=0x7fcbb0199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 0x7fcbb0199f60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00e9f0 tx=0x7fcb9c00eeb0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 shutdown_connections 2024-03-26T14:08:08.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb940a2800 0x7fcb940a4cc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbb0102030 0x7fcbb01a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcbb0101670 0x7fcbb0199f60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 --2- 172.21.15.137:0/1856630525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcbb0100480 0x7fcbb0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.090+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 >> 172.21.15.137:0/1856630525 conn(0x7fcbb00fba10 msgr2=0x7fcbb00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.091+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 shutdown_connections 2024-03-26T14:08:08.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.091+0000 7fcbb5400640 1 -- 172.21.15.137:0/1856630525 wait complete. 2024-03-26T14:08:08.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:08:08.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 -- 172.21.15.137:0/1620144130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e40ffe20 msgr2=0x7f23e4100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 --2- 172.21.15.137:0/1620144130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e40ffe20 0x7f23e4100240 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009900 tx=0x7f23d802bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 -- 172.21.15.137:0/1620144130 shutdown_connections 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 --2- 172.21.15.137:0/1620144130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e4101120 0x7f23e41015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 --2- 172.21.15.137:0/1620144130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e4100780 0x7f23e4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 --2- 172.21.15.137:0/1620144130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e40ffe20 0x7f23e4100240 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 -- 172.21.15.137:0/1620144130 >> 172.21.15.137:0/1620144130 conn(0x7f23e40fba20 msgr2=0x7f23e40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 -- 172.21.15.137:0/1620144130 shutdown_connections 2024-03-26T14:08:08.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.326+0000 7f23ea600640 1 -- 172.21.15.137:0/1620144130 wait complete. 2024-03-26T14:08:08.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.327+0000 7f23ea600640 1 Processor -- start 2024-03-26T14:08:08.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 -- start start 2024-03-26T14:08:08.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e40ffe20 0x7f23e41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e4100780 0x7f23e4199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 0x7f23e41a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e410f2c0 con 0x7f23e4101120 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e410f140 con 0x7f23e4100780 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23ea600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e410f440 con 0x7f23e40ffe20 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.328+0000 7f23e9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 0x7f23e41a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 0x7f23e41a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39524/0 (socket says 172.21.15.137:39524) 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 -- 172.21.15.137:0/1230540750 learned_addr learned my addr 172.21.15.137:0/1230540750 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e4100780 0x7f23e4199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9200640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e40ffe20 0x7f23e41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 -- 172.21.15.137:0/1230540750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e40ffe20 msgr2=0x7f23e41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e40ffe20 0x7f23e41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 -- 172.21.15.137:0/1230540750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e4100780 msgr2=0x7f23e4199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e4100780 0x7f23e4199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 -- 172.21.15.137:0/1230540750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e41a15e0 con 0x7f23e4101120 2024-03-26T14:08:08.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.329+0000 7f23e9c00640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 0x7f23e41a0f70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00eb10 tx=0x7f23dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.330+0000 7f23e1600640 1 -- 172.21.15.137:0/1230540750 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc004b20 con 0x7f23e4101120 2024-03-26T14:08:08.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.330+0000 7f23e1600640 1 -- 172.21.15.137:0/1230540750 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23dc004cc0 con 0x7f23e4101120 2024-03-26T14:08:08.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.330+0000 7f23e1600640 1 -- 172.21.15.137:0/1230540750 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc010690 con 0x7f23e4101120 2024-03-26T14:08:08.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.330+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e41a18d0 con 0x7f23e4101120 2024-03-26T14:08:08.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.330+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e41a1dd0 con 0x7f23e4101120 2024-03-26T14:08:08.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.333+0000 7f23e1600640 1 -- 172.21.15.137:0/1230540750 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f23dc005020 con 0x7f23e4101120 2024-03-26T14:08:08.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 Processor -- start 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 -- start start 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1307470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 0x5647c130c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c130f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c126c250 con 0x5647c1307a40 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1214a30 con 0x5647c130cbb0 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.334+0000 7f23ee99b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c12682f0 con 0x5647c1307050 2024-03-26T14:08:08.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 0x5647c130c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 0x5647c130c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39530/0 (socket says 172.21.15.137:39530) 2024-03-26T14:08:08.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 -- 172.21.15.137:0/2230397278 learned_addr learned my addr 172.21.15.137:0/2230397278 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e3e00640 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1307470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e3e00640 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c130f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 -- 172.21.15.137:0/2230397278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 msgr2=0x5647c1307470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1307470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 -- 172.21.15.137:0/2230397278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 msgr2=0x5647c130f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c130f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e9200640 1 -- 172.21.15.137:0/2230397278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c11ff690 con 0x5647c1307a40 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e3e00640 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c130f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.335+0000 7f23e1600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23cc0a2800 0x7f23cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.336+0000 7f23e1600640 1 -- 172.21.15.137:0/1230540750 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f23dc0cf470 con 0x7f23e4101120 2024-03-26T14:08:08.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.336+0000 7f23e9200640 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 0x5647c130c540 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ab0 tx=0x7f23d80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9f929b98285043 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.336+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23cc0a2800 0x7f23cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.336+0000 7f23d6000640 1 -- 172.21.15.137:0/2230397278 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8044020 con 0x5647c1307a40 2024-03-26T14:08:08.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.336+0000 7f23d6000640 1 -- 172.21.15.137:0/2230397278 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23d802cba0 con 0x5647c1307a40 2024-03-26T14:08:08.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.336+0000 7f23d6000640 1 -- 172.21.15.137:0/2230397278 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80046c0 con 0x5647c1307a40 2024-03-26T14:08:08.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 -- 172.21.15.137:0/2230397278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 msgr2=0x5647c130c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23cc0a2800 0x7f23cc0a4cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5647c13082a0 tx=0x7f23d00046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 0x5647c130c540 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ab0 tx=0x7f23d80095e0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 -- 172.21.15.137:0/2230397278 shutdown_connections 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c130f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307a40 0x5647c130c540 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 --2- 172.21.15.137:0/2230397278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1307470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 -- 172.21.15.137:0/2230397278 >> 172.21.15.137:0/2230397278 conn(0x5647c12cf200 msgr2=0x5647c13029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.337+0000 7f23ee99b740 1 -- 172.21.15.137:0/2230397278 shutdown_connections 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.338+0000 7f23ee99b740 1 -- 172.21.15.137:0/2230397278 wait complete. 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.338+0000 7f23ee99b740 1 -- start start 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.338+0000 7f23ee99b740 1 Processor -- start 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1311050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1311590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 0x5647c1318560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c130fb00 con 0x5647c130cbb0 2024-03-26T14:08:08.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c1227e80 con 0x5647c1307a40 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c130fc80 con 0x5647c1307050 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23ee99b740 10 client.0 _ll_register_callbacks cb 0x5647c123a130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23e9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 0x5647c1318560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1311590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23e9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 0x5647c1318560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39542/0 (socket says 172.21.15.137:39542) 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23e9c00640 1 -- 172.21.15.137:0/3338170031 learned_addr learned my addr 172.21.15.137:0/3338170031 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.339+0000 7f23e9200640 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1311050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23e9c00640 1 -- 172.21.15.137:0/3338170031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 msgr2=0x5647c1311050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23e9c00640 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1311050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23e9c00640 1 -- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 msgr2=0x5647c1311590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23e9c00640 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1311590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23e9c00640 1 -- 172.21.15.137:0/3338170031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647c104a7d0 con 0x5647c130cbb0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23e9c00640 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 0x5647c1318560 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f23dc098410 tx=0x7f23dc09b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9b53e254d48e9d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23caa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:08.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23caa00640 1 -- 172.21.15.137:0/3338170031 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0a41e0 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23caa00640 1 -- 172.21.15.137:0/3338170031 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f23dc0a3d50 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23d802cba0 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23caa00640 1 -- 172.21.15.137:0/3338170031 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23dc0a3f50 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23ee99b740 10 client.16932 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.340+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5647c0f803f0 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23caa00640 1 -- 172.21.15.137:0/3338170031 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23d802cba0 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23caa00640 1 -- 172.21.15.137:0/3338170031 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f23dc097790 con 0x5647c130cbb0 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23ee99b740 10 client.16932 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23ee99b740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23ee99b740 20 client.16932 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23ee99b740 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c131cee0 0x5647c131f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23ee99b740 4 client.16932 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:08.343 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.341+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647c1279790 con 0x5647c131cee0 2024-03-26T14:08:08.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.342+0000 7f23e9200640 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c131cee0 0x5647c131f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.342+0000 7f23caa00640 10 client.16932 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.342+0000 7f23e9200640 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c131cee0 0x5647c131f310 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a426fd3566f534a server_cookie=957321289d41b40c in_seq=0 out_seq=0 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.344+0000 7f23caa00640 1 -- 172.21.15.137:0/3338170031 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5647c1279790 con 0x5647c131cee0 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.344+0000 7f23caa00640 10 client.16932 handle_command_reply: tid=0 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.344+0000 7f23ee99b740 1 client.16932 shutdown 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.344+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 msgr2=0x5647c1318560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.344+0000 7f23ee99b740 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 0x5647c1318560 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f23dc098410 tx=0x7f23dc09b460 comp rx=0 tx=0).stop 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 shutdown_connections 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c131cee0 0x5647c131f310 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1311590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c130cbb0 0x5647c1318560 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 --2- 172.21.15.137:0/3338170031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307050 0x5647c1311050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23caa00640 0 client.16932 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 >> 172.21.15.137:0/3338170031 conn(0x5647c12cf200 msgr2=0x5647c1301bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 shutdown_connections 2024-03-26T14:08:08.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 1 -- 172.21.15.137:0/3338170031 wait complete. 2024-03-26T14:08:08.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.345+0000 7f23ee99b740 20 client.16932 trim_cache size 0 max 16384 2024-03-26T14:08:08.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 Processor -- start 2024-03-26T14:08:08.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 -- start start 2024-03-26T14:08:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1305c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307a40 0x5647c1318520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c1306140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41ae6c0 con 0x5647c1307050 2024-03-26T14:08:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41ac1c0 con 0x5647c130cbb0 2024-03-26T14:08:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.369+0000 7f23ee99b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41aa240 con 0x5647c1307a40 2024-03-26T14:08:08.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1305c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1305c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39558/0 (socket says 172.21.15.137:39558) 2024-03-26T14:08:08.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 -- 172.21.15.137:0/1655101036 learned_addr learned my addr 172.21.15.137:0/1655101036 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c1306140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307a40 0x5647c1318520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 -- 172.21.15.137:0/1655101036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307a40 msgr2=0x5647c1318520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307a40 0x5647c1318520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 -- 172.21.15.137:0/1655101036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 msgr2=0x5647c1306140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c1306140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.370+0000 7f23e9200640 1 -- 172.21.15.137:0/1655101036 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c0f400f0 con 0x5647c1307050 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.371+0000 7f23e9200640 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1305c00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ec0 tx=0x7f23d8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ecafb912583ac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.371+0000 7f23d5600640 1 -- 172.21.15.137:0/1655101036 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d804f070 con 0x5647c1307050 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.371+0000 7f23d5600640 1 -- 172.21.15.137:0/1655101036 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23d802cb80 con 0x5647c1307050 2024-03-26T14:08:08.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.371+0000 7f23d5600640 1 -- 172.21.15.137:0/1655101036 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8044040 con 0x5647c1307050 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 -- 172.21.15.137:0/1655101036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 msgr2=0x5647c1305c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1305c00 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f23d8005ec0 tx=0x7f23d8005e50 comp rx=0 tx=0).stop 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 -- 172.21.15.137:0/1655101036 shutdown_connections 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c130cbb0 0x5647c1306140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c1307a40 0x5647c1318520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1655101036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1305c00 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 -- 172.21.15.137:0/1655101036 >> 172.21.15.137:0/1655101036 conn(0x5647c12cf200 msgr2=0x5647c1308d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 -- 172.21.15.137:0/1655101036 shutdown_connections 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 -- 172.21.15.137:0/1655101036 wait complete. 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 -- start start 2024-03-26T14:08:08.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.372+0000 7f23ee99b740 1 Processor -- start 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1315df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1316330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c130cbb0 0x5647c1317340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41aa840 con 0x5647c1307050 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41abb60 con 0x5647c1307a40 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e41aaea0 con 0x5647c130cbb0 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23ee99b740 10 client.0 _ll_register_callbacks cb 0x7f23dc0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23e9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1315df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23e9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1315df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39574/0 (socket says 172.21.15.137:39574) 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23e9200640 1 -- 172.21.15.137:0/1121426198 learned_addr learned my addr 172.21.15.137:0/1121426198 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.373+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1316330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e9200640 1 -- 172.21.15.137:0/1121426198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c130cbb0 msgr2=0x5647c1317340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e9200640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c130cbb0 0x5647c1317340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e9200640 1 -- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 msgr2=0x5647c1316330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:08.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e9200640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1316330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e9200640 1 -- 172.21.15.137:0/1121426198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647c1317930 con 0x5647c1307050 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1316330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23e9200640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1315df0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f23d802c170 tx=0x7f23d80036e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e35b54f576870 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23caa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23caa00640 1 -- 172.21.15.137:0/1121426198 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8057070 con 0x5647c1307050 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23caa00640 1 -- 172.21.15.137:0/1121426198 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f23d8004040 con 0x5647c1307050 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.374+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23d802cb80 con 0x5647c1307050 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23caa00640 1 -- 172.21.15.137:0/1121426198 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23d804f040 con 0x5647c1307050 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 10 client.16938 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:08.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5647c13788c0 con 0x5647c1307050 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23caa00640 1 -- 172.21.15.137:0/1121426198 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23d802cb80 con 0x5647c1307050 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23caa00640 1 -- 172.21.15.137:0/1121426198 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f23d8040c50 con 0x5647c1307050 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 10 client.16938 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 20 client.16938 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c1378ef0 0x5647c137b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:08.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 4 client.16938 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.375+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647c1273ae0 con 0x5647c1378ef0 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.376+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c1378ef0 0x5647c137b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23e3e00640 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c1378ef0 0x5647c137b2e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0410eee3d8960b server_cookie=70417b93f52c302c in_seq=0 out_seq=0 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23caa00640 10 client.16938 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23caa00640 1 -- 172.21.15.137:0/1121426198 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5647c1273ae0 con 0x5647c1378ef0 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23caa00640 10 client.16938 handle_command_reply: tid=0 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23ee99b740 1 client.16938 shutdown 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 msgr2=0x5647c1315df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1315df0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f23d802c170 tx=0x7f23d80036e0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 shutdown_connections 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.377+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5647c1378ef0 0x5647c137b2e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647c1307a40 0x5647c1316330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647c130cbb0 0x5647c1317340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 1 --2- 172.21.15.137:0/1121426198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5647c1307050 0x5647c1315df0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 >> 172.21.15.137:0/1121426198 conn(0x5647c12cf200 msgr2=0x5647c1301330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23caa00640 0 client.16938 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 shutdown_connections 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 1 -- 172.21.15.137:0/1121426198 wait complete. 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ee99b740 20 client.16938 trim_cache size 0 max 16384 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: "3288ae6a-45b4-4f73-a987-6b9d73b9daad": { 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "3288ae6a-45b4-4f73-a987-6b9d73b9daad", 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:08:08.380 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:08:08.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23cc0a2800 msgr2=0x7f23cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ea600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23cc0a2800 0x7f23cc0a4cc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x5647c13082a0 tx=0x7f23d00046c0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 msgr2=0x7f23e41a0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:08.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.378+0000 7f23ea600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 0x7f23e41a0f70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00eb10 tx=0x7f23dc00c2d0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 shutdown_connections 2024-03-26T14:08:08.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23cc0a2800 0x7f23cc0a4cc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23e4100780 0x7f23e4199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23e4101120 0x7f23e41a0f70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 --2- 172.21.15.137:0/1230540750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23e40ffe20 0x7f23e41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:08.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 >> 172.21.15.137:0/1230540750 conn(0x7f23e40fba20 msgr2=0x7f23e40fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:08.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 shutdown_connections 2024-03-26T14:08:08.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:08.379+0000 7f23ea600640 1 -- 172.21.15.137:0/1230540750 wait complete. 2024-03-26T14:08:08.396 INFO:tasks.cephfs.filesystem:scrub status for tag:3288ae6a-45b4-4f73-a987-6b9d73b9daad - {'path': '/', 'tag': '3288ae6a-45b4-4f73-a987-6b9d73b9daad', 'options': 'recursive,force'} 2024-03-26T14:08:08.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:08 smithi179 ceph-mon[35472]: pgmap v631: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:08.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:08 smithi159 ceph-mon[35501]: pgmap v631: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:09.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:08 smithi137 ceph-mon[31539]: pgmap v631: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:09.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:09 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:08:09.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:09 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:09.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:09 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:08:09.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:09 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:09.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:09 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:08:09.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:09 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:09.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:09 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:08:09.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:09 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:10.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:09 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:08:10.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:09 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:10.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:09 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:08:10.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:09 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:10.825 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:10 smithi179 ceph-mon[35472]: pgmap v632: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:10.853 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:10 smithi159 ceph-mon[35501]: pgmap v632: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:11.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:10 smithi137 ceph-mon[31539]: pgmap v632: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:12.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:12 smithi179 ceph-mon[35472]: pgmap v633: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:12.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:12 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:08:12.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:12 smithi159 ceph-mon[35501]: pgmap v633: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:12.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:12 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:08:13.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:12 smithi137 ceph-mon[31539]: pgmap v633: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:13.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:12 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:08:14.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:14 smithi179 ceph-mon[35472]: pgmap v634: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:08:14.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:14 smithi159 ceph-mon[35501]: pgmap v634: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:08:15.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:14 smithi137 ceph-mon[31539]: pgmap v634: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:08:16.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:16 smithi179 ceph-mon[35472]: pgmap v635: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:16.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:16 smithi159 ceph-mon[35501]: pgmap v635: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:17.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:16 smithi137 ceph-mon[31539]: pgmap v635: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:18.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:18 smithi179 ceph-mon[35472]: pgmap v636: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:18.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:18 smithi159 ceph-mon[35501]: pgmap v636: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:19.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:18 smithi137 ceph-mon[31539]: pgmap v636: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:20.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:20 smithi179 ceph-mon[35472]: pgmap v637: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:20.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:20 smithi159 ceph-mon[35501]: pgmap v637: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:21.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:20 smithi137 ceph-mon[31539]: pgmap v637: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:22.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:22 smithi179 ceph-mon[35472]: pgmap v638: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:22.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:22 smithi159 ceph-mon[35501]: pgmap v638: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:23.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:22 smithi137 ceph-mon[31539]: pgmap v638: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:24.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:24 smithi179 ceph-mon[35472]: pgmap v639: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:24.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:24 smithi159 ceph-mon[35501]: pgmap v639: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:25.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:24 smithi137 ceph-mon[31539]: pgmap v639: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:25.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:25 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:08:25.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:25 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:08:26.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:25 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:08:26.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: pgmap v640: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:26.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi179 to 176.7M 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: Unable to set osd_memory_target on smithi179 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi159 to 176.8M 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: Unable to set osd_memory_target on smithi159 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:08:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:26 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:08:26.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: pgmap v640: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:26.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi179 to 176.7M 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: Unable to set osd_memory_target on smithi179 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi159 to 176.8M 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: Unable to set osd_memory_target on smithi159 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:08:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:08:26.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:26 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:08:27.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: pgmap v640: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:27.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi179 to 176.7M 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: Unable to set osd_memory_target on smithi179 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi159 to 176.8M 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: Unable to set osd_memory_target on smithi159 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:08:27.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:26 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:08:28.923 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:28 smithi179 ceph-mon[35472]: pgmap v641: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:28.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:28 smithi159 ceph-mon[35501]: pgmap v641: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:29.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:28 smithi137 ceph-mon[31539]: pgmap v641: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:31.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:30 smithi159 ceph-mon[35501]: pgmap v642: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:31.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:30 smithi137 ceph-mon[31539]: pgmap v642: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:31.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:30 smithi179 ceph-mon[35472]: pgmap v642: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:33.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:32 smithi159 ceph-mon[35501]: pgmap v643: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:33.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:32 smithi137 ceph-mon[31539]: pgmap v643: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:33.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:32 smithi179 ceph-mon[35472]: pgmap v643: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:35.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:34 smithi159 ceph-mon[35501]: pgmap v644: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:35.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:34 smithi137 ceph-mon[31539]: pgmap v644: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:35.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:34 smithi179 ceph-mon[35472]: pgmap v644: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:37.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:36 smithi159 ceph-mon[35501]: pgmap v645: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:37.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:36 smithi137 ceph-mon[31539]: pgmap v645: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:37.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:36 smithi179 ceph-mon[35472]: pgmap v645: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:38.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:08:38.397 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 -- 172.21.15.137:0/1377952288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0102760 msgr2=0x7f84d0104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 --2- 172.21.15.137:0/1377952288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0102760 0x7f84d0104bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009900 tx=0x7f84bc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 -- 172.21.15.137:0/1377952288 shutdown_connections 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 --2- 172.21.15.137:0/1377952288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0102760 0x7f84d0104bd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 --2- 172.21.15.137:0/1377952288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d0105110 0x7f84d0105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 --2- 172.21.15.137:0/1377952288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d00ffdf0 0x7f84d0102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 -- 172.21.15.137:0/1377952288 >> 172.21.15.137:0/1377952288 conn(0x7f84d00fba10 msgr2=0x7f84d00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 -- 172.21.15.137:0/1377952288 shutdown_connections 2024-03-26T14:08:38.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.625+0000 7f84d5400640 1 -- 172.21.15.137:0/1377952288 wait complete. 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.626+0000 7f84d5400640 1 Processor -- start 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.626+0000 7f84d5400640 1 -- start start 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d5400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d00ffdf0 0x7f84d019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 0x7f84d019e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0105110 0x7f84d01a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d5400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01136f0 con 0x7f84d0102760 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0113570 con 0x7f84d0105110 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d5400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d0113870 con 0x7f84d00ffdf0 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84cf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 0x7f84d019e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84cf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 0x7f84d019e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50326/0 (socket says 172.21.15.137:50326) 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84cf400640 1 -- 172.21.15.137:0/2643059621 learned_addr learned my addr 172.21.15.137:0/2643059621 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0105110 0x7f84d01a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.627+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d00ffdf0 0x7f84d019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84cf400640 1 -- 172.21.15.137:0/2643059621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d00ffdf0 msgr2=0x7f84d019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84cf400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d00ffdf0 0x7f84d019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84cf400640 1 -- 172.21.15.137:0/2643059621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0105110 msgr2=0x7f84d01a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84cf400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0105110 0x7f84d01a5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84cf400640 1 -- 172.21.15.137:0/2643059621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d01a5990 con 0x7f84d0102760 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84cf400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 0x7f84d019e2b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f84bc000c00 tx=0x7f84bc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84ccc00640 1 -- 172.21.15.137:0/2643059621 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc043020 con 0x7f84d0102760 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84ccc00640 1 -- 172.21.15.137:0/2643059621 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84bc0044c0 con 0x7f84d0102760 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d01a5c20 con 0x7f84d0102760 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.628+0000 7f84ccc00640 1 -- 172.21.15.137:0/2643059621 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc003c60 con 0x7f84d0102760 2024-03-26T14:08:38.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.629+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d01a61c0 con 0x7f84d0102760 2024-03-26T14:08:38.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 Processor -- start 2024-03-26T14:08:38.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 -- start start 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f0732470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f0737540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f073a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f0697250 con 0x55f5f0732050 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f063fa30 con 0x55f5f0732a40 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.632+0000 7f84d9612740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f06932f0 con 0x55f5f0737bb0 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84ccc00640 1 -- 172.21.15.137:0/2643059621 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f84bc046020 con 0x7f84d0102760 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f0732470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f0732470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50328/0 (socket says 172.21.15.137:50328) 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 -- 172.21.15.137:0/979873024 learned_addr learned my addr 172.21.15.137:0/979873024 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84cfe00640 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f0737540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 -- 172.21.15.137:0/979873024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 msgr2=0x55f5f073a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f073a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 -- 172.21.15.137:0/979873024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 msgr2=0x55f5f0737540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f0737540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84d4a00640 1 -- 172.21.15.137:0/979873024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f062a690 con 0x55f5f0732050 2024-03-26T14:08:38.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.633+0000 7f84cfe00640 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f0737540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:08:38.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.634+0000 7f84d4a00640 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f0732470 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009fd0 tx=0x7f84c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cae2289e6177ce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.634+0000 7f84c5200640 1 -- 172.21.15.137:0/979873024 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8004d20 con 0x55f5f0732050 2024-03-26T14:08:38.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.634+0000 7f84c5200640 1 -- 172.21.15.137:0/979873024 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84c8004050 con 0x55f5f0732050 2024-03-26T14:08:38.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.634+0000 7f84c5200640 1 -- 172.21.15.137:0/979873024 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8005280 con 0x55f5f0732050 2024-03-26T14:08:38.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84ccc00640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84b40a2880 0x7f84b40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 -- 172.21.15.137:0/979873024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 msgr2=0x55f5f0732470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f0732470 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009fd0 tx=0x7f84c80118d0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84ccc00640 1 -- 172.21.15.137:0/2643059621 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84bc0f10a0 con 0x7f84d0102760 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 -- 172.21.15.137:0/979873024 shutdown_connections 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f0737540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f073a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 --2- 172.21.15.137:0/979873024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f0732470 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 -- 172.21.15.137:0/979873024 >> 172.21.15.137:0/979873024 conn(0x55f5f06fa200 msgr2=0x55f5f072d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84b40a2880 0x7f84b40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 -- 172.21.15.137:0/979873024 shutdown_connections 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.635+0000 7f84d9612740 1 -- 172.21.15.137:0/979873024 wait complete. 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.636+0000 7f84d9612740 1 -- start start 2024-03-26T14:08:38.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.636+0000 7f84d9612740 1 Processor -- start 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.636+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84b40a2880 0x7f84b40a4d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55f5f072e910 tx=0x7f84c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.636+0000 7f84d9612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f073c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d9612740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f073c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d9612740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f0743610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d9612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f073ab60 con 0x55f5f0732050 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d9612740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f0652e80 con 0x55f5f0732a40 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d9612740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5f073ace0 con 0x55f5f0737bb0 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d9612740 10 client.0 _ll_register_callbacks cb 0x55f5f0665130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f073c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f073c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50334/0 (socket says 172.21.15.137:50334) 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 -- 172.21.15.137:0/497603993 learned_addr learned my addr 172.21.15.137:0/497603993 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cf400640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f073c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84d4a00640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f0743610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 -- 172.21.15.137:0/497603993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 msgr2=0x55f5f0743610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f0743610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 -- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 msgr2=0x55f5f073c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f073c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cfe00640 1 -- 172.21.15.137:0/497603993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5f04757d0 con 0x55f5f0732050 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.637+0000 7f84cf400640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f073c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84cfe00640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f073c090 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f84c00096d0 tx=0x7f84c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fd3e01aff00266 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84b9600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84b9600640 1 -- 172.21.15.137:0/497603993 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c0004260 con 0x55f5f0732050 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84b9600640 1 -- 172.21.15.137:0/497603993 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84c0004400 con 0x55f5f0732050 2024-03-26T14:08:38.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84c8004050 con 0x55f5f0732050 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84b9600640 1 -- 172.21.15.137:0/497603993 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c00035f0 con 0x55f5f0732050 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84d9612740 10 client.16950 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.638+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f5f03ab3f0 con 0x55f5f0732050 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84b9600640 1 -- 172.21.15.137:0/497603993 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84c8004050 con 0x55f5f0732050 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84b9600640 1 -- 172.21.15.137:0/497603993 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f84c0027070 con 0x55f5f0732050 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84d9612740 10 client.16950 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84d9612740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84d9612740 20 client.16950 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84d9612740 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f0747fb0 0x55f5f074a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84d9612740 4 client.16950 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.639+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5f06a4790 con 0x55f5f0747fb0 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.640+0000 7f84cf400640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f0747fb0 0x55f5f074a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.640+0000 7f84cf400640 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f0747fb0 0x55f5f074a450 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9e87392308a6ee server_cookie=3f29278c5124608 in_seq=0 out_seq=0 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.640+0000 7f84b9600640 10 client.16950 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84b9600640 1 -- 172.21.15.137:0/497603993 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f5f06a4790 con 0x55f5f0747fb0 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84b9600640 10 client.16950 handle_command_reply: tid=0 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 client.16950 shutdown 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 msgr2=0x55f5f073c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f073c090 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f84c00096d0 tx=0x7f84c0004730 comp rx=0 tx=0).stop 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 shutdown_connections 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f0747fb0 0x55f5f074a450 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f073c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0737bb0 0x55f5f0743610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 --2- 172.21.15.137:0/497603993 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0732050 0x55f5f073c090 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.641+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 >> 172.21.15.137:0/497603993 conn(0x55f5f06fa200 msgr2=0x55f5f072cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.642+0000 7f84b9600640 0 client.16950 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.642+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 shutdown_connections 2024-03-26T14:08:38.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.642+0000 7f84d9612740 1 -- 172.21.15.137:0/497603993 wait complete. 2024-03-26T14:08:38.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.642+0000 7f84d9612740 20 client.16950 trim_cache size 0 max 16384 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 Processor -- start 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 -- start start 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732050 0x55f5f073c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f07435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f0731310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01b2a40 con 0x55f5f0737bb0 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01b0540 con 0x55f5f0732a40 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d9612740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01ae5c0 con 0x55f5f0732050 2024-03-26T14:08:38.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.664+0000 7f84d4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f0731310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f0731310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50336/0 (socket says 172.21.15.137:50336) 2024-03-26T14:08:38.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 -- 172.21.15.137:0/2029564068 learned_addr learned my addr 172.21.15.137:0/2029564068 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:38.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84cf400640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f07435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732050 0x55f5f073c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 -- 172.21.15.137:0/2029564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732050 msgr2=0x55f5f073c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732050 0x55f5f073c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 -- 172.21.15.137:0/2029564068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 msgr2=0x55f5f07435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f07435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84d4a00640 1 -- 172.21.15.137:0/2029564068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5f036b0f0 con 0x55f5f0737bb0 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732050 0x55f5f073c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.665+0000 7f84cf400640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f07435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.666+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f0731310 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f84c8006010 tx=0x7f84c8004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928238467c72b9b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.666+0000 7f84bbe00640 1 -- 172.21.15.137:0/2029564068 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8029070 con 0x55f5f0737bb0 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.666+0000 7f84bbe00640 1 -- 172.21.15.137:0/2029564068 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84c8003e20 con 0x55f5f0737bb0 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.666+0000 7f84bbe00640 1 -- 172.21.15.137:0/2029564068 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8005040 con 0x55f5f0737bb0 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 -- 172.21.15.137:0/2029564068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 msgr2=0x55f5f0731310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f0731310 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f84c8006010 tx=0x7f84c8004a00 comp rx=0 tx=0).stop 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 -- 172.21.15.137:0/2029564068 shutdown_connections 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732a40 0x55f5f07435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f0731310 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 --2- 172.21.15.137:0/2029564068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732050 0x55f5f073c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 -- 172.21.15.137:0/2029564068 >> 172.21.15.137:0/2029564068 conn(0x55f5f06fa200 msgr2=0x55f5f0730c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 -- 172.21.15.137:0/2029564068 shutdown_connections 2024-03-26T14:08:38.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.667+0000 7f84d9612740 1 -- 172.21.15.137:0/2029564068 wait complete. 2024-03-26T14:08:38.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 -- start start 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 Processor -- start 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732050 0x55f5f0740c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732a40 0x55f5f0741190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f07421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01aebc0 con 0x55f5f0737bb0 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01afee0 con 0x55f5f0732050 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01af220 con 0x55f5f0732a40 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.668+0000 7f84d9612740 10 client.0 _ll_register_callbacks cb 0x7f84c00098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f07421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f07421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50344/0 (socket says 172.21.15.137:50344) 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 -- 172.21.15.137:0/2110912850 learned_addr learned my addr 172.21.15.137:0/2110912850 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:38.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732050 0x55f5f0740c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84cf400640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732a40 0x55f5f0741190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 -- 172.21.15.137:0/2110912850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732a40 msgr2=0x55f5f0741190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732a40 0x55f5f0741190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 -- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732050 msgr2=0x55f5f0740c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732050 0x55f5f0740c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 -- 172.21.15.137:0/2110912850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5f0742790 con 0x55f5f0737bb0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732050 0x55f5f0740c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84d4a00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f07421a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f84c8011dd0 tx=0x7f84c8004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d864e2552cfe9bac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.669+0000 7f84b9600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84b9600640 1 -- 172.21.15.137:0/2110912850 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8029040 con 0x55f5f0737bb0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84c8003e20 con 0x55f5f0737bb0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84b9600640 1 -- 172.21.15.137:0/2110912850 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f84c8003970 con 0x55f5f0737bb0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84b9600640 1 -- 172.21.15.137:0/2110912850 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c8042050 con 0x55f5f0737bb0 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84d9612740 10 client.16956 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:38.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f5f07a38e0 con 0x55f5f0737bb0 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.670+0000 7f84b9600640 1 -- 172.21.15.137:0/2110912850 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84c8003e20 con 0x55f5f0737bb0 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84b9600640 1 -- 172.21.15.137:0/2110912850 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f84c8005000 con 0x55f5f0737bb0 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84d9612740 10 client.16956 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84d9612740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84d9612740 20 client.16956 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84d9612740 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f07a4000 0x55f5f07a6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84d9612740 4 client.16956 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5f069eae0 con 0x55f5f07a4000 2024-03-26T14:08:38.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.671+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f07a4000 0x55f5f07a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.672+0000 7f84b9600640 10 client.16956 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:38.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.672+0000 7f84cfe00640 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f07a4000 0x55f5f07a6420 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dffb16123ae34f16 server_cookie=ea820c8656043510 in_seq=0 out_seq=0 2024-03-26T14:08:38.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84b9600640 1 -- 172.21.15.137:0/2110912850 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f5f069eae0 con 0x55f5f07a4000 2024-03-26T14:08:38.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84b9600640 10 client.16956 handle_command_reply: tid=0 2024-03-26T14:08:38.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 client.16956 shutdown 2024-03-26T14:08:38.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 msgr2=0x55f5f07421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f07421a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f84c8011dd0 tx=0x7f84c8004200 comp rx=0 tx=0).stop 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 shutdown_connections 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f5f07a4000 0x55f5f07a6420 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f5f0737bb0 0x55f5f07421a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5f0732a40 0x55f5f0741190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84b9600640 0 client.16956 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 --2- 172.21.15.137:0/2110912850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f5f0732050 0x55f5f0740c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 >> 172.21.15.137:0/2110912850 conn(0x55f5f06fa200 msgr2=0x55f5f073c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 shutdown_connections 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 1 -- 172.21.15.137:0/2110912850 wait complete. 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.673+0000 7f84d9612740 20 client.16956 trim_cache size 0 max 16384 2024-03-26T14:08:38.675 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:08:38.676 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:08:38.676 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:08:38.676 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:08:38.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84b40a2880 msgr2=0x7f84b40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84b40a2880 0x7f84b40a4d40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55f5f072e910 tx=0x7f84c0005f90 comp rx=0 tx=0).stop 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 msgr2=0x7f84d019e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 0x7f84d019e2b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f84bc000c00 tx=0x7f84bc0040c0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 shutdown_connections 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f84b40a2880 0x7f84b40a4d40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f84d0105110 0x7f84d01a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84d0102760 0x7f84d019e2b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 --2- 172.21.15.137:0/2643059621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d00ffdf0 0x7f84d019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 >> 172.21.15.137:0/2643059621 conn(0x7f84d00fba10 msgr2=0x7f84d0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 shutdown_connections 2024-03-26T14:08:38.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.674+0000 7f84d5400640 1 -- 172.21.15.137:0/2643059621 wait complete. 2024-03-26T14:08:38.695 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:08:38.695 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:08:38.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 -- 172.21.15.137:0/554957492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 msgr2=0x7f99281097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 --2- 172.21.15.137:0/554957492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281097e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9920009ca0 tx=0x7f992002c100 comp rx=0 tx=0).stop 2024-03-26T14:08:38.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 -- 172.21.15.137:0/554957492 shutdown_connections 2024-03-26T14:08:38.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 --2- 172.21.15.137:0/554957492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281097e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 --2- 172.21.15.137:0/554957492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9928104a60 0x7f9928106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 --2- 172.21.15.137:0/554957492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9928102130 0x7f9928104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.905+0000 7f9930400640 1 -- 172.21.15.137:0/554957492 >> 172.21.15.137:0/554957492 conn(0x7f99280fbb20 msgr2=0x7f99280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.906+0000 7f9930400640 1 -- 172.21.15.137:0/554957492 shutdown_connections 2024-03-26T14:08:38.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.906+0000 7f9930400640 1 -- 172.21.15.137:0/554957492 wait complete. 2024-03-26T14:08:38.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 Processor -- start 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 -- start start 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9928102130 0x7f9928199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9928104a60 0x7f9928199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992810d1e0 con 0x7f9928107390 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992810d060 con 0x7f9928102130 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f9930400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992810d360 con 0x7f9928104a60 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.907+0000 7f992e400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50366/0 (socket says 172.21.15.137:50366) 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 -- 172.21.15.137:0/3504525854 learned_addr learned my addr 172.21.15.137:0/3504525854 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992da00640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9928102130 0x7f9928199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992d000640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9928104a60 0x7f9928199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 -- 172.21.15.137:0/3504525854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9928104a60 msgr2=0x7f9928199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9928104a60 0x7f9928199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 -- 172.21.15.137:0/3504525854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9928102130 msgr2=0x7f9928199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9928102130 0x7f9928199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 -- 172.21.15.137:0/3504525854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99281a16b0 con 0x7f9928107390 2024-03-26T14:08:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.908+0000 7f992e400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281a1040 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9920032520 tx=0x7f9920035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.909+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920039040 con 0x7f9928107390 2024-03-26T14:08:38.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.909+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9920044070 con 0x7f9928107390 2024-03-26T14:08:38.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.909+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99281a1940 con 0x7f9928107390 2024-03-26T14:08:38.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.909+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200043c0 con 0x7f9928107390 2024-03-26T14:08:38.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.909+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99281a1e80 con 0x7f9928107390 2024-03-26T14:08:38.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.910+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9928108e80 con 0x7f9928107390 2024-03-26T14:08:38.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.912+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9920004560 con 0x7f9928107390 2024-03-26T14:08:38.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.914+0000 7f9926000640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f99100a2880 0x7f99100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.915+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f99200f1560 con 0x7f9928107390 2024-03-26T14:08:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.915+0000 7f992da00640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f99100a2880 0x7f99100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:38.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.915+0000 7f992da00640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f99100a2880 0x7f99100a4d40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f991c009bd0 tx=0x7f991c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:38.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:38.919+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f99200be2b0 con 0x7f9928107390 2024-03-26T14:08:39.034 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:38 smithi137 ceph-mon[31539]: pgmap v646: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:39.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.211+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99280622a0 con 0x7f9928107390 2024-03-26T14:08:39.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.212+0000 7f9926000640 1 -- 172.21.15.137:0/3504525854 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f99200bdbb0 con 0x7f9928107390 2024-03-26T14:08:39.212 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:08:39.212 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:08:39.213 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:08:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.215+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f99100a2880 msgr2=0x7f99100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.215+0000 7f9930400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f99100a2880 0x7f99100a4d40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f991c009bd0 tx=0x7f991c005f90 comp rx=0 tx=0).stop 2024-03-26T14:08:39.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.215+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 msgr2=0x7f99281a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.215+0000 7f9930400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281a1040 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9920032520 tx=0x7f9920035ce0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 shutdown_connections 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f99100a2880 0x7f99100a4d40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9928107390 0x7f99281a1040 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9928104a60 0x7f9928199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 --2- 172.21.15.137:0/3504525854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9928102130 0x7f9928199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 >> 172.21.15.137:0/3504525854 conn(0x7f99280fbb20 msgr2=0x7f9928101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 shutdown_connections 2024-03-26T14:08:39.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.216+0000 7f9930400640 1 -- 172.21.15.137:0/3504525854 wait complete. 2024-03-26T14:08:39.234 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:08:39.234 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:08:39.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:38 smithi179 ceph-mon[35472]: pgmap v646: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:39.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:38 smithi159 ceph-mon[35501]: pgmap v646: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3645040912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300102000 msgr2=0x7fd3001043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3645040912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300102000 0x7fd3001043f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009900 tx=0x7fd2f002bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3645040912 shutdown_connections 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3645040912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd300104930 0x7fd300106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3645040912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300107260 0x7fd3001096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3645040912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300102000 0x7fd3001043f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.454+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3645040912 >> 172.21.15.137:0/3645040912 conn(0x7fd3000fba10 msgr2=0x7fd3000fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.455+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3645040912 shutdown_connections 2024-03-26T14:08:39.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.455+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3645040912 wait complete. 2024-03-26T14:08:39.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 Processor -- start 2024-03-26T14:08:39.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 -- start start 2024-03-26T14:08:39.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300102000 0x7fd300199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 0x7fd300199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd300107260 0x7fd3001a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30010d110 con 0x7fd300104930 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30010cf90 con 0x7fd300107260 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.456+0000 7fd2ffe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30010d290 con 0x7fd300102000 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 0x7fd300199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 0x7fd300199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50386/0 (socket says 172.21.15.137:50386) 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 -- 172.21.15.137:0/3513610962 learned_addr learned my addr 172.21.15.137:0/3513610962 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd300107260 0x7fd3001a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fea00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300102000 0x7fd300199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 -- 172.21.15.137:0/3513610962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300102000 msgr2=0x7fd300199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300102000 0x7fd300199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 -- 172.21.15.137:0/3513610962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd300107260 msgr2=0x7fd3001a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd300107260 0x7fd3001a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 -- 172.21.15.137:0/3513610962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3001a1580 con 0x7fd300104930 2024-03-26T14:08:39.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.457+0000 7fd2fe000640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 0x7fd300199ea0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00e9c0 tx=0x7fd2ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.458+0000 7fd2f7400640 1 -- 172.21.15.137:0/3513610962 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00cd80 con 0x7fd300104930 2024-03-26T14:08:39.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.458+0000 7fd2f7400640 1 -- 172.21.15.137:0/3513610962 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2ec0054d0 con 0x7fd300104930 2024-03-26T14:08:39.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.458+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3001a1870 con 0x7fd300104930 2024-03-26T14:08:39.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.458+0000 7fd2f7400640 1 -- 172.21.15.137:0/3513610962 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec010430 con 0x7fd300104930 2024-03-26T14:08:39.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.458+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3001a1dc0 con 0x7fd300104930 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 Processor -- start 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 -- start start 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd2f7400640 1 -- 172.21.15.137:0/3513610962 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd2ec004fd0 con 0x7fd300104930 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf929e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf92b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf931e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582cf88e2c0 con 0x5582cf92abc0 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582cf836c90 con 0x5582cf9299f0 2024-03-26T14:08:39.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.462+0000 7fd308417740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582cf88a360 con 0x5582cf92b580 2024-03-26T14:08:39.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf929e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf92b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf929e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:45680/0 (socket says 172.21.15.137:45680) 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 -- 172.21.15.137:0/1006552951 learned_addr learned my addr 172.21.15.137:0/1006552951 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2fe000640 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf931e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 -- 172.21.15.137:0/1006552951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 msgr2=0x5582cf931e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf931e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 -- 172.21.15.137:0/1006552951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 msgr2=0x5582cf92b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf92b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.463+0000 7fd2ff400640 1 -- 172.21.15.137:0/1006552951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582cf85b080 con 0x5582cf9299f0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2ff400640 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf929e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8007f40 tx=0x7fd2f800f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdb566e32a19fea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2ebe00640 1 -- 172.21.15.137:0/1006552951 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8010900 con 0x5582cf9299f0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2f7400640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2e40a2800 0x7fd2e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2ebe00640 1 -- 172.21.15.137:0/1006552951 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2f80183f0 con 0x5582cf9299f0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2ebe00640 1 -- 172.21.15.137:0/1006552951 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8017600 con 0x5582cf9299f0 2024-03-26T14:08:39.464 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2f7400640 1 -- 172.21.15.137:0/3513610962 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd2ec0cf2b0 con 0x7fd300104930 2024-03-26T14:08:39.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.464+0000 7fd2fea00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2e40a2800 0x7fd2e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 -- 172.21.15.137:0/1006552951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 msgr2=0x5582cf929e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf929e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8007f40 tx=0x7fd2f800f850 comp rx=0 tx=0).stop 2024-03-26T14:08:39.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd2fea00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2e40a2800 0x7fd2e40a4cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 -- 172.21.15.137:0/1006552951 shutdown_connections 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf931e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf92b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 --2- 172.21.15.137:0/1006552951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf929e10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 -- 172.21.15.137:0/1006552951 >> 172.21.15.137:0/1006552951 conn(0x5582cf924de0 msgr2=0x5582cf9251e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 -- 172.21.15.137:0/1006552951 shutdown_connections 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.465+0000 7fd308417740 1 -- 172.21.15.137:0/1006552951 wait complete. 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.466+0000 7fd308417740 1 -- start start 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.466+0000 7fd308417740 1 Processor -- start 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf9299f0 0x5582cf933cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf934200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92b580 0x5582cf93b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582cf932590 con 0x5582cf92abc0 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582cf84a0d0 con 0x5582cf92b580 2024-03-26T14:08:39.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582cf932710 con 0x5582cf9299f0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd308417740 10 client.0 _ll_register_callbacks cb 0x5582cf84ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf934200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf934200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50398/0 (socket says 172.21.15.137:50398) 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 -- 172.21.15.137:0/51211696 learned_addr learned my addr 172.21.15.137:0/51211696 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2ff400640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92b580 0x5582cf93b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fea00640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf9299f0 0x5582cf933cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 -- 172.21.15.137:0/51211696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf9299f0 msgr2=0x5582cf933cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf9299f0 0x5582cf933cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 -- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92b580 msgr2=0x5582cf93b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92b580 0x5582cf93b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.467+0000 7fd2fe000640 1 -- 172.21.15.137:0/51211696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582cf5a23f0 con 0x5582cf92abc0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd2fe000640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf934200 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0a3af0 tx=0x7fd2ec004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d03f81aab1c4ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd2e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd2e8c00640 1 -- 172.21.15.137:0/51211696 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec09b1e0 con 0x5582cf92abc0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd2e8c00640 1 -- 172.21.15.137:0/51211696 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd2ec086070 con 0x5582cf92abc0 2024-03-26T14:08:39.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f80183f0 con 0x5582cf92abc0 2024-03-26T14:08:39.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd2e8c00640 1 -- 172.21.15.137:0/51211696 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2ec09bf80 con 0x5582cf92abc0 2024-03-26T14:08:39.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd308417740 10 client.16974 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:39.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.468+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5582cf66c7d0 con 0x5582cf92abc0 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd2e8c00640 1 -- 172.21.15.137:0/51211696 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2f80183f0 con 0x5582cf92abc0 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd2e8c00640 1 -- 172.21.15.137:0/51211696 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd2ec0a3b90 con 0x5582cf92abc0 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd308417740 10 client.16974 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd308417740 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd308417740 20 client.16974 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd308417740 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf93f740 0x5582cf941b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd308417740 4 client.16974 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:39.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.469+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582cf89b800 con 0x5582cf93f740 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.470+0000 7fd2fea00640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf93f740 0x5582cf941b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.470+0000 7fd2fea00640 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf93f740 0x5582cf941b70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fc4690bb089f0e5 server_cookie=e906d4595aa6ade3 in_seq=0 out_seq=0 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.470+0000 7fd2e8c00640 10 client.16974 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd2e8c00640 1 -- 172.21.15.137:0/51211696 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5582cf89b800 con 0x5582cf93f740 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd2e8c00640 10 client.16974 handle_command_reply: tid=0 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 client.16974 shutdown 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 msgr2=0x5582cf934200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf934200 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0a3af0 tx=0x7fd2ec004780 comp rx=0 tx=0).stop 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 shutdown_connections 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf93f740 0x5582cf941b70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92b580 0x5582cf93b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92abc0 0x5582cf934200 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 --2- 172.21.15.137:0/51211696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf9299f0 0x5582cf933cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd2e8c00640 0 client.16974 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 >> 172.21.15.137:0/51211696 conn(0x5582cf924de0 msgr2=0x5582cf924160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 shutdown_connections 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.472+0000 7fd308417740 1 -- 172.21.15.137:0/51211696 wait complete. 2024-03-26T14:08:39.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.473+0000 7fd308417740 20 client.16974 trim_cache size 0 max 16384 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.489+0000 7fd308417740 1 Processor -- start 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.489+0000 7fd308417740 1 -- start start 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd308417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 0x5582cf92dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd308417740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92abc0 0x5582cf93b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd308417740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf92e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd308417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001ae740 con 0x5582cf9299f0 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd308417740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001ac240 con 0x5582cf92abc0 2024-03-26T14:08:39.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd308417740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001aa2c0 con 0x5582cf92b580 2024-03-26T14:08:39.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 0x5582cf92dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 0x5582cf92dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50414/0 (socket says 172.21.15.137:50414) 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 -- 172.21.15.137:0/2793079034 learned_addr learned my addr 172.21.15.137:0/2793079034 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fe000640 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92abc0 0x5582cf93b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2ff400640 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf92e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 -- 172.21.15.137:0/2793079034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 msgr2=0x5582cf92e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf92e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 -- 172.21.15.137:0/2793079034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92abc0 msgr2=0x5582cf93b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92abc0 0x5582cf93b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2fea00640 1 -- 172.21.15.137:0/2793079034 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582cf925830 con 0x5582cf9299f0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.490+0000 7fd2ff400640 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf92e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.491+0000 7fd2fea00640 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 0x5582cf92dc60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0035000 tx=0x7fd2f0044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e41cc5dea9a9474 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.491+0000 7fd2eb400640 1 -- 172.21.15.137:0/2793079034 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00042a0 con 0x5582cf9299f0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.491+0000 7fd2eb400640 1 -- 172.21.15.137:0/2793079034 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2f0059070 con 0x5582cf9299f0 2024-03-26T14:08:39.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.491+0000 7fd2eb400640 1 -- 172.21.15.137:0/2793079034 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0004b60 con 0x5582cf9299f0 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 -- 172.21.15.137:0/2793079034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 msgr2=0x5582cf92dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 0x5582cf92dc60 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0035000 tx=0x7fd2f0044000 comp rx=0 tx=0).stop 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 -- 172.21.15.137:0/2793079034 shutdown_connections 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf92abc0 0x5582cf93b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92b580 0x5582cf92e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 --2- 172.21.15.137:0/2793079034 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf9299f0 0x5582cf92dc60 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 -- 172.21.15.137:0/2793079034 >> 172.21.15.137:0/2793079034 conn(0x5582cf924de0 msgr2=0x5582cf9288d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 -- 172.21.15.137:0/2793079034 shutdown_connections 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 -- 172.21.15.137:0/2793079034 wait complete. 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.492+0000 7fd308417740 1 -- start start 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 Processor -- start 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf9373b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92abc0 0x5582cf9378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 0x5582cf93a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001aa8c0 con 0x5582cf92b580 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001abbe0 con 0x5582cf9299f0 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001aaf20 con 0x5582cf92abc0 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd308417740 10 client.0 _ll_register_callbacks cb 0x7fd2ec00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd2ff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 0x5582cf93a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd2ff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 0x5582cf93a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50424/0 (socket says 172.21.15.137:50424) 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd2ff400640 1 -- 172.21.15.137:0/2292830357 learned_addr learned my addr 172.21.15.137:0/2292830357 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.493+0000 7fd2fea00640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf9373b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2fe000640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92abc0 0x5582cf9378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2ff400640 1 -- 172.21.15.137:0/2292830357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92abc0 msgr2=0x5582cf9378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2ff400640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92abc0 0x5582cf9378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2ff400640 1 -- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 msgr2=0x5582cf9373b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2ff400640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf9373b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2ff400640 1 -- 172.21.15.137:0/2292830357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582cf93af00 con 0x5582cf92b580 2024-03-26T14:08:39.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2ff400640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 0x5582cf93a910 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8007fd0 tx=0x7fd2f8003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e19b2ac5a040b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:39.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.494+0000 7fd2e8c00640 1 -- 172.21.15.137:0/2292830357 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8010700 con 0x5582cf92b580 2024-03-26T14:08:39.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd2e8c00640 1 -- 172.21.15.137:0/2292830357 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd2f80108a0 con 0x5582cf92b580 2024-03-26T14:08:39.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2f0059070 con 0x5582cf92b580 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd2e8c00640 1 -- 172.21.15.137:0/2292830357 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f8032050 con 0x5582cf92b580 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd308417740 10 client.16980 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5582cf99baa0 con 0x5582cf92b580 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd2e8c00640 1 -- 172.21.15.137:0/2292830357 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2f0059070 con 0x5582cf92b580 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd2e8c00640 1 -- 172.21.15.137:0/2292830357 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd2f8023440 con 0x5582cf92b580 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd308417740 10 client.16980 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd308417740 10 client.16980 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.495+0000 7fd308417740 20 client.16980 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.496+0000 7fd308417740 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf99c130 0x5582cf99e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.496+0000 7fd308417740 4 client.16980 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.496+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582cf895b50 con 0x5582cf99c130 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.496+0000 7fd2fea00640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf99c130 0x5582cf99e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd2fea00640 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf99c130 0x5582cf99e550 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=275505a36076b5c2 server_cookie=3384090074c693c9 in_seq=0 out_seq=0 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd2e8c00640 10 client.16980 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:39.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd2e8c00640 1 -- 172.21.15.137:0/2292830357 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582cf895b50 con 0x5582cf99c130 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd2e8c00640 10 client.16980 handle_command_reply: tid=0 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd308417740 1 client.16980 shutdown 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 msgr2=0x5582cf93a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd308417740 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 0x5582cf93a910 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8007fd0 tx=0x7fd2f8003c70 comp rx=0 tx=0).stop 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 shutdown_connections 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd308417740 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5582cf99c130 0x5582cf99e550 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd2e8c00640 0 client.16980 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.497+0000 7fd308417740 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5582cf92b580 0x5582cf93a910 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd308417740 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582cf92abc0 0x5582cf9378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd308417740 1 --2- 172.21.15.137:0/2292830357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5582cf9299f0 0x5582cf9373b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 >> 172.21.15.137:0/2292830357 conn(0x5582cf924de0 msgr2=0x5582cf9288d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 shutdown_connections 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd308417740 1 -- 172.21.15.137:0/2292830357 wait complete. 2024-03-26T14:08:39.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd308417740 20 client.16980 trim_cache size 0 max 16384 2024-03-26T14:08:39.500 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:08:39.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2e40a2800 msgr2=0x7fd2e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2e40a2800 0x7fd2e40a4cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0006010 tx=0x7fd2f0005a20 comp rx=0 tx=0).stop 2024-03-26T14:08:39.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 msgr2=0x7fd300199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.498+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 0x7fd300199ea0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00e9c0 tx=0x7fd2ec00ee80 comp rx=0 tx=0).stop 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 shutdown_connections 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd2e40a2800 0x7fd2e40a4cc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd300107260 0x7fd3001a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd300104930 0x7fd300199ea0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 --2- 172.21.15.137:0/3513610962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd300102000 0x7fd300199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 >> 172.21.15.137:0/3513610962 conn(0x7fd3000fba10 msgr2=0x7fd3001016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 shutdown_connections 2024-03-26T14:08:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.499+0000 7fd2ffe00640 1 -- 172.21.15.137:0/3513610962 wait complete. 2024-03-26T14:08:39.517 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:08:39.517 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:08:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 -- 172.21.15.137:0/1854342378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c101670 msgr2=0x7fbf8c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 --2- 172.21.15.137:0/1854342378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c101670 0x7fbf8c101af0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009900 tx=0x7fbf7802bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 -- 172.21.15.137:0/1854342378 shutdown_connections 2024-03-26T14:08:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 --2- 172.21.15.137:0/1854342378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf8c102030 0x7fbf8c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 --2- 172.21.15.137:0/1854342378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c101670 0x7fbf8c101af0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 --2- 172.21.15.137:0/1854342378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf8c100480 0x7fbf8c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.744+0000 7fbf90a00640 1 -- 172.21.15.137:0/1854342378 >> 172.21.15.137:0/1854342378 conn(0x7fbf8c0fba10 msgr2=0x7fbf8c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.745+0000 7fbf90a00640 1 -- 172.21.15.137:0/1854342378 shutdown_connections 2024-03-26T14:08:39.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.745+0000 7fbf90a00640 1 -- 172.21.15.137:0/1854342378 wait complete. 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 Processor -- start 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 -- start start 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 0x7fbf8c104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf8c101670 0x7fbf8c102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf8c102030 0x7fbf8c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c10c010 con 0x7fbf8c100480 2024-03-26T14:08:39.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c10be90 con 0x7fbf8c101670 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf90a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c10c190 con 0x7fbf8c102030 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf8a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 0x7fbf8c104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf8a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 0x7fbf8c104410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50450/0 (socket says 172.21.15.137:50450) 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.746+0000 7fbf8a800640 1 -- 172.21.15.137:0/3172656803 learned_addr learned my addr 172.21.15.137:0/3172656803 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf89e00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf8c101670 0x7fbf8c102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8b200640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf8c102030 0x7fbf8c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8a800640 1 -- 172.21.15.137:0/3172656803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf8c102030 msgr2=0x7fbf8c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf8c102030 0x7fbf8c102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8a800640 1 -- 172.21.15.137:0/3172656803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf8c101670 msgr2=0x7fbf8c102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf8c101670 0x7fbf8c102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8a800640 1 -- 172.21.15.137:0/3172656803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c1034d0 con 0x7fbf8c100480 2024-03-26T14:08:39.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 0x7fbf8c104410 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00d770 tx=0x7fbf7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.747+0000 7fbf83400640 1 -- 172.21.15.137:0/3172656803 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c017070 con 0x7fbf8c100480 2024-03-26T14:08:39.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.748+0000 7fbf83400640 1 -- 172.21.15.137:0/3172656803 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf7c009c30 con 0x7fbf8c100480 2024-03-26T14:08:39.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.748+0000 7fbf83400640 1 -- 172.21.15.137:0/3172656803 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c003d00 con 0x7fbf8c100480 2024-03-26T14:08:39.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.748+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8c1037c0 con 0x7fbf8c100480 2024-03-26T14:08:39.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.748+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c1a61c0 con 0x7fbf8c100480 2024-03-26T14:08:39.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.751+0000 7fbf83400640 1 -- 172.21.15.137:0/3172656803 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbf7c003ea0 con 0x7fbf8c100480 2024-03-26T14:08:39.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 Processor -- start 2024-03-26T14:08:39.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 -- start start 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c09f0 0x557ef35c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 0x557ef35c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c2580 0x557ef35c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef35252c0 con 0x557ef35c1bc0 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef34cdc90 con 0x557ef35c09f0 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.752+0000 7fbf9403d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef3521360 con 0x557ef35c2580 2024-03-26T14:08:39.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 0x557ef35c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 0x557ef35c2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50452/0 (socket says 172.21.15.137:50452) 2024-03-26T14:08:39.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 -- 172.21.15.137:0/886833207 learned_addr learned my addr 172.21.15.137:0/886833207 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8b200640 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c09f0 0x557ef35c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf89e00640 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c2580 0x557ef35c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf83400640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf700a2800 0x7fbf700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 -- 172.21.15.137:0/886833207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c2580 msgr2=0x557ef35c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c2580 0x557ef35c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 -- 172.21.15.137:0/886833207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c09f0 msgr2=0x557ef35c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c09f0 0x557ef35c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.753+0000 7fbf8a800640 1 -- 172.21.15.137:0/886833207 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef34f2080 con 0x557ef35c1bc0 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.754+0000 7fbf83400640 1 -- 172.21.15.137:0/3172656803 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf7c0ceb60 con 0x7fbf8c100480 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.754+0000 7fbf8b200640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf700a2800 0x7fbf700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.754+0000 7fbf8a800640 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 0x557ef35c2040 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c04f440 tx=0x7fbf7c05b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296265a150737149 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.754+0000 7fbf77400640 1 -- 172.21.15.137:0/886833207 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c09ac70 con 0x557ef35c1bc0 2024-03-26T14:08:39.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.754+0000 7fbf77400640 1 -- 172.21.15.137:0/886833207 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf7c04f470 con 0x557ef35c1bc0 2024-03-26T14:08:39.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.754+0000 7fbf77400640 1 -- 172.21.15.137:0/886833207 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c01e120 con 0x557ef35c1bc0 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 -- 172.21.15.137:0/886833207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 msgr2=0x557ef35c2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 0x557ef35c2040 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c04f440 tx=0x7fbf7c05b390 comp rx=0 tx=0).stop 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf8b200640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf700a2800 0x7fbf700a4cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x557ef35c1ac0 tx=0x7fbf84004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 -- 172.21.15.137:0/886833207 shutdown_connections 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c2580 0x557ef35c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c1bc0 0x557ef35c2040 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 --2- 172.21.15.137:0/886833207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c09f0 0x557ef35c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 -- 172.21.15.137:0/886833207 >> 172.21.15.137:0/886833207 conn(0x557ef35bbde0 msgr2=0x557ef35bc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 -- 172.21.15.137:0/886833207 shutdown_connections 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.755+0000 7fbf9403d740 1 -- 172.21.15.137:0/886833207 wait complete. 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.756+0000 7fbf9403d740 1 -- start start 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.756+0000 7fbf9403d740 1 Processor -- start 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef35c9530 con 0x557ef35c2580 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef34e10d0 con 0x557ef35c1bc0 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef35c96b0 con 0x557ef35c09f0 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf9403d740 10 client.0 _ll_register_callbacks cb 0x557ef34e5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50464/0 (socket says 172.21.15.137:50464) 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 -- 172.21.15.137:0/3381981085 learned_addr learned my addr 172.21.15.137:0/3381981085 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf89e00640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 -- 172.21.15.137:0/3381981085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 msgr2=0x557ef35cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 -- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 msgr2=0x557ef35cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8b200640 1 -- 172.21.15.137:0/3381981085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ef32393f0 con 0x557ef35c2580 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.757+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.758+0000 7fbf8b200640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d20f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400f8a0 tx=0x7fbf84004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81d219c20a196c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.758+0000 7fbf6be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.758+0000 7fbf6be00640 1 -- 172.21.15.137:0/3381981085 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8401ede0 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.758+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ef3532800 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.758+0000 7fbf6be00640 1 -- 172.21.15.137:0/3381981085 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf8401b410 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf6be00640 1 -- 172.21.15.137:0/3381981085 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf8400fe20 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 10 client.16992 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557ef33037d0 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf6be00640 1 -- 172.21.15.137:0/3381981085 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x557ef3532800 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf6be00640 1 -- 172.21.15.137:0/3381981085 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbf84025070 con 0x557ef35c2580 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 10 client.16992 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 20 client.16992 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef35d6670 0x557ef35d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 4 client.16992 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:39.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.759+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ef352cb50 con 0x557ef35d6670 2024-03-26T14:08:39.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.760+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef35d6670 0x557ef35d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.760+0000 7fbf8a800640 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef35d6670 0x557ef35d8af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6c42d15b7ca1306 server_cookie=ea523ab04fd0c95d in_seq=0 out_seq=0 2024-03-26T14:08:39.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.760+0000 7fbf6be00640 10 client.16992 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf6be00640 1 -- 172.21.15.137:0/3381981085 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557ef352cb50 con 0x557ef35d6670 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf6be00640 10 client.16992 handle_command_reply: tid=0 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 client.16992 shutdown 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 msgr2=0x557ef35d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d20f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400f8a0 tx=0x7fbf84004ab0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 shutdown_connections 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef35d6670 0x557ef35d8af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d20f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 --2- 172.21.15.137:0/3381981085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 >> 172.21.15.137:0/3381981085 conn(0x557ef35bbde0 msgr2=0x557ef35bb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.762+0000 7fbf6be00640 0 client.16992 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:39.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.763+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 shutdown_connections 2024-03-26T14:08:39.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.763+0000 7fbf9403d740 1 -- 172.21.15.137:0/3381981085 wait complete. 2024-03-26T14:08:39.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.763+0000 7fbf9403d740 20 client.16992 trim_cache size 0 max 16384 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.780+0000 7fbf9403d740 1 Processor -- start 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.780+0000 7fbf9403d740 1 -- start start 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35bafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf9403d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1b2c20 con 0x557ef35c2580 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf9403d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1b0720 con 0x557ef35c1bc0 2024-03-26T14:08:39.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf9403d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1ae7a0 con 0x557ef35c09f0 2024-03-26T14:08:39.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50478/0 (socket says 172.21.15.137:50478) 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 -- 172.21.15.137:0/214095109 learned_addr learned my addr 172.21.15.137:0/214095109 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8a800640 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35bafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf89e00640 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 -- 172.21.15.137:0/214095109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 msgr2=0x557ef35bafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35bafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 -- 172.21.15.137:0/214095109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 msgr2=0x557ef35d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.781+0000 7fbf8b200640 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.782+0000 7fbf8b200640 1 -- 172.21.15.137:0/214095109 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef35bbb50 con 0x557ef35c2580 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.782+0000 7fbf8b200640 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35dd3b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbf84014010 tx=0x7fbf84009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25782dcd542dcdfd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.782+0000 7fbf76a00640 1 -- 172.21.15.137:0/214095109 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8403a070 con 0x557ef35c2580 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.782+0000 7fbf76a00640 1 -- 172.21.15.137:0/214095109 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf84027440 con 0x557ef35c2580 2024-03-26T14:08:39.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.782+0000 7fbf76a00640 1 -- 172.21.15.137:0/214095109 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84035cf0 con 0x557ef35c2580 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 -- 172.21.15.137:0/214095109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 msgr2=0x557ef35dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35dd3b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbf84014010 tx=0x7fbf84009e40 comp rx=0 tx=0).stop 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 -- 172.21.15.137:0/214095109 shutdown_connections 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35dd3b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 --2- 172.21.15.137:0/214095109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35bafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 -- 172.21.15.137:0/214095109 >> 172.21.15.137:0/214095109 conn(0x557ef35bbde0 msgr2=0x557ef35ba8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 -- 172.21.15.137:0/214095109 shutdown_connections 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.783+0000 7fbf9403d740 1 -- 172.21.15.137:0/214095109 wait complete. 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 -- start start 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 Processor -- start 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1aeda0 con 0x557ef35c2580 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1b00c0 con 0x557ef35c1bc0 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1af400 con 0x557ef35c09f0 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.784+0000 7fbf9403d740 10 client.0 _ll_register_callbacks cb 0x7fbf8400f550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50488/0 (socket says 172.21.15.137:50488) 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 -- 172.21.15.137:0/34097451 learned_addr learned my addr 172.21.15.137:0/34097451 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8a800640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf89e00640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 -- 172.21.15.137:0/34097451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 msgr2=0x557ef35d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 -- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 msgr2=0x557ef35d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.785+0000 7fbf8b200640 1 -- 172.21.15.137:0/34097451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ef35d0b80 con 0x557ef35c2580 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf8b200640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d0350 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbf84027a90 tx=0x7fbf84009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fed80a0837a566a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:39.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf6be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf6be00640 1 -- 172.21.15.137:0/34097451 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8403a070 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf6be00640 1 -- 172.21.15.137:0/34097451 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbf84035a70 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf84027440 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf6be00640 1 -- 172.21.15.137:0/34097451 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf840275f0 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf9403d740 10 client.16998 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.786+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557ef3632ba0 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf6be00640 1 -- 172.21.15.137:0/34097451 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbf84027440 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf6be00640 1 -- 172.21.15.137:0/34097451 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbf84025040 con 0x557ef35c2580 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf9403d740 10 client.16998 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf9403d740 10 client.16998 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf9403d740 20 client.16998 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf9403d740 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef36334a0 0x557ef36358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf9403d740 4 client.16998 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.787+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ef34aa500 con 0x557ef36334a0 2024-03-26T14:08:39.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.788+0000 7fbf8a800640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef36334a0 0x557ef36358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:39.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.788+0000 7fbf8a800640 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef36334a0 0x557ef36358c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df9594472c20e900 server_cookie=17f21fddd24c2a63 in_seq=0 out_seq=0 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.788+0000 7fbf6be00640 10 client.16998 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf6be00640 1 -- 172.21.15.137:0/34097451 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ef34aa500 con 0x557ef36334a0 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf6be00640 10 client.16998 handle_command_reply: tid=0 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 client.16998 shutdown 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 msgr2=0x557ef35d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d0350 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbf84027a90 tx=0x7fbf84009a60 comp rx=0 tx=0).stop 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 shutdown_connections 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x557ef36334a0 0x557ef36358c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ef35c1bc0 0x557ef35d2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557ef35c2580 0x557ef35d0350 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 --2- 172.21.15.137:0/34097451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557ef35c09f0 0x557ef35d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 >> 172.21.15.137:0/34097451 conn(0x557ef35bbde0 msgr2=0x557ef35bb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf6be00640 0 client.16998 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:08:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 shutdown_connections 2024-03-26T14:08:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.789+0000 7fbf9403d740 1 -- 172.21.15.137:0/34097451 wait complete. 2024-03-26T14:08:39.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.790+0000 7fbf9403d740 20 client.16998 trim_cache size 0 max 16384 2024-03-26T14:08:39.791 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.790+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf700a2800 msgr2=0x7fbf700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.790+0000 7fbf90a00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf700a2800 0x7fbf700a4cc0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x557ef35c1ac0 tx=0x7fbf84004a40 comp rx=0 tx=0).stop 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.790+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 msgr2=0x7fbf8c104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.790+0000 7fbf90a00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 0x7fbf8c104410 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00d770 tx=0x7fbf7c00dc30 comp rx=0 tx=0).stop 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.790+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 shutdown_connections 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbf700a2800 0x7fbf700a4cc0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf8c101670 0x7fbf8c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf8c102030 0x7fbf8c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 --2- 172.21.15.137:0/3172656803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbf8c100480 0x7fbf8c104410 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 >> 172.21.15.137:0/3172656803 conn(0x7fbf8c0fba10 msgr2=0x7fbf8c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 shutdown_connections 2024-03-26T14:08:39.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:39.791+0000 7fbf90a00640 1 -- 172.21.15.137:0/3172656803 wait complete. 2024-03-26T14:08:39.807 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:08:39.807 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:08:40.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 -- 172.21.15.137:0/2924176059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 msgr2=0x7f686c109a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 --2- 172.21.15.137:0/2924176059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c109a60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6864009cd0 tx=0x7f686402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 -- 172.21.15.137:0/2924176059 shutdown_connections 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 --2- 172.21.15.137:0/2924176059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f686c102810 0x7f686c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 --2- 172.21.15.137:0/2924176059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c109a60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 --2- 172.21.15.137:0/2924176059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f686c0ff840 0x7f686c0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 -- 172.21.15.137:0/2924176059 >> 172.21.15.137:0/2924176059 conn(0x7f686c0fb190 msgr2=0x7f686c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 -- 172.21.15.137:0/2924176059 shutdown_connections 2024-03-26T14:08:40.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.038+0000 7f6870a00640 1 -- 172.21.15.137:0/2924176059 wait complete. 2024-03-26T14:08:40.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.039+0000 7f6870a00640 1 Processor -- start 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.039+0000 7f6870a00640 1 -- start start 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.039+0000 7f6870a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f686c0ff840 0x7f686c102180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.039+0000 7f6870a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f686c102810 0x7f686c1007c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f6870a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c100d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f6870a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c10d0b0 con 0x7f686c1031d0 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f6870a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c10cf30 con 0x7f686c102810 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f6870a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c10d230 con 0x7f686c0ff840 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c100d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50500/0 (socket says 172.21.15.137:50500) 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 -- 172.21.15.137:0/1999014554 learned_addr learned my addr 172.21.15.137:0/1999014554 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686aa00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f686c102810 0x7f686c1007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686b400640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f686c0ff840 0x7f686c102180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 -- 172.21.15.137:0/1999014554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f686c0ff840 msgr2=0x7f686c102180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f686c0ff840 0x7f686c102180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 -- 172.21.15.137:0/1999014554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f686c102810 msgr2=0x7f686c1007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f686c102810 0x7f686c1007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.040+0000 7f686be00640 1 -- 172.21.15.137:0/1999014554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c101240 con 0x7f686c1031d0 2024-03-26T14:08:40.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.041+0000 7f686be00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c100d00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f686402c630 tx=0x7f68640328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:40.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.041+0000 7f6863e00640 1 -- 172.21.15.137:0/1999014554 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864041020 con 0x7f686c1031d0 2024-03-26T14:08:40.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.041+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c1014d0 con 0x7f686c1031d0 2024-03-26T14:08:40.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.041+0000 7f6863e00640 1 -- 172.21.15.137:0/1999014554 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6864003970 con 0x7f686c1031d0 2024-03-26T14:08:40.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.041+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c1a6180 con 0x7f686c1031d0 2024-03-26T14:08:40.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.041+0000 7f6863e00640 1 -- 172.21.15.137:0/1999014554 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864039040 con 0x7f686c1031d0 2024-03-26T14:08:40.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.045+0000 7f6874c7f740 1 Processor -- start 2024-03-26T14:08:40.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.045+0000 7f6863e00640 1 -- 172.21.15.137:0/1999014554 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6864044020 con 0x7f686c1031d0 2024-03-26T14:08:40.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.045+0000 7f6874c7f740 1 -- start start 2024-03-26T14:08:40.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.045+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 0x55781d845e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.045+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d846bc0 0x55781d847040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.045+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d84de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f6874c7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781d7aa2c0 con 0x55781d8459f0 2024-03-26T14:08:40.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f6874c7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781d752c90 con 0x55781d846bc0 2024-03-26T14:08:40.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f6874c7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781d7a6360 con 0x55781d847580 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 0x55781d845e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 0x55781d845e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50506/0 (socket says 172.21.15.137:50506) 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f686aa00640 1 -- 172.21.15.137:0/837931681 learned_addr learned my addr 172.21.15.137:0/837931681 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f686b400640 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d846bc0 0x55781d847040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.046+0000 7f686aa00640 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d84de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686aa00640 1 -- 172.21.15.137:0/837931681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 msgr2=0x55781d84de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686aa00640 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d84de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686aa00640 1 -- 172.21.15.137:0/837931681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d846bc0 msgr2=0x55781d847040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686aa00640 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d846bc0 0x55781d847040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686aa00640 1 -- 172.21.15.137:0/837931681 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55781d777080 con 0x55781d8459f0 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686b400640 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d846bc0 0x55781d847040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f6863e00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68500a2800 0x7f68500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686aa00640 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 0x55781d845e10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f68580134a0 tx=0x7f6858009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc9b37fd329c9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:40.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f6863e00640 1 -- 172.21.15.137:0/1999014554 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f68640f0ff0 con 0x7f686c1031d0 2024-03-26T14:08:40.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f6857e00640 1 -- 172.21.15.137:0/837931681 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800f660 con 0x55781d8459f0 2024-03-26T14:08:40.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f686b400640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68500a2800 0x7f68500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f6857e00640 1 -- 172.21.15.137:0/837931681 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f685800fcb0 con 0x55781d8459f0 2024-03-26T14:08:40.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.047+0000 7f6857e00640 1 -- 172.21.15.137:0/837931681 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800ecb0 con 0x55781d8459f0 2024-03-26T14:08:40.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 -- 172.21.15.137:0/837931681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 msgr2=0x55781d845e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 0x55781d845e10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f68580134a0 tx=0x7f6858009790 comp rx=0 tx=0).stop 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f686b400640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68500a2800 0x7f68500a4cc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f686c068600 tx=0x7f685c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 -- 172.21.15.137:0/837931681 shutdown_connections 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d846bc0 0x55781d847040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d84de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 --2- 172.21.15.137:0/837931681 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d8459f0 0x55781d845e10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 -- 172.21.15.137:0/837931681 >> 172.21.15.137:0/837931681 conn(0x55781d840de0 msgr2=0x55781d8411e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 -- 172.21.15.137:0/837931681 shutdown_connections 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.048+0000 7f6874c7f740 1 -- 172.21.15.137:0/837931681 wait complete. 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.049+0000 7f6874c7f740 1 -- start start 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.049+0000 7f6874c7f740 1 Processor -- start 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d84fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d850240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d857280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781d84e590 con 0x55781d846bc0 2024-03-26T14:08:40.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781d7660d0 con 0x55781d8459f0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781d84e710 con 0x55781d847580 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f6874c7f740 10 client.0 _ll_register_callbacks cb 0x55781d76ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d850240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d850240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50518/0 (socket says 172.21.15.137:50518) 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 -- 172.21.15.137:0/4244633753 learned_addr learned my addr 172.21.15.137:0/4244633753 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686b400640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d84fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686be00640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d857280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 -- 172.21.15.137:0/4244633753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 msgr2=0x55781d857280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d857280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 -- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 msgr2=0x55781d84fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d84fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.050+0000 7f686aa00640 1 -- 172.21.15.137:0/4244633753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55781d4be3f0 con 0x55781d846bc0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.051+0000 7f686aa00640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d850240 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f685800e780 tx=0x7f6858004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646503c236210b4d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:40.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.051+0000 7f6854c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.051+0000 7f6854c00640 1 -- 172.21.15.137:0/4244633753 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800fe60 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.051+0000 7f6854c00640 1 -- 172.21.15.137:0/4244633753 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f685800ec00 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.051+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f685800fcb0 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.051+0000 7f6854c00640 1 -- 172.21.15.137:0/4244633753 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6858009800 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 10 client.17010 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55781d5887d0 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6854c00640 1 -- 172.21.15.137:0/4244633753 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f685800fcb0 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6854c00640 1 -- 172.21.15.137:0/4244633753 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f68580035f0 con 0x55781d846bc0 2024-03-26T14:08:40.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 10 client.17010 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 20 client.17010 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d85b440 0x55781d85d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 4 client.17010 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.052+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55781d7b7800 con 0x55781d85b440 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.053+0000 7f686b400640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d85b440 0x55781d85d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.054+0000 7f686b400640 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d85b440 0x55781d85d850 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471508f34faa3ec0 server_cookie=51f0beade12b82f2 in_seq=0 out_seq=0 2024-03-26T14:08:40.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.054+0000 7f6854c00640 10 client.17010 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.055+0000 7f6854c00640 1 -- 172.21.15.137:0/4244633753 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55781d7b7800 con 0x55781d85b440 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.055+0000 7f6854c00640 10 client.17010 handle_command_reply: tid=0 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.055+0000 7f6874c7f740 1 client.17010 shutdown 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 msgr2=0x55781d850240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d850240 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f685800e780 tx=0x7f6858004610 comp rx=0 tx=0).stop 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 shutdown_connections 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d85b440 0x55781d85d850 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d857280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d850240 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 --2- 172.21.15.137:0/4244633753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d84fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6854c00640 0 client.17010 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:40.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 >> 172.21.15.137:0/4244633753 conn(0x55781d840de0 msgr2=0x55781d840160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:40.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 shutdown_connections 2024-03-26T14:08:40.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 1 -- 172.21.15.137:0/4244633753 wait complete. 2024-03-26T14:08:40.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.056+0000 7f6874c7f740 20 client.17010 trim_cache size 0 max 16384 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.076+0000 7f6874c7f740 1 Processor -- start 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.076+0000 7f6874c7f740 1 -- start start 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.076+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d8459f0 0x55781d83bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.076+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d857240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.076+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d847580 0x55781d849d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f6874c7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1b2b40 con 0x55781d846bc0 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f6874c7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1b0640 con 0x55781d847580 2024-03-26T14:08:40.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f6874c7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1ae6c0 con 0x55781d8459f0 2024-03-26T14:08:40.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d857240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d857240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50532/0 (socket says 172.21.15.137:50532) 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 -- 172.21.15.137:0/480648817 learned_addr learned my addr 172.21.15.137:0/480648817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686be00640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d847580 0x55781d849d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686b400640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d8459f0 0x55781d83bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 -- 172.21.15.137:0/480648817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d8459f0 msgr2=0x55781d83bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d8459f0 0x55781d83bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 -- 172.21.15.137:0/480648817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d847580 msgr2=0x55781d849d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d847580 0x55781d849d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686aa00640 1 -- 172.21.15.137:0/480648817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55781d84d7f0 con 0x55781d846bc0 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686be00640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d847580 0x55781d849d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.077+0000 7f686b400640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d8459f0 0x55781d83bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.078+0000 7f686aa00640 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d857240 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6858003a40 tx=0x7f6858009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb6bfa964e88b3f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:40.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.078+0000 7f6857400640 1 -- 172.21.15.137:0/480648817 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6858013d30 con 0x55781d846bc0 2024-03-26T14:08:40.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.078+0000 7f6857400640 1 -- 172.21.15.137:0/480648817 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68580057f0 con 0x55781d846bc0 2024-03-26T14:08:40.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.078+0000 7f6857400640 1 -- 172.21.15.137:0/480648817 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685800ec00 con 0x55781d846bc0 2024-03-26T14:08:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 -- 172.21.15.137:0/480648817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 msgr2=0x55781d857240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d857240 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6858003a40 tx=0x7f6858009d50 comp rx=0 tx=0).stop 2024-03-26T14:08:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 -- 172.21.15.137:0/480648817 shutdown_connections 2024-03-26T14:08:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d847580 0x55781d849d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d857240 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 --2- 172.21.15.137:0/480648817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d8459f0 0x55781d83bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 -- 172.21.15.137:0/480648817 >> 172.21.15.137:0/480648817 conn(0x55781d840de0 msgr2=0x55781d8448d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 -- 172.21.15.137:0/480648817 shutdown_connections 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 -- 172.21.15.137:0/480648817 wait complete. 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.079+0000 7f6874c7f740 1 -- start start 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 Processor -- start 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d8532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d8537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d8567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1aecc0 con 0x55781d846bc0 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1affe0 con 0x55781d8459f0 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1af320 con 0x55781d847580 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f6874c7f740 10 client.0 _ll_register_callbacks cb 0x7f685800e6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:40.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d8537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f686aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d8537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50536/0 (socket says 172.21.15.137:50536) 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.080+0000 7f686aa00640 1 -- 172.21.15.137:0/878486989 learned_addr learned my addr 172.21.15.137:0/878486989 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686b400640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d8532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686aa00640 1 -- 172.21.15.137:0/878486989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 msgr2=0x55781d8567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686be00640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d8567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686aa00640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d8567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686aa00640 1 -- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 msgr2=0x55781d8532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686aa00640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d8532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686aa00640 1 -- 172.21.15.137:0/878486989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55781d856da0 con 0x55781d846bc0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686b400640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d8532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f686aa00640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d8537e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6858021040 tx=0x7f68580092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bfd55655a2b9a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f6854c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f6854c00640 1 -- 172.21.15.137:0/878486989 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685803b020 con 0x55781d846bc0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f6854c00640 1 -- 172.21.15.137:0/878486989 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6858003bb0 con 0x55781d846bc0 2024-03-26T14:08:40.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.081+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68580057f0 con 0x55781d846bc0 2024-03-26T14:08:40.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6854c00640 1 -- 172.21.15.137:0/878486989 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6858046050 con 0x55781d846bc0 2024-03-26T14:08:40.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 10 client.17016 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:40.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55781d8b7570 con 0x55781d846bc0 2024-03-26T14:08:40.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6854c00640 1 -- 172.21.15.137:0/878486989 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f68580057f0 con 0x55781d846bc0 2024-03-26T14:08:40.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6854c00640 1 -- 172.21.15.137:0/878486989 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6858004890 con 0x55781d846bc0 2024-03-26T14:08:40.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 10 client.17016 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:40.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 20 client.17016 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d8b7d20 0x55781d8ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 4 client.17016 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.082+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55781d7b1b50 con 0x55781d8b7d20 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.083+0000 7f686b400640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d8b7d20 0x55781d8ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f686b400640 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d8b7d20 0x55781d8ba140 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fb1fa73bc9c23b2 server_cookie=82ea07071940eeda in_seq=0 out_seq=0 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6854c00640 10 client.17016 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6854c00640 1 -- 172.21.15.137:0/878486989 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55781d7b1b50 con 0x55781d8b7d20 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6854c00640 10 client.17016 handle_command_reply: tid=0 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 client.17016 shutdown 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 msgr2=0x55781d8537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d8537e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6858021040 tx=0x7f68580092f0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 shutdown_connections 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55781d8b7d20 0x55781d8ba140 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d847580 0x55781d8567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55781d846bc0 0x55781d8537e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 --2- 172.21.15.137:0/878486989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55781d8459f0 0x55781d8532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6854c00640 0 client.17016 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.084+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 >> 172.21.15.137:0/878486989 conn(0x55781d840de0 msgr2=0x55781d83f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 shutdown_connections 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6874c7f740 1 -- 172.21.15.137:0/878486989 wait complete. 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6874c7f740 20 client.17016 trim_cache size 0 max 16384 2024-03-26T14:08:40.087 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68500a2800 msgr2=0x7f68500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6870a00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68500a2800 0x7f68500a4cc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f686c068600 tx=0x7f685c005f90 comp rx=0 tx=0).stop 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 msgr2=0x7f686c100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.085+0000 7f6870a00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c100d00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f686402c630 tx=0x7f68640328a0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 shutdown_connections 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f68500a2800 0x7f68500a4cc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f686c102810 0x7f686c1007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f686c1031d0 0x7f686c100d00 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 --2- 172.21.15.137:0/1999014554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f686c0ff840 0x7f686c102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:40.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 >> 172.21.15.137:0/1999014554 conn(0x7f686c0fb190 msgr2=0x7f686c105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:40.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 shutdown_connections 2024-03-26T14:08:40.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:40.086+0000 7f6870a00640 1 -- 172.21.15.137:0/1999014554 wait complete. 2024-03-26T14:08:40.101 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:39 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3504525854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:08:40.325 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:39 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3504525854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:08:40.353 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:39 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3504525854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:08:41.105 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:08:41.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:08:41.105 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:08:41.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:40 smithi137 ceph-mon[31539]: pgmap v647: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 -- 172.21.15.137:0/2301508850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 msgr2=0x7fe8fc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 --2- 172.21.15.137:0/2301508850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc104e10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009900 tx=0x7fe8e802bc30 comp rx=0 tx=0).stop 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 -- 172.21.15.137:0/2301508850 shutdown_connections 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 --2- 172.21.15.137:0/2301508850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8fc105350 0x7fe8fc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 --2- 172.21.15.137:0/2301508850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc104e10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 --2- 172.21.15.137:0/2301508850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8fc1037a0 0x7fe8fc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 -- 172.21.15.137:0/2301508850 >> 172.21.15.137:0/2301508850 conn(0x7fe8fc0fed10 msgr2=0x7fe8fc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 -- 172.21.15.137:0/2301508850 shutdown_connections 2024-03-26T14:08:41.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.331+0000 7fe8fb200640 1 -- 172.21.15.137:0/2301508850 wait complete. 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.332+0000 7fe8fb200640 1 Processor -- start 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.332+0000 7fe8fb200640 1 -- start start 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fb200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8fc1037a0 0x7fe8fc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8fc105350 0x7fe8fc1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fb200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc10f2b0 con 0x7fe8fc104990 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc10f130 con 0x7fe8fc105350 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fb200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc10f430 con 0x7fe8fc1037a0 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50554/0 (socket says 172.21.15.137:50554) 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 -- 172.21.15.137:0/1824788032 learned_addr learned my addr 172.21.15.137:0/1824788032 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8fc1037a0 0x7fe8fc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8fa800640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8fc105350 0x7fe8fc1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 -- 172.21.15.137:0/1824788032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8fc1037a0 msgr2=0x7fe8fc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8fc1037a0 0x7fe8fc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 -- 172.21.15.137:0/1824788032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8fc105350 msgr2=0x7fe8fc1a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.333+0000 7fe8f9400640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8fc105350 0x7fe8fc1a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8f9400640 1 -- 172.21.15.137:0/1824788032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc1a15c0 con 0x7fe8fc104990 2024-03-26T14:08:41.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8f9400640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc199ee0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006010 tx=0x7fe8e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8f2a00640 1 -- 172.21.15.137:0/1824788032 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8043020 con 0x7fe8fc104990 2024-03-26T14:08:41.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8f2a00640 1 -- 172.21.15.137:0/1824788032 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8e8004aa0 con 0x7fe8fc104990 2024-03-26T14:08:41.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8fc1a1850 con 0x7fe8fc104990 2024-03-26T14:08:41.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8f2a00640 1 -- 172.21.15.137:0/1824788032 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8003c60 con 0x7fe8fc104990 2024-03-26T14:08:41.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.334+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8fc1a1da0 con 0x7fe8fc104990 2024-03-26T14:08:41.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 Processor -- start 2024-03-26T14:08:41.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 -- start start 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe8f2a00640 1 -- 172.21.15.137:0/1824788032 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe8e8046020 con 0x7fe8fc104990 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd457a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd482a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8fca5eb0 con 0x563f8fd402b0 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8fc4ea60 con 0x563f8fd45e10 2024-03-26T14:08:41.338 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.338+0000 7fe903bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8fca1f50 con 0x563f8fd40ca0 2024-03-26T14:08:41.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50564/0 (socket says 172.21.15.137:50564) 2024-03-26T14:08:41.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 -- 172.21.15.137:0/2134862708 learned_addr learned my addr 172.21.15.137:0/2134862708 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8f9400640 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 -- 172.21.15.137:0/2134862708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 msgr2=0x563f8fd457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 -- 172.21.15.137:0/2134862708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 msgr2=0x563f8fd482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8fa800640 1 -- 172.21.15.137:0/2134862708 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8fa827e0 con 0x563f8fd402b0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.339+0000 7fe8f9400640 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8fa800640 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd406d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007e50 tx=0x7fe8f400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc39b85932094dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8f2a00640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8f2a00640 1 -- 172.21.15.137:0/1824788032 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe8e80f1330 con 0x7fe8fc104990 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8e7400640 1 -- 172.21.15.137:0/2134862708 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4014070 con 0x563f8fd402b0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8e7400640 1 -- 172.21.15.137:0/2134862708 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8f400ee90 con 0x563f8fd402b0 2024-03-26T14:08:41.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.340+0000 7fe8e7400640 1 -- 172.21.15.137:0/2134862708 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40053a0 con 0x563f8fd402b0 2024-03-26T14:08:41.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 -- 172.21.15.137:0/2134862708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 msgr2=0x563f8fd406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005fd0 tx=0x7fe8ec005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd406d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007e50 tx=0x7fe8f400e720 comp rx=0 tx=0).stop 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 -- 172.21.15.137:0/2134862708 shutdown_connections 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 --2- 172.21.15.137:0/2134862708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd406d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 -- 172.21.15.137:0/2134862708 >> 172.21.15.137:0/2134862708 conn(0x563f8fd3b820 msgr2=0x563f8fd3bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 -- 172.21.15.137:0/2134862708 shutdown_connections 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.341+0000 7fe903bac740 1 -- 172.21.15.137:0/2134862708 wait complete. 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.342+0000 7fe903bac740 1 -- start start 2024-03-26T14:08:41.342 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.342+0000 7fe903bac740 1 Processor -- start 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd51a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8fd48dc0 con 0x563f8fd45e10 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8fc61eb0 con 0x563f8fd40ca0 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f8fd48f40 con 0x563f8fd402b0 2024-03-26T14:08:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe903bac740 10 client.0 _ll_register_callbacks cb 0x563f8fc56150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd51a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd51a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50566/0 (socket says 172.21.15.137:50566) 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 -- 172.21.15.137:0/1347212519 learned_addr learned my addr 172.21.15.137:0/1347212519 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8f9400640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 -- 172.21.15.137:0/1347212519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 msgr2=0x563f8fd4a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd4a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 -- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 msgr2=0x563f8fd4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.343+0000 7fe8fa800640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.344+0000 7fe8fa800640 1 -- 172.21.15.137:0/1347212519 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f8fae2f70 con 0x563f8fd45e10 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.344+0000 7fe8fa800640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd51a50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40039a0 tx=0x7fe8f40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c212780f8d781a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.344+0000 7fe8dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.344+0000 7fe8dbe00640 1 -- 172.21.15.137:0/1347212519 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4014070 con 0x563f8fd45e10 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.344+0000 7fe8dbe00640 1 -- 172.21.15.137:0/1347212519 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe8f4010640 con 0x563f8fd45e10 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.344+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8f400ee90 con 0x563f8fd45e10 2024-03-26T14:08:41.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe8dbe00640 1 -- 172.21.15.137:0/1347212519 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f402f050 con 0x563f8fd45e10 2024-03-26T14:08:41.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 10 client.17028 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:41.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563f8f97a170 con 0x563f8fd45e10 2024-03-26T14:08:41.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe8dbe00640 1 -- 172.21.15.137:0/1347212519 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8f400ee90 con 0x563f8fd45e10 2024-03-26T14:08:41.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe8dbe00640 1 -- 172.21.15.137:0/1347212519 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe8f4005120 con 0x563f8fd45e10 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 10 client.17028 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 20 client.17028 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fd55ff0 0x563f8fd583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 4 client.17028 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.345+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f8fcb33f0 con 0x563f8fd55ff0 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.346+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fd55ff0 0x563f8fd583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.347+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fd55ff0 0x563f8fd583e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27643ba92d81658e server_cookie=9087e688d2431e66 in_seq=0 out_seq=0 2024-03-26T14:08:41.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.347+0000 7fe8dbe00640 10 client.17028 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe8dbe00640 1 -- 172.21.15.137:0/1347212519 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563f8fcb33f0 con 0x563f8fd55ff0 2024-03-26T14:08:41.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe8dbe00640 10 client.17028 handle_command_reply: tid=0 2024-03-26T14:08:41.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 client.17028 shutdown 2024-03-26T14:08:41.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 msgr2=0x563f8fd51a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd51a50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40039a0 tx=0x7fe8f40039d0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 shutdown_connections 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fd55ff0 0x563f8fd583e0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd51a50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe8dbe00640 0 client.17028 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 --2- 172.21.15.137:0/1347212519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 >> 172.21.15.137:0/1347212519 conn(0x563f8fd3b820 msgr2=0x563f8fd3aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 shutdown_connections 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 1 -- 172.21.15.137:0/1347212519 wait complete. 2024-03-26T14:08:41.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.349+0000 7fe903bac740 20 client.17028 trim_cache size 0 max 16384 2024-03-26T14:08:41.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 Processor -- start 2024-03-26T14:08:41.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 -- start start 2024-03-26T14:08:41.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd41740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd51a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd41c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1ae7c0 con 0x563f8fd402b0 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1ac2c0 con 0x563f8fd45e10 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.369+0000 7fe903bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1aa340 con 0x563f8fd40ca0 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd41740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd41740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50578/0 (socket says 172.21.15.137:50578) 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 -- 172.21.15.137:0/855160889 learned_addr learned my addr 172.21.15.137:0/855160889 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8fa800640 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd41c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9400640 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd51a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 -- 172.21.15.137:0/855160889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 msgr2=0x563f8fd51a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd51a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 -- 172.21.15.137:0/855160889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 msgr2=0x563f8fd41c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd41c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 -- 172.21.15.137:0/855160889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f8fd421c0 con 0x563f8fd402b0 2024-03-26T14:08:41.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.370+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd41740 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x563f8fd410a0 tx=0x7fe8ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b969e15cc870ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.371+0000 7fe8e6a00640 1 -- 172.21.15.137:0/855160889 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec029d70 con 0x563f8fd402b0 2024-03-26T14:08:41.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.371+0000 7fe8e6a00640 1 -- 172.21.15.137:0/855160889 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8ec012920 con 0x563f8fd402b0 2024-03-26T14:08:41.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.371+0000 7fe8e6a00640 1 -- 172.21.15.137:0/855160889 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec005280 con 0x563f8fd402b0 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 -- 172.21.15.137:0/855160889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 msgr2=0x563f8fd41740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd41740 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x563f8fd410a0 tx=0x7fe8ec014000 comp rx=0 tx=0).stop 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 -- 172.21.15.137:0/855160889 shutdown_connections 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd45e10 0x563f8fd41c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd40ca0 0x563f8fd51a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 --2- 172.21.15.137:0/855160889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd402b0 0x563f8fd41740 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 -- 172.21.15.137:0/855160889 >> 172.21.15.137:0/855160889 conn(0x563f8fd3b820 msgr2=0x563f8fd3f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 -- 172.21.15.137:0/855160889 shutdown_connections 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 -- 172.21.15.137:0/855160889 wait complete. 2024-03-26T14:08:41.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 -- start start 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.372+0000 7fe903bac740 1 Processor -- start 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1aa940 con 0x563f8fd45e10 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1abc60 con 0x563f8fd40ca0 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc1aafa0 con 0x563f8fd402b0 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe903bac740 10 client.0 _ll_register_callbacks cb 0x7fe8f4007ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe8fa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe8fa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50582/0 (socket says 172.21.15.137:50582) 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe8fa800640 1 -- 172.21.15.137:0/3329083713 learned_addr learned my addr 172.21.15.137:0/3329083713 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe8f9400640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.373+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd51840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8fa800640 1 -- 172.21.15.137:0/3329083713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 msgr2=0x563f8fd51840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8fa800640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd51840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8fa800640 1 -- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 msgr2=0x563f8fd4fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8fa800640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8fa800640 1 -- 172.21.15.137:0/3329083713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f8fd50b20 con 0x563f8fd45e10 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8fa800640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd503e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007be0 tx=0x7fe8f400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d46b237a3ddc8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe8dbe00640 1 -- 172.21.15.137:0/3329083713 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4014070 con 0x563f8fd45e10 2024-03-26T14:08:41.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.374+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8ec012920 con 0x563f8fd45e10 2024-03-26T14:08:41.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.375+0000 7fe8dbe00640 1 -- 172.21.15.137:0/3329083713 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe8f4004dc0 con 0x563f8fd45e10 2024-03-26T14:08:41.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.375+0000 7fe8dbe00640 1 -- 172.21.15.137:0/3329083713 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f4042050 con 0x563f8fd45e10 2024-03-26T14:08:41.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.375+0000 7fe903bac740 10 client.17034 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:41.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.375+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563f8fdb26e0 con 0x563f8fd45e10 2024-03-26T14:08:41.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.375+0000 7fe8dbe00640 1 -- 172.21.15.137:0/3329083713 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8ec012920 con 0x563f8fd45e10 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe8dbe00640 1 -- 172.21.15.137:0/3329083713 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe8f4045020 con 0x563f8fd45e10 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe903bac740 10 client.17034 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe903bac740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe903bac740 20 client.17034 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe903bac740 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fdb2fa0 0x563f8fdb53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe903bac740 4 client.17034 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563f8fcad740 con 0x563f8fdb2fa0 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.376+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fdb2fa0 0x563f8fdb53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.377+0000 7fe8f9e00640 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fdb2fa0 0x563f8fdb53c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5002625e5c7d1d server_cookie=fdbda572647f4102 in_seq=0 out_seq=0 2024-03-26T14:08:41.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.377+0000 7fe8dbe00640 10 client.17034 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe8dbe00640 1 -- 172.21.15.137:0/3329083713 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563f8fcad740 con 0x563f8fdb2fa0 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe8dbe00640 10 client.17034 handle_command_reply: tid=0 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 client.17034 shutdown 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 msgr2=0x563f8fd503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd503e0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4007be0 tx=0x7fe8f400ef10 comp rx=0 tx=0).stop 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 shutdown_connections 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563f8fdb2fa0 0x563f8fdb53c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563f8fd40ca0 0x563f8fd4fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563f8fd45e10 0x563f8fd503e0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 --2- 172.21.15.137:0/3329083713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563f8fd402b0 0x563f8fd51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe8dbe00640 0 client.17034 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 >> 172.21.15.137:0/3329083713 conn(0x563f8fd3b820 msgr2=0x563f8fd3f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 shutdown_connections 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 1 -- 172.21.15.137:0/3329083713 wait complete. 2024-03-26T14:08:41.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.380+0000 7fe903bac740 20 client.17034 trim_cache size 0 max 16384 2024-03-26T14:08:41.381 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:08:41.381 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:08:41.381 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "e30d16da-7ff1-4bb3-979c-ffe0294425c4", 2024-03-26T14:08:41.381 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:08:41.381 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8e00a2800 msgr2=0x7fe8e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec005fd0 tx=0x7fe8ec005c50 comp rx=0 tx=0).stop 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 msgr2=0x7fe8fc199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc199ee0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006010 tx=0x7fe8e80042f0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 shutdown_connections 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe8e00a2800 0x7fe8e00a4cc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8fc105350 0x7fe8fc1a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8fc104990 0x7fe8fc199ee0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 --2- 172.21.15.137:0/1824788032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8fc1037a0 0x7fe8fc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.381+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 >> 172.21.15.137:0/1824788032 conn(0x7fe8fc0fed10 msgr2=0x7fe8fc107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.382+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 shutdown_connections 2024-03-26T14:08:41.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.382+0000 7fe8fb200640 1 -- 172.21.15.137:0/1824788032 wait complete. 2024-03-26T14:08:41.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:08:41.399 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:08:41.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:40 smithi179 ceph-mon[35472]: pgmap v647: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:41.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:40 smithi159 ceph-mon[35501]: pgmap v647: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 -- 172.21.15.137:0/95418744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 msgr2=0x7f96e8073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 --2- 172.21.15.137:0/95418744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e8073b30 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009930 tx=0x7f96d802bc20 comp rx=0 tx=0).stop 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 -- 172.21.15.137:0/95418744 shutdown_connections 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 --2- 172.21.15.137:0/95418744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 0x7f96e8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 --2- 172.21.15.137:0/95418744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 0x7f96e810ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 --2- 172.21.15.137:0/95418744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e8073b30 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 -- 172.21.15.137:0/95418744 >> 172.21.15.137:0/95418744 conn(0x7f96e80fcb10 msgr2=0x7f96e80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 -- 172.21.15.137:0/95418744 shutdown_connections 2024-03-26T14:08:41.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.620+0000 7f96eca00640 1 -- 172.21.15.137:0/95418744 wait complete. 2024-03-26T14:08:41.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.621+0000 7f96eca00640 1 Processor -- start 2024-03-26T14:08:41.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 -- start start 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 0x7f96e819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 0x7f96e81a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8113710 con 0x7f96e80756e0 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8113590 con 0x7f96e8074100 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96eca00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8113890 con 0x7f96e810db70 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50596/0 (socket says 172.21.15.137:50596) 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96dde00640 1 -- 172.21.15.137:0/2202327776 learned_addr learned my addr 172.21.15.137:0/2202327776 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.622+0000 7f96e6800640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 0x7f96e819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96e7200640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 0x7f96e81a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dde00640 1 -- 172.21.15.137:0/2202327776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 msgr2=0x7f96e81a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dde00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 0x7f96e81a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dde00640 1 -- 172.21.15.137:0/2202327776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 msgr2=0x7f96e819de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dde00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 0x7f96e819de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dde00640 1 -- 172.21.15.137:0/2202327776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e81a5a10 con 0x7f96e80756e0 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96e7200640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 0x7f96e81a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:08:41.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96e6800640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 0x7f96e819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:08:41.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dde00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e819e380 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f96d000e9f0 tx=0x7f96d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dfe00640 1 -- 172.21.15.137:0/2202327776 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000cd80 con 0x7f96e80756e0 2024-03-26T14:08:41.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96dfe00640 1 -- 172.21.15.137:0/2202327776 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96d00108e0 con 0x7f96e80756e0 2024-03-26T14:08:41.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.623+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e81a5d00 con 0x7f96e80756e0 2024-03-26T14:08:41.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.624+0000 7f96dfe00640 1 -- 172.21.15.137:0/2202327776 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0018920 con 0x7f96e80756e0 2024-03-26T14:08:41.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.624+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e81a61f0 con 0x7f96e80756e0 2024-03-26T14:08:41.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 Processor -- start 2024-03-26T14:08:41.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96dfe00640 1 -- 172.21.15.137:0/2202327776 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f96d0010a90 con 0x7f96e80756e0 2024-03-26T14:08:41.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 -- start start 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9293470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b9298540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b929b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b91f8250 con 0x55f2b9298bb0 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b91a0a30 con 0x55f2b9293a40 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96f01f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b91f42f0 con 0x55f2b9293050 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.628+0000 7f96dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b929b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96e6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b9298540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b929b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50604/0 (socket says 172.21.15.137:50604) 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 -- 172.21.15.137:0/2538725862 learned_addr learned my addr 172.21.15.137:0/2538725862 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96e7200640 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9293470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 -- 172.21.15.137:0/2538725862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 msgr2=0x55f2b9293470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9293470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 -- 172.21.15.137:0/2538725862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 msgr2=0x55f2b9298540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b9298540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dde00640 1 -- 172.21.15.137:0/2538725862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b918b690 con 0x55f2b9298bb0 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96e6800640 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b9298540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.629+0000 7f96dfe00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f96cc0a2800 0x7f96cc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.630+0000 7f96dfe00640 1 -- 172.21.15.137:0/2202327776 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f96d00cf280 con 0x7f96e80756e0 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.630+0000 7f96dde00640 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b929b040 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f96d00d74b0 tx=0x7f96d00d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f82e1fc157af10f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.630+0000 7f96e7200640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f96cc0a2800 0x7f96cc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.630+0000 7f96d7400640 1 -- 172.21.15.137:0/2538725862 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0097af0 con 0x55f2b9298bb0 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.630+0000 7f96d7400640 1 -- 172.21.15.137:0/2538725862 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96d0010de0 con 0x55f2b9298bb0 2024-03-26T14:08:41.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.630+0000 7f96d7400640 1 -- 172.21.15.137:0/2538725862 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d009b1b0 con 0x55f2b9298bb0 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 -- 172.21.15.137:0/2538725862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 msgr2=0x55f2b929b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96e7200640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f96cc0a2800 0x7f96cc0a4cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f2b92942a0 tx=0x7f96e0009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b929b040 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f96d00d74b0 tx=0x7f96d00d2f90 comp rx=0 tx=0).stop 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 -- 172.21.15.137:0/2538725862 shutdown_connections 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b9298540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b929b040 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2538725862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9293470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 -- 172.21.15.137:0/2538725862 >> 172.21.15.137:0/2538725862 conn(0x55f2b925b200 msgr2=0x55f2b928e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 -- 172.21.15.137:0/2538725862 shutdown_connections 2024-03-26T14:08:41.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.631+0000 7f96f01f7740 1 -- 172.21.15.137:0/2538725862 wait complete. 2024-03-26T14:08:41.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 -- start start 2024-03-26T14:08:41.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 Processor -- start 2024-03-26T14:08:41.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b929d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b929d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b929bb60 con 0x55f2b9298bb0 2024-03-26T14:08:41.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b91b3e80 con 0x55f2b9293a40 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2b929bce0 con 0x55f2b9293050 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.632+0000 7f96f01f7740 10 client.0 _ll_register_callbacks cb 0x55f2b91c6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50610/0 (socket says 172.21.15.137:50610) 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 -- 172.21.15.137:0/1882335743 learned_addr learned my addr 172.21.15.137:0/1882335743 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96dde00640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b929d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e6800640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b929d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 -- 172.21.15.137:0/1882335743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 msgr2=0x55f2b929d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b929d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 -- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 msgr2=0x55f2b929d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b929d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 -- 172.21.15.137:0/1882335743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2b8fd67d0 con 0x55f2b9298bb0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96e7200640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a4510 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f96e0010940 tx=0x7f96e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817eb364446e5747 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.633+0000 7f96cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.634+0000 7f96cbe00640 1 -- 172.21.15.137:0/1882335743 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000fd50 con 0x55f2b9298bb0 2024-03-26T14:08:41.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.634+0000 7f96cbe00640 1 -- 172.21.15.137:0/1882335743 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f96e0027400 con 0x55f2b9298bb0 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.634+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2b9205790 con 0x55f2b9298bb0 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.634+0000 7f96cbe00640 1 -- 172.21.15.137:0/1882335743 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e0027600 con 0x55f2b9298bb0 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.634+0000 7f96f01f7740 10 client.17046 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.634+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f2b8f0c3f0 con 0x55f2b9298bb0 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96cbe00640 1 -- 172.21.15.137:0/1882335743 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f2b9205790 con 0x55f2b9298bb0 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96cbe00640 1 -- 172.21.15.137:0/1882335743 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f96e001f720 con 0x55f2b9298bb0 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96f01f7740 10 client.17046 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96f01f7740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96f01f7740 20 client.17046 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96f01f7740 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b92a8ed0 0x55f2b92ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96f01f7740 4 client.17046 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:08:41.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.635+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2b91ffae0 con 0x55f2b92a8ed0 2024-03-26T14:08:41.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.636+0000 7f96e6800640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b92a8ed0 0x55f2b92ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.636+0000 7f96cbe00640 10 client.17046 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.636+0000 7f96e6800640 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b92a8ed0 0x55f2b92ab2e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac53c90eb02747f server_cookie=9703a9aa86e4b7e in_seq=0 out_seq=0 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.637+0000 7f96cbe00640 1 -- 172.21.15.137:0/1882335743 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f2b91ffae0 con 0x55f2b92a8ed0 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.637+0000 7f96cbe00640 10 client.17046 handle_command_reply: tid=0 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 client.17046 shutdown 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 msgr2=0x55f2b92a4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a4510 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f96e0010940 tx=0x7f96e0004a40 comp rx=0 tx=0).stop 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 shutdown_connections 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b92a8ed0 0x55f2b92ab2e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293a40 0x55f2b929d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a4510 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 --2- 172.21.15.137:0/1882335743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b929d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 >> 172.21.15.137:0/1882335743 conn(0x55f2b925b200 msgr2=0x55f2b928dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96cbe00640 0 client.17046 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 shutdown_connections 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 1 -- 172.21.15.137:0/1882335743 wait complete. 2024-03-26T14:08:41.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.638+0000 7f96f01f7740 20 client.17046 trim_cache size 0 max 16384 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.661+0000 7f96f01f7740 1 Processor -- start 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 -- start start 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9292310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 0x55f2b92a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9298bb0 0x55f2b928d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81b2be0 con 0x55f2b9293a40 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81b06e0 con 0x55f2b9298bb0 2024-03-26T14:08:41.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96f01f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81ae760 con 0x55f2b9293050 2024-03-26T14:08:41.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 0x55f2b92a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 0x55f2b92a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50614/0 (socket says 172.21.15.137:50614) 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96dde00640 1 -- 172.21.15.137:0/2323558489 learned_addr learned my addr 172.21.15.137:0/2323558489 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96e7200640 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9298bb0 0x55f2b928d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.662+0000 7f96e6800640 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9292310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96dde00640 1 -- 172.21.15.137:0/2323558489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 msgr2=0x55f2b9292310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96dde00640 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9292310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96dde00640 1 -- 172.21.15.137:0/2323558489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9298bb0 msgr2=0x55f2b928d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96dde00640 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9298bb0 0x55f2b928d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96dde00640 1 -- 172.21.15.137:0/2323558489 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2b8ecc0f0 con 0x55f2b9293a40 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96dde00640 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 0x55f2b92a44d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f96d000c6a0 tx=0x7f96d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33c4afba9cf65f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96d6a00640 1 -- 172.21.15.137:0/2323558489 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d009b390 con 0x55f2b9293a40 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96d6a00640 1 -- 172.21.15.137:0/2323558489 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96d00a3ff0 con 0x55f2b9293a40 2024-03-26T14:08:41.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.663+0000 7f96d6a00640 1 -- 172.21.15.137:0/2323558489 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d009b620 con 0x55f2b9293a40 2024-03-26T14:08:41.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 -- 172.21.15.137:0/2323558489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 msgr2=0x55f2b92a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 0x55f2b92a44d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f96d000c6a0 tx=0x7f96d0000bc0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 -- 172.21.15.137:0/2323558489 shutdown_connections 2024-03-26T14:08:41.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9298bb0 0x55f2b928d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9293a40 0x55f2b92a44d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2323558489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293050 0x55f2b9292310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 -- 172.21.15.137:0/2323558489 >> 172.21.15.137:0/2323558489 conn(0x55f2b925b200 msgr2=0x55f2b9291c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 -- 172.21.15.137:0/2323558489 shutdown_connections 2024-03-26T14:08:41.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.664+0000 7f96f01f7740 1 -- 172.21.15.137:0/2323558489 wait complete. 2024-03-26T14:08:41.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.665+0000 7f96f01f7740 1 -- start start 2024-03-26T14:08:41.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.665+0000 7f96f01f7740 1 Processor -- start 2024-03-26T14:08:41.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.665+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293050 0x55f2b92a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293a40 0x55f2b92a2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96f01f7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96f01f7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81aed60 con 0x55f2b9298bb0 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96f01f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81b0080 con 0x55f2b9293050 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96f01f7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e81af3c0 con 0x55f2b9293a40 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96f01f7740 10 client.0 _ll_register_callbacks cb 0x7f96e0009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50622/0 (socket says 172.21.15.137:50622) 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 -- 172.21.15.137:0/2547382985 learned_addr learned my addr 172.21.15.137:0/2547382985 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e6800640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293050 0x55f2b92a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96dde00640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293a40 0x55f2b92a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 -- 172.21.15.137:0/2547382985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293a40 msgr2=0x55f2b92a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293a40 0x55f2b92a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 -- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293050 msgr2=0x55f2b92a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:08:41.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293050 0x55f2b92a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.666+0000 7f96e7200640 1 -- 172.21.15.137:0/2547382985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2b92a38c0 con 0x55f2b9298bb0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96e7200640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a32d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f96e0012fd0 tx=0x7f96e0004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f289f25a32e7e6ef server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96cbe00640 1 -- 172.21.15.137:0/2547382985 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0010520 con 0x55f2b9298bb0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96d00a3ff0 con 0x55f2b9298bb0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96cbe00640 1 -- 172.21.15.137:0/2547382985 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f96e001f600 con 0x55f2b9298bb0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96cbe00640 1 -- 172.21.15.137:0/2547382985 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e0046050 con 0x55f2b9298bb0 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96f01f7740 10 client.17052 fetch_fsmap learned FSMap version 24 2024-03-26T14:08:41.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.667+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f2b93048a0 con 0x55f2b9298bb0 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96cbe00640 1 -- 172.21.15.137:0/2547382985 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f96d00a3ff0 con 0x55f2b9298bb0 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96cbe00640 1 -- 172.21.15.137:0/2547382985 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f96e0034450 con 0x55f2b9298bb0 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96f01f7740 10 client.17052 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96f01f7740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96f01f7740 20 client.17052 populate_metadata read hostname 'smithi137' 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b9305220 0x55f2b9307640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96f01f7740 4 client.17052 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.668+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f2b917d3a0 con 0x55f2b9305220 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.669+0000 7f96e6800640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b9305220 0x55f2b9307640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96cbe00640 10 client.17052 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96e6800640 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b9305220 0x55f2b9307640 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea8b286901fc5a9 server_cookie=ad94d951583e454d in_seq=0 out_seq=0 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96cbe00640 1 -- 172.21.15.137:0/2547382985 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f2b917d3a0 con 0x55f2b9305220 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96cbe00640 10 client.17052 handle_command_reply: tid=0 2024-03-26T14:08:41.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 client.17052 shutdown 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 msgr2=0x55f2b92a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a32d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f96e0012fd0 tx=0x7f96e0004d40 comp rx=0 tx=0).stop 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 shutdown_connections 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f2b9305220 0x55f2b9307640 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f2b9298bb0 0x55f2b92a32d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f2b9293a40 0x55f2b92a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 --2- 172.21.15.137:0/2547382985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f2b9293050 0x55f2b92a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96cbe00640 0 client.17052 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 >> 172.21.15.137:0/2547382985 conn(0x55f2b925b200 msgr2=0x55f2b9294390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 shutdown_connections 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.670+0000 7f96f01f7740 1 -- 172.21.15.137:0/2547382985 wait complete. 2024-03-26T14:08:41.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.671+0000 7f96f01f7740 20 client.17052 trim_cache size 0 max 16384 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "e30d16da-7ff1-4bb3-979c-ffe0294425c4": { 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "e30d16da-7ff1-4bb3-979c-ffe0294425c4", 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:08:41.672 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.671+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f96cc0a2800 msgr2=0x7f96cc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.671+0000 7f96eca00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f96cc0a2800 0x7f96cc0a4cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f2b92942a0 tx=0x7f96e0009700 comp rx=0 tx=0).stop 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.671+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 msgr2=0x7f96e819e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.671+0000 7f96eca00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e819e380 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f96d000e9f0 tx=0x7f96d000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.671+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 shutdown_connections 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f96cc0a2800 0x7f96cc0a4cc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f96e810db70 0x7f96e81a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f96e80756e0 0x7f96e819e380 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 --2- 172.21.15.137:0/2202327776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f96e8074100 0x7f96e819de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 >> 172.21.15.137:0/2202327776 conn(0x7f96e80fcb10 msgr2=0x7f96e8100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 shutdown_connections 2024-03-26T14:08:41.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:08:41.672+0000 7f96eca00640 1 -- 172.21.15.137:0/2202327776 wait complete. 2024-03-26T14:08:41.689 INFO:tasks.cephfs.filesystem:scrub status for tag:e30d16da-7ff1-4bb3-979c-ffe0294425c4 - {'path': '/', 'tag': 'e30d16da-7ff1-4bb3-979c-ffe0294425c4', 'options': 'recursive,force'} 2024-03-26T14:08:43.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:42 smithi137 ceph-mon[31539]: pgmap v648: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:43.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:42 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:08:43.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:42 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:43.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:42 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:08:43.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:42 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:43.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:42 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:08:43.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:42 smithi179 ceph-mon[35472]: pgmap v648: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:43.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:42 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:08:43.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:42 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:43.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:42 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:08:43.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:42 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:43.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:42 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:08:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:42 smithi159 ceph-mon[35501]: pgmap v648: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:08:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:42 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:08:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:42 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:42 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:08:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:42 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:08:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:42 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:08:45.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:44 smithi137 ceph-mon[31539]: pgmap v649: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:45.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:44 smithi179 ceph-mon[35472]: pgmap v649: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:45.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:44 smithi159 ceph-mon[35501]: pgmap v649: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:08:47.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:46 smithi137 ceph-mon[31539]: pgmap v650: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:47.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:46 smithi179 ceph-mon[35472]: pgmap v650: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:47.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:46 smithi159 ceph-mon[35501]: pgmap v650: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:08:49.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:48 smithi137 ceph-mon[31539]: pgmap v651: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:49.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:48 smithi179 ceph-mon[35472]: pgmap v651: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:49.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:48 smithi159 ceph-mon[35501]: pgmap v651: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:51.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:50 smithi137 ceph-mon[31539]: pgmap v652: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:51.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:51 smithi179 ceph-mon[35472]: pgmap v652: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:51.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:51 smithi159 ceph-mon[35501]: pgmap v652: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:53.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:53 smithi137 ceph-mon[31539]: pgmap v653: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:53.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:53 smithi179 ceph-mon[35472]: pgmap v653: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:53.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:53 smithi159 ceph-mon[35501]: pgmap v653: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:08:55.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:55 smithi137 ceph-mon[31539]: pgmap v654: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:08:55.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:55 smithi179 ceph-mon[35472]: pgmap v654: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:08:55.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:55 smithi159 ceph-mon[35501]: pgmap v654: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:08:57.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:57 smithi137 ceph-mon[31539]: pgmap v655: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:57.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:57 smithi179 ceph-mon[35472]: pgmap v655: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:57.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:57 smithi159 ceph-mon[35501]: pgmap v655: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:08:59.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:08:59 smithi137 ceph-mon[31539]: pgmap v656: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:59.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:08:59 smithi179 ceph-mon[35472]: pgmap v656: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:08:59.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:08:59 smithi159 ceph-mon[35501]: pgmap v656: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:09:01.284 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:01 smithi137 ceph-mon[31539]: pgmap v657: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:01.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:01 smithi179 ceph-mon[35472]: pgmap v657: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:01.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:01 smithi159 ceph-mon[35501]: pgmap v657: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:03.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:03 smithi179 ceph-mon[35472]: pgmap v658: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:03.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:03 smithi159 ceph-mon[35501]: pgmap v658: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:03.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:03 smithi137 ceph-mon[31539]: pgmap v658: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:05.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:05 smithi179 ceph-mon[35472]: pgmap v659: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:05.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:05 smithi159 ceph-mon[35501]: pgmap v659: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:05.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:05 smithi137 ceph-mon[31539]: pgmap v659: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:07.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:07 smithi179 ceph-mon[35472]: pgmap v660: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:07.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:07 smithi159 ceph-mon[35501]: pgmap v660: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:07.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:07 smithi137 ceph-mon[31539]: pgmap v660: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:09.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:09 smithi179 ceph-mon[35472]: pgmap v661: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:09:09.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:09 smithi159 ceph-mon[35501]: pgmap v661: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:09:09.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:09 smithi137 ceph-mon[31539]: pgmap v661: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:09:11.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:11 smithi179 ceph-mon[35472]: pgmap v662: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:11.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:11 smithi159 ceph-mon[35501]: pgmap v662: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:11.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:11 smithi137 ceph-mon[31539]: pgmap v662: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:11.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:09:11.690 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 -- 172.21.15.137:0/46738661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c81038c0 msgr2=0x7f61c8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 --2- 172.21.15.137:0/46738661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c81038c0 0x7f61c8103ce0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009900 tx=0x7f61b802bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 -- 172.21.15.137:0/46738661 shutdown_connections 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 --2- 172.21.15.137:0/46738661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61c8104ab0 0x7f61c8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 --2- 172.21.15.137:0/46738661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 0x7f61c810bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 --2- 172.21.15.137:0/46738661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c81038c0 0x7f61c8103ce0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 -- 172.21.15.137:0/46738661 >> 172.21.15.137:0/46738661 conn(0x7f61c80fee50 msgr2=0x7f61c8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 -- 172.21.15.137:0/46738661 shutdown_connections 2024-03-26T14:09:11.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.935+0000 7f61cd000640 1 -- 172.21.15.137:0/46738661 wait complete. 2024-03-26T14:09:11.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.936+0000 7f61cd000640 1 Processor -- start 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.936+0000 7f61cd000640 1 -- start start 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.936+0000 7f61cd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61c81038c0 0x7f61c8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.936+0000 7f61cd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 0x7f61c819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61cd000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 0x7f61c81a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61cd000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c810f3f0 con 0x7f61c8104ab0 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61cd000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c810f270 con 0x7f61c81038c0 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61cd000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c810f570 con 0x7f61c8105470 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 0x7f61c819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 0x7f61c819a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39728/0 (socket says 172.21.15.137:39728) 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 -- 172.21.15.137:0/961677995 learned_addr learned my addr 172.21.15.137:0/961677995 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5e00640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61c81038c0 0x7f61c8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c6800640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 0x7f61c81a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 -- 172.21.15.137:0/961677995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 msgr2=0x7f61c81a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 0x7f61c81a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 -- 172.21.15.137:0/961677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61c81038c0 msgr2=0x7f61c8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61c81038c0 0x7f61c8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c5400640 1 -- 172.21.15.137:0/961677995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c81a1710 con 0x7f61c8104ab0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.937+0000 7f61c6800640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 0x7f61c81a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.938+0000 7f61c5400640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 0x7f61c819a030 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e9f0 tx=0x7f61b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.938+0000 7f61bea00640 1 -- 172.21.15.137:0/961677995 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b400cd80 con 0x7f61c8104ab0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.938+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c81a1a00 con 0x7f61c8104ab0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.938+0000 7f61bea00640 1 -- 172.21.15.137:0/961677995 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61b40108e0 con 0x7f61c8104ab0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.938+0000 7f61bea00640 1 -- 172.21.15.137:0/961677995 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4018920 con 0x7f61c8104ab0 2024-03-26T14:09:11.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.938+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c81a1f50 con 0x7f61c8104ab0 2024-03-26T14:09:11.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 Processor -- start 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61bea00640 1 -- 172.21.15.137:0/961677995 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f61b4010a90 con 0x7f61c8104ab0 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 -- start start 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x5611728602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x561172865370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172867e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611727c5250 con 0x56117285fe80 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56117276da30 con 0x5611728659e0 2024-03-26T14:09:11.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61cfda3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611727c12f0 con 0x561172860870 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.942+0000 7f61c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x5611728602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x5611728602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39742/0 (socket says 172.21.15.137:39742) 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 -- 172.21.15.137:0/3653244436 learned_addr learned my addr 172.21.15.137:0/3653244436 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c5e00640 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x561172865370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c5400640 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172867e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 -- 172.21.15.137:0/3653244436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 msgr2=0x561172865370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x561172865370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 -- 172.21.15.137:0/3653244436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 msgr2=0x561172867e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172867e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61c6800640 1 -- 172.21.15.137:0/3653244436 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561172758690 con 0x56117285fe80 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.943+0000 7f61bea00640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61ac0a2800 0x7f61ac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.944+0000 7f61c6800640 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x5611728602a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009ea0 tx=0x7f61c0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16106ecac9a6d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:11.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.944+0000 7f61bea00640 1 -- 172.21.15.137:0/961677995 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61b40cf1d0 con 0x7f61c8104ab0 2024-03-26T14:09:11.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.944+0000 7f61c5e00640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61ac0a2800 0x7f61ac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.944+0000 7f61b3400640 1 -- 172.21.15.137:0/3653244436 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0004d20 con 0x56117285fe80 2024-03-26T14:09:11.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.944+0000 7f61b3400640 1 -- 172.21.15.137:0/3653244436 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61c00054d0 con 0x56117285fe80 2024-03-26T14:09:11.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.944+0000 7f61b3400640 1 -- 172.21.15.137:0/3653244436 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00118a0 con 0x56117285fe80 2024-03-26T14:09:11.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61c5e00640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61ac0a2800 0x7f61ac0a4cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005ab0 tx=0x7f61b8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 -- 172.21.15.137:0/3653244436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 msgr2=0x5611728602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x5611728602a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f61c0009ea0 tx=0x7f61c0009f90 comp rx=0 tx=0).stop 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 -- 172.21.15.137:0/3653244436 shutdown_connections 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172867e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x561172865370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 --2- 172.21.15.137:0/3653244436 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x5611728602a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 -- 172.21.15.137:0/3653244436 >> 172.21.15.137:0/3653244436 conn(0x561172828200 msgr2=0x56117285b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 -- 172.21.15.137:0/3653244436 shutdown_connections 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.945+0000 7f61cfda3740 1 -- 172.21.15.137:0/3653244436 wait complete. 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 -- start start 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 Processor -- start 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x561172869e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172871410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172868990 con 0x56117285fe80 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172780e80 con 0x5611728659e0 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561172868b10 con 0x561172860870 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.946+0000 7f61cfda3740 10 client.0 _ll_register_callbacks cb 0x561172793130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x561172869e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x561172869e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39756/0 (socket says 172.21.15.137:39756) 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 -- 172.21.15.137:0/2261235635 learned_addr learned my addr 172.21.15.137:0/2261235635 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c6800640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172871410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5400640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 -- 172.21.15.137:0/2261235635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 msgr2=0x56117286a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 -- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 msgr2=0x561172871410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172871410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.947+0000 7f61c5e00640 1 -- 172.21.15.137:0/2261235635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611725a37d0 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61c5e00640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x561172869e90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009230 tx=0x7f61b802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474d389fa04d4c57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61a7e00640 1 -- 172.21.15.137:0/2261235635 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8051b50 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61a7e00640 1 -- 172.21.15.137:0/2261235635 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61b8051cf0 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61c00054d0 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61a7e00640 1 -- 172.21.15.137:0/2261235635 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b8005040 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61cfda3740 10 client.17064 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.948+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5611724d93f0 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61a7e00640 1 -- 172.21.15.137:0/2261235635 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61c00054d0 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61a7e00640 1 -- 172.21.15.137:0/2261235635 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f61b8004c60 con 0x56117285fe80 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61cfda3740 10 client.17064 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61cfda3740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:11.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61cfda3740 20 client.17064 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:11.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561172875d20 0x561172878190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61cfda3740 4 client.17064 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:11.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611727d2790 con 0x561172875d20 2024-03-26T14:09:11.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.949+0000 7f61c5400640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561172875d20 0x561172878190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.950+0000 7f61a7e00640 10 client.17064 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.950+0000 7f61c5400640 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561172875d20 0x561172878190 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a754ea1cd25afa91 server_cookie=38ca7b360794fd29 in_seq=0 out_seq=0 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61a7e00640 1 -- 172.21.15.137:0/2261235635 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5611727d2790 con 0x561172875d20 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61a7e00640 10 client.17064 handle_command_reply: tid=0 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 client.17064 shutdown 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 msgr2=0x561172869e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x561172869e90 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009230 tx=0x7f61b802bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 shutdown_connections 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x561172875d20 0x561172878190 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611728659e0 0x561172871410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2261235635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56117285fe80 0x561172869e90 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61a7e00640 0 client.17064 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 >> 172.21.15.137:0/2261235635 conn(0x561172828200 msgr2=0x56117285aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 shutdown_connections 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.951+0000 7f61cfda3740 1 -- 172.21.15.137:0/2261235635 wait complete. 2024-03-26T14:09:11.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.952+0000 7f61cfda3740 20 client.17064 trim_cache size 0 max 16384 2024-03-26T14:09:11.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.970+0000 7f61cfda3740 1 Processor -- start 2024-03-26T14:09:11.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.970+0000 7f61cfda3740 1 -- start start 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x5611728713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x56117285a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61cfda3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81aa560 con 0x5611728659e0 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61cfda3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a8060 con 0x56117285fe80 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61cfda3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a60e0 con 0x561172860870 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x56117285a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x56117285a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39764/0 (socket says 172.21.15.137:39764) 2024-03-26T14:09:11.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 -- 172.21.15.137:0/2119592844 learned_addr learned my addr 172.21.15.137:0/2119592844 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c5e00640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c5400640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x5611728713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 -- 172.21.15.137:0/2119592844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 msgr2=0x5611728713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x5611728713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 -- 172.21.15.137:0/2119592844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 msgr2=0x56117286a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c6800640 1 -- 172.21.15.137:0/2119592844 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611724990f0 con 0x5611728659e0 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.972+0000 7f61c5e00640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.971+0000 7f61c5400640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x5611728713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:11.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.972+0000 7f61c6800640 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x56117285a910 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006010 tx=0x7f61c0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7538839a4dafdaa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.972+0000 7f61b2a00640 1 -- 172.21.15.137:0/2119592844 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0004d60 con 0x5611728659e0 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.972+0000 7f61b2a00640 1 -- 172.21.15.137:0/2119592844 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61c00054f0 con 0x5611728659e0 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.972+0000 7f61b2a00640 1 -- 172.21.15.137:0/2119592844 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00119d0 con 0x5611728659e0 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 -- 172.21.15.137:0/2119592844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 msgr2=0x56117285a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x56117285a910 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006010 tx=0x7f61c0005c80 comp rx=0 tx=0).stop 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 -- 172.21.15.137:0/2119592844 shutdown_connections 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x56117285a910 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x5611728713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 --2- 172.21.15.137:0/2119592844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 -- 172.21.15.137:0/2119592844 >> 172.21.15.137:0/2119592844 conn(0x561172828200 msgr2=0x56117285a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 -- 172.21.15.137:0/2119592844 shutdown_connections 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.973+0000 7f61cfda3740 1 -- 172.21.15.137:0/2119592844 wait complete. 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 -- start start 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 Processor -- start 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x5611728703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a66e0 con 0x5611728659e0 2024-03-26T14:09:11.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a7a00 con 0x56117285fe80 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c81a6d40 con 0x561172860870 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.974+0000 7f61cfda3740 10 client.0 _ll_register_callbacks cb 0x7f61b8043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x5611728703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x5611728703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39770/0 (socket says 172.21.15.137:39770) 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 -- 172.21.15.137:0/1172721147 learned_addr learned my addr 172.21.15.137:0/1172721147 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c5e00640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c5400640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 -- 172.21.15.137:0/1172721147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 msgr2=0x56117286f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 -- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 msgr2=0x56117286ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.975+0000 7f61c6800640 1 -- 172.21.15.137:0/1172721147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611728709d0 con 0x5611728659e0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61c6800640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x5611728703e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f61c000fbc0 tx=0x7f61c0005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9328714e3d1d88 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61a7e00640 1 -- 172.21.15.137:0/1172721147 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0031020 con 0x5611728659e0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61a7e00640 1 -- 172.21.15.137:0/1172721147 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f61c0011ae0 con 0x5611728659e0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61c00054f0 con 0x5611728659e0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61a7e00640 1 -- 172.21.15.137:0/1172721147 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61c003c050 con 0x5611728659e0 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61cfda3740 10 client.17070 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:11.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.976+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5611728d18b0 con 0x5611728659e0 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61a7e00640 1 -- 172.21.15.137:0/1172721147 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f61c00054f0 con 0x5611728659e0 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61a7e00640 1 -- 172.21.15.137:0/1172721147 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f61c002d510 con 0x5611728659e0 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61cfda3740 10 client.17070 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61cfda3740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61cfda3740 20 client.17070 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61cfda3740 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5611728d1fb0 0x5611728d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61cfda3740 4 client.17070 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:09:11.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.977+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5611727ccae0 con 0x5611728d1fb0 2024-03-26T14:09:11.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.978+0000 7f61c5e00640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5611728d1fb0 0x5611728d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:11.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.978+0000 7f61c5e00640 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5611728d1fb0 0x5611728d43d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f37b0cbc593cd1 server_cookie=4da5c032c344bc57 in_seq=0 out_seq=0 2024-03-26T14:09:11.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.978+0000 7f61a7e00640 10 client.17070 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61a7e00640 1 -- 172.21.15.137:0/1172721147 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5611727ccae0 con 0x5611728d1fb0 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61a7e00640 10 client.17070 handle_command_reply: tid=0 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 client.17070 shutdown 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 msgr2=0x5611728703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x5611728703e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f61c000fbc0 tx=0x7f61c0005950 comp rx=0 tx=0).stop 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 shutdown_connections 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5611728d1fb0 0x5611728d43d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5611728659e0 0x5611728703e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561172860870 0x56117286f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 --2- 172.21.15.137:0/1172721147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117285fe80 0x56117286ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 >> 172.21.15.137:0/1172721147 conn(0x561172828200 msgr2=0x56117286a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:11.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61a7e00640 0 client.17070 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:11.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 shutdown_connections 2024-03-26T14:09:11.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.979+0000 7f61cfda3740 1 -- 172.21.15.137:0/1172721147 wait complete. 2024-03-26T14:09:11.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cfda3740 20 client.17070 trim_cache size 0 max 16384 2024-03-26T14:09:11.987 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:09:11.987 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:09:11.988 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:09:11.988 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:09:11.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61ac0a2800 msgr2=0x7f61ac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61ac0a2800 0x7f61ac0a4cc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61b8005ab0 tx=0x7f61b8005fd0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 msgr2=0x7f61c819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 0x7f61c819a030 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f61b400e9f0 tx=0x7f61b400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 shutdown_connections 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f61ac0a2800 0x7f61ac0a4cc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f61c8105470 0x7f61c81a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f61c8104ab0 0x7f61c819a030 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 --2- 172.21.15.137:0/961677995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61c81038c0 0x7f61c8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:11.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.980+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 >> 172.21.15.137:0/961677995 conn(0x7f61c80fee50 msgr2=0x7f61c8107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:11.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.981+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 shutdown_connections 2024-03-26T14:09:11.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:11.981+0000 7f61cd000640 1 -- 172.21.15.137:0/961677995 wait complete. 2024-03-26T14:09:11.997 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:09:11.997 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 -- 172.21.15.137:0/1427721286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 msgr2=0x7f80f8104d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 --2- 172.21.15.137:0/1427721286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8104d60 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009900 tx=0x7f80e402bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 -- 172.21.15.137:0/1427721286 shutdown_connections 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 --2- 172.21.15.137:0/1427721286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80f81052a0 0x7f80f810bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 --2- 172.21.15.137:0/1427721286 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8104d60 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 --2- 172.21.15.137:0/1427721286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80f81036f0 0x7f80f8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 -- 172.21.15.137:0/1427721286 >> 172.21.15.137:0/1427721286 conn(0x7f80f80fec80 msgr2=0x7f80f81010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 -- 172.21.15.137:0/1427721286 shutdown_connections 2024-03-26T14:09:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.222+0000 7f80fea00640 1 -- 172.21.15.137:0/1427721286 wait complete. 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.223+0000 7f80fea00640 1 Processor -- start 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.223+0000 7f80fea00640 1 -- start start 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.223+0000 7f80fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80f81036f0 0x7f80f8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.223+0000 7f80fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80f81052a0 0x7f80f81a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f810f270 con 0x7f80f81048e0 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f810f0f0 con 0x7f80f81036f0 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f810f3f0 con 0x7f80f81052a0 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39798/0 (socket says 172.21.15.137:39798) 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 -- 172.21.15.137:0/50465721 learned_addr learned my addr 172.21.15.137:0/50465721 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fd600640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80f81036f0 0x7f80f8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fe000640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80f81052a0 0x7f80f81a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 -- 172.21.15.137:0/50465721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80f81052a0 msgr2=0x7f80f81a0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80f81052a0 0x7f80f81a0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 -- 172.21.15.137:0/50465721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80f81036f0 msgr2=0x7f80f8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80f81036f0 0x7f80f8199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.224+0000 7f80fcc00640 1 -- 172.21.15.137:0/50465721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f81a1530 con 0x7f80f81048e0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.225+0000 7f80fcc00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8199e50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.225+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4043020 con 0x7f80f81048e0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.225+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f80e4004aa0 con 0x7f80f81048e0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.225+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f81a17c0 con 0x7f80f81048e0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.225+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003d00 con 0x7f80f81048e0 2024-03-26T14:09:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.225+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f81a1d10 con 0x7f80f81048e0 2024-03-26T14:09:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.227+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f810aa10 con 0x7f80f81048e0 2024-03-26T14:09:12.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.229+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f80e4046020 con 0x7f80f81048e0 2024-03-26T14:09:12.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.231+0000 7f80f6000640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80e00a2800 0x7f80e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.231+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f80e40f1300 con 0x7f80f81048e0 2024-03-26T14:09:12.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.231+0000 7f80fd600640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80e00a2800 0x7f80e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.232+0000 7f80fd600640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80e00a2800 0x7f80e00a4cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80ec005fd0 tx=0x7f80ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.236+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f80e404d020 con 0x7f80f81048e0 2024-03-26T14:09:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.490+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80f80622a0 con 0x7f80f81048e0 2024-03-26T14:09:12.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.493+0000 7f80f6000640 1 -- 172.21.15.137:0/50465721 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f80e40be0b0 con 0x7f80f81048e0 2024-03-26T14:09:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:09:12.493 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:09:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80e00a2800 msgr2=0x7f80e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80e00a2800 0x7f80e00a4cc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80ec005fd0 tx=0x7f80ec00b390 comp rx=0 tx=0).stop 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 msgr2=0x7f80f8199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8199e50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f80e4000c00 tx=0x7f80e40042f0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 shutdown_connections 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f80e00a2800 0x7f80e00a4cc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80f81052a0 0x7f80f81a0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f80f81048e0 0x7f80f8199e50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 --2- 172.21.15.137:0/50465721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80f81036f0 0x7f80f8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 >> 172.21.15.137:0/50465721 conn(0x7f80f80fec80 msgr2=0x7f80f8107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 shutdown_connections 2024-03-26T14:09:12.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.496+0000 7f80fea00640 1 -- 172.21.15.137:0/50465721 wait complete. 2024-03-26T14:09:12.516 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:09:12.516 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:09:12.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 -- 172.21.15.137:0/884696089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180102120 msgr2=0x7fa180104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 --2- 172.21.15.137:0/884696089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180102120 0x7fa180104510 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa170009900 tx=0x7fa17002bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:12.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 -- 172.21.15.137:0/884696089 shutdown_connections 2024-03-26T14:09:12.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 --2- 172.21.15.137:0/884696089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa180107380 0x7fa1801097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 --2- 172.21.15.137:0/884696089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180104a50 0x7fa180106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 --2- 172.21.15.137:0/884696089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180102120 0x7fa180104510 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 -- 172.21.15.137:0/884696089 >> 172.21.15.137:0/884696089 conn(0x7fa1800fbb30 msgr2=0x7fa1800fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 -- 172.21.15.137:0/884696089 shutdown_connections 2024-03-26T14:09:12.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.745+0000 7fa186000640 1 -- 172.21.15.137:0/884696089 wait complete. 2024-03-26T14:09:12.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.746+0000 7fa186000640 1 Processor -- start 2024-03-26T14:09:12.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.746+0000 7fa186000640 1 -- start start 2024-03-26T14:09:12.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa186000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa180102120 0x7fa180197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa186000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180104a50 0x7fa180197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa186000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 0x7fa18019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa186000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18010d230 con 0x7fa180107380 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa186000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18010d0b0 con 0x7fa180102120 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa186000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18010d3b0 con 0x7fa180104a50 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa17fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 0x7fa18019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa17fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 0x7fa18019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39822/0 (socket says 172.21.15.137:39822) 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa17f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa180102120 0x7fa180197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa17fe00640 1 -- 172.21.15.137:0/181018996 learned_addr learned my addr 172.21.15.137:0/181018996 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.747+0000 7fa17ea00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180104a50 0x7fa180197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa17fe00640 1 -- 172.21.15.137:0/181018996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180104a50 msgr2=0x7fa180197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa17fe00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180104a50 0x7fa180197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa17fe00640 1 -- 172.21.15.137:0/181018996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa180102120 msgr2=0x7fa180197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa17fe00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa180102120 0x7fa180197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa17fe00640 1 -- 172.21.15.137:0/181018996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18019f5a0 con 0x7fa180107380 2024-03-26T14:09:12.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa17fe00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 0x7fa18019ef30 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa178009fd0 tx=0x7fa17800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa177e00640 1 -- 172.21.15.137:0/181018996 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178017070 con 0x7fa180107380 2024-03-26T14:09:12.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18019f890 con 0x7fa180107380 2024-03-26T14:09:12.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18019fde0 con 0x7fa180107380 2024-03-26T14:09:12.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa177e00640 1 -- 172.21.15.137:0/181018996 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa178003e20 con 0x7fa180107380 2024-03-26T14:09:12.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.748+0000 7fa177e00640 1 -- 172.21.15.137:0/181018996 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178010460 con 0x7fa180107380 2024-03-26T14:09:12.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.752+0000 7fa177e00640 1 -- 172.21.15.137:0/181018996 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa178004020 con 0x7fa180107380 2024-03-26T14:09:12.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.752+0000 7fa188d6b740 1 Processor -- start 2024-03-26T14:09:12.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.752+0000 7fa188d6b740 1 -- start start 2024-03-26T14:09:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580414c20 0x560580415040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x560580416270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x56058041d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa188d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605803752c0 con 0x560580415df0 2024-03-26T14:09:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa188d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058031dc90 con 0x5605804167b0 2024-03-26T14:09:12.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa188d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560580371360 con 0x560580414c20 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa17f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x560580416270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa17f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x560580416270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39838/0 (socket says 172.21.15.137:39838) 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa17f400640 1 -- 172.21.15.137:0/2914703691 learned_addr learned my addr 172.21.15.137:0/2914703691 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.753+0000 7fa17ea00640 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x56058041d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17ea00640 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580414c20 0x560580415040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17f400640 1 -- 172.21.15.137:0/2914703691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580414c20 msgr2=0x560580415040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17f400640 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580414c20 0x560580415040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa177e00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1640a2880 0x7fa1640a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17f400640 1 -- 172.21.15.137:0/2914703691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 msgr2=0x56058041d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17f400640 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x56058041d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17f400640 1 -- 172.21.15.137:0/2914703691 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560580342080 con 0x560580415df0 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa177e00640 1 -- 172.21.15.137:0/181018996 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa1780ceae0 con 0x7fa180107380 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17fe00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1640a2880 0x7fa1640a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.754+0000 7fa17f400640 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x560580416270 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa170005ab0 tx=0x7fa1700095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0486b6703e782c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.755+0000 7fa16be00640 1 -- 172.21.15.137:0/2914703691 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170045020 con 0x560580415df0 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.755+0000 7fa16be00640 1 -- 172.21.15.137:0/2914703691 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa17002caa0 con 0x560580415df0 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.755+0000 7fa16be00640 1 -- 172.21.15.137:0/2914703691 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700046c0 con 0x560580415df0 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 -- 172.21.15.137:0/2914703691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 msgr2=0x560580416270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x560580416270 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa170005ab0 tx=0x7fa1700095e0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa17fe00640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1640a2880 0x7fa1640a4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa1780a3320 tx=0x7fa17800b910 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 -- 172.21.15.137:0/2914703691 shutdown_connections 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x56058041d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x560580416270 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2914703691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580414c20 0x560580415040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 -- 172.21.15.137:0/2914703691 >> 172.21.15.137:0/2914703691 conn(0x56058040bc00 msgr2=0x56058040c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 -- 172.21.15.137:0/2914703691 shutdown_connections 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 -- 172.21.15.137:0/2914703691 wait complete. 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.756+0000 7fa188d6b740 1 -- start start 2024-03-26T14:09:12.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 Processor -- start 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x56058041ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580415df0 0x56058041f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580426360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058041d760 con 0x560580414c20 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605803310d0 con 0x5605804167b0 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56058041d8e0 con 0x560580415df0 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa188d6b740 10 client.0 _ll_register_callbacks cb 0x560580335e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.757+0000 7fa17ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x56058041ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x56058041ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39850/0 (socket says 172.21.15.137:39850) 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 -- 172.21.15.137:0/444588680 learned_addr learned my addr 172.21.15.137:0/444588680 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580426360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.760 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17f400640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580415df0 0x56058041f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 -- 172.21.15.137:0/444588680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580415df0 msgr2=0x56058041f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580415df0 0x56058041f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 -- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 msgr2=0x560580426360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580426360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 -- 172.21.15.137:0/444588680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605800893f0 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa17ea00640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x56058041ede0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa16c004630 tx=0x7fa16c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d7ca1d28ce9ffb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa168c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.758+0000 7fa168c00640 1 -- 172.21.15.137:0/444588680 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c011e90 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa168c00640 1 -- 172.21.15.137:0/444588680 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa16c0140f0 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa17002caa0 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa168c00640 1 -- 172.21.15.137:0/444588680 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa16c028050 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa188d6b740 10 client.17088 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5605801537d0 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa168c00640 1 -- 172.21.15.137:0/444588680 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa17002caa0 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.759+0000 7fa168c00640 1 -- 172.21.15.137:0/444588680 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa16c0103d0 con 0x560580414c20 2024-03-26T14:09:12.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa188d6b740 10 client.17088 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa188d6b740 10 client.17088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:09:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa188d6b740 20 client.17088 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa188d6b740 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56058042a870 0x56058042ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa188d6b740 4 client.17088 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560580382800 con 0x56058042a870 2024-03-26T14:09:12.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.760+0000 7fa17f400640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56058042a870 0x56058042ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.761+0000 7fa168c00640 10 client.17088 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.761+0000 7fa17f400640 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56058042a870 0x56058042ccf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c43f37d393dab76d server_cookie=47f8175a41ab1e6f in_seq=0 out_seq=0 2024-03-26T14:09:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.762+0000 7fa168c00640 1 -- 172.21.15.137:0/444588680 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560580382800 con 0x56058042a870 2024-03-26T14:09:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.762+0000 7fa168c00640 10 client.17088 handle_command_reply: tid=0 2024-03-26T14:09:12.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.762+0000 7fa188d6b740 1 client.17088 shutdown 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 msgr2=0x56058041ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x56058041ede0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa16c004630 tx=0x7fa16c0046c0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 shutdown_connections 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56058042a870 0x56058042ccf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580426360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560580415df0 0x56058041f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 --2- 172.21.15.137:0/444588680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x56058041ede0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa168c00640 0 client.17088 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 >> 172.21.15.137:0/444588680 conn(0x56058040bc00 msgr2=0x56058040aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 shutdown_connections 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 1 -- 172.21.15.137:0/444588680 wait complete. 2024-03-26T14:09:12.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.763+0000 7fa188d6b740 20 client.17088 trim_cache size 0 max 16384 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 Processor -- start 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 -- start start 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560580414c20 0x56058040a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x5605804262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804167b0 0x56058040ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600120e0 con 0x560580415df0 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000fbe0 con 0x560580414c20 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa188d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000dc60 con 0x5605804167b0 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa17f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x5605804262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa17f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x5605804262f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39866/0 (socket says 172.21.15.137:39866) 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.781+0000 7fa17f400640 1 -- 172.21.15.137:0/2758343714 learned_addr learned my addr 172.21.15.137:0/2758343714 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:12.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17ea00640 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560580414c20 0x56058040a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17f400640 1 -- 172.21.15.137:0/2758343714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804167b0 msgr2=0x56058040ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17f400640 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804167b0 0x56058040ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17f400640 1 -- 172.21.15.137:0/2758343714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560580414c20 msgr2=0x56058040a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17f400640 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560580414c20 0x56058040a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17f400640 1 -- 172.21.15.137:0/2758343714 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56058040c670 con 0x560580415df0 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa17f400640 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x5605804262f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa170009900 tx=0x7fa170009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a494273c62e9c159 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.782+0000 7fa16b400640 1 -- 172.21.15.137:0/2758343714 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170045040 con 0x560580415df0 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa16b400640 1 -- 172.21.15.137:0/2758343714 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1700035f0 con 0x560580415df0 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa16b400640 1 -- 172.21.15.137:0/2758343714 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700346a0 con 0x560580415df0 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 -- 172.21.15.137:0/2758343714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 msgr2=0x5605804262f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x5605804262f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa170009900 tx=0x7fa170009260 comp rx=0 tx=0).stop 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 -- 172.21.15.137:0/2758343714 shutdown_connections 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804167b0 0x56058040ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580415df0 0x5605804262f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 --2- 172.21.15.137:0/2758343714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560580414c20 0x56058040a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 -- 172.21.15.137:0/2758343714 >> 172.21.15.137:0/2758343714 conn(0x56058040bc00 msgr2=0x56058040f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 -- 172.21.15.137:0/2758343714 shutdown_connections 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.783+0000 7fa188d6b740 1 -- 172.21.15.137:0/2758343714 wait complete. 2024-03-26T14:09:12.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.784+0000 7fa188d6b740 1 -- start start 2024-03-26T14:09:12.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.784+0000 7fa188d6b740 1 Processor -- start 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.784+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x560580425e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580424450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa188d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804263d0 0x560580424990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa188d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000e260 con 0x560580414c20 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa188d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000f580 con 0x5605804167b0 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa188d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16000e8c0 con 0x5605804263d0 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa188d6b740 10 client.0 _ll_register_callbacks cb 0x7fa16c005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x560580425e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x560580425e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39874/0 (socket says 172.21.15.137:39874) 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 -- 172.21.15.137:0/398490001 learned_addr learned my addr 172.21.15.137:0/398490001 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:12.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17f400640 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580424450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 -- 172.21.15.137:0/398490001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804263d0 msgr2=0x560580424990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804263d0 0x560580424990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 -- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 msgr2=0x560580424450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580424450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.785+0000 7fa17ea00640 1 -- 172.21.15.137:0/398490001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560580425190 con 0x560580414c20 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa17ea00640 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x560580425e40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005fd0 tx=0x7fa16c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f3d2d21d5cbedf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa168c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa168c00640 1 -- 172.21.15.137:0/398490001 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00ce10 con 0x560580414c20 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa168c00640 1 -- 172.21.15.137:0/398490001 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa16c0103d0 con 0x560580414c20 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1700035f0 con 0x560580414c20 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa168c00640 1 -- 172.21.15.137:0/398490001 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa16c03c050 con 0x560580414c20 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa188d6b740 10 client.17094 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.786+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560580487220 con 0x560580414c20 2024-03-26T14:09:12.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa168c00640 1 -- 172.21.15.137:0/398490001 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1700035f0 con 0x560580414c20 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa168c00640 1 -- 172.21.15.137:0/398490001 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa16c012b80 con 0x560580414c20 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa188d6b740 10 client.17094 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa188d6b740 10 client.17094 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa188d6b740 20 client.17094 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa188d6b740 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560580487ad0 0x560580489ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa188d6b740 4 client.17094 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56058037cb50 con 0x560580487ad0 2024-03-26T14:09:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.787+0000 7fa17f400640 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560580487ad0 0x560580489ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:12.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.788+0000 7fa17f400640 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560580487ad0 0x560580489ef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=970f8cba9f8483e7 server_cookie=1a22f4617128e525 in_seq=0 out_seq=0 2024-03-26T14:09:12.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.788+0000 7fa168c00640 10 client.17094 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:12.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa168c00640 1 -- 172.21.15.137:0/398490001 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56058037cb50 con 0x560580487ad0 2024-03-26T14:09:12.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa168c00640 10 client.17094 handle_command_reply: tid=0 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 client.17094 shutdown 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 msgr2=0x560580425e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x560580425e40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005fd0 tx=0x7fa16c011d10 comp rx=0 tx=0).stop 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 shutdown_connections 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x560580487ad0 0x560580489ef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5605804167b0 0x560580424450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5605804263d0 0x560580424990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 --2- 172.21.15.137:0/398490001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560580414c20 0x560580425e40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 >> 172.21.15.137:0/398490001 conn(0x56058040bc00 msgr2=0x560580419020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa168c00640 0 client.17094 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 shutdown_connections 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 1 -- 172.21.15.137:0/398490001 wait complete. 2024-03-26T14:09:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.789+0000 7fa188d6b740 20 client.17094 trim_cache size 0 max 16384 2024-03-26T14:09:12.792 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:09:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1640a2880 msgr2=0x7fa1640a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1640a2880 0x7fa1640a4d40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa1780a3320 tx=0x7fa17800b910 comp rx=0 tx=0).stop 2024-03-26T14:09:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 msgr2=0x7fa18019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 0x7fa18019ef30 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa178009fd0 tx=0x7fa17800b260 comp rx=0 tx=0).stop 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 shutdown_connections 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa1640a2880 0x7fa1640a4d40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa180107380 0x7fa18019ef30 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa180104a50 0x7fa180197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 --2- 172.21.15.137:0/181018996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa180102120 0x7fa180197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 >> 172.21.15.137:0/181018996 conn(0x7fa1800fbb30 msgr2=0x7fa1801029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 shutdown_connections 2024-03-26T14:09:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:12.790+0000 7fa186000640 1 -- 172.21.15.137:0/181018996 wait complete. 2024-03-26T14:09:12.807 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:09:12.808 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 -- 172.21.15.137:0/3221808280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 msgr2=0x7f5e30104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 --2- 172.21.15.137:0/3221808280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30104450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009860 tx=0x7f5e2002bb70 comp rx=0 tx=0).stop 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 -- 172.21.15.137:0/3221808280 shutdown_connections 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 --2- 172.21.15.137:0/3221808280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e301072c0 0x7f5e301096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 --2- 172.21.15.137:0/3221808280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e30104990 0x7f5e30106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 --2- 172.21.15.137:0/3221808280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30104450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.034+0000 7f5e35c00640 1 -- 172.21.15.137:0/3221808280 >> 172.21.15.137:0/3221808280 conn(0x7f5e300fba10 msgr2=0x7f5e300fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.035+0000 7f5e35c00640 1 -- 172.21.15.137:0/3221808280 shutdown_connections 2024-03-26T14:09:13.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.035+0000 7f5e35c00640 1 -- 172.21.15.137:0/3221808280 wait complete. 2024-03-26T14:09:13.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.035+0000 7f5e35c00640 1 Processor -- start 2024-03-26T14:09:13.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 -- start start 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 0x7f5e30199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e301072c0 0x7f5e301a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3010d0c0 con 0x7f5e30104990 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3010cf40 con 0x7f5e30102060 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e35c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3010d240 con 0x7f5e301072c0 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 0x7f5e30199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 0x7f5e30199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39888/0 (socket says 172.21.15.137:39888) 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e2f400640 1 -- 172.21.15.137:0/1751796917 learned_addr learned my addr 172.21.15.137:0/1751796917 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.036+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e35200640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e301072c0 0x7f5e301a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2f400640 1 -- 172.21.15.137:0/1751796917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e301072c0 msgr2=0x7f5e301a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2f400640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e301072c0 0x7f5e301a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2f400640 1 -- 172.21.15.137:0/1751796917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 msgr2=0x7f5e30199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2f400640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2f400640 1 -- 172.21.15.137:0/1751796917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e301a1640 con 0x7f5e30104990 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e35200640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e301072c0 0x7f5e301a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.037+0000 7f5e2f400640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 0x7f5e30199f60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e9c0 tx=0x7f5e1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.038+0000 7f5e2cc00640 1 -- 172.21.15.137:0/1751796917 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00cd80 con 0x7f5e30104990 2024-03-26T14:09:13.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.038+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e301a1930 con 0x7f5e30104990 2024-03-26T14:09:13.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.038+0000 7f5e2cc00640 1 -- 172.21.15.137:0/1751796917 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e1c0054d0 con 0x7f5e30104990 2024-03-26T14:09:13.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.038+0000 7f5e2cc00640 1 -- 172.21.15.137:0/1751796917 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c010430 con 0x7f5e30104990 2024-03-26T14:09:13.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.038+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e301a1ed0 con 0x7f5e30104990 2024-03-26T14:09:13.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.041+0000 7f5e39fb3740 1 Processor -- start 2024-03-26T14:09:13.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.041+0000 7f5e39fb3740 1 -- start start 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91677e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91679040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d9167fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d915dc2c0 con 0x561d916779f0 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d91584c90 con 0x561d91679580 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d915d8360 con 0x561d91678bc0 2024-03-26T14:09:13.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e35200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91677e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e35200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91677e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39896/0 (socket says 172.21.15.137:39896) 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e35200640 1 -- 172.21.15.137:0/1336606415 learned_addr learned my addr 172.21.15.137:0/1336606415 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e2cc00640 1 -- 172.21.15.137:0/1751796917 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5e1c004fd0 con 0x7f5e30104990 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.042+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91679040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e35200640 1 -- 172.21.15.137:0/1336606415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 msgr2=0x561d91679040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e35200640 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91679040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e35200640 1 -- 172.21.15.137:0/1336606415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 msgr2=0x561d9167fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e35200640 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d9167fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e35200640 1 -- 172.21.15.137:0/1336606415 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d915a9080 con 0x561d916779f0 2024-03-26T14:09:13.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e35200640 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91677e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5e28010460 tx=0x7f5e28010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27ca3d3dfe0a226 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e25600640 1 -- 172.21.15.137:0/1336606415 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2801d070 con 0x561d916779f0 2024-03-26T14:09:13.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e25600640 1 -- 172.21.15.137:0/1336606415 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e2800eb10 con 0x561d916779f0 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.043+0000 7f5e25600640 1 -- 172.21.15.137:0/1336606415 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28018ba0 con 0x561d916779f0 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.044+0000 7f5e2cc00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5e140a2800 0x7f5e140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.044+0000 7f5e2cc00640 1 -- 172.21.15.137:0/1751796917 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5e1c0cf2f0 con 0x7f5e30104990 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.044+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1336606415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 msgr2=0x561d91677e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.044+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91677e10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5e28010460 tx=0x7f5e28010860 comp rx=0 tx=0).stop 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.044+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5e140a2800 0x7f5e140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1336606415 shutdown_connections 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d9167fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91679040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1336606415 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91677e10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1336606415 >> 172.21.15.137:0/1336606415 conn(0x561d91672de0 msgr2=0x561d916731e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1336606415 shutdown_connections 2024-03-26T14:09:13.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1336606415 wait complete. 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e39fb3740 1 -- start start 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.045+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5e140a2800 0x7f5e140a4cc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e200349a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 Processor -- start 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91681cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91682210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91689250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d91680590 con 0x561d916779f0 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d915980d0 con 0x561d91679580 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d91680710 con 0x561d91678bc0 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e39fb3740 10 client.0 _ll_register_callbacks cb 0x561d9159ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:13.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91681cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91681cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39902/0 (socket says 172.21.15.137:39902) 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.046+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3497459875 learned_addr learned my addr 172.21.15.137:0/3497459875 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e35200640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91689250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2f400640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91682210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3497459875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 msgr2=0x561d91682210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91682210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 msgr2=0x561d91689250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91689250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3497459875 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d912f03f0 con 0x561d916779f0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e35200640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91689250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91681cd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009860 tx=0x7f5e2002bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eeafe3750059c90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.047+0000 7f5e1a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e1a000640 1 -- 172.21.15.137:0/3497459875 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2004d020 con 0x561d916779f0 2024-03-26T14:09:13.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e1a000640 1 -- 172.21.15.137:0/3497459875 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5e200037f0 con 0x561d916779f0 2024-03-26T14:09:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e2800eb10 con 0x561d916779f0 2024-03-26T14:09:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e1a000640 1 -- 172.21.15.137:0/3497459875 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e20058050 con 0x561d916779f0 2024-03-26T14:09:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e39fb3740 10 client.17106 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561d913ba7d0 con 0x561d916779f0 2024-03-26T14:09:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.048+0000 7f5e1a000640 1 -- 172.21.15.137:0/3497459875 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e2800eb10 con 0x561d916779f0 2024-03-26T14:09:13.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e1a000640 1 -- 172.21.15.137:0/3497459875 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5e2003fa30 con 0x561d916779f0 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e39fb3740 10 client.17106 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e39fb3740 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e39fb3740 20 client.17106 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d9168d7a0 0x561d9168fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e39fb3740 4 client.17106 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.049+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d915e9800 con 0x561d9168d7a0 2024-03-26T14:09:13.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.050+0000 7f5e2f400640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d9168d7a0 0x561d9168fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.050+0000 7f5e2f400640 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d9168d7a0 0x561d9168fbd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec84f72ee8f04555 server_cookie=3e14aa6e8530926b in_seq=0 out_seq=0 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.050+0000 7f5e1a000640 10 client.17106 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e1a000640 1 -- 172.21.15.137:0/3497459875 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x561d915e9800 con 0x561d9168d7a0 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e1a000640 10 client.17106 handle_command_reply: tid=0 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 client.17106 shutdown 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 msgr2=0x561d91681cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91681cd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5e20009860 tx=0x7f5e2002bb70 comp rx=0 tx=0).stop 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 shutdown_connections 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d9168d7a0 0x561d9168fbd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91689250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d91682210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e1a000640 0 client.17106 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3497459875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91681cd0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 >> 172.21.15.137:0/3497459875 conn(0x561d91672de0 msgr2=0x561d91672160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 shutdown_connections 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.052+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3497459875 wait complete. 2024-03-26T14:09:13.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.053+0000 7f5e39fb3740 20 client.17106 trim_cache size 0 max 16384 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 Processor -- start 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 -- start start 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91671fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d916891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91694490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301ae810 con 0x561d916779f0 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301ac310 con 0x561d91679580 2024-03-26T14:09:13.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.067+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301aa390 con 0x561d91678bc0 2024-03-26T14:09:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91671fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91671fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39912/0 (socket says 172.21.15.137:39912) 2024-03-26T14:09:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3106892744 learned_addr learned my addr 172.21.15.137:0/3106892744 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e35200640 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91694490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2f400640 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d916891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3106892744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 msgr2=0x561d916891e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d916891e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3106892744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 msgr2=0x561d91694490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91694490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.068+0000 7f5e2fe00640 1 -- 172.21.15.137:0/3106892744 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d9167f7a0 con 0x561d916779f0 2024-03-26T14:09:13.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.069+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91671fe0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5e2003fdf0 tx=0x7f5e20004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b95980e8639d93 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e24c00640 1 -- 172.21.15.137:0/3106892744 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20061070 con 0x561d916779f0 2024-03-26T14:09:13.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e24c00640 1 -- 172.21.15.137:0/3106892744 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e20044040 con 0x561d916779f0 2024-03-26T14:09:13.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e24c00640 1 -- 172.21.15.137:0/3106892744 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2004d040 con 0x561d916779f0 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3106892744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 msgr2=0x561d91671fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91671fe0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5e2003fdf0 tx=0x7f5e20004610 comp rx=0 tx=0).stop 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3106892744 shutdown_connections 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91679580 0x561d91694490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91678bc0 0x561d916891e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/3106892744 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91671fe0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.071+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3106892744 >> 172.21.15.137:0/3106892744 conn(0x561d91672de0 msgr2=0x561d916718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.072+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3106892744 shutdown_connections 2024-03-26T14:09:13.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.072+0000 7f5e39fb3740 1 -- 172.21.15.137:0/3106892744 wait complete. 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.072+0000 7f5e39fb3740 1 -- start start 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.072+0000 7f5e39fb3740 1 Processor -- start 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91685380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91678bc0 0x561d916858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91679580 0x561d91688890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301aa990 con 0x561d916779f0 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301abcb0 con 0x561d91678bc0 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301aaff0 con 0x561d91679580 2024-03-26T14:09:13.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e39fb3740 10 client.0 _ll_register_callbacks cb 0x7f5e28009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91685380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91685380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39928/0 (socket says 172.21.15.137:39928) 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 -- 172.21.15.137:0/1639790684 learned_addr learned my addr 172.21.15.137:0/1639790684 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2f400640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91678bc0 0x561d916858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e35200640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91679580 0x561d91688890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 -- 172.21.15.137:0/1639790684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91679580 msgr2=0x561d91688890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91679580 0x561d91688890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 -- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91678bc0 msgr2=0x561d916858c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.073+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91678bc0 0x561d916858c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e2fe00640 1 -- 172.21.15.137:0/1639790684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d91688e80 con 0x561d916779f0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e35200640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91679580 0x561d91688890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e2fe00640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91685380 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5e20005770 tx=0x7f5e20003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3db9ef1e00c336 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e1a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e1a000640 1 -- 172.21.15.137:0/1639790684 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20061070 con 0x561d916779f0 2024-03-26T14:09:13.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e1a000640 1 -- 172.21.15.137:0/1639790684 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5e2003f650 con 0x561d916779f0 2024-03-26T14:09:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e20044040 con 0x561d916779f0 2024-03-26T14:09:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.074+0000 7f5e1a000640 1 -- 172.21.15.137:0/1639790684 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2006d050 con 0x561d916779f0 2024-03-26T14:09:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 10 client.17112 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:13.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561d916e9ac0 con 0x561d916779f0 2024-03-26T14:09:13.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e1a000640 1 -- 172.21.15.137:0/1639790684 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5e20044040 con 0x561d916779f0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e1a000640 1 -- 172.21.15.137:0/1639790684 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5e20049970 con 0x561d916779f0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 10 client.17112 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 10 client.17112 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 20 client.17112 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d916ea2e0 0x561d916ec700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 4 client.17112 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.075+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561d915e3b50 con 0x561d916ea2e0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.076+0000 7f5e2f400640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d916ea2e0 0x561d916ec700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.076+0000 7f5e2f400640 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d916ea2e0 0x561d916ec700 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff68968fd94b0dbf server_cookie=56a477b534cdda70 in_seq=0 out_seq=0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.076+0000 7f5e1a000640 10 client.17112 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e1a000640 1 -- 172.21.15.137:0/1639790684 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561d915e3b50 con 0x561d916ea2e0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e1a000640 10 client.17112 handle_command_reply: tid=0 2024-03-26T14:09:13.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 client.17112 shutdown 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 msgr2=0x561d91685380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91685380 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5e20005770 tx=0x7f5e20003c00 comp rx=0 tx=0).stop 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 shutdown_connections 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x561d916ea2e0 0x561d916ec700 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561d91678bc0 0x561d916858c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561d91679580 0x561d91688890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 --2- 172.21.15.137:0/1639790684 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x561d916779f0 0x561d91685380 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 >> 172.21.15.137:0/1639790684 conn(0x561d91672de0 msgr2=0x561d91672160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e1a000640 0 client.17112 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 shutdown_connections 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 1 -- 172.21.15.137:0/1639790684 wait complete. 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.077+0000 7f5e39fb3740 20 client.17112 trim_cache size 0 max 16384 2024-03-26T14:09:13.079 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:09:13.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5e140a2800 msgr2=0x7f5e140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5e140a2800 0x7f5e140a4cc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5e20000c00 tx=0x7f5e200349a0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 msgr2=0x7f5e30199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 0x7f5e30199f60 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e9c0 tx=0x7f5e1c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:09:13.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 shutdown_connections 2024-03-26T14:09:13.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5e140a2800 0x7f5e140a4cc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e301072c0 0x7f5e301a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5e30104990 0x7f5e30199f60 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 --2- 172.21.15.137:0/1751796917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e30102060 0x7f5e30199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 >> 172.21.15.137:0/1751796917 conn(0x7f5e300fba10 msgr2=0x7f5e30101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 shutdown_connections 2024-03-26T14:09:13.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.078+0000 7f5e35c00640 1 -- 172.21.15.137:0/1751796917 wait complete. 2024-03-26T14:09:13.095 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:09:13.095 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 -- 172.21.15.137:0/3280518570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17901016d0 msgr2=0x7f1790101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 --2- 172.21.15.137:0/3280518570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17901016d0 0x7f1790101af0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f1784009900 tx=0x7f178402bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 -- 172.21.15.137:0/3280518570 shutdown_connections 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 --2- 172.21.15.137:0/3280518570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790103280 0x7f1790109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 --2- 172.21.15.137:0/3280518570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17901028c0 0x7f1790102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 --2- 172.21.15.137:0/3280518570 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f17901016d0 0x7f1790101af0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 -- 172.21.15.137:0/3280518570 >> 172.21.15.137:0/3280518570 conn(0x7f17900fcc40 msgr2=0x7f17900ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 -- 172.21.15.137:0/3280518570 shutdown_connections 2024-03-26T14:09:13.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.316+0000 7f1797400640 1 -- 172.21.15.137:0/3280518570 wait complete. 2024-03-26T14:09:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 Processor -- start 2024-03-26T14:09:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 -- start start 2024-03-26T14:09:13.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17901016d0 0x7f1790199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901028c0 0x7f179019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 0x7f17901a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179010d240 con 0x7f1790103280 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179010d0c0 con 0x7f17901028c0 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.317+0000 7f1797400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179010d3c0 con 0x7f17901016d0 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 0x7f17901a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 0x7f17901a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39954/0 (socket says 172.21.15.137:39954) 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 -- 172.21.15.137:0/62745992 learned_addr learned my addr 172.21.15.137:0/62745992 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f178fe00640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901028c0 0x7f179019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1794a00640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17901016d0 0x7f1790199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 -- 172.21.15.137:0/62745992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17901016d0 msgr2=0x7f1790199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17901016d0 0x7f1790199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 -- 172.21.15.137:0/62745992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901028c0 msgr2=0x7f179019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901028c0 0x7f179019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.318+0000 7f1795400640 1 -- 172.21.15.137:0/62745992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17901a17b0 con 0x7f1790103280 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.319+0000 7f1795400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 0x7f17901a1140 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f178800eb10 tx=0x7f178800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.319+0000 7f178d600640 1 -- 172.21.15.137:0/62745992 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788004b20 con 0x7f1790103280 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.319+0000 7f178d600640 1 -- 172.21.15.137:0/62745992 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1788004cc0 con 0x7f1790103280 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.319+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a1aa0 con 0x7f1790103280 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.319+0000 7f178d600640 1 -- 172.21.15.137:0/62745992 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788010690 con 0x7f1790103280 2024-03-26T14:09:13.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.319+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a1fa0 con 0x7f1790103280 2024-03-26T14:09:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.322+0000 7f179a1c9740 1 Processor -- start 2024-03-26T14:09:13.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.323+0000 7f178d600640 1 -- 172.21.15.137:0/62745992 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1788005020 con 0x7f1790103280 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.323+0000 7f179a1c9740 1 -- start start 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddb4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f179a1c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653dd182c0 con 0x55653ddb53a0 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f179a1c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653dcc0c90 con 0x55653ddb49e0 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f179a1c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653dd14360 con 0x55653ddb3810 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f178fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f1794a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddb4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f1794a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddb4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:58676/0 (socket says 172.21.15.137:58676) 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f1794a00640 1 -- 172.21.15.137:0/3878359331 learned_addr learned my addr 172.21.15.137:0/3878359331 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.324+0000 7f178fe00640 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f1794a00640 1 -- 172.21.15.137:0/3878359331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 msgr2=0x55653ddb3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f1794a00640 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f1794a00640 1 -- 172.21.15.137:0/3878359331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 msgr2=0x55653ddbbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f1794a00640 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddbbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f1794a00640 1 -- 172.21.15.137:0/3878359331 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653dce5080 con 0x55653ddb49e0 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f178fe00640 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddbbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f178d600640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17780a2800 0x7f17780a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.326 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f178d600640 1 -- 172.21.15.137:0/62745992 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f17880cf410 con 0x7f1790103280 2024-03-26T14:09:13.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f178fe00640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17780a2800 0x7f17780a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.325+0000 7f1794a00640 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddb4e60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1784005ab0 tx=0x7f1784009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d1a2c900e952dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.326+0000 7f1781c00640 1 -- 172.21.15.137:0/3878359331 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1784044020 con 0x55653ddb49e0 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.326+0000 7f1781c00640 1 -- 172.21.15.137:0/3878359331 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f178402cae0 con 0x55653ddb49e0 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.326+0000 7f1781c00640 1 -- 172.21.15.137:0/3878359331 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840046c0 con 0x55653ddb49e0 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 -- 172.21.15.137:0/3878359331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 msgr2=0x55653ddb4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f178fe00640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17780a2800 0x7f17780a4cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f177c00bfd0 tx=0x7f177c005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddb4e60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1784005ab0 tx=0x7f1784009670 comp rx=0 tx=0).stop 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 -- 172.21.15.137:0/3878359331 shutdown_connections 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddb4e60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddbbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 --2- 172.21.15.137:0/3878359331 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 -- 172.21.15.137:0/3878359331 >> 172.21.15.137:0/3878359331 conn(0x55653ddaec00 msgr2=0x55653ddaf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 -- 172.21.15.137:0/3878359331 shutdown_connections 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 -- 172.21.15.137:0/3878359331 wait complete. 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.327+0000 7f179a1c9740 1 -- start start 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 Processor -- start 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddbda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddbdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb53a0 0x55653ddc4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ddbc350 con 0x55653ddb49e0 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653dcd40d0 con 0x55653ddb53a0 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653ddbc4d0 con 0x55653ddb3810 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f179a1c9740 10 client.0 _ll_register_callbacks cb 0x55653dcd8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.328+0000 7f178fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddbdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddbdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39966/0 (socket says 172.21.15.137:39966) 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 -- 172.21.15.137:0/1868905802 learned_addr learned my addr 172.21.15.137:0/1868905802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f1795400640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb53a0 0x55653ddc4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f1794a00640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddbda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 -- 172.21.15.137:0/1868905802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 msgr2=0x55653ddbda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddbda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 -- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb53a0 msgr2=0x55653ddc4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb53a0 0x55653ddc4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 -- 172.21.15.137:0/1868905802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653da2c3f0 con 0x55653ddb49e0 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f1795400640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb53a0 0x55653ddc4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f178fe00640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddbdf90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f177c005b60 tx=0x7f177c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3375b7441ccf7dbd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.329+0000 7f1776000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f1776000640 1 -- 172.21.15.137:0/1868905802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c005020 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f1776000640 1 -- 172.21.15.137:0/1868905802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f177c0051c0 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178402cae0 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f1776000640 1 -- 172.21.15.137:0/1868905802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f177c032050 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f179a1c9740 10 client.17124 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55653daf67d0 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.330+0000 7f1776000640 1 -- 172.21.15.137:0/1868905802 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f178402cae0 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f1776000640 1 -- 172.21.15.137:0/1868905802 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f177c0035f0 con 0x55653ddb49e0 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f179a1c9740 10 client.17124 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f179a1c9740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f179a1c9740 20 client.17124 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653ddc9530 0x55653ddcb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f179a1c9740 4 client.17124 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55653dd25800 con 0x55653ddc9530 2024-03-26T14:09:13.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.331+0000 7f1794a00640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653ddc9530 0x55653ddcb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.332+0000 7f1794a00640 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653ddc9530 0x55653ddcb9b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ca803a8e2aaad server_cookie=bed66e75e485210d in_seq=0 out_seq=0 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.332+0000 7f1776000640 10 client.17124 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.333+0000 7f1776000640 1 -- 172.21.15.137:0/1868905802 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55653dd25800 con 0x55653ddc9530 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.333+0000 7f1776000640 10 client.17124 handle_command_reply: tid=0 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 client.17124 shutdown 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 msgr2=0x55653ddbdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddbdf90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f177c005b60 tx=0x7f177c004730 comp rx=0 tx=0).stop 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 shutdown_connections 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653ddc9530 0x55653ddcb9b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb53a0 0x55653ddc4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddbdf90 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1868905802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddbda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 >> 172.21.15.137:0/1868905802 conn(0x55653ddaec00 msgr2=0x55653ddadeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f1776000640 0 client.17124 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:13.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 shutdown_connections 2024-03-26T14:09:13.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 1 -- 172.21.15.137:0/1868905802 wait complete. 2024-03-26T14:09:13.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.334+0000 7f179a1c9740 20 client.17124 trim_cache size 0 max 16384 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.349+0000 7f179a1c9740 1 Processor -- start 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.349+0000 7f179a1c9740 1 -- start start 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddc4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddb7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f179a1c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901aea00 con 0x55653ddb53a0 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f179a1c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901ac500 con 0x55653ddb49e0 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f179a1c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901aa580 con 0x55653ddb3810 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddb7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddb7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39970/0 (socket says 172.21.15.137:39970) 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 -- 172.21.15.137:0/249803365 learned_addr learned my addr 172.21.15.137:0/249803365 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f178fe00640 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddc4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1794a00640 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 -- 172.21.15.137:0/249803365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 msgr2=0x55653ddb7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 -- 172.21.15.137:0/249803365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 msgr2=0x55653ddc4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddc4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.350+0000 7f1795400640 1 -- 172.21.15.137:0/249803365 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653ddaa510 con 0x55653ddb53a0 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.351+0000 7f1795400640 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddb7f60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f17880a3c90 tx=0x7f178809b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2a866105118df7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.351+0000 7f1781200640 1 -- 172.21.15.137:0/249803365 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17880b4070 con 0x55653ddb53a0 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.351+0000 7f1781200640 1 -- 172.21.15.137:0/249803365 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f178809ba20 con 0x55653ddb53a0 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.351+0000 7f1781200640 1 -- 172.21.15.137:0/249803365 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1788098090 con 0x55653ddb53a0 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 -- 172.21.15.137:0/249803365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 msgr2=0x55653ddb7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddb7f60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f17880a3c90 tx=0x7f178809b560 comp rx=0 tx=0).stop 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 -- 172.21.15.137:0/249803365 shutdown_connections 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb49e0 0x55653ddc4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb53a0 0x55653ddb7f60 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 --2- 172.21.15.137:0/249803365 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb3810 0x55653ddb7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 -- 172.21.15.137:0/249803365 >> 172.21.15.137:0/249803365 conn(0x55653ddaec00 msgr2=0x55653ddb26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 -- 172.21.15.137:0/249803365 shutdown_connections 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 -- 172.21.15.137:0/249803365 wait complete. 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 -- start start 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.352+0000 7f179a1c9740 1 Processor -- start 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb3810 0x55653ddc0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddc14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb53a0 0x55653ddc44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901aab80 con 0x55653ddb49e0 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901abea0 con 0x55653ddb3810 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17901ab1e0 con 0x55653ddb53a0 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f179a1c9740 10 client.0 _ll_register_callbacks cb 0x7f177c00b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f178fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddc14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f178fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddc14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39974/0 (socket says 172.21.15.137:39974) 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f178fe00640 1 -- 172.21.15.137:0/1160017550 learned_addr learned my addr 172.21.15.137:0/1160017550 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f1794a00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb3810 0x55653ddc0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.353+0000 7f1795400640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb53a0 0x55653ddc44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f178fe00640 1 -- 172.21.15.137:0/1160017550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb53a0 msgr2=0x55653ddc44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f178fe00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb53a0 0x55653ddc44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f178fe00640 1 -- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb3810 msgr2=0x55653ddc0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f178fe00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb3810 0x55653ddc0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f178fe00640 1 -- 172.21.15.137:0/1160017550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653ddc4ab0 con 0x55653ddb49e0 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f1795400640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb53a0 0x55653ddc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f1794a00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb3810 0x55653ddc0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f178fe00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddc14a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f177c003a90 tx=0x7f177c003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ea717803d58bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f1776000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f1776000640 1 -- 172.21.15.137:0/1160017550 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c03b060 con 0x55653ddb49e0 2024-03-26T14:09:13.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f1776000640 1 -- 172.21.15.137:0/1160017550 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f177c0047a0 con 0x55653ddb49e0 2024-03-26T14:09:13.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.354+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f178809ba20 con 0x55653ddb49e0 2024-03-26T14:09:13.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f1776000640 1 -- 172.21.15.137:0/1160017550 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f177c042050 con 0x55653ddb49e0 2024-03-26T14:09:13.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 10 client.17130 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:13.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55653de25710 con 0x55653ddb49e0 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f1776000640 1 -- 172.21.15.137:0/1160017550 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f178809ba20 con 0x55653ddb49e0 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f1776000640 1 -- 172.21.15.137:0/1160017550 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f177c005040 con 0x55653ddb49e0 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 10 client.17130 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 10 client.17130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 20 client.17130 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653de26060 0x55653de28480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 4 client.17130 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.355+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55653dd1fb50 con 0x55653de26060 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.356+0000 7f1794a00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653de26060 0x55653de28480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f1794a00640 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653de26060 0x55653de28480 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4247bd90debf07da server_cookie=5f6648896ab0bcb3 in_seq=0 out_seq=0 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f1776000640 10 client.17130 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f1776000640 1 -- 172.21.15.137:0/1160017550 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55653dd1fb50 con 0x55653de26060 2024-03-26T14:09:13.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f1776000640 10 client.17130 handle_command_reply: tid=0 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 client.17130 shutdown 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 msgr2=0x55653ddc14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddc14a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f177c003a90 tx=0x7f177c003e90 comp rx=0 tx=0).stop 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 shutdown_connections 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55653de26060 0x55653de28480 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653ddb53a0 0x55653ddc44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55653ddb49e0 0x55653ddc14a0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 --2- 172.21.15.137:0/1160017550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55653ddb3810 0x55653ddc0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 >> 172.21.15.137:0/1160017550 conn(0x55653ddaec00 msgr2=0x55653ddad620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f1776000640 0 client.17130 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.357+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 shutdown_connections 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f179a1c9740 1 -- 172.21.15.137:0/1160017550 wait complete. 2024-03-26T14:09:13.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f179a1c9740 20 client.17130 trim_cache size 0 max 16384 2024-03-26T14:09:13.361 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:09:13.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17780a2800 msgr2=0x7f17780a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17780a2800 0x7f17780a4cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f177c00bfd0 tx=0x7f177c005c90 comp rx=0 tx=0).stop 2024-03-26T14:09:13.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 msgr2=0x7f17901a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 0x7f17901a1140 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f178800eb10 tx=0x7f178800c2d0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 shutdown_connections 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f17780a2800 0x7f17780a4cc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901028c0 0x7f179019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1790103280 0x7f17901a1140 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 --2- 172.21.15.137:0/62745992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f17901016d0 0x7f1790199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.358+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 >> 172.21.15.137:0/62745992 conn(0x7f17900fcc40 msgr2=0x7f17900738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.359+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 shutdown_connections 2024-03-26T14:09:13.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:13.359+0000 7f1797400640 1 -- 172.21.15.137:0/62745992 wait complete. 2024-03-26T14:09:13.374 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:13 smithi137 ceph-mon[31539]: pgmap v663: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:13.374 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:13 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/50465721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:09:13.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:13 smithi179 ceph-mon[35472]: pgmap v663: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:13.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:13 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/50465721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:09:13.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:13 smithi159 ceph-mon[35501]: pgmap v663: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:13.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:13 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/50465721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:09:14.377 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:09:14.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:09:14.378 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:09:14.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 -- 172.21.15.137:0/1175125843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8103330 msgr2=0x7fe1d8109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 --2- 172.21.15.137:0/1175125843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8103330 0x7fe1d8109bc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009cd0 tx=0x7fe1d002c0f0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 -- 172.21.15.137:0/1175125843 shutdown_connections 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 --2- 172.21.15.137:0/1175125843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8102970 0x7fe1d8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 --2- 172.21.15.137:0/1175125843 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8103330 0x7fe1d8109bc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 --2- 172.21.15.137:0/1175125843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d80ff9a0 0x7fe1d80ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 -- 172.21.15.137:0/1175125843 >> 172.21.15.137:0/1175125843 conn(0x7fe1d80fb2d0 msgr2=0x7fe1d80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.618+0000 7fe1e0400640 1 -- 172.21.15.137:0/1175125843 shutdown_connections 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.619+0000 7fe1e0400640 1 -- 172.21.15.137:0/1175125843 wait complete. 2024-03-26T14:09:14.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.620+0000 7fe1e0400640 1 Processor -- start 2024-03-26T14:09:14.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.620+0000 7fe1e0400640 1 -- start start 2024-03-26T14:09:14.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1e0400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d80ff9a0 0x7fe1d81022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1e0400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 0x7fe1d8100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1e0400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8103330 0x7fe1d8100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1e0400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d810d220 con 0x7fe1d8102970 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1e0400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d810d0a0 con 0x7fe1d8103330 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1e0400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d810d3a0 con 0x7fe1d80ff9a0 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 0x7fe1d8100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 0x7fe1d8100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39984/0 (socket says 172.21.15.137:39984) 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 -- 172.21.15.137:0/3620857367 learned_addr learned my addr 172.21.15.137:0/3620857367 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1de400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8103330 0x7fe1d8100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dda00640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d80ff9a0 0x7fe1d81022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 -- 172.21.15.137:0/3620857367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d80ff9a0 msgr2=0x7fe1d81022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d80ff9a0 0x7fe1d81022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 -- 172.21.15.137:0/3620857367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8103330 msgr2=0x7fe1d8100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8103330 0x7fe1d8100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1dd000640 1 -- 172.21.15.137:0/3620857367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d81016f0 con 0x7fe1d8102970 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.621+0000 7fe1de400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8103330 0x7fe1d8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.622+0000 7fe1dd000640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 0x7fe1d8100920 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400e9f0 tx=0x7fe1c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.622+0000 7fe1d6000640 1 -- 172.21.15.137:0/3620857367 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400cd80 con 0x7fe1d8102970 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.622+0000 7fe1d6000640 1 -- 172.21.15.137:0/3620857367 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1c40108e0 con 0x7fe1d8102970 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.622+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d81a1a10 con 0x7fe1d8102970 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.622+0000 7fe1d6000640 1 -- 172.21.15.137:0/3620857367 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4018920 con 0x7fe1d8102970 2024-03-26T14:09:14.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.622+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d81a1f60 con 0x7fe1d8102970 2024-03-26T14:09:14.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.626+0000 7fe1d6000640 1 -- 172.21.15.137:0/3620857367 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe1c4010a90 con 0x7fe1d8102970 2024-03-26T14:09:14.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.626+0000 7fe1e3029740 1 Processor -- start 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 -- start start 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956cd64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956cdb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956cde090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562956c3beb0 con 0x562956cd6a90 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562956be4a60 con 0x562956cdbc00 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1e3029740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562956c37f50 con 0x562956cd60a0 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956cdb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956cdb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:39988/0 (socket says 172.21.15.137:39988) 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1dda00640 1 -- 172.21.15.137:0/2193648987 learned_addr learned my addr 172.21.15.137:0/2193648987 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1dd000640 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956cde090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.627+0000 7fe1de400640 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956cd64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1dda00640 1 -- 172.21.15.137:0/2193648987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 msgr2=0x562956cd64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956cd64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1dda00640 1 -- 172.21.15.137:0/2193648987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 msgr2=0x562956cde090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956cde090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1dda00640 1 -- 172.21.15.137:0/2193648987 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562956a187e0 con 0x562956cd6a90 2024-03-26T14:09:14.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1d6000640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1c00a2800 0x7fe1c00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956cdb590 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00bc80 tx=0x7fe1cc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613f75029539c638 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1d6000640 1 -- 172.21.15.137:0/3620857367 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe1c40cf2c0 con 0x7fe1d8102970 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1caa00640 1 -- 172.21.15.137:0/2193648987 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc018be0 con 0x562956cd6a90 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1de400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1c00a2800 0x7fe1c00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1caa00640 1 -- 172.21.15.137:0/2193648987 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1cc018d80 con 0x562956cd6a90 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.628+0000 7fe1caa00640 1 -- 172.21.15.137:0/2193648987 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc01f5e0 con 0x562956cd6a90 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 -- 172.21.15.137:0/2193648987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 msgr2=0x562956cdb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956cdb590 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00bc80 tx=0x7fe1cc00f730 comp rx=0 tx=0).stop 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1de400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1c00a2800 0x7fe1c00a4cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0032420 tx=0x7fe1d002e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 -- 172.21.15.137:0/2193648987 shutdown_connections 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956cde090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956cdb590 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2193648987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956cd64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.629+0000 7fe1e3029740 1 -- 172.21.15.137:0/2193648987 >> 172.21.15.137:0/2193648987 conn(0x562956cd1610 msgr2=0x562956cd1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.630+0000 7fe1e3029740 1 -- 172.21.15.137:0/2193648987 shutdown_connections 2024-03-26T14:09:14.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.630+0000 7fe1e3029740 1 -- 172.21.15.137:0/2193648987 wait complete. 2024-03-26T14:09:14.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.630+0000 7fe1e3029740 1 -- start start 2024-03-26T14:09:14.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.630+0000 7fe1e3029740 1 Processor -- start 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cd6a90 0x562956ce0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 0x562956ce77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562956cdebb0 con 0x562956cdbc00 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562956bf7eb0 con 0x562956cd6a90 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562956cded30 con 0x562956cd60a0 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1e3029740 10 client.0 _ll_register_callbacks cb 0x562956bec150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1de400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 0x562956ce77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1de400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 0x562956ce77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40002/0 (socket says 172.21.15.137:40002) 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1de400640 1 -- 172.21.15.137:0/2777307779 learned_addr learned my addr 172.21.15.137:0/2777307779 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1dd000640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cd6a90 0x562956ce0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.631+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1de400640 1 -- 172.21.15.137:0/2777307779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 msgr2=0x562956ce0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1de400640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1de400640 1 -- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cd6a90 msgr2=0x562956ce0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1de400640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cd6a90 0x562956ce0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1de400640 1 -- 172.21.15.137:0/2777307779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562956a78f70 con 0x562956cdbc00 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1de400640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 0x562956ce77b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0006010 tx=0x7fe1d002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a1bcaf6e5217f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.632+0000 7fe1bf400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1bf400640 1 -- 172.21.15.137:0/2777307779 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0039040 con 0x562956cdbc00 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1cc018d80 con 0x562956cdbc00 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1bf400640 1 -- 172.21.15.137:0/2777307779 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe1d004b460 con 0x562956cdbc00 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1bf400640 1 -- 172.21.15.137:0/2777307779 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1d0058050 con 0x562956cdbc00 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1e3029740 10 client.17142 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562956910170 con 0x562956cdbc00 2024-03-26T14:09:14.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1bf400640 1 -- 172.21.15.137:0/2777307779 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1cc018d80 con 0x562956cdbc00 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.633+0000 7fe1bf400640 1 -- 172.21.15.137:0/2777307779 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe1d00347c0 con 0x562956cdbc00 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1e3029740 10 client.17142 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1e3029740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1e3029740 20 client.17142 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956cec130 0x562956cee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1e3029740 4 client.17142 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562956c493f0 con 0x562956cec130 2024-03-26T14:09:14.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.634+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956cec130 0x562956cee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.635+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956cec130 0x562956cee5d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a177b107ec32b57b server_cookie=8a3f53b2de252418 in_seq=0 out_seq=0 2024-03-26T14:09:14.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.635+0000 7fe1bf400640 10 client.17142 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.636+0000 7fe1bf400640 1 -- 172.21.15.137:0/2777307779 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562956c493f0 con 0x562956cec130 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1bf400640 10 client.17142 handle_command_reply: tid=0 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 client.17142 shutdown 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 msgr2=0x562956ce77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 0x562956ce77b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0006010 tx=0x7fe1d002c0f0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 shutdown_connections 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956cec130 0x562956cee5d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cd6a90 0x562956ce0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cdbc00 0x562956ce77b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1bf400640 0 client.17142 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2777307779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 >> 172.21.15.137:0/2777307779 conn(0x562956cd1610 msgr2=0x562956cd0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 shutdown_connections 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 1 -- 172.21.15.137:0/2777307779 wait complete. 2024-03-26T14:09:14.638 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.637+0000 7fe1e3029740 20 client.17142 trim_cache size 0 max 16384 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.656+0000 7fe1e3029740 1 Processor -- start 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.656+0000 7fe1e3029740 1 -- start start 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 0x562956cd06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd6a90 0x562956cd0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1e3029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81aa4c0 con 0x562956cd60a0 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1e3029740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81a7fc0 con 0x562956cdbc00 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1e3029740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81a6060 con 0x562956cd6a90 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 0x562956cd06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1de400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1dda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 0x562956cd06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40012/0 (socket says 172.21.15.137:40012) 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1dda00640 1 -- 172.21.15.137:0/1877778168 learned_addr learned my addr 172.21.15.137:0/1877778168 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.657+0000 7fe1dd000640 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd6a90 0x562956cd0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1dda00640 1 -- 172.21.15.137:0/1877778168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd6a90 msgr2=0x562956cd0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1dda00640 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd6a90 0x562956cd0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1dda00640 1 -- 172.21.15.137:0/1877778168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 msgr2=0x562956ce7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1dda00640 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1dda00640 1 -- 172.21.15.137:0/1877778168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562956cd2080 con 0x562956cd60a0 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1dda00640 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 0x562956cd06b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00bc80 tx=0x7fe1cc00ffc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de5f04c4e700287 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1ca000640 1 -- 172.21.15.137:0/1877778168 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc038070 con 0x562956cd60a0 2024-03-26T14:09:14.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1ca000640 1 -- 172.21.15.137:0/1877778168 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1cc0054f0 con 0x562956cd60a0 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.658+0000 7fe1ca000640 1 -- 172.21.15.137:0/1877778168 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc020630 con 0x562956cd60a0 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 -- 172.21.15.137:0/1877778168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 msgr2=0x562956cd06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 0x562956cd06b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00bc80 tx=0x7fe1cc00ffc0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 -- 172.21.15.137:0/1877778168 shutdown_connections 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd6a90 0x562956cd0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 --2- 172.21.15.137:0/1877778168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd60a0 0x562956cd06b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 -- 172.21.15.137:0/1877778168 >> 172.21.15.137:0/1877778168 conn(0x562956cd1610 msgr2=0x562956ccffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 -- 172.21.15.137:0/1877778168 shutdown_connections 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.659+0000 7fe1e3029740 1 -- 172.21.15.137:0/1877778168 wait complete. 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 -- start start 2024-03-26T14:09:14.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 Processor -- start 2024-03-26T14:09:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956ce6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81a6660 con 0x562956cd6a90 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81a7960 con 0x562956cdbc00 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d81a6ca0 con 0x562956cd60a0 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.660+0000 7fe1e3029740 10 client.0 _ll_register_callbacks cb 0x7fe1d0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956ce6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956ce6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40016/0 (socket says 172.21.15.137:40016) 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 -- 172.21.15.137:0/2284846850 learned_addr learned my addr 172.21.15.137:0/2284846850 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1de400640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 -- 172.21.15.137:0/2284846850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 msgr2=0x562956ce60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 -- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 msgr2=0x562956ce6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 -- 172.21.15.137:0/2284846850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562956d47c00 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1de400640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1dd000640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956ce6600 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4098220 tx=0x7fe1c40d2900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e125c95f6f490 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.661+0000 7fe1bf400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1bf400640 1 -- 172.21.15.137:0/2284846850 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4010de0 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1bf400640 1 -- 172.21.15.137:0/2284846850 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe1c409cb80 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1cc0054f0 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1bf400640 1 -- 172.21.15.137:0/2284846850 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1c409bf50 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 10 client.17148 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562956d48f90 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1bf400640 1 -- 172.21.15.137:0/2284846850 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1cc0054f0 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1bf400640 1 -- 172.21.15.137:0/2284846850 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe1c40d2b50 con 0x562956cd6a90 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 10 client.17148 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 20 client.17148 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:14.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956d498f0 0x562956d4bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 4 client.17148 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:09:14.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.662+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562956c43740 con 0x562956d498f0 2024-03-26T14:09:14.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.663+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956d498f0 0x562956d4bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.664+0000 7fe1bf400640 10 client.17148 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.664+0000 7fe1dda00640 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956d498f0 0x562956d4bd10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546e224cc364ae76 server_cookie=c295cf116846e5ff in_seq=0 out_seq=0 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.666+0000 7fe1bf400640 1 -- 172.21.15.137:0/2284846850 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562956c43740 con 0x562956d498f0 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.666+0000 7fe1bf400640 10 client.17148 handle_command_reply: tid=0 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.666+0000 7fe1e3029740 1 client.17148 shutdown 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.666+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 msgr2=0x562956ce6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956ce6600 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4098220 tx=0x7fe1c40d2900 comp rx=0 tx=0).stop 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 shutdown_connections 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562956d498f0 0x562956d4bd10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562956cdbc00 0x562956ce6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562956cd6a90 0x562956ce6600 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 --2- 172.21.15.137:0/2284846850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562956cd60a0 0x562956ce60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 >> 172.21.15.137:0/2284846850 conn(0x562956cd1610 msgr2=0x562956cd7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1bf400640 0 client.17148 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 shutdown_connections 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 1 -- 172.21.15.137:0/2284846850 wait complete. 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.667+0000 7fe1e3029740 20 client.17148 trim_cache size 0 max 16384 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "4b159800-169f-4070-9949-5540fbea10a6", 2024-03-26T14:09:14.668 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:09:14.669 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:09:14.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1c00a2800 msgr2=0x7fe1c00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1c00a2800 0x7fe1c00a4cc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0032420 tx=0x7fe1d002e040 comp rx=0 tx=0).stop 2024-03-26T14:09:14.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 msgr2=0x7fe1d8100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 0x7fe1d8100920 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400e9f0 tx=0x7fe1c400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 shutdown_connections 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1c00a2800 0x7fe1c00a4cc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1d8103330 0x7fe1d8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1d8102970 0x7fe1d8100920 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 --2- 172.21.15.137:0/3620857367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1d80ff9a0 0x7fe1d81022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 >> 172.21.15.137:0/3620857367 conn(0x7fe1d80fb2d0 msgr2=0x7fe1d8105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 shutdown_connections 2024-03-26T14:09:14.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.668+0000 7fe1e0400640 1 -- 172.21.15.137:0/3620857367 wait complete. 2024-03-26T14:09:14.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:09:14.687 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 -- 172.21.15.137:0/314803864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 msgr2=0x7f0a841019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 --2- 172.21.15.137:0/314803864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a841019c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009900 tx=0x7f0a7402bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 -- 172.21.15.137:0/314803864 shutdown_connections 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 --2- 172.21.15.137:0/314803864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84102790 0x7f0a84102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 --2- 172.21.15.137:0/314803864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a84103150 0x7f0a841099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 --2- 172.21.15.137:0/314803864 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a841019c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 -- 172.21.15.137:0/314803864 >> 172.21.15.137:0/314803864 conn(0x7f0a840fcb10 msgr2=0x7f0a840fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 -- 172.21.15.137:0/314803864 shutdown_connections 2024-03-26T14:09:14.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.903+0000 7f0a89200640 1 -- 172.21.15.137:0/314803864 wait complete. 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.904+0000 7f0a89200640 1 Processor -- start 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.904+0000 7f0a89200640 1 -- start start 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.904+0000 7f0a89200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a84199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a89200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a84102790 0x7f0a84199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a89200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84103150 0x7f0a841a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a89200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8410d0e0 con 0x7f0a841015a0 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a89200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8410cf60 con 0x7f0a84103150 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a89200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8410d260 con 0x7f0a84102790 2024-03-26T14:09:14.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a84199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a84199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40044/0 (socket says 172.21.15.137:40044) 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 -- 172.21.15.137:0/1089819848 learned_addr learned my addr 172.21.15.137:0/1089819848 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a84102790 0x7f0a84199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 -- 172.21.15.137:0/1089819848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a84102790 msgr2=0x7f0a84199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a84102790 0x7f0a84199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83e00640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84103150 0x7f0a841a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 -- 172.21.15.137:0/1089819848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84103150 msgr2=0x7f0a841a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84103150 0x7f0a841a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83400640 1 -- 172.21.15.137:0/1089819848 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a841a1640 con 0x7f0a841015a0 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.905+0000 7f0a83e00640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84103150 0x7f0a841a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:14.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.906+0000 7f0a83400640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a84199a20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0a74000c00 tx=0x7f0a74004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.906+0000 7f0a7be00640 1 -- 172.21.15.137:0/1089819848 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74043020 con 0x7f0a841015a0 2024-03-26T14:09:14.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.906+0000 7f0a7be00640 1 -- 172.21.15.137:0/1089819848 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a74004440 con 0x7f0a841015a0 2024-03-26T14:09:14.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.906+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a841a18d0 con 0x7f0a841015a0 2024-03-26T14:09:14.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.906+0000 7f0a7be00640 1 -- 172.21.15.137:0/1089819848 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74003c90 con 0x7f0a841015a0 2024-03-26T14:09:14.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.906+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841a1e20 con 0x7f0a841015a0 2024-03-26T14:09:14.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.910+0000 7f0a8d564740 1 Processor -- start 2024-03-26T14:09:14.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.910+0000 7f0a8d564740 1 -- start start 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c482d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c4832540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c4835040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a8d564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c4792250 con 0x5639c4832bb0 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a8d564740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c473aa30 con 0x5639c482d050 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a8d564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c478e2f0 con 0x5639c482da40 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a7be00640 1 -- 172.21.15.137:0/1089819848 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0a74046020 con 0x7f0a841015a0 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a82a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c4835040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c482d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a82a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c4835040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40058/0 (socket says 172.21.15.137:40058) 2024-03-26T14:09:14.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.911+0000 7f0a82a00640 1 -- 172.21.15.137:0/1029371438 learned_addr learned my addr 172.21.15.137:0/1029371438 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a82a00640 1 -- 172.21.15.137:0/1029371438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 msgr2=0x5639c4832540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c4832540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a82a00640 1 -- 172.21.15.137:0/1029371438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 msgr2=0x5639c482d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c482d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a82a00640 1 -- 172.21.15.137:0/1029371438 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639c4725690 con 0x5639c4832bb0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c4835040 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009c00 tx=0x7f0a70010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9227bf7c96adad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a78c00640 1 -- 172.21.15.137:0/1029371438 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70004020 con 0x5639c4832bb0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a78c00640 1 -- 172.21.15.137:0/1029371438 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a700041c0 con 0x5639c4832bb0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.912+0000 7f0a78c00640 1 -- 172.21.15.137:0/1029371438 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70012690 con 0x5639c4832bb0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a7be00640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0a680a2800 0x7f0a680a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a7be00640 1 -- 172.21.15.137:0/1089819848 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0a740f1200 con 0x7f0a841015a0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a83e00640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0a680a2800 0x7f0a680a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 -- 172.21.15.137:0/1029371438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 msgr2=0x5639c4835040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c4835040 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009c00 tx=0x7f0a70010740 comp rx=0 tx=0).stop 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 -- 172.21.15.137:0/1029371438 shutdown_connections 2024-03-26T14:09:14.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c4835040 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c4832540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1029371438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c482d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.913+0000 7f0a8d564740 1 -- 172.21.15.137:0/1029371438 >> 172.21.15.137:0/1029371438 conn(0x5639c47f5200 msgr2=0x5639c48289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.914+0000 7f0a8d564740 1 -- 172.21.15.137:0/1029371438 shutdown_connections 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.914+0000 7f0a8d564740 1 -- 172.21.15.137:0/1029371438 wait complete. 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.914+0000 7f0a83e00640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0a680a2800 0x7f0a680a4cc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5639c482e2a0 tx=0x7f0a7c008630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.914+0000 7f0a8d564740 1 -- start start 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.914+0000 7f0a8d564740 1 Processor -- start 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 0x5639c4837090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482da40 0x5639c48375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c4832bb0 0x5639c483e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c4835b60 con 0x5639c482d050 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c474de80 con 0x5639c482da40 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639c4835ce0 con 0x5639c4832bb0 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a8d564740 10 client.0 _ll_register_callbacks cb 0x5639c4760130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a83400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 0x5639c4837090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a83400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 0x5639c4837090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40060/0 (socket says 172.21.15.137:40060) 2024-03-26T14:09:14.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a83400640 1 -- 172.21.15.137:0/1148308339 learned_addr learned my addr 172.21.15.137:0/1148308339 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.915+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482da40 0x5639c48375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83e00640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c4832bb0 0x5639c483e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83400640 1 -- 172.21.15.137:0/1148308339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c4832bb0 msgr2=0x5639c483e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83400640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c4832bb0 0x5639c483e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83400640 1 -- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482da40 msgr2=0x5639c48375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83400640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482da40 0x5639c48375d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83400640 1 -- 172.21.15.137:0/1148308339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639c45707d0 con 0x5639c482d050 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a83400640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 0x5639c4837090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0a740bd490 tx=0x7f0a740bd390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bf0ffbc66ba86c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a6d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a6d600640 1 -- 172.21.15.137:0/1148308339 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74003e30 con 0x5639c482d050 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a6d600640 1 -- 172.21.15.137:0/1148308339 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0a740c5aa0 con 0x5639c482d050 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.916+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a700041c0 con 0x5639c482d050 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a6d600640 1 -- 172.21.15.137:0/1148308339 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7403c350 con 0x5639c482d050 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 10 client.17160 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:14.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5639c44a63f0 con 0x5639c482d050 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a6d600640 1 -- 172.21.15.137:0/1148308339 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a700041c0 con 0x5639c482d050 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a6d600640 1 -- 172.21.15.137:0/1148308339 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0a740f3110 con 0x5639c482d050 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 10 client.17160 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 20 client.17160 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c4842dd0 0x5639c4845230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 4 client.17160 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.917+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639c479f790 con 0x5639c4842dd0 2024-03-26T14:09:14.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.918+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c4842dd0 0x5639c4845230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.919+0000 7f0a82a00640 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c4842dd0 0x5639c4845230 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327b32c9229a402e server_cookie=c9fd0d7ae10f363c in_seq=0 out_seq=0 2024-03-26T14:09:14.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.919+0000 7f0a6d600640 10 client.17160 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.920+0000 7f0a6d600640 1 -- 172.21.15.137:0/1148308339 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5639c479f790 con 0x5639c4842dd0 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.920+0000 7f0a6d600640 10 client.17160 handle_command_reply: tid=0 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.920+0000 7f0a8d564740 1 client.17160 shutdown 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.920+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 msgr2=0x5639c4837090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.920+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 0x5639c4837090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0a740bd490 tx=0x7f0a740bd390 comp rx=0 tx=0).stop 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 shutdown_connections 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c4842dd0 0x5639c4845230 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482da40 0x5639c48375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c4832bb0 0x5639c483e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 --2- 172.21.15.137:0/1148308339 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482d050 0x5639c4837090 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 >> 172.21.15.137:0/1148308339 conn(0x5639c47f5200 msgr2=0x5639c4827bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a6d600640 0 client.17160 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 shutdown_connections 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 1 -- 172.21.15.137:0/1148308339 wait complete. 2024-03-26T14:09:14.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.921+0000 7f0a8d564740 20 client.17160 trim_cache size 0 max 16384 2024-03-26T14:09:14.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 Processor -- start 2024-03-26T14:09:14.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 -- start start 2024-03-26T14:09:14.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482d050 0x5639c4837530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 0x5639c483e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c4832bb0 0x5639c482c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841ae750 con 0x5639c482da40 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.945+0000 7f0a8d564740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841ac250 con 0x5639c4832bb0 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a8d564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841aa2d0 con 0x5639c482d050 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 0x5639c483e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 0x5639c483e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:40070/0 (socket says 172.21.15.137:40070) 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 -- 172.21.15.137:0/386764353 learned_addr learned my addr 172.21.15.137:0/386764353 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a83e00640 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c4832bb0 0x5639c482c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a83400640 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482d050 0x5639c4837530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 -- 172.21.15.137:0/386764353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482d050 msgr2=0x5639c4837530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482d050 0x5639c4837530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 -- 172.21.15.137:0/386764353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c4832bb0 msgr2=0x5639c482c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c4832bb0 0x5639c482c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.946+0000 7f0a82a00640 1 -- 172.21.15.137:0/386764353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639c44660f0 con 0x5639c482da40 2024-03-26T14:09:14.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.947+0000 7f0a82a00640 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 0x5639c483e5a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x5639c4829910 tx=0x7f0a70010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4958f41fbc5810 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.947+0000 7f0a6fe00640 1 -- 172.21.15.137:0/386764353 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700292c0 con 0x5639c482da40 2024-03-26T14:09:14.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.947+0000 7f0a6fe00640 1 -- 172.21.15.137:0/386764353 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a700054f0 con 0x5639c482da40 2024-03-26T14:09:14.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.947+0000 7f0a6fe00640 1 -- 172.21.15.137:0/386764353 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a700048c0 con 0x5639c482da40 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 -- 172.21.15.137:0/386764353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 msgr2=0x5639c483e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 0x5639c483e5a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x5639c4829910 tx=0x7f0a70010fd0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 -- 172.21.15.137:0/386764353 shutdown_connections 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c4832bb0 0x5639c482c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c482da40 0x5639c483e5a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 --2- 172.21.15.137:0/386764353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482d050 0x5639c4837530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 -- 172.21.15.137:0/386764353 >> 172.21.15.137:0/386764353 conn(0x5639c47f5200 msgr2=0x5639c482bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 -- 172.21.15.137:0/386764353 shutdown_connections 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.948+0000 7f0a8d564740 1 -- 172.21.15.137:0/386764353 wait complete. 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.949+0000 7f0a8d564740 1 -- start start 2024-03-26T14:09:14.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.949+0000 7f0a8d564740 1 Processor -- start 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c483bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c483c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c483d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841aa8d0 con 0x5639c4832bb0 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841abbf0 con 0x5639c482d050 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841aaf30 con 0x5639c482da40 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a8d564740 10 client.0 _ll_register_callbacks cb 0x7f0a740b9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c483d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c483bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c483bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:58798/0 (socket says 172.21.15.137:58798) 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83400640 1 -- 172.21.15.137:0/2208658228 learned_addr learned my addr 172.21.15.137:0/2208658228 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a82a00640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c483c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83e00640 1 -- 172.21.15.137:0/2208658228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 msgr2=0x5639c483c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83e00640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c483c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83e00640 1 -- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 msgr2=0x5639c483bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:14.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83e00640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c483bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83e00640 1 -- 172.21.15.137:0/2208658228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639c483da90 con 0x5639c4832bb0 2024-03-26T14:09:14.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.950+0000 7f0a83400640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c483bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:09:14.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a83e00640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c483d4a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c008cc0 tx=0x7f0a7c012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7e3e967686d289 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:14.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a6d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:14.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a6d600640 1 -- 172.21.15.137:0/2208658228 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c029060 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a6d600640 1 -- 172.21.15.137:0/2208658228 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0a7c024600 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a700054f0 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a6d600640 1 -- 172.21.15.137:0/2208658228 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a7c030050 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a8d564740 10 client.17166 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.951+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5639c489eac0 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a6d600640 1 -- 172.21.15.137:0/2208658228 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a700054f0 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a6d600640 1 -- 172.21.15.137:0/2208658228 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0a7c017040 con 0x5639c4832bb0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a8d564740 10 client.17166 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a8d564740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a8d564740 20 client.17166 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a8d564740 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c489f4d0 0x5639c48a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a8d564740 4 client.17166 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.952+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5639c4799ae0 con 0x5639c489f4d0 2024-03-26T14:09:14.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.953+0000 7f0a83400640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c489f4d0 0x5639c48a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.953+0000 7f0a83400640 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c489f4d0 0x5639c48a18f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695418fdfb1ae7b5 server_cookie=bbb2142761ccd7a3 in_seq=0 out_seq=0 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.953+0000 7f0a6d600640 10 client.17166 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a6d600640 1 -- 172.21.15.137:0/2208658228 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5639c4799ae0 con 0x5639c489f4d0 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a6d600640 10 client.17166 handle_command_reply: tid=0 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 client.17166 shutdown 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 msgr2=0x5639c483d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c483d4a0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c008cc0 tx=0x7f0a7c012040 comp rx=0 tx=0).stop 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 shutdown_connections 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5639c489f4d0 0x5639c48a18f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5639c4832bb0 0x5639c483d4a0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c482da40 0x5639c483c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 --2- 172.21.15.137:0/2208658228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5639c482d050 0x5639c483bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 >> 172.21.15.137:0/2208658228 conn(0x5639c47f5200 msgr2=0x5639c482c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a6d600640 0 client.17166 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 shutdown_connections 2024-03-26T14:09:14.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 1 -- 172.21.15.137:0/2208658228 wait complete. 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.954+0000 7f0a8d564740 20 client.17166 trim_cache size 0 max 16384 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: "4b159800-169f-4070-9949-5540fbea10a6": { 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "4b159800-169f-4070-9949-5540fbea10a6", 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:09:14.957 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0a680a2800 msgr2=0x7f0a680a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0a680a2800 0x7f0a680a4cc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5639c482e2a0 tx=0x7f0a7c008630 comp rx=0 tx=0).stop 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 msgr2=0x7f0a84199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a84199a20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0a74000c00 tx=0x7f0a74004040 comp rx=0 tx=0).stop 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 shutdown_connections 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0a680a2800 0x7f0a680a4cc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a84103150 0x7f0a841a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a84102790 0x7f0a84199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 --2- 172.21.15.137:0/1089819848 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0a841015a0 0x7f0a84199a20 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:14.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 >> 172.21.15.137:0/1089819848 conn(0x7f0a840fcb10 msgr2=0x7f0a84107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:14.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 shutdown_connections 2024-03-26T14:09:14.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:14.955+0000 7f0a89200640 1 -- 172.21.15.137:0/1089819848 wait complete. 2024-03-26T14:09:14.977 INFO:tasks.cephfs.filesystem:scrub status for tag:4b159800-169f-4070-9949-5540fbea10a6 - {'path': '/', 'tag': '4b159800-169f-4070-9949-5540fbea10a6', 'options': 'recursive,force'} 2024-03-26T14:09:15.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:15 smithi179 ceph-mon[35472]: pgmap v664: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:15.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:15 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:09:15.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:15 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:15.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:15 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:09:15.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:15 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:15.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:15 smithi159 ceph-mon[35501]: pgmap v664: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:15.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:15 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:09:15.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:15 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:15.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:15 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:09:15.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:15 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:15.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:15 smithi137 ceph-mon[31539]: pgmap v664: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:15.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:15 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:09:15.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:15 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:15.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:15 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:09:15.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:15 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:17.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:17 smithi179 ceph-mon[35472]: pgmap v665: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:17.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:17 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:09:17.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:17 smithi159 ceph-mon[35501]: pgmap v665: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:17.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:17 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:09:17.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:17 smithi137 ceph-mon[31539]: pgmap v665: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:17.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:17 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:09:19.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:19 smithi179 ceph-mon[35472]: pgmap v666: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:09:19.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:19 smithi159 ceph-mon[35501]: pgmap v666: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:09:19.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:19 smithi137 ceph-mon[31539]: pgmap v666: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:09:21.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:21 smithi179 ceph-mon[35472]: pgmap v667: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:21.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:21 smithi159 ceph-mon[35501]: pgmap v667: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:21.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:21 smithi137 ceph-mon[31539]: pgmap v667: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:23.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:23 smithi179 ceph-mon[35472]: pgmap v668: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:09:23.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:23 smithi159 ceph-mon[35501]: pgmap v668: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:09:23.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:23 smithi137 ceph-mon[31539]: pgmap v668: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:09:25.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:25 smithi179 ceph-mon[35472]: pgmap v669: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:09:25.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:25 smithi159 ceph-mon[35501]: pgmap v669: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:09:25.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:25 smithi137 ceph-mon[31539]: pgmap v669: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:09:27.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:27 smithi179 ceph-mon[35472]: pgmap v670: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:27.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:27 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:09:27.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:27 smithi159 ceph-mon[35501]: pgmap v670: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:27.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:27 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:09:27.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:27 smithi137 ceph-mon[31539]: pgmap v670: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:27.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:27 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:09:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:09:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:09:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:28 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:09:28.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:09:28.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:09:28.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:28 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:09:28.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:09:28.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:09:28.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:28 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:09:29.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:29 smithi179 ceph-mon[35472]: pgmap v671: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:09:29.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:29 smithi159 ceph-mon[35501]: pgmap v671: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:09:29.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:29 smithi137 ceph-mon[31539]: pgmap v671: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:09:31.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:31 smithi179 ceph-mon[35472]: pgmap v672: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:09:31.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:31 smithi159 ceph-mon[35501]: pgmap v672: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:09:31.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:31 smithi137 ceph-mon[31539]: pgmap v672: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:09:33.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:33 smithi179 ceph-mon[35472]: pgmap v673: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:09:33.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:33 smithi159 ceph-mon[35501]: pgmap v673: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:09:33.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:33 smithi137 ceph-mon[31539]: pgmap v673: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:09:35.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:35 smithi179 ceph-mon[35472]: pgmap v674: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:09:35.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:35 smithi159 ceph-mon[35501]: pgmap v674: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:09:35.534 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:35 smithi137 ceph-mon[31539]: pgmap v674: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:09:37.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:37 smithi179 ceph-mon[35472]: pgmap v675: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:09:37.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:37 smithi159 ceph-mon[35501]: pgmap v675: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:09:37.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:37 smithi137 ceph-mon[31539]: pgmap v675: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:09:39.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:39 smithi179 ceph-mon[35472]: pgmap v676: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:09:39.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:39 smithi159 ceph-mon[35501]: pgmap v676: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:09:39.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:39 smithi137 ceph-mon[31539]: pgmap v676: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:09:41.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:41 smithi179 ceph-mon[35472]: pgmap v677: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:41.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:41 smithi159 ceph-mon[35501]: pgmap v677: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:41.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:41 smithi137 ceph-mon[31539]: pgmap v677: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:43.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:43 smithi179 ceph-mon[35472]: pgmap v678: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:43.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:43 smithi159 ceph-mon[35501]: pgmap v678: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:43.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:43 smithi137 ceph-mon[31539]: pgmap v678: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:09:44.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:09:44.978 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 -- 172.21.15.137:0/1547934747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc102000 msgr2=0x7f93bc1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 --2- 172.21.15.137:0/1547934747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc102000 0x7f93bc1043f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f93ac009860 tx=0x7f93ac02bb70 comp rx=0 tx=0).stop 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 -- 172.21.15.137:0/1547934747 shutdown_connections 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 --2- 172.21.15.137:0/1547934747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc104930 0x7f93bc106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 --2- 172.21.15.137:0/1547934747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc107260 0x7f93bc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 --2- 172.21.15.137:0/1547934747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc102000 0x7f93bc1043f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.208+0000 7f93c1400640 1 -- 172.21.15.137:0/1547934747 >> 172.21.15.137:0/1547934747 conn(0x7f93bc0fba10 msgr2=0x7f93bc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.209+0000 7f93c1400640 1 -- 172.21.15.137:0/1547934747 shutdown_connections 2024-03-26T14:09:45.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.209+0000 7f93c1400640 1 -- 172.21.15.137:0/1547934747 wait complete. 2024-03-26T14:09:45.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 Processor -- start 2024-03-26T14:09:45.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 -- start start 2024-03-26T14:09:45.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 0x7f93bc101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc104930 0x7f93bc0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc107260 0x7f93bc1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc10d0c0 con 0x7f93bc102000 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc10cf40 con 0x7f93bc104930 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.210+0000 7f93c1400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc10d240 con 0x7f93bc107260 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 0x7f93bc101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 0x7f93bc101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57768/0 (socket says 172.21.15.137:57768) 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 -- 172.21.15.137:0/2098105017 learned_addr learned my addr 172.21.15.137:0/2098105017 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bb400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc104930 0x7f93bc0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.214 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93c0a00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc107260 0x7f93bc1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 -- 172.21.15.137:0/2098105017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc107260 msgr2=0x7f93bc1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc107260 0x7f93bc1004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 -- 172.21.15.137:0/2098105017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc104930 msgr2=0x7f93bc0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc104930 0x7f93bc0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93bbe00640 1 -- 172.21.15.137:0/2098105017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc100d80 con 0x7f93bc102000 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.211+0000 7f93c0a00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc107260 0x7f93bc1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.212+0000 7f93bbe00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 0x7f93bc101970 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005940 tx=0x7f93ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.212+0000 7f93b8c00640 1 -- 172.21.15.137:0/2098105017 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac041020 con 0x7f93bc102000 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.212+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1a1860 con 0x7f93bc102000 2024-03-26T14:09:45.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.212+0000 7f93b8c00640 1 -- 172.21.15.137:0/2098105017 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93ac02c9c0 con 0x7f93bc102000 2024-03-26T14:09:45.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.212+0000 7f93b8c00640 1 -- 172.21.15.137:0/2098105017 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac039040 con 0x7f93bc102000 2024-03-26T14:09:45.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.212+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc1a1e00 con 0x7f93bc102000 2024-03-26T14:09:45.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.216+0000 7f93c57db740 1 Processor -- start 2024-03-26T14:09:45.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 -- start start 2024-03-26T14:09:45.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93b8c00640 1 -- 172.21.15.137:0/2098105017 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f93ac044020 con 0x7f93bc102000 2024-03-26T14:09:45.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b35a40 0x55df90b3a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 0x55df90b3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df90a9a250 con 0x55df90b3abb0 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df90a42a30 con 0x55df90b35a40 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93c57db740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df90a962f0 con 0x55df90b35050 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 0x55df90b3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 0x55df90b3d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57770/0 (socket says 172.21.15.137:57770) 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.217+0000 7f93bb400640 1 -- 172.21.15.137:0/3085512366 learned_addr learned my addr 172.21.15.137:0/3085512366 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bbe00640 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b35a40 0x55df90b3a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93c0a00640 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bb400640 1 -- 172.21.15.137:0/3085512366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 msgr2=0x55df90b35470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bb400640 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b35470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bb400640 1 -- 172.21.15.137:0/3085512366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b35a40 msgr2=0x55df90b3a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bb400640 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b35a40 0x55df90b3a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bb400640 1 -- 172.21.15.137:0/3085512366 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df90a2d690 con 0x55df90b3abb0 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.218+0000 7f93bb400640 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 0x55df90b3d040 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009be0 tx=0x7f93a80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439b1a70695e4051 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.219+0000 7f93b8c00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93a00a2800 0x7f93a00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.219+0000 7f93b1200640 1 -- 172.21.15.137:0/3085512366 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004020 con 0x55df90b3abb0 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.219+0000 7f93b1200640 1 -- 172.21.15.137:0/3085512366 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93a80041c0 con 0x55df90b3abb0 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.219+0000 7f93b1200640 1 -- 172.21.15.137:0/3085512366 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80126b0 con 0x55df90b3abb0 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.219+0000 7f93b8c00640 1 -- 172.21.15.137:0/2098105017 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f93ac0f0900 con 0x7f93bc102000 2024-03-26T14:09:45.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.219+0000 7f93c0a00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93a00a2800 0x7f93a00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 -- 172.21.15.137:0/3085512366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 msgr2=0x55df90b3d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 0x55df90b3d040 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009be0 tx=0x7f93a80117b0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 -- 172.21.15.137:0/3085512366 shutdown_connections 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b35a40 0x55df90b3a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b3abb0 0x55df90b3d040 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 --2- 172.21.15.137:0/3085512366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b35470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 -- 172.21.15.137:0/3085512366 >> 172.21.15.137:0/3085512366 conn(0x55df90afd200 msgr2=0x55df90b309c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c0a00640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93a00a2800 0x7f93a00a4cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55df90b362a0 tx=0x7f93b4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 -- 172.21.15.137:0/3085512366 shutdown_connections 2024-03-26T14:09:45.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 -- 172.21.15.137:0/3085512366 wait complete. 2024-03-26T14:09:45.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.220+0000 7f93c57db740 1 -- start start 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 Processor -- start 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 0x55df90b3f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df90b3db60 con 0x55df90b35a40 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df90a55e80 con 0x55df90b3abb0 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df90b3dce0 con 0x55df90b35050 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.221+0000 7f93c57db740 10 client.0 _ll_register_callbacks cb 0x55df90a68130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:45.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 0x55df90b3f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 0x55df90b3f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57778/0 (socket says 172.21.15.137:57778) 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 -- 172.21.15.137:0/4159629970 learned_addr learned my addr 172.21.15.137:0/4159629970 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bbe00640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93c0a00640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 -- 172.21.15.137:0/4159629970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 msgr2=0x55df90b3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b3f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 -- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 msgr2=0x55df90b465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 -- 172.21.15.137:0/4159629970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df908787d0 con 0x55df90b35a40 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93bb400640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 0x55df90b3f5f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f93a8000c00 tx=0x7f93a8009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2b49a561365b3b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.222+0000 7f93a5600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.223+0000 7f93a5600640 1 -- 172.21.15.137:0/4159629970 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8028020 con 0x55df90b35a40 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.223+0000 7f93a5600640 1 -- 172.21.15.137:0/4159629970 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f93a80037f0 con 0x55df90b35a40 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.223+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a80041c0 con 0x55df90b35a40 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.223+0000 7f93a5600640 1 -- 172.21.15.137:0/4159629970 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a8033050 con 0x55df90b35a40 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.223+0000 7f93c57db740 10 client.17178 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.223+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55df907ae3f0 con 0x55df90b35a40 2024-03-26T14:09:45.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93a5600640 1 -- 172.21.15.137:0/4159629970 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93a80041c0 con 0x55df90b35a40 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93a5600640 1 -- 172.21.15.137:0/4159629970 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f93a80244a0 con 0x55df90b35a40 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93c57db740 10 client.17178 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93c57db740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93c57db740 20 client.17178 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93c57db740 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90b4ab60 0x55df90b4cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93c57db740 4 client.17178 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.224+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df90aa7790 con 0x55df90b4ab60 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.225+0000 7f93bbe00640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90b4ab60 0x55df90b4cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.226+0000 7f93a5600640 10 client.17178 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.225+0000 7f93bbe00640 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90b4ab60 0x55df90b4cf50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c090aa03b0d924e server_cookie=6b2c588eac243ade in_seq=0 out_seq=0 2024-03-26T14:09:45.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93a5600640 1 -- 172.21.15.137:0/4159629970 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df90aa7790 con 0x55df90b4ab60 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93a5600640 10 client.17178 handle_command_reply: tid=0 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 client.17178 shutdown 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 msgr2=0x55df90b3f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 0x55df90b3f5f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f93a8000c00 tx=0x7f93a8009930 comp rx=0 tx=0).stop 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 shutdown_connections 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90b4ab60 0x55df90b4cf50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35a40 0x55df90b3f5f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 --2- 172.21.15.137:0/4159629970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35050 0x55df90b3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93a5600640 0 client.17178 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:45.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.228+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 >> 172.21.15.137:0/4159629970 conn(0x55df90afd200 msgr2=0x55df90b2fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.229+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 shutdown_connections 2024-03-26T14:09:45.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.229+0000 7f93c57db740 1 -- 172.21.15.137:0/4159629970 wait complete. 2024-03-26T14:09:45.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.229+0000 7f93c57db740 20 client.17178 trim_cache size 0 max 16384 2024-03-26T14:09:45.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.249+0000 7f93c57db740 1 Processor -- start 2024-03-26T14:09:45.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 -- start start 2024-03-26T14:09:45.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b30bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b33c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1ae710 con 0x55df90b35050 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1ac210 con 0x55df90b3abb0 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93c57db740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1aa290 con 0x55df90b35a40 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b30bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b30bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57792/0 (socket says 172.21.15.137:57792) 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.250+0000 7f93bbe00640 1 -- 172.21.15.137:0/2964724634 learned_addr learned my addr 172.21.15.137:0/2964724634 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bb400640 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93c0a00640 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b33c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bbe00640 1 -- 172.21.15.137:0/2964724634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 msgr2=0x55df90b46580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bbe00640 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b46580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bbe00640 1 -- 172.21.15.137:0/2964724634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 msgr2=0x55df90b33c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bbe00640 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b33c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bbe00640 1 -- 172.21.15.137:0/2964724634 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df9076e0f0 con 0x55df90b35050 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93c0a00640 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93bbe00640 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b30bf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005b60 tx=0x7f93ac0edec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c06e26f5fde4a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93a7e00640 1 -- 172.21.15.137:0/2964724634 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0bd590 con 0x55df90b35050 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93a7e00640 1 -- 172.21.15.137:0/2964724634 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93ac0bce90 con 0x55df90b35050 2024-03-26T14:09:45.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.251+0000 7f93a7e00640 1 -- 172.21.15.137:0/2964724634 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0b9500 con 0x55df90b35050 2024-03-26T14:09:45.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 -- 172.21.15.137:0/2964724634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 msgr2=0x55df90b30bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b30bf0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005b60 tx=0x7f93ac0edec0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 -- 172.21.15.137:0/2964724634 shutdown_connections 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b33c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 --2- 172.21.15.137:0/2964724634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b30bf0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 -- 172.21.15.137:0/2964724634 >> 172.21.15.137:0/2964724634 conn(0x55df90afd200 msgr2=0x55df90b304e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 -- 172.21.15.137:0/2964724634 shutdown_connections 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.252+0000 7f93c57db740 1 -- 172.21.15.137:0/2964724634 wait complete. 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 -- start start 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 Processor -- start 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b43cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b451b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1aa890 con 0x55df90b35050 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1abbb0 con 0x55df90b3abb0 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1aaef0 con 0x55df90b35a40 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.253+0000 7f93c57db740 10 client.0 _ll_register_callbacks cb 0x7f93a8009a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b43cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b43cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57804/0 (socket says 172.21.15.137:57804) 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 -- 172.21.15.137:0/1534693727 learned_addr learned my addr 172.21.15.137:0/1534693727 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bb400640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93c0a00640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 -- 172.21.15.137:0/1534693727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 msgr2=0x55df90b441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 -- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 msgr2=0x55df90b451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 -- 172.21.15.137:0/1534693727 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df90b457a0 con 0x55df90b35050 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93bbe00640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b43cb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0bb3e0 tx=0x7f93ac005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240a22e81acb7754 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.254+0000 7f93a5600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93a5600640 1 -- 172.21.15.137:0/1534693727 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0bd590 con 0x55df90b35050 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93a5600640 1 -- 172.21.15.137:0/1534693727 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f93ac0c51a0 con 0x55df90b35050 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93ac0bce90 con 0x55df90b35050 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93a5600640 1 -- 172.21.15.137:0/1534693727 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93ac0bb190 con 0x55df90b35050 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93c57db740 10 client.17184 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55df90ba63d0 con 0x55df90b35050 2024-03-26T14:09:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93a5600640 1 -- 172.21.15.137:0/1534693727 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f93ac0bce90 con 0x55df90b35050 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93a5600640 1 -- 172.21.15.137:0/1534693727 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f93ac03c030 con 0x55df90b35050 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93c57db740 10 client.17184 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93c57db740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.255+0000 7f93c57db740 20 client.17184 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.256+0000 7f93c57db740 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90ba6bb0 0x55df90ba8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.256+0000 7f93c57db740 4 client.17184 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.256+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df90aa1ae0 con 0x55df90ba6bb0 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.256+0000 7f93bb400640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90ba6bb0 0x55df90ba8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93bb400640 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90ba6bb0 0x55df90ba8fd0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff9580fde23d1bb9 server_cookie=64ba8b731349d7cf in_seq=0 out_seq=0 2024-03-26T14:09:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93a5600640 10 client.17184 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93a5600640 1 -- 172.21.15.137:0/1534693727 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55df90aa1ae0 con 0x55df90ba6bb0 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93a5600640 10 client.17184 handle_command_reply: tid=0 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 client.17184 shutdown 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 msgr2=0x55df90b43cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b43cb0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0bb3e0 tx=0x7f93ac005750 comp rx=0 tx=0).stop 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 shutdown_connections 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55df90ba6bb0 0x55df90ba8fd0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55df90b3abb0 0x55df90b451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.257+0000 7f93c57db740 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55df90b35a40 0x55df90b441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.258+0000 7f93c57db740 1 --2- 172.21.15.137:0/1534693727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55df90b35050 0x55df90b43cb0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.258+0000 7f93a5600640 0 client.17184 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.258+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 >> 172.21.15.137:0/1534693727 conn(0x55df90afd200 msgr2=0x55df90b33c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.258+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 shutdown_connections 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.258+0000 7f93c57db740 1 -- 172.21.15.137:0/1534693727 wait complete. 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.258+0000 7f93c57db740 20 client.17184 trim_cache size 0 max 16384 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:09:45.264 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:09:45.265 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:09:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93a00a2800 msgr2=0x7f93a00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93a00a2800 0x7f93a00a4cc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x55df90b362a0 tx=0x7f93b4009500 comp rx=0 tx=0).stop 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 msgr2=0x7f93bc101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 0x7f93bc101970 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005940 tx=0x7f93ac004270 comp rx=0 tx=0).stop 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 shutdown_connections 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f93a00a2800 0x7f93a00a4cc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93bc104930 0x7f93bc0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93bc107260 0x7f93bc1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 --2- 172.21.15.137:0/2098105017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93bc102000 0x7f93bc101970 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 >> 172.21.15.137:0/2098105017 conn(0x7f93bc0fba10 msgr2=0x7f93bc102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 shutdown_connections 2024-03-26T14:09:45.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.259+0000 7f93c1400640 1 -- 172.21.15.137:0/2098105017 wait complete. 2024-03-26T14:09:45.277 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:09:45.277 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:09:45.347 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:45 smithi137 ceph-mon[31539]: pgmap v679: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:45.423 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:45 smithi179 ceph-mon[35472]: pgmap v679: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:45.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:45 smithi159 ceph-mon[35501]: pgmap v679: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:09:45.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 -- 172.21.15.137:0/2660467532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c107420 msgr2=0x7fd16c109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 --2- 172.21.15.137:0/2660467532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c107420 0x7fd16c109870 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd164009ca0 tx=0x7fd16402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 -- 172.21.15.137:0/2660467532 shutdown_connections 2024-03-26T14:09:45.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 --2- 172.21.15.137:0/2660467532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c107420 0x7fd16c109870 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 --2- 172.21.15.137:0/2660467532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c104af0 0x7fd16c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 --2- 172.21.15.137:0/2660467532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c1021c0 0x7fd16c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 -- 172.21.15.137:0/2660467532 >> 172.21.15.137:0/2660467532 conn(0x7fd16c0fbbd0 msgr2=0x7fd16c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 -- 172.21.15.137:0/2660467532 shutdown_connections 2024-03-26T14:09:45.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.485+0000 7fd170a00640 1 -- 172.21.15.137:0/2660467532 wait complete. 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.486+0000 7fd170a00640 1 Processor -- start 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.486+0000 7fd170a00640 1 -- start start 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.486+0000 7fd170a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c1021c0 0x7fd16c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.486+0000 7fd170a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 0x7fd16c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.486+0000 7fd170a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c107420 0x7fd16c1a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd170a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c10d220 con 0x7fd16c104af0 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd170a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c10d0a0 con 0x7fd16c107420 2024-03-26T14:09:45.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd170a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c10d3a0 con 0x7fd16c1021c0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 0x7fd16c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 0x7fd16c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57816/0 (socket says 172.21.15.137:57816) 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 -- 172.21.15.137:0/3043086592 learned_addr learned my addr 172.21.15.137:0/3043086592 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd16a800640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c107420 0x7fd16c1a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169e00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c1021c0 0x7fd16c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 -- 172.21.15.137:0/3043086592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c1021c0 msgr2=0x7fd16c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c1021c0 0x7fd16c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 -- 172.21.15.137:0/3043086592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c107420 msgr2=0x7fd16c1a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c107420 0x7fd16c1a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd169400640 1 -- 172.21.15.137:0/3043086592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c1a16f0 con 0x7fd16c104af0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.487+0000 7fd16a800640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c107420 0x7fd16c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.488+0000 7fd169400640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 0x7fd16c19a010 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e9f0 tx=0x7fd15800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.488+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15800ccc0 con 0x7fd16c104af0 2024-03-26T14:09:45.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.488+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c1a19e0 con 0x7fd16c104af0 2024-03-26T14:09:45.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.488+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd1580054d0 con 0x7fd16c104af0 2024-03-26T14:09:45.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.488+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd158010430 con 0x7fd16c104af0 2024-03-26T14:09:45.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.488+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c1a1f30 con 0x7fd16c104af0 2024-03-26T14:09:45.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.489+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c108f10 con 0x7fd16c104af0 2024-03-26T14:09:45.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.493+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd158005000 con 0x7fd16c104af0 2024-03-26T14:09:45.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.494+0000 7fd162a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1500a2800 0x7fd1500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:45.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.494+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd1580cf4a0 con 0x7fd16c104af0 2024-03-26T14:09:45.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.494+0000 7fd169e00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1500a2800 0x7fd1500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:45.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.495+0000 7fd169e00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1500a2800 0x7fd1500a4cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009bd0 tx=0x7fd15c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:45.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.496+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd158053740 con 0x7fd16c104af0 2024-03-26T14:09:45.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.764+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd16c02d2c0 con 0x7fd16c104af0 2024-03-26T14:09:45.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.766+0000 7fd162a00640 1 -- 172.21.15.137:0/3043086592 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fd1580052f0 con 0x7fd16c104af0 2024-03-26T14:09:45.766 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:09:45.767 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:09:45.767 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:09:45.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1500a2800 msgr2=0x7fd1500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1500a2800 0x7fd1500a4cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009bd0 tx=0x7fd15c005f90 comp rx=0 tx=0).stop 2024-03-26T14:09:45.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 msgr2=0x7fd16c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:45.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 0x7fd16c19a010 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd15800e9f0 tx=0x7fd15800eeb0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 shutdown_connections 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1500a2800 0x7fd1500a4cc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd16c107420 0x7fd16c1a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd16c104af0 0x7fd16c19a010 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 --2- 172.21.15.137:0/3043086592 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd16c1021c0 0x7fd16c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 >> 172.21.15.137:0/3043086592 conn(0x7fd16c0fbbd0 msgr2=0x7fd16c1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 shutdown_connections 2024-03-26T14:09:45.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:45.769+0000 7fd170a00640 1 -- 172.21.15.137:0/3043086592 wait complete. 2024-03-26T14:09:45.789 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:09:45.789 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:09:46.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 -- 172.21.15.137:0/3149301551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 msgr2=0x7f6f84105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 --2- 172.21.15.137:0/3149301551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f84105590 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009ca0 tx=0x7f6f7c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 -- 172.21.15.137:0/3149301551 shutdown_connections 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 --2- 172.21.15.137:0/3149301551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 0x7f6f84104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 --2- 172.21.15.137:0/3149301551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f84105590 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 --2- 172.21.15.137:0/3149301551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f840ffdf0 0x7f6f84102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.006+0000 7f6f83e00640 1 -- 172.21.15.137:0/3149301551 >> 172.21.15.137:0/3149301551 conn(0x7f6f840fba10 msgr2=0x7f6f840fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.007+0000 7f6f83e00640 1 -- 172.21.15.137:0/3149301551 shutdown_connections 2024-03-26T14:09:46.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.007+0000 7f6f83e00640 1 -- 172.21.15.137:0/3149301551 wait complete. 2024-03-26T14:09:46.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.007+0000 7f6f83e00640 1 Processor -- start 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 -- start start 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f840ffdf0 0x7f6f8419dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 0x7f6f8419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f841a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84113680 con 0x7f6f84105110 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84113500 con 0x7f6f84102760 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f84113800 con 0x7f6f840ffdf0 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f841a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f841a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57848/0 (socket says 172.21.15.137:57848) 2024-03-26T14:09:46.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.008+0000 7f6f83400640 1 -- 172.21.15.137:0/629191511 learned_addr learned my addr 172.21.15.137:0/629191511 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f82000640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 0x7f6f8419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f82a00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f840ffdf0 0x7f6f8419dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f83400640 1 -- 172.21.15.137:0/629191511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f840ffdf0 msgr2=0x7f6f8419dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f83400640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f840ffdf0 0x7f6f8419dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f83400640 1 -- 172.21.15.137:0/629191511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 msgr2=0x7f6f8419e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f83400640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 0x7f6f8419e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f83400640 1 -- 172.21.15.137:0/629191511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f841a5960 con 0x7f6f84105110 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f82000640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 0x7f6f8419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.009+0000 7f6f83400640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f841a52c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02c700 tx=0x7f6f7c0328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.010+0000 7f6f88a00640 1 -- 172.21.15.137:0/629191511 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c041020 con 0x7f6f84105110 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.010+0000 7f6f88a00640 1 -- 172.21.15.137:0/629191511 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f7c003e20 con 0x7f6f84105110 2024-03-26T14:09:46.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.010+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f841a5bf0 con 0x7f6f84105110 2024-03-26T14:09:46.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.010+0000 7f6f88a00640 1 -- 172.21.15.137:0/629191511 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c039040 con 0x7f6f84105110 2024-03-26T14:09:46.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.010+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f841a6130 con 0x7f6f84105110 2024-03-26T14:09:46.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.013+0000 7f6f88a00640 1 -- 172.21.15.137:0/629191511 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6f7c044020 con 0x7f6f84105110 2024-03-26T14:09:46.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 Processor -- start 2024-03-26T14:09:46.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 -- start start 2024-03-26T14:09:46.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ce6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0ce8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a0c4b2c0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a0bf3c90 con 0x55f3a0ce7bc0 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.014+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a0c47360 con 0x55f3a0ce8580 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ce6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ce6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57862/0 (socket says 172.21.15.137:57862) 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 -- 172.21.15.137:0/138762427 learned_addr learned my addr 172.21.15.137:0/138762427 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82a00640 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0ce8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f88a00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f680a2880 0x7f6f680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f88a00640 1 -- 172.21.15.137:0/629191511 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6f7c0f1370 con 0x7f6f84105110 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 -- 172.21.15.137:0/138762427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 msgr2=0x55f3a0ceee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 -- 172.21.15.137:0/138762427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 msgr2=0x55f3a0ce8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0ce8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f83400640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f680a2880 0x7f6f680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.015+0000 7f6f82000640 1 -- 172.21.15.137:0/138762427 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a0c18080 con 0x55f3a0ce69f0 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.016+0000 7f6f82000640 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ce6e10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000bfd0 tx=0x7f6f7000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35803e80170e4b2d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.016+0000 7f6f6fe00640 1 -- 172.21.15.137:0/138762427 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70019070 con 0x55f3a0ce69f0 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.016+0000 7f6f83400640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f680a2880 0x7f6f680a4d40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0ba280 tx=0x7f6f7c0bb950 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.016+0000 7f6f6fe00640 1 -- 172.21.15.137:0/138762427 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f700054d0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.090 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.016+0000 7f6f6fe00640 1 -- 172.21.15.137:0/138762427 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70004c40 con 0x55f3a0ce69f0 2024-03-26T14:09:46.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/138762427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 msgr2=0x55f3a0ce6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ce6e10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000bfd0 tx=0x7f6f7000b6f0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/138762427 shutdown_connections 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0ce8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/138762427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ce6e10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/138762427 >> 172.21.15.137:0/138762427 conn(0x55f3a0ce1de0 msgr2=0x55f3a0ce21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/138762427 shutdown_connections 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.017+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/138762427 wait complete. 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 -- start start 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 Processor -- start 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a0cef590 con 0x55f3a0ce8580 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a0c070d0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.018+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3a0cef710 con 0x55f3a0ce7bc0 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f8cbf3740 10 client.0 _ll_register_callbacks cb 0x55f3a0c0be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57868/0 (socket says 172.21.15.137:57868) 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 -- 172.21.15.137:0/1302007444 learned_addr learned my addr 172.21.15.137:0/1302007444 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82a00640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 -- 172.21.15.137:0/1302007444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce7bc0 msgr2=0x55f3a0cf11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 -- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce69f0 msgr2=0x55f3a0cf0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.019+0000 7f6f82000640 1 -- 172.21.15.137:0/1302007444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3a095f3f0 con 0x55f3a0ce8580 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f82000640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf8210 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000bee0 tx=0x7f6f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02799e20f34d964 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f6cc00640 1 -- 172.21.15.137:0/1302007444 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70019070 con 0x55f3a0ce8580 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f6cc00640 1 -- 172.21.15.137:0/1302007444 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6f70004bc0 con 0x55f3a0ce8580 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f700054d0 con 0x55f3a0ce8580 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f6cc00640 1 -- 172.21.15.137:0/1302007444 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f70034050 con 0x55f3a0ce8580 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f8cbf3740 10 client.17202 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.020+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f3a0a297d0 con 0x55f3a0ce8580 2024-03-26T14:09:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f6cc00640 1 -- 172.21.15.137:0/1302007444 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f700054d0 con 0x55f3a0ce8580 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f6cc00640 1 -- 172.21.15.137:0/1302007444 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6f70029020 con 0x55f3a0ce8580 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f8cbf3740 10 client.17202 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f8cbf3740 10 client.17202 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f8cbf3740 20 client.17202 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0cfc520 0x55f3a0cfe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f8cbf3740 4 client.17202 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.021+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f3a0c58800 con 0x55f3a0cfc520 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.022+0000 7f6f82a00640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0cfc520 0x55f3a0cfe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.023+0000 7f6f82a00640 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0cfc520 0x55f3a0cfe910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8978427246c5e381 server_cookie=c43154a67497055d in_seq=0 out_seq=0 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.023+0000 7f6f6cc00640 10 client.17202 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f6cc00640 1 -- 172.21.15.137:0/1302007444 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f3a0c58800 con 0x55f3a0cfc520 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f6cc00640 10 client.17202 handle_command_reply: tid=0 2024-03-26T14:09:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 client.17202 shutdown 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 msgr2=0x55f3a0cf8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf8210 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000bee0 tx=0x7f6f700046c0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 shutdown_connections 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0cfc520 0x55f3a0cfe910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf8210 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1302007444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 >> 172.21.15.137:0/1302007444 conn(0x55f3a0ce1de0 msgr2=0x55f3a0ce1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f6cc00640 0 client.17202 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 shutdown_connections 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1302007444 wait complete. 2024-03-26T14:09:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.026+0000 7f6f8cbf3740 20 client.17202 trim_cache size 0 max 16384 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.043+0000 7f6f8cbf3740 1 Processor -- start 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.043+0000 7f6f8cbf3740 1 -- start start 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ceb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841b2a00 con 0x55f3a0ce69f0 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841b0500 con 0x55f3a0ce7bc0 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841ae580 con 0x55f3a0ce8580 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ceb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ceb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57880/0 (socket says 172.21.15.137:57880) 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f82000640 1 -- 172.21.15.137:0/1910985258 learned_addr learned my addr 172.21.15.137:0/1910985258 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.044+0000 7f6f82a00640 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f82000640 1 -- 172.21.15.137:0/1910985258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 msgr2=0x55f3a0ceb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f82000640 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f82000640 1 -- 172.21.15.137:0/1910985258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 msgr2=0x55f3a0cf81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f82000640 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f82000640 1 -- 172.21.15.137:0/1910985258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3a0cfef90 con 0x55f3a0ce69f0 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f82000640 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ceb2d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6f700059f0 tx=0x7f6f70011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f50aeff36f571e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f6f400640 1 -- 172.21.15.137:0/1910985258 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70019070 con 0x55f3a0ce69f0 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f6f400640 1 -- 172.21.15.137:0/1910985258 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f700116c0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.045+0000 7f6f6f400640 1 -- 172.21.15.137:0/1910985258 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7001ebc0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1910985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 msgr2=0x55f3a0ceb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ceb2d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6f700059f0 tx=0x7f6f70011b80 comp rx=0 tx=0).stop 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1910985258 shutdown_connections 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0ceb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/1910985258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0ceb2d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1910985258 >> 172.21.15.137:0/1910985258 conn(0x55f3a0ce1de0 msgr2=0x55f3a0ceabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1910985258 shutdown_connections 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.046+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/1910985258 wait complete. 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.047+0000 7f6f8cbf3740 1 -- start start 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.047+0000 7f6f8cbf3740 1 Processor -- start 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841aeb80 con 0x55f3a0ce69f0 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841afea0 con 0x55f3a0ce7bc0 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f841af1e0 con 0x55f3a0ce8580 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f8cbf3740 10 client.0 _ll_register_callbacks cb 0x7f6f7400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57886/0 (socket says 172.21.15.137:57886) 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 -- 172.21.15.137:0/2677845249 learned_addr learned my addr 172.21.15.137:0/2677845249 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82a00640 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 -- 172.21.15.137:0/2677845249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 msgr2=0x55f3a0cf7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 -- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 msgr2=0x55f3a0cf4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.048+0000 7f6f82000640 1 -- 172.21.15.137:0/2677845249 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3a0cf7f50 con 0x55f3a0ce69f0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f82000640 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf4450 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000b8a0 tx=0x7f6f7001e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9708c822a6f62996 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f6cc00640 1 -- 172.21.15.137:0/2677845249 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70019030 con 0x55f3a0ce69f0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f6cc00640 1 -- 172.21.15.137:0/2677845249 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6f70013250 con 0x55f3a0ce69f0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f700116c0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f6cc00640 1 -- 172.21.15.137:0/2677845249 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f7000bd70 con 0x55f3a0ce69f0 2024-03-26T14:09:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f8cbf3740 10 client.17208 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.049+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f3a0d58900 con 0x55f3a0ce69f0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f6cc00640 1 -- 172.21.15.137:0/2677845249 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f700116c0 con 0x55f3a0ce69f0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f6cc00640 1 -- 172.21.15.137:0/2677845249 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6f70029040 con 0x55f3a0ce69f0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f8cbf3740 10 client.17208 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f8cbf3740 10 client.17208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f8cbf3740 20 client.17208 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0d591f0 0x55f3a0d5b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f8cbf3740 4 client.17208 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.050+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f3a0c52b50 con 0x55f3a0d591f0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.051+0000 7f6f82a00640 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0d591f0 0x55f3a0d5b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f82a00640 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0d591f0 0x55f3a0d5b610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d753cff621cbd00 server_cookie=4fed80b8a386f76a in_seq=0 out_seq=0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f6cc00640 10 client.17208 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f6cc00640 1 -- 172.21.15.137:0/2677845249 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f3a0c52b50 con 0x55f3a0d591f0 2024-03-26T14:09:46.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f6cc00640 10 client.17208 handle_command_reply: tid=0 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f8cbf3740 1 client.17208 shutdown 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 msgr2=0x55f3a0cf4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf4450 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000b8a0 tx=0x7f6f7001e6a0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 shutdown_connections 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.052+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f3a0d591f0 0x55f3a0d5b610 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3a0ce7bc0 0x55f3a0cf4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3a0ce8580 0x55f3a0cf7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 1 --2- 172.21.15.137:0/2677845249 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f3a0ce69f0 0x55f3a0cf4450 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 >> 172.21.15.137:0/2677845249 conn(0x55f3a0ce1de0 msgr2=0x55f3a0d03aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f6cc00640 0 client.17208 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 shutdown_connections 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 1 -- 172.21.15.137:0/2677845249 wait complete. 2024-03-26T14:09:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.053+0000 7f6f8cbf3740 20 client.17208 trim_cache size 0 max 16384 2024-03-26T14:09:46.111 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f680a2880 msgr2=0x7f6f680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f680a2880 0x7f6f680a4d40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0ba280 tx=0x7f6f7c0bb950 comp rx=0 tx=0).stop 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 msgr2=0x7f6f841a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f841a52c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02c700 tx=0x7f6f7c0328a0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 shutdown_connections 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6f680a2880 0x7f6f680a4d40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f84102760 0x7f6f8419e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6f84105110 0x7f6f841a52c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 --2- 172.21.15.137:0/629191511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f840ffdf0 0x7f6f8419dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 >> 172.21.15.137:0/629191511 conn(0x7f6f840fba10 msgr2=0x7f6f84100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 shutdown_connections 2024-03-26T14:09:46.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.054+0000 7f6f83e00640 1 -- 172.21.15.137:0/629191511 wait complete. 2024-03-26T14:09:46.113 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:09:46.113 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:09:46.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 -- 172.21.15.137:0/2604739639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981020c0 msgr2=0x7efd981044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 --2- 172.21.15.137:0/2604739639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981020c0 0x7efd981044b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009900 tx=0x7efd8c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:46.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 -- 172.21.15.137:0/2604739639 shutdown_connections 2024-03-26T14:09:46.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 --2- 172.21.15.137:0/2604739639 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd98107320 0x7efd98109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 --2- 172.21.15.137:0/2604739639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd981049f0 0x7efd98106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 --2- 172.21.15.137:0/2604739639 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981020c0 0x7efd981044b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 -- 172.21.15.137:0/2604739639 >> 172.21.15.137:0/2604739639 conn(0x7efd980fba10 msgr2=0x7efd980fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 -- 172.21.15.137:0/2604739639 shutdown_connections 2024-03-26T14:09:46.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.340+0000 7efd9de00640 1 -- 172.21.15.137:0/2604739639 wait complete. 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.341+0000 7efd9de00640 1 Processor -- start 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.341+0000 7efd9de00640 1 -- start start 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd981020c0 0x7efd98196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 0x7efd981966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98107320 0x7efd98192ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9de00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9810d120 con 0x7efd981049f0 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9810cfa0 con 0x7efd981020c0 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9810d2a0 con 0x7efd98107320 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 0x7efd981966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 0x7efd981966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57908/0 (socket says 172.21.15.137:57908) 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd97e00640 1 -- 172.21.15.137:0/1823451979 learned_addr learned my addr 172.21.15.137:0/1823451979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.342+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd981020c0 0x7efd98196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd9d400640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98107320 0x7efd98192ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd97e00640 1 -- 172.21.15.137:0/1823451979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98107320 msgr2=0x7efd98192ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd97e00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98107320 0x7efd98192ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd97e00640 1 -- 172.21.15.137:0/1823451979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd981020c0 msgr2=0x7efd98196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd97e00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd981020c0 0x7efd98196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd97e00640 1 -- 172.21.15.137:0/1823451979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd981935a0 con 0x7efd981049f0 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd97e00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 0x7efd981966b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efd8400d740 tx=0x7efd8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd95600640 1 -- 172.21.15.137:0/1823451979 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84017070 con 0x7efd981049f0 2024-03-26T14:09:46.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.343+0000 7efd95600640 1 -- 172.21.15.137:0/1823451979 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd8400de50 con 0x7efd981049f0 2024-03-26T14:09:46.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.344+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd981a5ea0 con 0x7efd981049f0 2024-03-26T14:09:46.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.344+0000 7efd95600640 1 -- 172.21.15.137:0/1823451979 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84004d60 con 0x7efd981049f0 2024-03-26T14:09:46.346 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.344+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd981a6250 con 0x7efd981049f0 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.347+0000 7efda2232740 1 Processor -- start 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.347+0000 7efda2232740 1 -- start start 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.347+0000 7efd95600640 1 -- 172.21.15.137:0/1823451979 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7efd84004f00 con 0x7efd981049f0 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efda2232740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 0x55618c911e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efda2232740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c912bc0 0x55618c913040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efda2232740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c913580 0x55618c919e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.350 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efda2232740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c8762c0 con 0x55618c9119f0 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efda2232740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c81ec90 con 0x55618c912bc0 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efda2232740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c872360 con 0x55618c913580 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efd9d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 0x55618c911e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efd9d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 0x55618c911e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57922/0 (socket says 172.21.15.137:57922) 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efd9d400640 1 -- 172.21.15.137:0/2527511335 learned_addr learned my addr 172.21.15.137:0/2527511335 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efd9ca00640 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c912bc0 0x55618c913040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.348+0000 7efd97e00640 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c913580 0x55618c919e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9d400640 1 -- 172.21.15.137:0/2527511335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c913580 msgr2=0x55618c919e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9d400640 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c913580 0x55618c919e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.351 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9d400640 1 -- 172.21.15.137:0/2527511335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c912bc0 msgr2=0x55618c913040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9d400640 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c912bc0 0x55618c913040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9d400640 1 -- 172.21.15.137:0/2527511335 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618c843080 con 0x55618c9119f0 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9ca00640 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c912bc0 0x55618c913040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd95600640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd800a2880 0x7efd800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd9d400640 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 0x55618c911e10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efd90009ee0 tx=0x7efd90010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ccae0cd43d5cfc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.349+0000 7efd95600640 1 -- 172.21.15.137:0/1823451979 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd840ce770 con 0x7efd981049f0 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.350+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd800a2880 0x7efd800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.350+0000 7efd89c00640 1 -- 172.21.15.137:0/2527511335 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90016070 con 0x55618c9119f0 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.350+0000 7efd89c00640 1 -- 172.21.15.137:0/2527511335 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd90004d20 con 0x55618c9119f0 2024-03-26T14:09:46.352 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.350+0000 7efd89c00640 1 -- 172.21.15.137:0/2527511335 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90012740 con 0x55618c9119f0 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 -- 172.21.15.137:0/2527511335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 msgr2=0x55618c911e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd800a2880 0x7efd800a4d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efd8c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 0x55618c911e10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7efd90009ee0 tx=0x7efd90010740 comp rx=0 tx=0).stop 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 -- 172.21.15.137:0/2527511335 shutdown_connections 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c912bc0 0x55618c913040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c913580 0x55618c919e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 --2- 172.21.15.137:0/2527511335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c9119f0 0x55618c911e10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 -- 172.21.15.137:0/2527511335 >> 172.21.15.137:0/2527511335 conn(0x55618c90cde0 msgr2=0x55618c90d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 -- 172.21.15.137:0/2527511335 shutdown_connections 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 -- 172.21.15.137:0/2527511335 wait complete. 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.351+0000 7efda2232740 1 -- start start 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 Processor -- start 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c91bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c91c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c923240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c91a590 con 0x55618c913580 2024-03-26T14:09:46.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c8320d0 con 0x55618c9119f0 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55618c91a710 con 0x55618c912bc0 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efda2232740 10 client.0 _ll_register_callbacks cb 0x55618c836e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efd9d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c923240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efd9d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c923240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57936/0 (socket says 172.21.15.137:57936) 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.352+0000 7efd9d400640 1 -- 172.21.15.137:0/1884614603 learned_addr learned my addr 172.21.15.137:0/1884614603 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c91bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd97e00640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c91c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9d400640 1 -- 172.21.15.137:0/1884614603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 msgr2=0x55618c91c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9d400640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c91c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9d400640 1 -- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 msgr2=0x55618c91bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9d400640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c91bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9d400640 1 -- 172.21.15.137:0/1884614603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55618c58a3f0 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd9d400640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c923240 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efd90006010 tx=0x7efd90004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775523c48feba6f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.353+0000 7efd7e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efd7e000640 1 -- 172.21.15.137:0/1884614603 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90016040 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efd7e000640 1 -- 172.21.15.137:0/1884614603 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd900052e0 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd90004d20 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efd7e000640 1 -- 172.21.15.137:0/1884614603 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd90030050 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efda2232740 10 client.17220 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55618c6547d0 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.354+0000 7efd7e000640 1 -- 172.21.15.137:0/1884614603 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd90004d20 con 0x55618c913580 2024-03-26T14:09:46.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efd7e000640 1 -- 172.21.15.137:0/1884614603 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7efd9002b030 con 0x55618c913580 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efda2232740 10 client.17220 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efda2232740 10 client.17220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efda2232740 20 client.17220 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efda2232740 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c927450 0x55618c929840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efda2232740 4 client.17220 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55618c883800 con 0x55618c927450 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.355+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c927450 0x55618c929840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.356+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c927450 0x55618c929840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56d7640652739d0e server_cookie=d77cf271ba31d87b in_seq=0 out_seq=0 2024-03-26T14:09:46.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.356+0000 7efd7e000640 10 client.17220 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.358+0000 7efd7e000640 1 -- 172.21.15.137:0/1884614603 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55618c883800 con 0x55618c927450 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.358+0000 7efd7e000640 10 client.17220 handle_command_reply: tid=0 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 client.17220 shutdown 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 msgr2=0x55618c923240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c923240 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7efd90006010 tx=0x7efd90004ac0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 shutdown_connections 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c927450 0x55618c929840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c923240 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c91c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 --2- 172.21.15.137:0/1884614603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c91bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efd7e000640 0 client.17220 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 >> 172.21.15.137:0/1884614603 conn(0x55618c90cde0 msgr2=0x55618c90c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 shutdown_connections 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 1 -- 172.21.15.137:0/1884614603 wait complete. 2024-03-26T14:09:46.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.359+0000 7efda2232740 20 client.17220 trim_cache size 0 max 16384 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.377+0000 7efda2232740 1 Processor -- start 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.377+0000 7efda2232740 1 -- start start 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efda2232740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c9119f0 0x55618c910750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efda2232740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 0x55618c923160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efda2232740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c913580 0x55618c915c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efda2232740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981b2bd0 con 0x55618c912bc0 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efda2232740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981b06d0 con 0x55618c913580 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efda2232740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981ae750 con 0x55618c9119f0 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 0x55618c923160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 0x55618c923160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57940/0 (socket says 172.21.15.137:57940) 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 -- 172.21.15.137:0/1674280703 learned_addr learned my addr 172.21.15.137:0/1674280703 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd9ca00640 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c9119f0 0x55618c910750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd9d400640 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c913580 0x55618c915c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 -- 172.21.15.137:0/1674280703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c9119f0 msgr2=0x55618c910750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c9119f0 0x55618c910750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 -- 172.21.15.137:0/1674280703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c913580 msgr2=0x55618c915c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c913580 0x55618c915c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.378+0000 7efd97e00640 1 -- 172.21.15.137:0/1674280703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55618c9115c0 con 0x55618c912bc0 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.379+0000 7efd97e00640 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 0x55618c923160 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efd840976d0 tx=0x7efd840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0314de4891f230f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.379+0000 7efd89200640 1 -- 172.21.15.137:0/1674280703 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84020040 con 0x55618c912bc0 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.379+0000 7efd89200640 1 -- 172.21.15.137:0/1674280703 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd8409b400 con 0x55618c912bc0 2024-03-26T14:09:46.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.379+0000 7efd89200640 1 -- 172.21.15.137:0/1674280703 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84097370 con 0x55618c912bc0 2024-03-26T14:09:46.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 -- 172.21.15.137:0/1674280703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 msgr2=0x55618c923160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 0x55618c923160 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efd840976d0 tx=0x7efd840046c0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 -- 172.21.15.137:0/1674280703 shutdown_connections 2024-03-26T14:09:46.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c913580 0x55618c915c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c912bc0 0x55618c923160 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 --2- 172.21.15.137:0/1674280703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c9119f0 0x55618c910750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 -- 172.21.15.137:0/1674280703 >> 172.21.15.137:0/1674280703 conn(0x55618c90cde0 msgr2=0x55618c910040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 -- 172.21.15.137:0/1674280703 shutdown_connections 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 -- 172.21.15.137:0/1674280703 wait complete. 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 -- start start 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.380+0000 7efda2232740 1 Processor -- start 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c922b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c923060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c91f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981aed50 con 0x55618c913580 2024-03-26T14:09:46.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981b0070 con 0x55618c9119f0 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd981af3b0 con 0x55618c912bc0 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efda2232740 10 client.0 _ll_register_callbacks cb 0x7efd90009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efd9d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c91f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efd9d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c91f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57948/0 (socket says 172.21.15.137:57948) 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.381+0000 7efd9d400640 1 -- 172.21.15.137:0/3653217376 learned_addr learned my addr 172.21.15.137:0/3653217376 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9ca00640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c922b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd97e00640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c923060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9d400640 1 -- 172.21.15.137:0/3653217376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 msgr2=0x55618c923060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9d400640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c923060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9d400640 1 -- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 msgr2=0x55618c922b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9d400640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c922b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9d400640 1 -- 172.21.15.137:0/3653217376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55618c90d850 con 0x55618c913580 2024-03-26T14:09:46.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9ca00640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c922b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd9d400640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c91f5c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efd90010780 tx=0x7efd9001a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc66d395943c735 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd7e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd7e000640 1 -- 172.21.15.137:0/3653217376 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90016030 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efd7e000640 1 -- 172.21.15.137:0/3653217376 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efd90004b30 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.382+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd8409b400 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efd7e000640 1 -- 172.21.15.137:0/3653217376 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd90040050 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efda2232740 10 client.17226 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55618c983590 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efd7e000640 1 -- 172.21.15.137:0/3653217376 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd8409b400 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efd7e000640 1 -- 172.21.15.137:0/3653217376 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7efd9002b040 con 0x55618c913580 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efda2232740 10 client.17226 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:46.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efda2232740 10 client.17226 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.383+0000 7efda2232740 20 client.17226 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.384+0000 7efda2232740 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c983da0 0x55618c9861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.384+0000 7efda2232740 4 client.17226 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.384+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55618c87db50 con 0x55618c983da0 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.384+0000 7efd9ca00640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c983da0 0x55618c9861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.385+0000 7efd9ca00640 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c983da0 0x55618c9861c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7c787e31a3a3080 server_cookie=f1b5685232d5e7c5 in_seq=0 out_seq=0 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.385+0000 7efd7e000640 10 client.17226 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.385+0000 7efd7e000640 1 -- 172.21.15.137:0/3653217376 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55618c87db50 con 0x55618c983da0 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.385+0000 7efd7e000640 10 client.17226 handle_command_reply: tid=0 2024-03-26T14:09:46.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.385+0000 7efda2232740 1 client.17226 shutdown 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 msgr2=0x55618c91f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c91f5c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efd90010780 tx=0x7efd9001a730 comp rx=0 tx=0).stop 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 shutdown_connections 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55618c983da0 0x55618c9861c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55618c913580 0x55618c91f5c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efd7e000640 0 client.17226 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55618c912bc0 0x55618c923060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 --2- 172.21.15.137:0/3653217376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55618c9119f0 0x55618c922b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 >> 172.21.15.137:0/3653217376 conn(0x55618c90cde0 msgr2=0x55618c9108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 shutdown_connections 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 1 -- 172.21.15.137:0/3653217376 wait complete. 2024-03-26T14:09:46.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.386+0000 7efda2232740 20 client.17226 trim_cache size 0 max 16384 2024-03-26T14:09:46.392 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:09:46.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd800a2880 msgr2=0x7efd800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd800a2880 0x7efd800a4d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efd8c000c00 tx=0x7efd8c02c7e0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 msgr2=0x7efd981966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 0x7efd981966b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efd8400d740 tx=0x7efd8400dc00 comp rx=0 tx=0).stop 2024-03-26T14:09:46.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 shutdown_connections 2024-03-26T14:09:46.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efd800a2880 0x7efd800a4d40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd98107320 0x7efd98192ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd981049f0 0x7efd981966b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 --2- 172.21.15.137:0/1823451979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd981020c0 0x7efd98196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 >> 172.21.15.137:0/1823451979 conn(0x7efd980fba10 msgr2=0x7efd981018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 shutdown_connections 2024-03-26T14:09:46.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.387+0000 7efd9de00640 1 -- 172.21.15.137:0/1823451979 wait complete. 2024-03-26T14:09:46.402 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:46 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3043086592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:09:46.405 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:09:46.405 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:09:46.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:46 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3043086592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:09:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 -- 172.21.15.137:0/679154571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940109430 msgr2=0x7f894010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 --2- 172.21.15.137:0/679154571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940109430 0x7f894010b8d0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ca0 tx=0x7f893802c180 comp rx=0 tx=0).stop 2024-03-26T14:09:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 -- 172.21.15.137:0/679154571 shutdown_connections 2024-03-26T14:09:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 --2- 172.21.15.137:0/679154571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89400740a0 0x7f8940074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 --2- 172.21.15.137:0/679154571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940109430 0x7f894010b8d0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 --2- 172.21.15.137:0/679154571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8940075680 0x7f8940073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 -- 172.21.15.137:0/679154571 >> 172.21.15.137:0/679154571 conn(0x7f89400fcb10 msgr2=0x7f89400fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 -- 172.21.15.137:0/679154571 shutdown_connections 2024-03-26T14:09:46.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.602+0000 7f8946a00640 1 -- 172.21.15.137:0/679154571 wait complete. 2024-03-26T14:09:46.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.603+0000 7f8946a00640 1 Processor -- start 2024-03-26T14:09:46.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.603+0000 7f8946a00640 1 -- start start 2024-03-26T14:09:46.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89400740a0 0x7f8940199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 0x7f8940199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8940109430 0x7f89401a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894010f2d0 con 0x7f8940075680 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894010f150 con 0x7f89400740a0 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894010f450 con 0x7f8940109430 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8944c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 0x7f8940199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8945600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89400740a0 0x7f8940199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8944c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 0x7f8940199e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57968/0 (socket says 172.21.15.137:57968) 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8944c00640 1 -- 172.21.15.137:0/4095364359 learned_addr learned my addr 172.21.15.137:0/4095364359 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.604+0000 7f8946000640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8940109430 0x7f89401a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8944c00640 1 -- 172.21.15.137:0/4095364359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8940109430 msgr2=0x7f89401a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8944c00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8940109430 0x7f89401a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8944c00640 1 -- 172.21.15.137:0/4095364359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89400740a0 msgr2=0x7f8940199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8944c00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89400740a0 0x7f8940199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8944c00640 1 -- 172.21.15.137:0/4095364359 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89401a1540 con 0x7f8940075680 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8944c00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 0x7f8940199e60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f892c00e9f0 tx=0x7f892c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f893e000640 1 -- 172.21.15.137:0/4095364359 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c00cd80 con 0x7f8940075680 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89401a1830 con 0x7f8940075680 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f893e000640 1 -- 172.21.15.137:0/4095364359 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f892c0108e0 con 0x7f8940075680 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.605+0000 7f893e000640 1 -- 172.21.15.137:0/4095364359 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c018920 con 0x7f8940075680 2024-03-26T14:09:46.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.606+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89401a1dd0 con 0x7f8940075680 2024-03-26T14:09:46.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.609+0000 7f894ac13740 1 Processor -- start 2024-03-26T14:09:46.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.609+0000 7f894ac13740 1 -- start start 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f894ac13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effbde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f894ac13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effbf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f894ac13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effc5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f894ac13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644eff222c0 con 0x5644effbd9f0 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f894ac13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644efecac90 con 0x5644effbf580 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f894ac13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644eff1e360 con 0x5644effbebc0 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f8946000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effbde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f8946000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effbde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57974/0 (socket says 172.21.15.137:57974) 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f8946000640 1 -- 172.21.15.137:0/1002367783 learned_addr learned my addr 172.21.15.137:0/1002367783 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f893e000640 1 -- 172.21.15.137:0/4095364359 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f892c010a90 con 0x7f8940075680 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f8945600640 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effbf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.610+0000 7f8946000640 1 -- 172.21.15.137:0/1002367783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 msgr2=0x5644effbf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8946000640 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effbf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8946000640 1 -- 172.21.15.137:0/1002367783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 msgr2=0x5644effc5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8946000640 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effc5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8946000640 1 -- 172.21.15.137:0/1002367783 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644efeef080 con 0x5644effbd9f0 2024-03-26T14:09:46.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8945600640 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effbf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:46.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8946000640 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effbde10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8938005e30 tx=0x7f8938005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd01f1e7e18fdeb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8936600640 1 -- 172.21.15.137:0/1002367783 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938043020 con 0x5644effbd9f0 2024-03-26T14:09:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8936600640 1 -- 172.21.15.137:0/1002367783 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8938003e20 con 0x5644effbd9f0 2024-03-26T14:09:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.611+0000 7f8936600640 1 -- 172.21.15.137:0/1002367783 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938039040 con 0x5644effbd9f0 2024-03-26T14:09:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f893e000640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f89240a2880 0x7f89240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 -- 172.21.15.137:0/1002367783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 msgr2=0x5644effbde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effbde10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8938005e30 tx=0x7f8938005f80 comp rx=0 tx=0).stop 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f893e000640 1 -- 172.21.15.137:0/4095364359 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f892c0cf1c0 con 0x7f8940075680 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f8945600640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f89240a2880 0x7f89240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 -- 172.21.15.137:0/1002367783 shutdown_connections 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effc5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effbf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 --2- 172.21.15.137:0/1002367783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effbde10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.612+0000 7f894ac13740 1 -- 172.21.15.137:0/1002367783 >> 172.21.15.137:0/1002367783 conn(0x5644effb8de0 msgr2=0x5644effb91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.613+0000 7f894ac13740 1 -- 172.21.15.137:0/1002367783 shutdown_connections 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.613+0000 7f894ac13740 1 -- 172.21.15.137:0/1002367783 wait complete. 2024-03-26T14:09:46.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.613+0000 7f894ac13740 1 -- start start 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.613+0000 7f8945600640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f89240a2880 0x7f89240a4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5644effb9d50 tx=0x7f893000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 Processor -- start 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbd9f0 0x5644effc7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 0x5644effc81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644effc6530 con 0x5644effbebc0 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644efede0d0 con 0x5644effbf580 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644effc66b0 con 0x5644effbd9f0 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f894ac13740 10 client.0 _ll_register_callbacks cb 0x5644efee2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f8944c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 0x5644effc81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f8944c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 0x5644effc81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57980/0 (socket says 172.21.15.137:57980) 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f8944c00640 1 -- 172.21.15.137:0/210462466 learned_addr learned my addr 172.21.15.137:0/210462466 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.614+0000 7f8946000640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8945600640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbd9f0 0x5644effc7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8944c00640 1 -- 172.21.15.137:0/210462466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbd9f0 msgr2=0x5644effc7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8944c00640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbd9f0 0x5644effc7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8944c00640 1 -- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 msgr2=0x5644effcf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8944c00640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8944c00640 1 -- 172.21.15.137:0/210462466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644efc363f0 con 0x5644effbebc0 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f8944c00640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 0x5644effc81a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f892c0a39c0 tx=0x7f892c010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb1376dee9d033 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f892aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f892aa00640 1 -- 172.21.15.137:0/210462466 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c09f8a0 con 0x5644effbebc0 2024-03-26T14:09:46.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f892aa00640 1 -- 172.21.15.137:0/210462466 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f892c09f470 con 0x5644effbebc0 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.615+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8938003e20 con 0x5644effbebc0 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f892aa00640 1 -- 172.21.15.137:0/210462466 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f892c09f670 con 0x5644effbebc0 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 10 client.17238 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5644efd007d0 con 0x5644effbebc0 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f892aa00640 1 -- 172.21.15.137:0/210462466 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8938003e20 con 0x5644effbebc0 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f892aa00640 1 -- 172.21.15.137:0/210462466 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f892c084ce0 con 0x5644effbebc0 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 10 client.17238 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 20 client.17238 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644effd3790 0x5644effd5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 4 client.17238 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.616+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644eff2f800 con 0x5644effd3790 2024-03-26T14:09:46.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.617+0000 7f8945600640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644effd3790 0x5644effd5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.617+0000 7f8945600640 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644effd3790 0x5644effd5bb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5faf083939cf35de server_cookie=44ee1be8c51d5802 in_seq=0 out_seq=0 2024-03-26T14:09:46.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.617+0000 7f892aa00640 10 client.17238 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f892aa00640 1 -- 172.21.15.137:0/210462466 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5644eff2f800 con 0x5644effd3790 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f892aa00640 10 client.17238 handle_command_reply: tid=0 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 client.17238 shutdown 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 msgr2=0x5644effc81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 0x5644effc81a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f892c0a39c0 tx=0x7f892c010de0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 shutdown_connections 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644effd3790 0x5644effd5bb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbebc0 0x5644effc81a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 --2- 172.21.15.137:0/210462466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbd9f0 0x5644effc7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f892aa00640 0 client.17238 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:46.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 >> 172.21.15.137:0/210462466 conn(0x5644effb8de0 msgr2=0x5644effb8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.619+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 shutdown_connections 2024-03-26T14:09:46.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.620+0000 7f894ac13740 1 -- 172.21.15.137:0/210462466 wait complete. 2024-03-26T14:09:46.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.620+0000 7f894ac13740 20 client.17238 trim_cache size 0 max 16384 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 Processor -- start 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 -- start start 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effb3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbebc0 0x5644effcf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbf580 0x5644effda470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401ae6c0 con 0x5644effbd9f0 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401ac1c0 con 0x5644effbebc0 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.635+0000 7f894ac13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401aa240 con 0x5644effbf580 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effb3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effb3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:57996/0 (socket says 172.21.15.137:57996) 2024-03-26T14:09:46.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 -- 172.21.15.137:0/3980182090 learned_addr learned my addr 172.21.15.137:0/3980182090 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8944c00640 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbebc0 0x5644effcf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 -- 172.21.15.137:0/3980182090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbf580 msgr2=0x5644effda470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbf580 0x5644effda470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 -- 172.21.15.137:0/3980182090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbebc0 msgr2=0x5644effcf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbebc0 0x5644effcf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 -- 172.21.15.137:0/3980182090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644effbd5f0 con 0x5644effbd9f0 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.636+0000 7f8945600640 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effb3ff0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8930005f00 tx=0x7f8930015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16198308716d9247 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.637+0000 7f8935c00640 1 -- 172.21.15.137:0/3980182090 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930004300 con 0x5644effbd9f0 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.637+0000 7f8935c00640 1 -- 172.21.15.137:0/3980182090 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8930028230 con 0x5644effbd9f0 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.637+0000 7f8935c00640 1 -- 172.21.15.137:0/3980182090 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930004e50 con 0x5644effbd9f0 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 -- 172.21.15.137:0/3980182090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 msgr2=0x5644effb3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effb3ff0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8930005f00 tx=0x7f8930015000 comp rx=0 tx=0).stop 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 -- 172.21.15.137:0/3980182090 shutdown_connections 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbebc0 0x5644effcf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbf580 0x5644effda470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 --2- 172.21.15.137:0/3980182090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effb3ff0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 -- 172.21.15.137:0/3980182090 >> 172.21.15.137:0/3980182090 conn(0x5644effb8de0 msgr2=0x5644effbc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 -- 172.21.15.137:0/3980182090 shutdown_connections 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 -- 172.21.15.137:0/3980182090 wait complete. 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 -- start start 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.638+0000 7f894ac13740 1 Processor -- start 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effcee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401aa840 con 0x5644effbd9f0 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401abb60 con 0x5644effbf580 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89401aaea0 con 0x5644effbebc0 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f894ac13740 10 client.0 _ll_register_callbacks cb 0x7f892c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f8945600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f8945600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58008/0 (socket says 172.21.15.137:58008) 2024-03-26T14:09:46.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f8945600640 1 -- 172.21.15.137:0/2668323105 learned_addr learned my addr 172.21.15.137:0/2668323105 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.639+0000 7f8946000640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8944c00640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effcee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8945600640 1 -- 172.21.15.137:0/2668323105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 msgr2=0x5644effcee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8945600640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effcee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8945600640 1 -- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 msgr2=0x5644effcb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8945600640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8945600640 1 -- 172.21.15.137:0/2668323105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644effcb9b0 con 0x5644effbd9f0 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f8945600640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effce920 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8930028480 tx=0x7f8930003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef9035360b018a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:46.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f892aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.640+0000 7f892aa00640 1 -- 172.21.15.137:0/2668323105 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930039020 con 0x5644effbd9f0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f892aa00640 1 -- 172.21.15.137:0/2668323105 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f89300046c0 con 0x5644effbd9f0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8930028230 con 0x5644effbd9f0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f892aa00640 1 -- 172.21.15.137:0/2668323105 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89300048c0 con 0x5644effbd9f0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 10 client.17244 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5644f002fb80 con 0x5644effbd9f0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f892aa00640 1 -- 172.21.15.137:0/2668323105 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8930028230 con 0x5644effbd9f0 2024-03-26T14:09:46.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f892aa00640 1 -- 172.21.15.137:0/2668323105 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8930019040 con 0x5644effbd9f0 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 10 client.17244 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 20 client.17244 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644f0030500 0x5644f0032920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 4 client.17244 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.641+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5644eff29b50 con 0x5644f0030500 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.642+0000 7f8944c00640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644f0030500 0x5644f0032920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f8944c00640 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644f0030500 0x5644f0032920 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6efbdeddf7dbc042 server_cookie=814e3633665a13eb in_seq=0 out_seq=0 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f892aa00640 10 client.17244 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f892aa00640 1 -- 172.21.15.137:0/2668323105 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5644eff29b50 con 0x5644f0030500 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f892aa00640 10 client.17244 handle_command_reply: tid=0 2024-03-26T14:09:46.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 client.17244 shutdown 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 msgr2=0x5644effce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effce920 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8930028480 tx=0x7f8930003cc0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 shutdown_connections 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5644f0030500 0x5644f0032920 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644effbf580 0x5644effcb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644effbebc0 0x5644effcee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f892aa00640 0 client.17244 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 --2- 172.21.15.137:0/2668323105 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5644effbd9f0 0x5644effce920 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 >> 172.21.15.137:0/2668323105 conn(0x5644effb8de0 msgr2=0x5644effc1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.643+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 shutdown_connections 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f894ac13740 1 -- 172.21.15.137:0/2668323105 wait complete. 2024-03-26T14:09:46.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f894ac13740 20 client.17244 trim_cache size 0 max 16384 2024-03-26T14:09:46.648 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:09:46.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f89240a2880 msgr2=0x7f89240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f89240a2880 0x7f89240a4d40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x5644effb9d50 tx=0x7f893000b390 comp rx=0 tx=0).stop 2024-03-26T14:09:46.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 msgr2=0x7f8940199e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 0x7f8940199e60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f892c00e9f0 tx=0x7f892c00eeb0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 shutdown_connections 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f89240a2880 0x7f89240a4d40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8940109430 0x7f89401a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8940075680 0x7f8940199e60 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 --2- 172.21.15.137:0/4095364359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89400740a0 0x7f8940199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.644+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 >> 172.21.15.137:0/4095364359 conn(0x7f89400fcb10 msgr2=0x7f89401053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.645+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 shutdown_connections 2024-03-26T14:09:46.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:46.645+0000 7f8946a00640 1 -- 172.21.15.137:0/4095364359 wait complete. 2024-03-26T14:09:46.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:46 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3043086592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:09:47.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:47 smithi159 ceph-mon[35501]: pgmap v680: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:47.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:47 smithi137 ceph-mon[31539]: pgmap v680: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:47.666 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:09:47.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:09:47.667 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:09:47.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:47 smithi179 ceph-mon[35472]: pgmap v680: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:47.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 -- 172.21.15.137:0/3238604768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c107400 msgr2=0x7f637c1097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 --2- 172.21.15.137:0/3238604768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c107400 0x7f637c1097f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6374009ca0 tx=0x7f637402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 -- 172.21.15.137:0/3238604768 shutdown_connections 2024-03-26T14:09:47.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 --2- 172.21.15.137:0/3238604768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c107400 0x7f637c1097f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 --2- 172.21.15.137:0/3238604768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f637c104ad0 0x7f637c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 --2- 172.21.15.137:0/3238604768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f637c1021a0 0x7f637c104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 -- 172.21.15.137:0/3238604768 >> 172.21.15.137:0/3238604768 conn(0x7f637c0fbb50 msgr2=0x7f637c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 -- 172.21.15.137:0/3238604768 shutdown_connections 2024-03-26T14:09:47.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.891+0000 7f6382600640 1 -- 172.21.15.137:0/3238604768 wait complete. 2024-03-26T14:09:47.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.892+0000 7f6382600640 1 Processor -- start 2024-03-26T14:09:47.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 -- start start 2024-03-26T14:09:47.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f637c1021a0 0x7f637c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 0x7f637c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f637c107400 0x7f637c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c10d210 con 0x7f637c104ad0 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c10d090 con 0x7f637c107400 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f6382600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c10d390 con 0x7f637c1021a0 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f637aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 0x7f637c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f637aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 0x7f637c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58016/0 (socket says 172.21.15.137:58016) 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.893+0000 7f637aa00640 1 -- 172.21.15.137:0/1976103448 learned_addr learned my addr 172.21.15.137:0/1976103448 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637b400640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f637c1021a0 0x7f637c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637be00640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f637c107400 0x7f637c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637aa00640 1 -- 172.21.15.137:0/1976103448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f637c1021a0 msgr2=0x7f637c1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637aa00640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f637c1021a0 0x7f637c1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637aa00640 1 -- 172.21.15.137:0/1976103448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f637c107400 msgr2=0x7f637c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637aa00640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f637c107400 0x7f637c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637aa00640 1 -- 172.21.15.137:0/1976103448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c19f4f0 con 0x7f637c104ad0 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f637aa00640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 0x7f637c197e10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f636800d740 tx=0x7f636800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f6373e00640 1 -- 172.21.15.137:0/1976103448 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368017070 con 0x7f637c104ad0 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f6373e00640 1 -- 172.21.15.137:0/1976103448 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f636800de50 con 0x7f637c104ad0 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c19f7e0 con 0x7f637c104ad0 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.894+0000 7f6373e00640 1 -- 172.21.15.137:0/1976103448 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368004d60 con 0x7f637c104ad0 2024-03-26T14:09:47.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.895+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c19fce0 con 0x7f637c104ad0 2024-03-26T14:09:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.898+0000 7f638537d740 1 Processor -- start 2024-03-26T14:09:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.898+0000 7f638537d740 1 -- start start 2024-03-26T14:09:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.898+0000 7f638537d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f638537d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f638537d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f638537d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768d83eeb0 con 0x55768d8dd4b0 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f638537d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768d7e7a60 con 0x55768d8ddea0 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f638537d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768d83af50 con 0x55768d8e3010 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f6373e00640 1 -- 172.21.15.137:0/1976103448 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6368004f00 con 0x7f637c104ad0 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8dd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8dd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58018/0 (socket says 172.21.15.137:58018) 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 -- 172.21.15.137:0/2358469447 learned_addr learned my addr 172.21.15.137:0/2358469447 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637b400640 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 -- 172.21.15.137:0/2358469447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 msgr2=0x55768d8e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 -- 172.21.15.137:0/2358469447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 msgr2=0x55768d8e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637be00640 1 -- 172.21.15.137:0/2358469447 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55768d61b7e0 con 0x55768d8dd4b0 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.899+0000 7f637b400640 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.900+0000 7f637be00640 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8dd8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6374005e30 tx=0x7f6374005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50fdf2a3dc936915 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.900+0000 7f6370c00640 1 -- 172.21.15.137:0/2358469447 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374043020 con 0x55768d8dd4b0 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.900+0000 7f6370c00640 1 -- 172.21.15.137:0/2358469447 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6374032db0 con 0x55768d8dd4b0 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.900+0000 7f6370c00640 1 -- 172.21.15.137:0/2358469447 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374039040 con 0x55768d8dd4b0 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f6373e00640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63600a2800 0x7f63600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f6373e00640 1 -- 172.21.15.137:0/1976103448 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f63680ce8d0 con 0x7f637c104ad0 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f637b400640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63600a2800 0x7f63600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 -- 172.21.15.137:0/2358469447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 msgr2=0x55768d8dd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8dd8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6374005e30 tx=0x7f6374005f80 comp rx=0 tx=0).stop 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 -- 172.21.15.137:0/2358469447 shutdown_connections 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 --2- 172.21.15.137:0/2358469447 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8dd8d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 -- 172.21.15.137:0/2358469447 >> 172.21.15.137:0/2358469447 conn(0x55768d8d4610 msgr2=0x55768d8d4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.901+0000 7f638537d740 1 -- 172.21.15.137:0/2358469447 shutdown_connections 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.902+0000 7f638537d740 1 -- 172.21.15.137:0/2358469447 wait complete. 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.902+0000 7f637b400640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63600a2800 0x7f63600a4cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55768d8d5580 tx=0x7f636c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:47.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.902+0000 7f638537d740 1 -- start start 2024-03-26T14:09:47.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.902+0000 7f638537d740 1 Processor -- start 2024-03-26T14:09:47.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8eec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768d8e5f60 con 0x55768d8dd4b0 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768d7faeb0 con 0x55768d8ddea0 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55768d8e60e0 con 0x55768d8e3010 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f638537d740 10 client.0 _ll_register_callbacks cb 0x55768d7ef150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f637b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f637b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58028/0 (socket says 172.21.15.137:58028) 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f637b400640 1 -- 172.21.15.137:0/163033303 learned_addr learned my addr 172.21.15.137:0/163033303 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f637aa00640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.903+0000 7f637be00640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637b400640 1 -- 172.21.15.137:0/163033303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 msgr2=0x55768d8eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637b400640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637b400640 1 -- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 msgr2=0x55768d8e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637b400640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637b400640 1 -- 172.21.15.137:0/163033303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55768d67bf70 con 0x55768d8dd4b0 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637be00640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f637b400640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7690 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f636c011cd0 tx=0x7f636c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58b7f075c575cac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f6365600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f6365600640 1 -- 172.21.15.137:0/163033303 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c02b060 con 0x55768d8dd4b0 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f6365600640 1 -- 172.21.15.137:0/163033303 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f636c01c460 con 0x55768d8dd4b0 2024-03-26T14:09:47.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.904+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6374032db0 con 0x55768d8dd4b0 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f6365600640 1 -- 172.21.15.137:0/163033303 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636c032050 con 0x55768d8dd4b0 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f638537d740 10 client.17256 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55768d513170 con 0x55768d8dd4b0 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f6365600640 1 -- 172.21.15.137:0/163033303 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6374032db0 con 0x55768d8dd4b0 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f6365600640 1 -- 172.21.15.137:0/163033303 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f636c01a040 con 0x55768d8dd4b0 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f638537d740 10 client.17256 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f638537d740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f638537d740 20 client.17256 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.905+0000 7f638537d740 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d8f3510 0x55768d8f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.906+0000 7f638537d740 4 client.17256 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.906+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55768d84c3f0 con 0x55768d8f3510 2024-03-26T14:09:47.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.906+0000 7f637aa00640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d8f3510 0x55768d8f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.907+0000 7f637aa00640 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d8f3510 0x55768d8f5940 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb549bab12542c server_cookie=31cac34b268c8b0d in_seq=0 out_seq=0 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.907+0000 7f6365600640 10 client.17256 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.908+0000 7f6365600640 1 -- 172.21.15.137:0/163033303 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55768d84c3f0 con 0x55768d8f3510 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.908+0000 7f6365600640 10 client.17256 handle_command_reply: tid=0 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.908+0000 7f638537d740 1 client.17256 shutdown 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 msgr2=0x55768d8e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7690 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f636c011cd0 tx=0x7f636c0046c0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 shutdown_connections 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d8f3510 0x55768d8f5940 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 --2- 172.21.15.137:0/163033303 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7690 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 >> 172.21.15.137:0/163033303 conn(0x55768d8d4610 msgr2=0x55768d8d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:47.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f6365600640 0 client.17256 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:47.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 shutdown_connections 2024-03-26T14:09:47.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 1 -- 172.21.15.137:0/163033303 wait complete. 2024-03-26T14:09:47.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.909+0000 7f638537d740 20 client.17256 trim_cache size 0 max 16384 2024-03-26T14:09:47.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 Processor -- start 2024-03-26T14:09:47.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 -- start start 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1aa530 con 0x55768d8dd4b0 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a8030 con 0x55768d8ddea0 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.935+0000 7f638537d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a60b0 con 0x55768d8e3010 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41224/0 (socket says 172.21.15.137:41224) 2024-03-26T14:09:47.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 -- 172.21.15.137:0/1473668042 learned_addr learned my addr 172.21.15.137:0/1473668042 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637aa00640 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637be00640 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 -- 172.21.15.137:0/1473668042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 msgr2=0x55768d8d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8d7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 -- 172.21.15.137:0/1473668042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 msgr2=0x55768d8eeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8eeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 -- 172.21.15.137:0/1473668042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55768d8d3680 con 0x55768d8dd4b0 2024-03-26T14:09:47.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.936+0000 7f637b400640 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7b30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f636c018040 tx=0x7f636c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3dfe26d8fd401f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:47.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f6367e00640 1 -- 172.21.15.137:0/1473668042 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c01ec60 con 0x55768d8dd4b0 2024-03-26T14:09:47.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f6367e00640 1 -- 172.21.15.137:0/1473668042 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f636c026db0 con 0x55768d8dd4b0 2024-03-26T14:09:47.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f6367e00640 1 -- 172.21.15.137:0/1473668042 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c004990 con 0x55768d8dd4b0 2024-03-26T14:09:47.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f638537d740 1 -- 172.21.15.137:0/1473668042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 msgr2=0x55768d8e7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f638537d740 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7b30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f636c018040 tx=0x7f636c005850 comp rx=0 tx=0).stop 2024-03-26T14:09:47.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f638537d740 1 -- 172.21.15.137:0/1473668042 shutdown_connections 2024-03-26T14:09:47.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f638537d740 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8ddea0 0x55768d8eeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.937+0000 7f638537d740 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8e3010 0x55768d8d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.938+0000 7f638537d740 1 --2- 172.21.15.137:0/1473668042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8dd4b0 0x55768d8e7b30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.938+0000 7f638537d740 1 -- 172.21.15.137:0/1473668042 >> 172.21.15.137:0/1473668042 conn(0x55768d8d4610 msgr2=0x55768d8d7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.938+0000 7f638537d740 1 -- 172.21.15.137:0/1473668042 shutdown_connections 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.938+0000 7f638537d740 1 -- 172.21.15.137:0/1473668042 wait complete. 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.938+0000 7f638537d740 1 -- start start 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.938+0000 7f638537d740 1 Processor -- start 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8dd4b0 0x55768d8ee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8ddea0 0x55768d8ecf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 0x55768d8ed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.944 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a66b0 con 0x55768d8e3010 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a79d0 con 0x55768d8dd4b0 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c1a6d10 con 0x55768d8ddea0 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f638537d740 10 client.0 _ll_register_callbacks cb 0x7f636c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f637be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 0x55768d8ed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f637be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 0x55768d8ed470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41232/0 (socket says 172.21.15.137:41232) 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f637be00640 1 -- 172.21.15.137:0/322770721 learned_addr learned my addr 172.21.15.137:0/322770721 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:47.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f637b400640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8dd4b0 0x55768d8ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.939+0000 7f637aa00640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8ddea0 0x55768d8ecf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637be00640 1 -- 172.21.15.137:0/322770721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8ddea0 msgr2=0x55768d8ecf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637be00640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8ddea0 0x55768d8ecf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637be00640 1 -- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8dd4b0 msgr2=0x55768d8ee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637be00640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8dd4b0 0x55768d8ee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637be00640 1 -- 172.21.15.137:0/322770721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55768d8eda30 con 0x55768d8e3010 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637b400640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8dd4b0 0x55768d8ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637aa00640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8ddea0 0x55768d8ecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:47.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f637be00640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 0x55768d8ed470 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6374006010 tx=0x7f6374009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a6d1156d39331b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f6365600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f6365600640 1 -- 172.21.15.137:0/322770721 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374039040 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f636c026db0 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.940+0000 7f6365600640 1 -- 172.21.15.137:0/322770721 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6374003970 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f6365600640 1 -- 172.21.15.137:0/322770721 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6374056050 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 10 client.17262 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55768d94fcc0 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f6365600640 1 -- 172.21.15.137:0/322770721 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f636c026db0 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f6365600640 1 -- 172.21.15.137:0/322770721 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f63740044f0 con 0x55768d8e3010 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 10 client.17262 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 20 client.17262 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:47.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d950170 0x55768d952560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:47.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 4 client.17262 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:09:47.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.941+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55768d846740 con 0x55768d950170 2024-03-26T14:09:47.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.942+0000 7f637b400640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d950170 0x55768d952560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:47.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.943+0000 7f6365600640 10 client.17262 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:47.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.943+0000 7f637b400640 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d950170 0x55768d952560 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf8de3a43ac030f server_cookie=710a2101dbd34cc6 in_seq=0 out_seq=0 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.945+0000 7f6365600640 1 -- 172.21.15.137:0/322770721 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55768d846740 con 0x55768d950170 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.945+0000 7f6365600640 10 client.17262 handle_command_reply: tid=0 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.945+0000 7f638537d740 1 client.17262 shutdown 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 msgr2=0x55768d8ed470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 0x55768d8ed470 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6374006010 tx=0x7f6374009540 comp rx=0 tx=0).stop 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 shutdown_connections 2024-03-26T14:09:47.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55768d950170 0x55768d952560 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55768d8e3010 0x55768d8ed470 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55768d8ddea0 0x55768d8ecf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 --2- 172.21.15.137:0/322770721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55768d8dd4b0 0x55768d8ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f6365600640 0 client.17262 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 >> 172.21.15.137:0/322770721 conn(0x55768d8d4610 msgr2=0x55768d8d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 shutdown_connections 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 1 -- 172.21.15.137:0/322770721 wait complete. 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.946+0000 7f638537d740 20 client.17262 trim_cache size 0 max 16384 2024-03-26T14:09:47.950 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:09:47.951 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:09:47.951 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "427e9577-c3fc-4310-a8a6-698a562ace9c", 2024-03-26T14:09:47.951 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:09:47.951 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63600a2800 msgr2=0x7f63600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63600a2800 0x7f63600a4cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x55768d8d5580 tx=0x7f636c005f90 comp rx=0 tx=0).stop 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 msgr2=0x7f637c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 0x7f637c197e10 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f636800d740 tx=0x7f636800dc00 comp rx=0 tx=0).stop 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 shutdown_connections 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f63600a2800 0x7f63600a4cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f637c107400 0x7f637c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f637c104ad0 0x7f637c197e10 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 --2- 172.21.15.137:0/1976103448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f637c1021a0 0x7f637c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 >> 172.21.15.137:0/1976103448 conn(0x7f637c0fbb50 msgr2=0x7f637c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:47.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 shutdown_connections 2024-03-26T14:09:47.953 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:47.947+0000 7f6382600640 1 -- 172.21.15.137:0/1976103448 wait complete. 2024-03-26T14:09:47.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:09:47.967 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:09:48.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.181+0000 7f4aa9200640 1 -- 172.21.15.137:0/2746716336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4107bc0 msgr2=0x7f4aa4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.181+0000 7f4aa9200640 1 --2- 172.21.15.137:0/2746716336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4107bc0 0x7f4aa4107fe0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009900 tx=0x7f4a9402bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 -- 172.21.15.137:0/2746716336 shutdown_connections 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 --2- 172.21.15.137:0/2746716336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa4108db0 0x7f4aa4109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 --2- 172.21.15.137:0/2746716336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa4109770 0x7f4aa4110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 --2- 172.21.15.137:0/2746716336 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4107bc0 0x7f4aa4107fe0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 -- 172.21.15.137:0/2746716336 >> 172.21.15.137:0/2746716336 conn(0x7f4aa4076b70 msgr2=0x7f4aa4078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 -- 172.21.15.137:0/2746716336 shutdown_connections 2024-03-26T14:09:48.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.182+0000 7f4aa9200640 1 -- 172.21.15.137:0/2746716336 wait complete. 2024-03-26T14:09:48.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.183+0000 7f4aa9200640 1 Processor -- start 2024-03-26T14:09:48.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.183+0000 7f4aa9200640 1 -- start start 2024-03-26T14:09:48.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa9200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa4107bc0 0x7f4aa419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 0x7f4aa419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa4109770 0x7f4aa41a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa9200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa4113740 con 0x7f4aa4108db0 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa9200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41135c0 con 0x7f4aa4109770 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa9200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41138c0 con 0x7f4aa4107bc0 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 0x7f4aa419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa2a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 0x7f4aa419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41250/0 (socket says 172.21.15.137:41250) 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa2a00640 1 -- 172.21.15.137:0/3811408051 learned_addr learned my addr 172.21.15.137:0/3811408051 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.184+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa4109770 0x7f4aa41a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa3400640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa4107bc0 0x7f4aa419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa2a00640 1 -- 172.21.15.137:0/3811408051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa4107bc0 msgr2=0x7f4aa419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa4107bc0 0x7f4aa419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa2a00640 1 -- 172.21.15.137:0/3811408051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa4109770 msgr2=0x7f4aa41a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa4109770 0x7f4aa41a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa2a00640 1 -- 172.21.15.137:0/3811408051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa41a5a00 con 0x7f4aa4108db0 2024-03-26T14:09:48.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.185+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 0x7f4aa419e370 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000e9f0 tx=0x7f4a9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:48.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.187+0000 7f4a9be00640 1 -- 172.21.15.137:0/3811408051 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000cd80 con 0x7f4aa4108db0 2024-03-26T14:09:48.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.187+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa41a5cf0 con 0x7f4aa4108db0 2024-03-26T14:09:48.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.187+0000 7f4a9be00640 1 -- 172.21.15.137:0/3811408051 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a900108e0 con 0x7f4aa4108db0 2024-03-26T14:09:48.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.187+0000 7f4a9be00640 1 -- 172.21.15.137:0/3811408051 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90018920 con 0x7f4aa4108db0 2024-03-26T14:09:48.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.187+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa41a6230 con 0x7f4aa4108db0 2024-03-26T14:09:48.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.191+0000 7f4aad5be740 1 Processor -- start 2024-03-26T14:09:48.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.191+0000 7f4aad5be740 1 -- start start 2024-03-26T14:09:48.192 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baebe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baec6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aad5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625bae23250 con 0x5625baebe050 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aad5be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625badcba30 con 0x5625baec3bb0 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aad5be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625bae1f2f0 con 0x5625baebea40 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baebe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4a9be00640 1 -- 172.21.15.137:0/3811408051 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4a90010a90 con 0x7f4aa4108db0 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baebe470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41258/0 (socket says 172.21.15.137:41258) 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aa3e00640 1 -- 172.21.15.137:0/3922761692 learned_addr learned my addr 172.21.15.137:0/3922761692 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aa3400640 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.192+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baec6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.193+0000 7f4aa3e00640 1 -- 172.21.15.137:0/3922761692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 msgr2=0x5625baec3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.193+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.193+0000 7f4aa3e00640 1 -- 172.21.15.137:0/3922761692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 msgr2=0x5625baec6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.193+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baec6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.193+0000 7f4aa3e00640 1 -- 172.21.15.137:0/3922761692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625badb6690 con 0x5625baebe050 2024-03-26T14:09:48.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.193+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baebe470 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ee0 tx=0x7f4a9c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6626d92368599fca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:48.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.194+0000 7f4a9be00640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a880a2880 0x7f4a880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.194+0000 7f4a9be00640 1 -- 172.21.15.137:0/3811408051 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4a900cf200 con 0x7f4aa4108db0 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.194+0000 7f4aa3400640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a880a2880 0x7f4a880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.195+0000 7f4a98c00640 1 -- 172.21.15.137:0/3922761692 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c016070 con 0x5625baebe050 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.195+0000 7f4a98c00640 1 -- 172.21.15.137:0/3922761692 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a9c004d20 con 0x5625baebe050 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.195+0000 7f4a98c00640 1 -- 172.21.15.137:0/3922761692 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c012740 con 0x5625baebe050 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 -- 172.21.15.137:0/3922761692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 msgr2=0x5625baebe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aa3400640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a880a2880 0x7f4a880a4d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4a9402c240 tx=0x7f4a94005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baebe470 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ee0 tx=0x7f4a9c010740 comp rx=0 tx=0).stop 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 -- 172.21.15.137:0/3922761692 shutdown_connections 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baec6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 --2- 172.21.15.137:0/3922761692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baebe470 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 -- 172.21.15.137:0/3922761692 >> 172.21.15.137:0/3922761692 conn(0x5625bae86200 msgr2=0x5625baeb99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:48.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 -- 172.21.15.137:0/3922761692 shutdown_connections 2024-03-26T14:09:48.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 -- 172.21.15.137:0/3922761692 wait complete. 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.196+0000 7f4aad5be740 1 -- start start 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 Processor -- start 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baec8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baecf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625baec6b60 con 0x5625baebe050 2024-03-26T14:09:48.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625baddee80 con 0x5625baec3bb0 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625baec6ce0 con 0x5625baebea40 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.197+0000 7f4aad5be740 10 client.0 _ll_register_callbacks cb 0x5625badf1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baec8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baec8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41274/0 (socket says 172.21.15.137:41274) 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 -- 172.21.15.137:0/1956739281 learned_addr learned my addr 172.21.15.137:0/1956739281 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baecf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 -- 172.21.15.137:0/1956739281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 msgr2=0x5625baec8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 -- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 msgr2=0x5625baecf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baecf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.198+0000 7f4aa3400640 1 -- 172.21.15.137:0/1956739281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625bac017d0 con 0x5625baebe050 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4aa3400640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baec8000 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009900 tx=0x7f4a9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9247c77d08c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:48.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4a8d600640 1 -- 172.21.15.137:0/1956739281 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9404e030 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4a8d600640 1 -- 172.21.15.137:0/1956739281 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4a940046c0 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a9c004d20 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4a8d600640 1 -- 172.21.15.137:0/1956739281 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9405b050 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4aad5be740 10 client.17274 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.199+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5625bab373f0 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4a8d600640 1 -- 172.21.15.137:0/1956739281 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a9c004d20 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4a8d600640 1 -- 172.21.15.137:0/1956739281 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4a94003ab0 con 0x5625baebe050 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aad5be740 10 client.17274 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aad5be740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aad5be740 20 client.17274 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:48.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baed3eb0 0x5625baed6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aad5be740 4 client.17274 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:09:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625bae30790 con 0x5625baed3eb0 2024-03-26T14:09:48.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.200+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baed3eb0 0x5625baed6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.201+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baed3eb0 0x5625baed6330 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343186306dfc545b server_cookie=ff6650ad2edff2c4 in_seq=0 out_seq=0 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.201+0000 7f4a8d600640 10 client.17274 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4a8d600640 1 -- 172.21.15.137:0/1956739281 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5625bae30790 con 0x5625baed3eb0 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4a8d600640 10 client.17274 handle_command_reply: tid=0 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 client.17274 shutdown 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 msgr2=0x5625baec8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baec8000 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009900 tx=0x7f4a9402bc30 comp rx=0 tx=0).stop 2024-03-26T14:09:48.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 shutdown_connections 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baed3eb0 0x5625baed6330 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baec3bb0 0x5625baecf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baec8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1956739281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baebe050 0x5625baec8000 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 >> 172.21.15.137:0/1956739281 conn(0x5625bae86200 msgr2=0x5625baeb8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4a8d600640 0 client.17274 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 shutdown_connections 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 1 -- 172.21.15.137:0/1956739281 wait complete. 2024-03-26T14:09:48.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.203+0000 7f4aad5be740 20 client.17274 trim_cache size 0 max 16384 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 Processor -- start 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 -- start start 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baebfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baeb8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41b2ab0 con 0x5625baec3bb0 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41b05b0 con 0x5625baebe050 2024-03-26T14:09:48.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.221+0000 7f4aad5be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41ae630 con 0x5625baebea40 2024-03-26T14:09:48.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baeb8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baebfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baeb8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41286/0 (socket says 172.21.15.137:41286) 2024-03-26T14:09:48.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 -- 172.21.15.137:0/1250791981 learned_addr learned my addr 172.21.15.137:0/1250791981 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 -- 172.21.15.137:0/1250791981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 msgr2=0x5625baecf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 -- 172.21.15.137:0/1250791981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 msgr2=0x5625baebfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baebfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 -- 172.21.15.137:0/1250791981 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625baaf70f0 con 0x5625baec3bb0 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3400640 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baebfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.222+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baeb8a40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c019000 tx=0x7f4a9c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67efc4fdbd1e5cb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.223+0000 7f4a8fe00640 1 -- 172.21.15.137:0/1250791981 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c016050 con 0x5625baec3bb0 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.223+0000 7f4a8fe00640 1 -- 172.21.15.137:0/1250791981 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a9c029070 con 0x5625baec3bb0 2024-03-26T14:09:48.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.223+0000 7f4a8fe00640 1 -- 172.21.15.137:0/1250791981 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c01a670 con 0x5625baec3bb0 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 -- 172.21.15.137:0/1250791981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 msgr2=0x5625baeb8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baeb8a40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c019000 tx=0x7f4a9c005c80 comp rx=0 tx=0).stop 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 -- 172.21.15.137:0/1250791981 shutdown_connections 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baeb8a40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 --2- 172.21.15.137:0/1250791981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baebfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 -- 172.21.15.137:0/1250791981 >> 172.21.15.137:0/1250791981 conn(0x5625bae86200 msgr2=0x5625baeb8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 -- 172.21.15.137:0/1250791981 shutdown_connections 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 -- 172.21.15.137:0/1250791981 wait complete. 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.224+0000 7f4aad5be740 1 -- start start 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 Processor -- start 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baecd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baecea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41aec30 con 0x5625baec3bb0 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41aff50 con 0x5625baebe050 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41af290 con 0x5625baebea40 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aad5be740 10 client.0 _ll_register_callbacks cb 0x7f4a94042050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baecea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aa3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baecd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aa3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baecea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41296/0 (socket says 172.21.15.137:41296) 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.225+0000 7f4aa3e00640 1 -- 172.21.15.137:0/2772955949 learned_addr learned my addr 172.21.15.137:0/2772955949 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa3e00640 1 -- 172.21.15.137:0/2772955949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 msgr2=0x5625baecdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa3e00640 1 -- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 msgr2=0x5625baecd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:09:48.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baecd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa3e00640 1 -- 172.21.15.137:0/2772955949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625baecf040 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa2a00640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4aa3e00640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baecea50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ee0 tx=0x7f4a9c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15d5ebaec79841e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4a8d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.226+0000 7f4a8d600640 1 -- 172.21.15.137:0/2772955949 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c012860 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4a8d600640 1 -- 172.21.15.137:0/2772955949 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4a9c012a00 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a9c029070 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4a8d600640 1 -- 172.21.15.137:0/2772955949 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a9c016020 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 10 client.17280 fetch_fsmap learned FSMap version 24 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5625baf2ffd0 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4a8d600640 1 -- 172.21.15.137:0/2772955949 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a9c029070 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4a8d600640 1 -- 172.21.15.137:0/2772955949 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4a9c00f040 con 0x5625baec3bb0 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 10 client.17280 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:09:48.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 20 client.17280 populate_metadata read hostname 'smithi137' 2024-03-26T14:09:48.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baf30730 0x5625baf32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:09:48.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 4 client.17280 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:09:48.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.227+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5625bae2aae0 con 0x5625baf30730 2024-03-26T14:09:48.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.228+0000 7f4aa3400640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baf30730 0x5625baf32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:09:48.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aa3400640 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baf30730 0x5625baf32b50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38346f3a141391b7 server_cookie=8ce2e567d4fce41f in_seq=0 out_seq=0 2024-03-26T14:09:48.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4a8d600640 10 client.17280 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:48.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4a8d600640 1 -- 172.21.15.137:0/2772955949 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5625bae2aae0 con 0x5625baf30730 2024-03-26T14:09:48.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4a8d600640 10 client.17280 handle_command_reply: tid=0 2024-03-26T14:09:48.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 client.17280 shutdown 2024-03-26T14:09:48.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 msgr2=0x5625baecea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baecea50 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009ee0 tx=0x7f4a9c005a90 comp rx=0 tx=0).stop 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 shutdown_connections 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5625baf30730 0x5625baf32b50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5625baec3bb0 0x5625baecea50 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5625baebea40 0x5625baecdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4a8d600640 0 client.17280 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 --2- 172.21.15.137:0/2772955949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625baebe050 0x5625baecd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 >> 172.21.15.137:0/2772955949 conn(0x5625bae86200 msgr2=0x5625baebfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 shutdown_connections 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.229+0000 7f4aad5be740 1 -- 172.21.15.137:0/2772955949 wait complete. 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aad5be740 20 client.17280 trim_cache size 0 max 16384 2024-03-26T14:09:48.235 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: "427e9577-c3fc-4310-a8a6-698a562ace9c": { 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "427e9577-c3fc-4310-a8a6-698a562ace9c", 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:09:48.236 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a880a2880 msgr2=0x7f4a880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a880a2880 0x7f4a880a4d40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4a9402c240 tx=0x7f4a94005fd0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 msgr2=0x7f4aa419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 0x7f4aa419e370 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000e9f0 tx=0x7f4a9000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 shutdown_connections 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4a880a2880 0x7f4a880a4d40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa4109770 0x7f4aa41a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4aa4108db0 0x7f4aa419e370 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.230+0000 7f4aa9200640 1 --2- 172.21.15.137:0/3811408051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4aa4107bc0 0x7f4aa419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.231+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 >> 172.21.15.137:0/3811408051 conn(0x7f4aa4076b70 msgr2=0x7f4aa410c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:09:48.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.231+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 shutdown_connections 2024-03-26T14:09:48.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:09:48.231+0000 7f4aa9200640 1 -- 172.21.15.137:0/3811408051 wait complete. 2024-03-26T14:09:48.247 INFO:tasks.cephfs.filesystem:scrub status for tag:427e9577-c3fc-4310-a8a6-698a562ace9c - {'path': '/', 'tag': '427e9577-c3fc-4310-a8a6-698a562ace9c', 'options': 'recursive,force'} 2024-03-26T14:09:48.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:48 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:09:48.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:48 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:48.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:48 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:09:48.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:48 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:48.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:48 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:09:48.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:48 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:48.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:48 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:09:48.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:48 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:48.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:48 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:09:48.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:48 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:48.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:48 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:09:48.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:48 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:09:49.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:49 smithi159 ceph-mon[35501]: pgmap v681: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:09:49.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:49 smithi137 ceph-mon[31539]: pgmap v681: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:09:49.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:49 smithi179 ceph-mon[35472]: pgmap v681: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:09:51.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:51 smithi159 ceph-mon[35501]: pgmap v682: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:51.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:51 smithi137 ceph-mon[31539]: pgmap v682: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:51.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:51 smithi179 ceph-mon[35472]: pgmap v682: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:09:52.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:52 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:09:52.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:52 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:09:52.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:52 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:09:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:53 smithi137 ceph-mon[31539]: pgmap v683: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:09:53.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:53 smithi179 ceph-mon[35472]: pgmap v683: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:09:53.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:53 smithi159 ceph-mon[35501]: pgmap v683: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:09:55.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:55 smithi137 ceph-mon[31539]: pgmap v684: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:09:55.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:55 smithi179 ceph-mon[35472]: pgmap v684: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:09:55.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:55 smithi159 ceph-mon[35501]: pgmap v684: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:09:57.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:57 smithi137 ceph-mon[31539]: pgmap v685: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:57.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:57 smithi179 ceph-mon[35472]: pgmap v685: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:57.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:57 smithi159 ceph-mon[35501]: pgmap v685: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:09:59.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:09:59 smithi137 ceph-mon[31539]: pgmap v686: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:09:59.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:09:59 smithi179 ceph-mon[35472]: pgmap v686: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:09:59.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:09:59 smithi159 ceph-mon[35501]: pgmap v686: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:10:00.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:00 smithi137 ceph-mon[31539]: overall HEALTH_OK 2024-03-26T14:10:00.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:00 smithi179 ceph-mon[35472]: overall HEALTH_OK 2024-03-26T14:10:00.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:00 smithi159 ceph-mon[35501]: overall HEALTH_OK 2024-03-26T14:10:01.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:01 smithi137 ceph-mon[31539]: pgmap v687: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:10:01.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:01 smithi179 ceph-mon[35472]: pgmap v687: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:10:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:01 smithi159 ceph-mon[35501]: pgmap v687: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:10:03.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:03 smithi137 ceph-mon[31539]: pgmap v688: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:10:03.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:03 smithi179 ceph-mon[35472]: pgmap v688: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:10:03.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:03 smithi159 ceph-mon[35501]: pgmap v688: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:10:05.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:05 smithi137 ceph-mon[31539]: pgmap v689: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:05.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:05 smithi179 ceph-mon[35472]: pgmap v689: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:05.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:05 smithi159 ceph-mon[35501]: pgmap v689: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:07.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:07 smithi137 ceph-mon[31539]: pgmap v690: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:07.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:07 smithi179 ceph-mon[35472]: pgmap v690: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:07.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:07 smithi159 ceph-mon[35501]: pgmap v690: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:09.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:09 smithi137 ceph-mon[31539]: pgmap v691: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:09.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:09 smithi179 ceph-mon[35472]: pgmap v691: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:09.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:09 smithi159 ceph-mon[35501]: pgmap v691: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:11.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:11 smithi137 ceph-mon[31539]: pgmap v692: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:11.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:11 smithi179 ceph-mon[35472]: pgmap v692: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:11.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:11 smithi159 ceph-mon[35501]: pgmap v692: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:13.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:13 smithi179 ceph-mon[35472]: pgmap v693: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:10:13.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:13 smithi159 ceph-mon[35501]: pgmap v693: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:10:13.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:13 smithi137 ceph-mon[31539]: pgmap v693: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:10:15.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:15 smithi179 ceph-mon[35472]: pgmap v694: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:15.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:15 smithi159 ceph-mon[35501]: pgmap v694: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:15.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:15 smithi137 ceph-mon[31539]: pgmap v694: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:17.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:17 smithi179 ceph-mon[35472]: pgmap v695: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:17.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:17 smithi159 ceph-mon[35501]: pgmap v695: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:17.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:17 smithi137 ceph-mon[31539]: pgmap v695: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:18.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:10:18.248 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:10:18.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 -- 172.21.15.137:0/32116739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 msgr2=0x7fa320110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 --2- 172.21.15.137:0/32116739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa320110000 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa318009900 tx=0x7fa31802bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 -- 172.21.15.137:0/32116739 shutdown_connections 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 --2- 172.21.15.137:0/32116739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa320110000 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 --2- 172.21.15.137:0/32116739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa320108db0 0x7fa320109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 --2- 172.21.15.137:0/32116739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa320107bc0 0x7fa320107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 -- 172.21.15.137:0/32116739 >> 172.21.15.137:0/32116739 conn(0x7fa320076b70 msgr2=0x7fa320078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 -- 172.21.15.137:0/32116739 shutdown_connections 2024-03-26T14:10:18.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.476+0000 7fa324c00640 1 -- 172.21.15.137:0/32116739 wait complete. 2024-03-26T14:10:18.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 Processor -- start 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 -- start start 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa320107bc0 0x7fa32019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa320108db0 0x7fa32019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa3201a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320113720 con 0x7fa320109770 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201135a0 con 0x7fa320108db0 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.477+0000 7fa324c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201138a0 con 0x7fa320107bc0 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa3201a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa3201a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37806/0 (socket says 172.21.15.137:37806) 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 -- 172.21.15.137:0/781459785 learned_addr learned my addr 172.21.15.137:0/781459785 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa316a00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa320108db0 0x7fa32019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31f400640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa320107bc0 0x7fa32019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 -- 172.21.15.137:0/781459785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa320107bc0 msgr2=0x7fa32019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa320107bc0 0x7fa32019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 -- 172.21.15.137:0/781459785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa320108db0 msgr2=0x7fa32019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa320108db0 0x7fa32019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.478+0000 7fa31fe00640 1 -- 172.21.15.137:0/781459785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3201a5960 con 0x7fa320109770 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.479+0000 7fa31fe00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa3201a52f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa318036040 tx=0x7fa31802cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.479+0000 7fa31cc00640 1 -- 172.21.15.137:0/781459785 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318043020 con 0x7fa320109770 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.479+0000 7fa31cc00640 1 -- 172.21.15.137:0/781459785 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3180044c0 con 0x7fa320109770 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.479+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3201a5bf0 con 0x7fa320109770 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.479+0000 7fa31cc00640 1 -- 172.21.15.137:0/781459785 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318003bf0 con 0x7fa320109770 2024-03-26T14:10:18.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.479+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3201a6140 con 0x7fa320109770 2024-03-26T14:10:18.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa31cc00640 1 -- 172.21.15.137:0/781459785 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa318046020 con 0x7fa320109770 2024-03-26T14:10:18.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 Processor -- start 2024-03-26T14:10:18.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 -- start start 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace3720470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace3725540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace3728040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace3685250 con 0x55ace3725bb0 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace362da30 con 0x55ace3720a40 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.483+0000 7fa328fb4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace36812f0 con 0x55ace3720050 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.484+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace3728040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.484+0000 7fa31f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace3725540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.484+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace3728040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37810/0 (socket says 172.21.15.137:37810) 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.484+0000 7fa316a00640 1 -- 172.21.15.137:0/3474171204 learned_addr learned my addr 172.21.15.137:0/3474171204 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:18.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.484+0000 7fa316a00640 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace3720470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.484+0000 7fa31cc00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3040a2800 0x7fa3040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa316a00640 1 -- 172.21.15.137:0/3474171204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 msgr2=0x55ace3720470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa316a00640 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace3720470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa316a00640 1 -- 172.21.15.137:0/3474171204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 msgr2=0x55ace3725540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa316a00640 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace3725540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa316a00640 1 -- 172.21.15.137:0/3474171204 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace3618690 con 0x55ace3725bb0 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa31cc00640 1 -- 172.21.15.137:0/781459785 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa3180f1220 con 0x7fa320109770 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa31fe00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3040a2800 0x7fa3040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa316a00640 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace3728040 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa3080058c0 tx=0x7fa30800c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75cf612b2cdf5d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa30fe00640 1 -- 172.21.15.137:0/3474171204 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080142d0 con 0x55ace3725bb0 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa30fe00640 1 -- 172.21.15.137:0/3474171204 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa30800fc10 con 0x55ace3725bb0 2024-03-26T14:10:18.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.485+0000 7fa30fe00640 1 -- 172.21.15.137:0/3474171204 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080129f0 con 0x55ace3725bb0 2024-03-26T14:10:18.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 -- 172.21.15.137:0/3474171204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 msgr2=0x55ace3728040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace3728040 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa3080058c0 tx=0x7fa30800c4d0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa31fe00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3040a2800 0x7fa3040a4cc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3180bd4b0 tx=0x7fa318005a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 -- 172.21.15.137:0/3474171204 shutdown_connections 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace3725540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace3728040 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 --2- 172.21.15.137:0/3474171204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace3720470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.486+0000 7fa328fb4740 1 -- 172.21.15.137:0/3474171204 >> 172.21.15.137:0/3474171204 conn(0x55ace36e8200 msgr2=0x55ace371b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.487+0000 7fa328fb4740 1 -- 172.21.15.137:0/3474171204 shutdown_connections 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.487+0000 7fa328fb4740 1 -- 172.21.15.137:0/3474171204 wait complete. 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.487+0000 7fa328fb4740 1 -- start start 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.487+0000 7fa328fb4740 1 Processor -- start 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace372a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace372a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3731660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace3728b60 con 0x55ace3720050 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace3640e80 con 0x55ace3725bb0 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace3728ce0 con 0x55ace3720a40 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa328fb4740 10 client.0 _ll_register_callbacks cb 0x55ace3653130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace372a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace372a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37814/0 (socket says 172.21.15.137:37814) 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa316a00640 1 -- 172.21.15.137:0/1438576225 learned_addr learned my addr 172.21.15.137:0/1438576225 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa31f400640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace372a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.488+0000 7fa316a00640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3731660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa316a00640 1 -- 172.21.15.137:0/1438576225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 msgr2=0x55ace372a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa316a00640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace372a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa316a00640 1 -- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 msgr2=0x55ace3731660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa316a00640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3731660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa316a00640 1 -- 172.21.15.137:0/1438576225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ace34637d0 con 0x55ace3720050 2024-03-26T14:10:18.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa316a00640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace372a0e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa308005740 tx=0x7fa308014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23fb002fd7e2414 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa30cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa30cc00640 1 -- 172.21.15.137:0/1438576225 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800f760 con 0x55ace3720050 2024-03-26T14:10:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa30cc00640 1 -- 172.21.15.137:0/1438576225 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa30800f900 con 0x55ace3720050 2024-03-26T14:10:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.489+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa30800fc10 con 0x55ace3720050 2024-03-26T14:10:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa30cc00640 1 -- 172.21.15.137:0/1438576225 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa308034050 con 0x55ace3720050 2024-03-26T14:10:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 10 client.17292 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:18.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ace33993f0 con 0x55ace3720050 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa30cc00640 1 -- 172.21.15.137:0/1438576225 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa30800fc10 con 0x55ace3720050 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa30cc00640 1 -- 172.21.15.137:0/1438576225 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa308011830 con 0x55ace3720050 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 10 client.17292 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 20 client.17292 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3735b50 0x55ace3737f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 4 client.17292 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.490+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ace3692790 con 0x55ace3735b50 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.491+0000 7fa31f400640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3735b50 0x55ace3737f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.492+0000 7fa31f400640 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3735b50 0x55ace3737f40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8273d842fd291db9 server_cookie=98c588cfbe1e6460 in_seq=0 out_seq=0 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.492+0000 7fa30cc00640 10 client.17292 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.492+0000 7fa30cc00640 1 -- 172.21.15.137:0/1438576225 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ace3692790 con 0x55ace3735b50 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.492+0000 7fa30cc00640 10 client.17292 handle_command_reply: tid=0 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.492+0000 7fa328fb4740 1 client.17292 shutdown 2024-03-26T14:10:18.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 msgr2=0x55ace372a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace372a0e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fa308005740 tx=0x7fa308014f10 comp rx=0 tx=0).stop 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 shutdown_connections 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3735b50 0x55ace3737f40 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3731660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace372a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 --2- 172.21.15.137:0/1438576225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace372a0e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 >> 172.21.15.137:0/1438576225 conn(0x55ace36e8200 msgr2=0x55ace371abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa30cc00640 0 client.17292 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 shutdown_connections 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 1 -- 172.21.15.137:0/1438576225 wait complete. 2024-03-26T14:10:18.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.493+0000 7fa328fb4740 20 client.17292 trim_cache size 0 max 16384 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 Processor -- start 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 -- start start 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace371f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace3731620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3721bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201b2a60 con 0x55ace3720050 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201b0560 con 0x55ace3725bb0 2024-03-26T14:10:18.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.516+0000 7fa328fb4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201ae5e0 con 0x55ace3720a40 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace371f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace371f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37818/0 (socket says 172.21.15.137:37818) 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 -- 172.21.15.137:0/986999992 learned_addr learned my addr 172.21.15.137:0/986999992 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3721bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa31f400640 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace3731620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 -- 172.21.15.137:0/986999992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 msgr2=0x55ace3731620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace3731620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 -- 172.21.15.137:0/986999992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 msgr2=0x55ace3721bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3721bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.517+0000 7fa316a00640 1 -- 172.21.15.137:0/986999992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace33590f0 con 0x55ace3720050 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.518+0000 7fa316a00640 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace371f490 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa308014950 tx=0x7fa308001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49316c957312db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.518+0000 7fa30f400640 1 -- 172.21.15.137:0/986999992 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800c990 con 0x55ace3720050 2024-03-26T14:10:18.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.518+0000 7fa30f400640 1 -- 172.21.15.137:0/986999992 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa30800cdc0 con 0x55ace3720050 2024-03-26T14:10:18.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.518+0000 7fa30f400640 1 -- 172.21.15.137:0/986999992 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30800f990 con 0x55ace3720050 2024-03-26T14:10:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 -- 172.21.15.137:0/986999992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 msgr2=0x55ace371f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace371f490 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa308014950 tx=0x7fa308001e50 comp rx=0 tx=0).stop 2024-03-26T14:10:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 -- 172.21.15.137:0/986999992 shutdown_connections 2024-03-26T14:10:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3725bb0 0x55ace3721bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720a40 0x55ace3731620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 --2- 172.21.15.137:0/986999992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3720050 0x55ace371f490 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 -- 172.21.15.137:0/986999992 >> 172.21.15.137:0/986999992 conn(0x55ace36e8200 msgr2=0x55ace37214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 -- 172.21.15.137:0/986999992 shutdown_connections 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 -- 172.21.15.137:0/986999992 wait complete. 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.519+0000 7fa328fb4740 1 -- start start 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 Processor -- start 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace372f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace372f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace37306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201aebe0 con 0x55ace3725bb0 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201aff00 con 0x55ace3720a40 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3201af240 con 0x55ace3720050 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa328fb4740 10 client.0 _ll_register_callbacks cb 0x7fa3100099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.520+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace37306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace37306e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37824/0 (socket says 172.21.15.137:37824) 2024-03-26T14:10:18.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 -- 172.21.15.137:0/2970357961 learned_addr learned my addr 172.21.15.137:0/2970357961 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa31f400640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace372f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace372f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 -- 172.21.15.137:0/2970357961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 msgr2=0x55ace372f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace372f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 -- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 msgr2=0x55ace372f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace372f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.524 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 -- 172.21.15.137:0/2970357961 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ace3730cd0 con 0x55ace3725bb0 2024-03-26T14:10:18.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa316a00640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace37306e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa308005f50 tx=0x7fa308000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53914def831e2b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.521+0000 7fa30cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:18.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa30cc00640 1 -- 172.21.15.137:0/2970357961 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30803d070 con 0x55ace3725bb0 2024-03-26T14:10:18.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa30cc00640 1 -- 172.21.15.137:0/2970357961 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa308011640 con 0x55ace3725bb0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa30800cdc0 con 0x55ace3725bb0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa30cc00640 1 -- 172.21.15.137:0/2970357961 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa30800fb70 con 0x55ace3725bb0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa328fb4740 10 client.17298 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ace3791760 con 0x55ace3725bb0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.522+0000 7fa30cc00640 1 -- 172.21.15.137:0/2970357961 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa30800cdc0 con 0x55ace3725bb0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa30cc00640 1 -- 172.21.15.137:0/2970357961 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa3080037f0 con 0x55ace3725bb0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa328fb4740 10 client.17298 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa328fb4740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa328fb4740 20 client.17298 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa328fb4740 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3791ed0 0x55ace37942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa328fb4740 4 client.17298 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ace368cae0 con 0x55ace3791ed0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.523+0000 7fa31f400640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3791ed0 0x55ace37942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa30cc00640 10 client.17298 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:18.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa31f400640 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3791ed0 0x55ace37942f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a0faea2aee650ea server_cookie=38fe733a2e84fc2c in_seq=0 out_seq=0 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa30cc00640 1 -- 172.21.15.137:0/2970357961 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ace368cae0 con 0x55ace3791ed0 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa30cc00640 10 client.17298 handle_command_reply: tid=0 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 client.17298 shutdown 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 msgr2=0x55ace37306e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace37306e0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa308005f50 tx=0x7fa308000bc0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 shutdown_connections 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ace3791ed0 0x55ace37942f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ace3720a40 0x55ace372f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ace3725bb0 0x55ace37306e0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 --2- 172.21.15.137:0/2970357961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ace3720050 0x55ace372f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.524+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 >> 172.21.15.137:0/2970357961 conn(0x55ace36e8200 msgr2=0x55ace3721d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa30cc00640 0 client.17298 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 shutdown_connections 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa328fb4740 1 -- 172.21.15.137:0/2970357961 wait complete. 2024-03-26T14:10:18.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa328fb4740 20 client.17298 trim_cache size 0 max 16384 2024-03-26T14:10:18.529 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:10:18.529 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:10:18.529 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:10:18.529 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3040a2800 msgr2=0x7fa3040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa324c00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3040a2800 0x7fa3040a4cc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3180bd4b0 tx=0x7fa318005a40 comp rx=0 tx=0).stop 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 msgr2=0x7fa3201a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.525+0000 7fa324c00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa3201a52f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa318036040 tx=0x7fa31802cdd0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 shutdown_connections 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3040a2800 0x7fa3040a4cc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa320108db0 0x7fa32019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa320109770 0x7fa3201a52f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 --2- 172.21.15.137:0/781459785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa320107bc0 0x7fa32019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 >> 172.21.15.137:0/781459785 conn(0x7fa320076b70 msgr2=0x7fa32010c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 shutdown_connections 2024-03-26T14:10:18.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.526+0000 7fa324c00640 1 -- 172.21.15.137:0/781459785 wait complete. 2024-03-26T14:10:18.543 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:10:18.544 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 -- 172.21.15.137:0/2496168729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0075660 msgr2=0x7fbac0073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 --2- 172.21.15.137:0/2496168729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0075660 0x7fbac0073ab0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009900 tx=0x7fbab402bcc0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 -- 172.21.15.137:0/2496168729 shutdown_connections 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 --2- 172.21.15.137:0/2496168729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac0074080 0x7fbac00744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 --2- 172.21.15.137:0/2496168729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac01093b0 0x7fbac010b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 --2- 172.21.15.137:0/2496168729 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0075660 0x7fbac0073ab0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 -- 172.21.15.137:0/2496168729 >> 172.21.15.137:0/2496168729 conn(0x7fbac00fca70 msgr2=0x7fbac00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 -- 172.21.15.137:0/2496168729 shutdown_connections 2024-03-26T14:10:18.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.758+0000 7fbac6800640 1 -- 172.21.15.137:0/2496168729 wait complete. 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.759+0000 7fbac6800640 1 Processor -- start 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.759+0000 7fbac6800640 1 -- start start 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 0x7fbac0199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac6800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0075660 0x7fbac0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac01093b0 0x7fbac01a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac6800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac010f230 con 0x7fbac0074080 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac6800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac010f0b0 con 0x7fbac01093b0 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac6800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac010f3b0 con 0x7fbac0075660 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 0x7fbac0199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 0x7fbac0199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37846/0 (socket says 172.21.15.137:37846) 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 -- 172.21.15.137:0/896688324 learned_addr learned my addr 172.21.15.137:0/896688324 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac4a00640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0075660 0x7fbac0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5e00640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac01093b0 0x7fbac01a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 -- 172.21.15.137:0/896688324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0075660 msgr2=0x7fbac0199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0075660 0x7fbac0199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 -- 172.21.15.137:0/896688324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac01093b0 msgr2=0x7fbac01a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac01093b0 0x7fbac01a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.760+0000 7fbac5400640 1 -- 172.21.15.137:0/896688324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac01a1560 con 0x7fbac0074080 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.761+0000 7fbac5400640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 0x7fbac0199940 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbab4006010 tx=0x7fbab40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.761+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4043020 con 0x7fbac0074080 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.761+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbab402dd10 con 0x7fbac0074080 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.761+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01a17f0 con 0x7fbac0074080 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.761+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4003b10 con 0x7fbac0074080 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.761+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac01a1d40 con 0x7fbac0074080 2024-03-26T14:10:18.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.763+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac0108170 con 0x7fbac0074080 2024-03-26T14:10:18.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.766+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbab4046020 con 0x7fbac0074080 2024-03-26T14:10:18.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.767+0000 7fbabe000640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa80a2800 0x7fbaa80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:18.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.767+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbab40f1140 con 0x7fbac0074080 2024-03-26T14:10:18.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.767+0000 7fbac4a00640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa80a2800 0x7fbaa80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:18.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.768+0000 7fbac4a00640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa80a2800 0x7fbaa80a4cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbac019ae50 tx=0x7fbaac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:18.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:18.773+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbab404d020 con 0x7fbac0074080 2024-03-26T14:10:19.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.104+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbac00622a0 con 0x7fbac0074080 2024-03-26T14:10:19.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.106+0000 7fbabe000640 1 -- 172.21.15.137:0/896688324 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fbab403c350 con 0x7fbac0074080 2024-03-26T14:10:19.107 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:10:19.107 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:10:19.107 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:10:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa80a2800 msgr2=0x7fbaa80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa80a2800 0x7fbaa80a4cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbac019ae50 tx=0x7fbaac005f90 comp rx=0 tx=0).stop 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 msgr2=0x7fbac0199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 0x7fbac0199940 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbab4006010 tx=0x7fbab40045e0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 shutdown_connections 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbaa80a2800 0x7fbaa80a4cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbac01093b0 0x7fbac01a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0075660 0x7fbac0199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 --2- 172.21.15.137:0/896688324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbac0074080 0x7fbac0199940 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 >> 172.21.15.137:0/896688324 conn(0x7fbac00fca70 msgr2=0x7fbac0105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 shutdown_connections 2024-03-26T14:10:19.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.109+0000 7fbac6800640 1 -- 172.21.15.137:0/896688324 wait complete. 2024-03-26T14:10:19.129 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:10:19.129 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:10:19.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 -- 172.21.15.137:0/1039109375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04104ab0 msgr2=0x7f0b04106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 --2- 172.21.15.137:0/1039109375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04104ab0 0x7f0b04106ea0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0af0009900 tx=0x7f0af002bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 -- 172.21.15.137:0/1039109375 shutdown_connections 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 --2- 172.21.15.137:0/1039109375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b041073e0 0x7f0b041097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 --2- 172.21.15.137:0/1039109375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04104ab0 0x7f0b04106ea0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 --2- 172.21.15.137:0/1039109375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b04102180 0x7f0b04104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 -- 172.21.15.137:0/1039109375 >> 172.21.15.137:0/1039109375 conn(0x7f0b040fbb10 msgr2=0x7f0b040fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 -- 172.21.15.137:0/1039109375 shutdown_connections 2024-03-26T14:10:19.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.359+0000 7f0b08c00640 1 -- 172.21.15.137:0/1039109375 wait complete. 2024-03-26T14:10:19.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.360+0000 7f0b08c00640 1 Processor -- start 2024-03-26T14:10:19.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 -- start start 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 0x7f0b04199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b04104ab0 0x7f0b0419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b041073e0 0x7f0b041a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0410d1d0 con 0x7f0b04102180 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0410d050 con 0x7f0b041073e0 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b08c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0410d350 con 0x7f0b04104ab0 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b01e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 0x7f0b04199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b01e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 0x7f0b04199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37864/0 (socket says 172.21.15.137:37864) 2024-03-26T14:10:19.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b01e00640 1 -- 172.21.15.137:0/3498453712 learned_addr learned my addr 172.21.15.137:0/3498453712 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b02800640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b041073e0 0x7f0b041a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.361+0000 7f0b01400640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b04104ab0 0x7f0b0419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0b01e00640 1 -- 172.21.15.137:0/3498453712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b04104ab0 msgr2=0x7f0b0419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0b01e00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b04104ab0 0x7f0b0419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0b01e00640 1 -- 172.21.15.137:0/3498453712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b041073e0 msgr2=0x7f0b041a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0b01e00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b041073e0 0x7f0b041a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0b01e00640 1 -- 172.21.15.137:0/3498453712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b041a17d0 con 0x7f0b04102180 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0b01e00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 0x7f0b04199bb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d770 tx=0x7f0af400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0afaa00640 1 -- 172.21.15.137:0/3498453712 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4017070 con 0x7f0b04102180 2024-03-26T14:10:19.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0afaa00640 1 -- 172.21.15.137:0/3498453712 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0af40054d0 con 0x7f0b04102180 2024-03-26T14:10:19.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.362+0000 7f0afaa00640 1 -- 172.21.15.137:0/3498453712 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4004cf0 con 0x7f0b04102180 2024-03-26T14:10:19.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.363+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b041a1ac0 con 0x7f0b04102180 2024-03-26T14:10:19.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.363+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b041a1f90 con 0x7f0b04102180 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.366+0000 7f0b0b86c740 1 Processor -- start 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.366+0000 7f0afaa00640 1 -- 172.21.15.137:0/3498453712 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0af4004e90 con 0x7f0b04102180 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.366+0000 7f0b0b86c740 1 -- start start 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0f9e9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0f94e2c0 con 0x555a0f9e9810 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0f8f6c90 con 0x555a0f9eb3a0 2024-03-26T14:10:19.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0f94a360 con 0x555a0f9ea9e0 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0f9e9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0f9e9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37876/0 (socket says 172.21.15.137:37876) 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b02800640 1 -- 172.21.15.137:0/3113984090 learned_addr learned my addr 172.21.15.137:0/3113984090 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b01400640 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.367+0000 7f0b01e00640 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0b02800640 1 -- 172.21.15.137:0/3113984090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 msgr2=0x555a0f9eae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0b02800640 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9eae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0b02800640 1 -- 172.21.15.137:0/3113984090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 msgr2=0x555a0f9f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0b02800640 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0b02800640 1 -- 172.21.15.137:0/3113984090 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0f91b080 con 0x555a0f9e9810 2024-03-26T14:10:19.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0b02800640 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0f9e9c30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009fd0 tx=0x7f0afc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883e7f33dd2a76ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0afaa00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ae80a2800 0x7f0ae80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.368+0000 7f0aef000640 1 -- 172.21.15.137:0/3113984090 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc004d20 con 0x555a0f9e9810 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0aef000640 1 -- 172.21.15.137:0/3113984090 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0afc004050 con 0x555a0f9e9810 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0aef000640 1 -- 172.21.15.137:0/3113984090 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005280 con 0x555a0f9e9810 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0afaa00640 1 -- 172.21.15.137:0/3498453712 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0af40ce890 con 0x7f0b04102180 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b01e00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ae80a2800 0x7f0ae80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3113984090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 msgr2=0x555a0f9e9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0f9e9c30 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009fd0 tx=0x7f0afc0118d0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b01e00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ae80a2800 0x7f0ae80a4cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0af4097830 tx=0x7f0af40ceb10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3113984090 shutdown_connections 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9eae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3113984090 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0f9e9c30 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.369+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3113984090 >> 172.21.15.137:0/3113984090 conn(0x555a0f9e4c00 msgr2=0x555a0f9e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.370+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3113984090 shutdown_connections 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.370+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3113984090 wait complete. 2024-03-26T14:10:19.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.370+0000 7f0b0b86c740 1 -- start start 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.370+0000 7f0b0b86c740 1 Processor -- start 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9e9810 0x555a0f9f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0f9f23b0 con 0x555a0f9ea9e0 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0f90a0d0 con 0x555a0f9eb3a0 2024-03-26T14:10:19.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0f9f2530 con 0x555a0f9e9810 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b0b86c740 10 client.0 _ll_register_callbacks cb 0x555a0f90ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b01400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b01400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37882/0 (socket says 172.21.15.137:37882) 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b01400640 1 -- 172.21.15.137:0/3414101229 learned_addr learned my addr 172.21.15.137:0/3414101229 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.371+0000 7f0b02800640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b02800640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9e9810 0x555a0f9f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b01400640 1 -- 172.21.15.137:0/3414101229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9e9810 msgr2=0x555a0f9f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b01400640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9e9810 0x555a0f9f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b01400640 1 -- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 msgr2=0x555a0f9fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b01400640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b01400640 1 -- 172.21.15.137:0/3414101229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0f6623f0 con 0x555a0f9ea9e0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b02800640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0b01400640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f3f90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0af0005ab0 tx=0x7f0af0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ccc2417c9b7dcd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.372+0000 7f0ae3400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.373+0000 7f0ae3400640 1 -- 172.21.15.137:0/3414101229 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0005d50 con 0x555a0f9ea9e0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.373+0000 7f0ae3400640 1 -- 172.21.15.137:0/3414101229 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0af0005230 con 0x555a0f9ea9e0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.373+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0afc004050 con 0x555a0f9ea9e0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.373+0000 7f0ae3400640 1 -- 172.21.15.137:0/3414101229 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0af0051050 con 0x555a0f9ea9e0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.373+0000 7f0b0b86c740 10 client.17316 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.373+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555a0f72c7d0 con 0x555a0f9ea9e0 2024-03-26T14:10:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0ae3400640 1 -- 172.21.15.137:0/3414101229 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0afc004050 con 0x555a0f9ea9e0 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0ae3400640 1 -- 172.21.15.137:0/3414101229 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0af00038e0 con 0x555a0f9ea9e0 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0b0b86c740 10 client.17316 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0b0b86c740 10 client.17316 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0b0b86c740 20 client.17316 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0f9ff5c0 0x555a0fa019b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0b0b86c740 4 client.17316 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.374+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a0f95b800 con 0x555a0f9ff5c0 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.375+0000 7f0b02800640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0f9ff5c0 0x555a0fa019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.375+0000 7f0b02800640 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0f9ff5c0 0x555a0fa019b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31b62a5c96564a87 server_cookie=8485be1b3ef31848 in_seq=0 out_seq=0 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.375+0000 7f0ae3400640 10 client.17316 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0ae3400640 1 -- 172.21.15.137:0/3414101229 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555a0f95b800 con 0x555a0f9ff5c0 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0ae3400640 10 client.17316 handle_command_reply: tid=0 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 client.17316 shutdown 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 msgr2=0x555a0f9f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f3f90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0af0005ab0 tx=0x7f0af0004270 comp rx=0 tx=0).stop 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 shutdown_connections 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0f9ff5c0 0x555a0fa019b0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f3f90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/3414101229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9e9810 0x555a0f9f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 >> 172.21.15.137:0/3414101229 conn(0x555a0f9e4c00 msgr2=0x555a0f9e3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0ae3400640 0 client.17316 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 shutdown_connections 2024-03-26T14:10:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.377+0000 7f0b0b86c740 1 -- 172.21.15.137:0/3414101229 wait complete. 2024-03-26T14:10:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.378+0000 7f0b0b86c740 20 client.17316 trim_cache size 0 max 16384 2024-03-26T14:10:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 Processor -- start 2024-03-26T14:10:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 -- start start 2024-03-26T14:10:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0fa015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9faf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9e3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041ae970 con 0x555a0f9e9810 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041ac470 con 0x555a0f9eb3a0 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041aa4f0 con 0x555a0f9ea9e0 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0fa015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0fa015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37886/0 (socket says 172.21.15.137:37886) 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.395+0000 7f0b02800640 1 -- 172.21.15.137:0/1789595575 learned_addr learned my addr 172.21.15.137:0/1789595575 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b01400640 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b02800640 1 -- 172.21.15.137:0/1789595575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 msgr2=0x555a0f9faf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b02800640 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9faf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b02800640 1 -- 172.21.15.137:0/1789595575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 msgr2=0x555a0f9e3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b02800640 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b02800640 1 -- 172.21.15.137:0/1789595575 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0f9f4160 con 0x555a0f9e9810 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0b02800640 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0fa015d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0afc01b040 tx=0x7f0afc003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39783ad0a2dab1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.396+0000 7f0aee600640 1 -- 172.21.15.137:0/1789595575 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02a0f0 con 0x555a0f9e9810 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0aee600640 1 -- 172.21.15.137:0/1789595575 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0afc02a290 con 0x555a0f9e9810 2024-03-26T14:10:19.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0aee600640 1 -- 172.21.15.137:0/1789595575 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc005470 con 0x555a0f9e9810 2024-03-26T14:10:19.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1789595575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 msgr2=0x555a0fa015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0fa015d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f0afc01b040 tx=0x7f0afc003990 comp rx=0 tx=0).stop 2024-03-26T14:10:19.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1789595575 shutdown_connections 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9eb3a0 0x555a0f9e3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1789595575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9e9810 0x555a0fa015d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1789595575 >> 172.21.15.137:0/1789595575 conn(0x555a0f9e4c00 msgr2=0x555a0f9f3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1789595575 shutdown_connections 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.397+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1789595575 wait complete. 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.398+0000 7f0b0b86c740 1 -- start start 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.398+0000 7f0b0b86c740 1 Processor -- start 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9e9810 0x555a0f9f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041aaaf0 con 0x555a0f9eb3a0 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041abe10 con 0x555a0f9e9810 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b041ab150 con 0x555a0f9ea9e0 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b0b86c740 10 client.0 _ll_register_callbacks cb 0x7f0af0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b02800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37888/0 (socket says 172.21.15.137:37888) 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b02800640 1 -- 172.21.15.137:0/1578448170 learned_addr learned my addr 172.21.15.137:0/1578448170 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b02800640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9e9810 0x555a0f9f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b01400640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b02800640 1 -- 172.21.15.137:0/1578448170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 msgr2=0x555a0f9f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.399+0000 7f0b02800640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b02800640 1 -- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9e9810 msgr2=0x555a0f9f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b02800640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9e9810 0x555a0f9f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b02800640 1 -- 172.21.15.137:0/1578448170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0f9fa590 con 0x555a0f9eb3a0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b02800640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f9fa0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0afc006010 tx=0x7f0afc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a3e9f042191fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0ae3400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0ae3400640 1 -- 172.21.15.137:0/1578448170 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02aa30 con 0x555a0f9eb3a0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0ae3400640 1 -- 172.21.15.137:0/1578448170 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0afc02abd0 con 0x555a0f9eb3a0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0afc02a290 con 0x555a0f9eb3a0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0ae3400640 1 -- 172.21.15.137:0/1578448170 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0afc02add0 con 0x555a0f9eb3a0 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b0b86c740 10 client.17322 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:19.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.400+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555a0fa5b200 con 0x555a0f9eb3a0 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0ae3400640 1 -- 172.21.15.137:0/1578448170 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0afc02a290 con 0x555a0f9eb3a0 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0ae3400640 1 -- 172.21.15.137:0/1578448170 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0afc00f740 con 0x555a0f9eb3a0 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0b0b86c740 10 client.17322 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0b0b86c740 10 client.17322 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0b0b86c740 20 client.17322 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0fa5b960 0x555a0fa5dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0b0b86c740 4 client.17322 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.401+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555a0f955b50 con 0x555a0fa5b960 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.402+0000 7f0b01400640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0fa5b960 0x555a0fa5dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.402+0000 7f0b01400640 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0fa5b960 0x555a0fa5dd80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9967f5b88afe17 server_cookie=6b435f2afb2e39b3 in_seq=0 out_seq=0 2024-03-26T14:10:19.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.402+0000 7f0ae3400640 10 client.17322 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0ae3400640 1 -- 172.21.15.137:0/1578448170 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555a0f955b50 con 0x555a0fa5b960 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0ae3400640 10 client.17322 handle_command_reply: tid=0 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 client.17322 shutdown 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 msgr2=0x555a0f9f9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f9fa0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f0afc006010 tx=0x7f0afc0096c0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 shutdown_connections 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555a0fa5b960 0x555a0fa5dd80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a0f9eb3a0 0x555a0f9f9fa0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a0f9ea9e0 0x555a0f9f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 --2- 172.21.15.137:0/1578448170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a0f9e9810 0x555a0f9f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0ae3400640 0 client.17322 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 >> 172.21.15.137:0/1578448170 conn(0x555a0f9e4c00 msgr2=0x555a0f9e3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 shutdown_connections 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.403+0000 7f0b0b86c740 1 -- 172.21.15.137:0/1578448170 wait complete. 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b0b86c740 20 client.17322 trim_cache size 0 max 16384 2024-03-26T14:10:19.410 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:10:19.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ae80a2800 msgr2=0x7f0ae80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ae80a2800 0x7f0ae80a4cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0af4097830 tx=0x7f0af40ceb10 comp rx=0 tx=0).stop 2024-03-26T14:10:19.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 msgr2=0x7f0b04199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 0x7f0b04199bb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d770 tx=0x7f0af400dc30 comp rx=0 tx=0).stop 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 shutdown_connections 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0ae80a2800 0x7f0ae80a4cc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b041073e0 0x7f0b041a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b04104ab0 0x7f0b0419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 --2- 172.21.15.137:0/3498453712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b04102180 0x7f0b04199bb0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.404+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 >> 172.21.15.137:0/3498453712 conn(0x7f0b040fbb10 msgr2=0x7f0b04100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.405+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 shutdown_connections 2024-03-26T14:10:19.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.405+0000 7f0b08c00640 1 -- 172.21.15.137:0/3498453712 wait complete. 2024-03-26T14:10:19.421 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:19 smithi137 ceph-mon[31539]: pgmap v696: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:19.421 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:19 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/896688324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:10:19.424 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:10:19.425 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:10:19.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 -- 172.21.15.137:0/3333704291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214100480 msgr2=0x7f82141008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 --2- 172.21.15.137:0/3333704291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214100480 0x7f82141008a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8204009900 tx=0x7f820402bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 -- 172.21.15.137:0/3333704291 shutdown_connections 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 --2- 172.21.15.137:0/3333704291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 0x7f82141088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 --2- 172.21.15.137:0/3333704291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8214101670 0x7f8214101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 --2- 172.21.15.137:0/3333704291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214100480 0x7f82141008a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 -- 172.21.15.137:0/3333704291 >> 172.21.15.137:0/3333704291 conn(0x7f82140fba10 msgr2=0x7f82140fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 -- 172.21.15.137:0/3333704291 shutdown_connections 2024-03-26T14:10:19.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.643+0000 7f8213200640 1 -- 172.21.15.137:0/3333704291 wait complete. 2024-03-26T14:10:19.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 Processor -- start 2024-03-26T14:10:19.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 -- start start 2024-03-26T14:10:19.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8214100480 0x7f8214104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 0x7f8214102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 0x7f8214102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821410c010 con 0x7f8214101670 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821410be90 con 0x7f8214102030 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.645+0000 7f8213200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821410c190 con 0x7f8214100480 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 0x7f8214102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 0x7f8214102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37912/0 (socket says 172.21.15.137:37912) 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 -- 172.21.15.137:0/1017167884 learned_addr learned my addr 172.21.15.137:0/1017167884 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f8212800640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 0x7f8214102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f8211e00640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8214100480 0x7f8214104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 -- 172.21.15.137:0/1017167884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8214100480 msgr2=0x7f8214104410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8214100480 0x7f8214104410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 -- 172.21.15.137:0/1017167884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 msgr2=0x7f8214102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 0x7f8214102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 -- 172.21.15.137:0/1017167884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214103820 con 0x7f8214101670 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f8212800640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 0x7f8214102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.646+0000 7f820be00640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 0x7f8214102a50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00d770 tx=0x7f81fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.647+0000 7f820aa00640 1 -- 172.21.15.137:0/1017167884 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc017070 con 0x7f8214101670 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.647+0000 7f820aa00640 1 -- 172.21.15.137:0/1017167884 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81fc0054d0 con 0x7f8214101670 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.647+0000 7f820aa00640 1 -- 172.21.15.137:0/1017167884 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc004cf0 con 0x7f8214101670 2024-03-26T14:10:19.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.647+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82141a1930 con 0x7f8214101670 2024-03-26T14:10:19.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.647+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141a1e30 con 0x7f8214101670 2024-03-26T14:10:19.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 Processor -- start 2024-03-26T14:10:19.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f820aa00640 1 -- 172.21.15.137:0/1017167884 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f81fc004e90 con 0x7f8214101670 2024-03-26T14:10:19.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 -- start start 2024-03-26T14:10:19.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43422ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43422e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c434234e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4341912c0 con 0x55c43422e580 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c434139c90 con 0x55c43422c9f0 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f821bdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43418d360 con 0x55c43422dbc0 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.652+0000 7f820be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c434234e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c434234e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37918/0 (socket says 172.21.15.137:37918) 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 -- 172.21.15.137:0/784584710 learned_addr learned my addr 172.21.15.137:0/784584710 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f8212800640 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43422ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f8211e00640 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43422e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 -- 172.21.15.137:0/784584710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 msgr2=0x55c43422e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43422e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 -- 172.21.15.137:0/784584710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 msgr2=0x55c43422ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43422ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820be00640 1 -- 172.21.15.137:0/784584710 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c43415e080 con 0x55c43422e580 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.653+0000 7f820aa00640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81f80a2800 0x7f81f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.654+0000 7f820be00640 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c434234e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0d4c20 tx=0x7f81fc0d6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0f2d3ff48b34fc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.654+0000 7f820aa00640 1 -- 172.21.15.137:0/1017167884 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f81fc0ce980 con 0x7f8214101670 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.654+0000 7f8212800640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81f80a2800 0x7f81f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.654+0000 7f8203400640 1 -- 172.21.15.137:0/784584710 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc020070 con 0x55c43422e580 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.654+0000 7f8203400640 1 -- 172.21.15.137:0/784584710 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f81fc009c30 con 0x55c43422e580 2024-03-26T14:10:19.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.654+0000 7f8203400640 1 -- 172.21.15.137:0/784584710 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0971b0 con 0x55c43422e580 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 -- 172.21.15.137:0/784584710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 msgr2=0x55c434234e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f8212800640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81f80a2800 0x7f81f80a4cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55c43422dac0 tx=0x7f820c0098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c434234e10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0d4c20 tx=0x7f81fc0d6670 comp rx=0 tx=0).stop 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 -- 172.21.15.137:0/784584710 shutdown_connections 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c434234e10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43422e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 --2- 172.21.15.137:0/784584710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43422ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 -- 172.21.15.137:0/784584710 >> 172.21.15.137:0/784584710 conn(0x55c434227de0 msgr2=0x55c4342281e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 -- 172.21.15.137:0/784584710 shutdown_connections 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.655+0000 7f821bdfc740 1 -- 172.21.15.137:0/784584710 wait complete. 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 -- start start 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 Processor -- start 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c434236b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422dbc0 0x55c4342370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422e580 0x55c43423e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c434235530 con 0x55c43422c9f0 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43414d0d0 con 0x55c43422dbc0 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.656+0000 7f821bdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4342356b0 con 0x55c43422e580 2024-03-26T14:10:19.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f821bdfc740 10 client.0 _ll_register_callbacks cb 0x55c434151e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c434236b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c434236b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37930/0 (socket says 172.21.15.137:37930) 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 -- 172.21.15.137:0/3792407851 learned_addr learned my addr 172.21.15.137:0/3792407851 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f820be00640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422dbc0 0x55c4342370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8212800640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422e580 0x55c43423e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 -- 172.21.15.137:0/3792407851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422e580 msgr2=0x55c43423e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422e580 0x55c43423e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 -- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422dbc0 msgr2=0x55c4342370b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422dbc0 0x55c4342370b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.657+0000 7f8211e00640 1 -- 172.21.15.137:0/3792407851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c433ea53f0 con 0x55c43422c9f0 2024-03-26T14:10:19.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f8211e00640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c434236b70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8204009900 tx=0x7f8204004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5c4393bf40b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f81f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f81f7e00640 1 -- 172.21.15.137:0/3792407851 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204005450 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c43419e800 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f81f7e00640 1 -- 172.21.15.137:0/3792407851 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f82040055f0 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f81f7e00640 1 -- 172.21.15.137:0/3792407851 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82040044a0 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f821bdfc740 10 client.17334 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.658+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c433f6f7d0 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f81f7e00640 1 -- 172.21.15.137:0/3792407851 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55c43419e800 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f81f7e00640 1 -- 172.21.15.137:0/3792407851 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f82040038f0 con 0x55c43422c9f0 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f821bdfc740 10 client.17334 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:19.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f821bdfc740 10 client.17334 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f821bdfc740 20 client.17334 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c434242600 0x55c434244a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f821bdfc740 4 client.17334 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.659+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c434198b50 con 0x55c434242600 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.660+0000 7f820be00640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c434242600 0x55c434244a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.660+0000 7f820be00640 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c434242600 0x55c434244a30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d16b5e267a5d73d server_cookie=e33b4a50062e2658 in_seq=0 out_seq=0 2024-03-26T14:10:19.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.660+0000 7f81f7e00640 10 client.17334 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:19.667 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f81f7e00640 1 -- 172.21.15.137:0/3792407851 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c434198b50 con 0x55c434242600 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f81f7e00640 10 client.17334 handle_command_reply: tid=0 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 client.17334 shutdown 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 msgr2=0x55c434236b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c434236b70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8204009900 tx=0x7f8204004270 comp rx=0 tx=0).stop 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 shutdown_connections 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c434242600 0x55c434244a30 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422dbc0 0x55c4342370b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422e580 0x55c43423e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3792407851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c434236b70 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f81f7e00640 0 client.17334 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:19.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.662+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 >> 172.21.15.137:0/3792407851 conn(0x55c434227de0 msgr2=0x55c434227160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.663+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 shutdown_connections 2024-03-26T14:10:19.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.663+0000 7f821bdfc740 1 -- 172.21.15.137:0/3792407851 wait complete. 2024-03-26T14:10:19.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.663+0000 7f821bdfc740 20 client.17334 trim_cache size 0 max 16384 2024-03-26T14:10:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:19 smithi179 ceph-mon[35472]: pgmap v696: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:19 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/896688324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:10:19.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.680+0000 7f821bdfc740 1 Processor -- start 2024-03-26T14:10:19.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.680+0000 7f821bdfc740 1 -- start start 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c43422b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422e580 0x55c4342492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f821bdfc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141ae780 con 0x55c43422c9f0 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f821bdfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141ac280 con 0x55c43422e580 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f821bdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141aa300 con 0x55c43422dbc0 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c43422b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c43422b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37936/0 (socket says 172.21.15.137:37936) 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 -- 172.21.15.137:0/1848105772 learned_addr learned my addr 172.21.15.137:0/1848105772 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8212800640 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422e580 0x55c4342492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f820be00640 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 -- 172.21.15.137:0/1848105772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 msgr2=0x55c43423e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 -- 172.21.15.137:0/1848105772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422e580 msgr2=0x55c4342492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422e580 0x55c4342492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.681+0000 7f8211e00640 1 -- 172.21.15.137:0/1848105772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c43422c5f0 con 0x55c43422c9f0 2024-03-26T14:10:19.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.682+0000 7f8211e00640 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c43422b040 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8204003d10 tx=0x7f820402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9ac235ea212a06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.682+0000 7f8202a00640 1 -- 172.21.15.137:0/1848105772 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820405b070 con 0x55c43422c9f0 2024-03-26T14:10:19.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.682+0000 7f8202a00640 1 -- 172.21.15.137:0/1848105772 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8204034440 con 0x55c43422c9f0 2024-03-26T14:10:19.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.682+0000 7f8202a00640 1 -- 172.21.15.137:0/1848105772 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204034970 con 0x55c43422c9f0 2024-03-26T14:10:19.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 -- 172.21.15.137:0/1848105772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 msgr2=0x55c43422b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c43422b040 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8204003d10 tx=0x7f820402bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 -- 172.21.15.137:0/1848105772 shutdown_connections 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422e580 0x55c4342492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 --2- 172.21.15.137:0/1848105772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422c9f0 0x55c43422b040 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 -- 172.21.15.137:0/1848105772 >> 172.21.15.137:0/1848105772 conn(0x55c434227de0 msgr2=0x55c434227160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 -- 172.21.15.137:0/1848105772 shutdown_connections 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.683+0000 7f821bdfc740 1 -- 172.21.15.137:0/1848105772 wait complete. 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 -- start start 2024-03-26T14:10:19.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 Processor -- start 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43423b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c43423cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141aa900 con 0x55c43422e580 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141abc20 con 0x55c43422c9f0 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82141aaf60 con 0x55c43422dbc0 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f821bdfc740 10 client.0 _ll_register_callbacks cb 0x7f8204009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.684+0000 7f8212800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c43423cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c43423cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37950/0 (socket says 172.21.15.137:37950) 2024-03-26T14:10:19.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 -- 172.21.15.137:0/3750596081 learned_addr learned my addr 172.21.15.137:0/3750596081 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8211e00640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43423b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f820be00640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 -- 172.21.15.137:0/3750596081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 msgr2=0x55c43423beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 -- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 msgr2=0x55c43423b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43423b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 -- 172.21.15.137:0/3750596081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c43423d4b0 con 0x55c43422e580 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f8212800640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c43423cec0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f820c005c00 tx=0x7f820c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d28f985c5c54be server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.685+0000 7f81f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f81f7e00640 1 -- 172.21.15.137:0/3750596081 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c02c060 con 0x55c43422e580 2024-03-26T14:10:19.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f81f7e00640 1 -- 172.21.15.137:0/3750596081 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f820c01eb20 con 0x55c43422e580 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8204034440 con 0x55c43422e580 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f81f7e00640 1 -- 172.21.15.137:0/3750596081 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820c033050 con 0x55c43422e580 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f821bdfc740 10 client.17340 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c43429e0d0 con 0x55c43422e580 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.686+0000 7f81f7e00640 1 -- 172.21.15.137:0/3750596081 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8204034440 con 0x55c43422e580 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f81f7e00640 1 -- 172.21.15.137:0/3750596081 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f820c026070 con 0x55c43422e580 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f821bdfc740 10 client.17340 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f821bdfc740 10 client.17340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f821bdfc740 20 client.17340 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c43429eae0 0x55c4342a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f821bdfc740 4 client.17340 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:10:19.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c434116500 con 0x55c43429eae0 2024-03-26T14:10:19.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.687+0000 7f8211e00640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c43429eae0 0x55c4342a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.688+0000 7f8211e00640 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c43429eae0 0x55c4342a0f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94cca5073b9ceed8 server_cookie=d2047dd181b1faf9 in_seq=0 out_seq=0 2024-03-26T14:10:19.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.688+0000 7f81f7e00640 10 client.17340 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:19.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f81f7e00640 1 -- 172.21.15.137:0/3750596081 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c434116500 con 0x55c43429eae0 2024-03-26T14:10:19.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f81f7e00640 10 client.17340 handle_command_reply: tid=0 2024-03-26T14:10:19.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 client.17340 shutdown 2024-03-26T14:10:19.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 msgr2=0x55c43423cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c43423cec0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f820c005c00 tx=0x7f820c004a40 comp rx=0 tx=0).stop 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 shutdown_connections 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c43429eae0 0x55c4342a0f00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c43422e580 0x55c43423cec0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c43422dbc0 0x55c43423beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 --2- 172.21.15.137:0/3750596081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c43422c9f0 0x55c43423b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 >> 172.21.15.137:0/3750596081 conn(0x55c434227de0 msgr2=0x55c434230df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f81f7e00640 0 client.17340 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 shutdown_connections 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 1 -- 172.21.15.137:0/3750596081 wait complete. 2024-03-26T14:10:19.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.689+0000 7f821bdfc740 20 client.17340 trim_cache size 0 max 16384 2024-03-26T14:10:19.696 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81f80a2800 msgr2=0x7f81f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81f80a2800 0x7f81f80a4cc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55c43422dac0 tx=0x7f820c0098a0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 msgr2=0x7f8214102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 0x7f8214102a50 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00d770 tx=0x7f81fc00dc30 comp rx=0 tx=0).stop 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 shutdown_connections 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f81f80a2800 0x7f81f80a4cc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8214102030 0x7f8214102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8214101670 0x7f8214102a50 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 --2- 172.21.15.137:0/1017167884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8214100480 0x7f8214104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 >> 172.21.15.137:0/1017167884 conn(0x7f82140fba10 msgr2=0x7f8214068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 shutdown_connections 2024-03-26T14:10:19.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.690+0000 7f8213200640 1 -- 172.21.15.137:0/1017167884 wait complete. 2024-03-26T14:10:19.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:19 smithi159 ceph-mon[35501]: pgmap v696: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:19.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:19 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/896688324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:10:19.711 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:10:19.711 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 -- 172.21.15.137:0/4293461571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 msgr2=0x7f8254074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 --2- 172.21.15.137:0/4293461571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f8254074500 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002bcd0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 -- 172.21.15.137:0/4293461571 shutdown_connections 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 --2- 172.21.15.137:0/4293461571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8254109430 0x7f825410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 --2- 172.21.15.137:0/4293461571 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f8254074500 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 --2- 172.21.15.137:0/4293461571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8254075680 0x7f8254073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.930+0000 7f825b000640 1 -- 172.21.15.137:0/4293461571 >> 172.21.15.137:0/4293461571 conn(0x7f82540fcb10 msgr2=0x7f82540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.931+0000 7f825b000640 1 -- 172.21.15.137:0/4293461571 shutdown_connections 2024-03-26T14:10:19.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.931+0000 7f825b000640 1 -- 172.21.15.137:0/4293461571 wait complete. 2024-03-26T14:10:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.931+0000 7f825b000640 1 Processor -- start 2024-03-26T14:10:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 -- start start 2024-03-26T14:10:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f82541999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8254075680 0x7f8254199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8254109430 0x7f82541a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825410f300 con 0x7f82540740a0 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825410f180 con 0x7f8254109430 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f825b000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825410f480 con 0x7f8254075680 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f8259c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f82541999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f8259c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f82541999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37962/0 (socket says 172.21.15.137:37962) 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.932+0000 7f8259c00640 1 -- 172.21.15.137:0/3431170731 learned_addr learned my addr 172.21.15.137:0/3431170731 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f825a600640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8254109430 0x7f82541a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259200640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8254075680 0x7f8254199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259c00640 1 -- 172.21.15.137:0/3431170731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8254075680 msgr2=0x7f8254199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259c00640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8254075680 0x7f8254199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259c00640 1 -- 172.21.15.137:0/3431170731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8254109430 msgr2=0x7f82541a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259c00640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8254109430 0x7f82541a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259c00640 1 -- 172.21.15.137:0/3431170731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82541a15c0 con 0x7f82540740a0 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8259c00640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f82541999a0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f824800e9f0 tx=0x7f824800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.933+0000 7f8252000640 1 -- 172.21.15.137:0/3431170731 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824800cd80 con 0x7f82540740a0 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.934+0000 7f8252000640 1 -- 172.21.15.137:0/3431170731 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82480108e0 con 0x7f82540740a0 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.934+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82541a18b0 con 0x7f82540740a0 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.934+0000 7f8252000640 1 -- 172.21.15.137:0/3431170731 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248018920 con 0x7f82540740a0 2024-03-26T14:10:19.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.934+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82541a1dd0 con 0x7f82540740a0 2024-03-26T14:10:19.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 Processor -- start 2024-03-26T14:10:19.945 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 -- start start 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b88620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b89850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 0x55e868b90620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868ae42c0 con 0x55e868b89d90 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868a8cc90 con 0x55e868b893d0 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.937+0000 7f825f376740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868ae0360 con 0x55e868b88200 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 0x55e868b90620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8252000640 1 -- 172.21.15.137:0/3431170731 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8248010a90 con 0x7f82540740a0 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 0x55e868b90620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37968/0 (socket says 172.21.15.137:37968) 2024-03-26T14:10:19.946 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 -- 172.21.15.137:0/1539972440 learned_addr learned my addr 172.21.15.137:0/1539972440 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f825a600640 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b88620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 -- 172.21.15.137:0/1539972440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 msgr2=0x55e868b88620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b88620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259c00640 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b89850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 -- 172.21.15.137:0/1539972440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 msgr2=0x55e868b89850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b89850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259200640 1 -- 172.21.15.137:0/1539972440 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e868ab1080 con 0x55e868b89d90 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.938+0000 7f8259c00640 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b89850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.939+0000 7f8259200640 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 0x55e868b90620 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f824002c8e0 tx=0x7f8240009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dab3ea3148eda62 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.939+0000 7f8246a00640 1 -- 172.21.15.137:0/1539972440 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240004240 con 0x55e868b89d90 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.939+0000 7f8246a00640 1 -- 172.21.15.137:0/1539972440 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f824002ce30 con 0x55e868b89d90 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.939+0000 7f8246a00640 1 -- 172.21.15.137:0/1539972440 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240004920 con 0x55e868b89d90 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f8252000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f823c0a2800 0x7f823c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f8252000640 1 -- 172.21.15.137:0/3431170731 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f82480cf2c0 con 0x7f82540740a0 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 -- 172.21.15.137:0/1539972440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 msgr2=0x55e868b90620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 0x55e868b90620 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f824002c8e0 tx=0x7f8240009670 comp rx=0 tx=0).stop 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825a600640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f823c0a2800 0x7f823c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 -- 172.21.15.137:0/1539972440 shutdown_connections 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b89850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b89d90 0x55e868b90620 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 --2- 172.21.15.137:0/1539972440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b88620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.940+0000 7f825f376740 1 -- 172.21.15.137:0/1539972440 >> 172.21.15.137:0/1539972440 conn(0x55e868b7ade0 msgr2=0x55e868b7b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.941+0000 7f825f376740 1 -- 172.21.15.137:0/1539972440 shutdown_connections 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.941+0000 7f825f376740 1 -- 172.21.15.137:0/1539972440 wait complete. 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.941+0000 7f825a600640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f823c0a2800 0x7f823c0a4cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55e868b892d0 tx=0x7f824c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.941+0000 7f825f376740 1 -- start start 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.941+0000 7f825f376740 1 Processor -- start 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b88200 0x55e868b92390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b99910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.949 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868b90d40 con 0x55e868b893d0 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868aa00d0 con 0x55e868b88200 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e868b90ec0 con 0x55e868b89d90 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f825f376740 10 client.0 _ll_register_callbacks cb 0x55e868aa4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f8259200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f8259200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b928d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37984/0 (socket says 172.21.15.137:37984) 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f8259200640 1 -- 172.21.15.137:0/828956458 learned_addr learned my addr 172.21.15.137:0/828956458 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.942+0000 7f8259c00640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b88200 0x55e868b92390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f825a600640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b99910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259200640 1 -- 172.21.15.137:0/828956458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 msgr2=0x55e868b99910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259200640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b99910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259200640 1 -- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b88200 msgr2=0x55e868b92390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259200640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b88200 0x55e868b92390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259200640 1 -- 172.21.15.137:0/828956458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8687f83f0 con 0x55e868b893d0 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259c00640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b88200 0x55e868b92390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f8259200640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b928d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8240036040 tx=0x7f8240004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5601116168354980 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f823b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.943+0000 7f823b400640 1 -- 172.21.15.137:0/828956458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824004e070 con 0x55e868b893d0 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f824002ce30 con 0x55e868b893d0 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f823b400640 1 -- 172.21.15.137:0/828956458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f82400037f0 con 0x55e868b893d0 2024-03-26T14:10:19.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f823b400640 1 -- 172.21.15.137:0/828956458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f824005a050 con 0x55e868b893d0 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f825f376740 10 client.17352 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e8688c27d0 con 0x55e868b893d0 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f823b400640 1 -- 172.21.15.137:0/828956458 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f824002ce30 con 0x55e868b893d0 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f823b400640 1 -- 172.21.15.137:0/828956458 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8240034a20 con 0x55e868b893d0 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f825f376740 10 client.17352 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.944+0000 7f825f376740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.945+0000 7f825f376740 20 client.17352 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.945+0000 7f825f376740 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868b9d9e0 0x55e868b9fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.945+0000 7f825f376740 4 client.17352 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:19.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.945+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e868af1800 con 0x55e868b9d9e0 2024-03-26T14:10:19.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.945+0000 7f8259c00640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868b9d9e0 0x55e868b9fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.946+0000 7f8259c00640 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868b9d9e0 0x55e868b9fdd0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e823a27f567a882 server_cookie=9fb4c696abe31c03 in_seq=0 out_seq=0 2024-03-26T14:10:19.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.946+0000 7f823b400640 10 client.17352 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:19.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f823b400640 1 -- 172.21.15.137:0/828956458 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e868af1800 con 0x55e868b9d9e0 2024-03-26T14:10:19.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f823b400640 10 client.17352 handle_command_reply: tid=0 2024-03-26T14:10:19.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 client.17352 shutdown 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 msgr2=0x55e868b928d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b928d0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8240036040 tx=0x7f8240004270 comp rx=0 tx=0).stop 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 shutdown_connections 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868b9d9e0 0x55e868b9fdd0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b99910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b928d0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 --2- 172.21.15.137:0/828956458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b88200 0x55e868b92390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f823b400640 0 client.17352 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.947+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 >> 172.21.15.137:0/828956458 conn(0x55e868b7ade0 msgr2=0x55e868b7a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.948+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 shutdown_connections 2024-03-26T14:10:19.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.948+0000 7f825f376740 1 -- 172.21.15.137:0/828956458 wait complete. 2024-03-26T14:10:19.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.948+0000 7f825f376740 20 client.17352 trim_cache size 0 max 16384 2024-03-26T14:10:19.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 Processor -- start 2024-03-26T14:10:19.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 -- start start 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 0x55e868b75ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b8c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541aa2c0 con 0x55e868b88200 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541a7dc0 con 0x55e868b893d0 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f825f376740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541a5e40 con 0x55e868b89d90 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f8259c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 0x55e868b75ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f8259c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 0x55e868b75ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37994/0 (socket says 172.21.15.137:37994) 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.966+0000 7f8259c00640 1 -- 172.21.15.137:0/46241520 learned_addr learned my addr 172.21.15.137:0/46241520 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259200640 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f825a600640 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b8c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259c00640 1 -- 172.21.15.137:0/46241520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 msgr2=0x55e868b8c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259c00640 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b8c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259c00640 1 -- 172.21.15.137:0/46241520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 msgr2=0x55e868b998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259c00640 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259c00640 1 -- 172.21.15.137:0/46241520 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e868b768b0 con 0x55e868b88200 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f825a600640 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b8c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8259c00640 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 0x55e868b75ff0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8248005b60 tx=0x7f82480cc880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce50b9c6f26bf9e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8246000640 1 -- 172.21.15.137:0/46241520 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824809bf50 con 0x55e868b88200 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8246000640 1 -- 172.21.15.137:0/46241520 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f824809b850 con 0x55e868b88200 2024-03-26T14:10:19.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.967+0000 7f8246000640 1 -- 172.21.15.137:0/46241520 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824809cb80 con 0x55e868b88200 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 -- 172.21.15.137:0/46241520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 msgr2=0x55e868b75ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 0x55e868b75ff0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8248005b60 tx=0x7f82480cc880 comp rx=0 tx=0).stop 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 -- 172.21.15.137:0/46241520 shutdown_connections 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b893d0 0x55e868b998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b89d90 0x55e868b8c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 --2- 172.21.15.137:0/46241520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b88200 0x55e868b75ff0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 -- 172.21.15.137:0/46241520 >> 172.21.15.137:0/46241520 conn(0x55e868b7ade0 msgr2=0x55e868b7e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.976 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 -- 172.21.15.137:0/46241520 shutdown_connections 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.968+0000 7f825f376740 1 -- 172.21.15.137:0/46241520 wait complete. 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 -- start start 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 Processor -- start 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b99700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b97d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b89d90 0x55e868b982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541a6440 con 0x55e868b893d0 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541a7760 con 0x55e868b89d90 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.969+0000 7f825f376740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82541a6aa0 con 0x55e868b88200 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f825f376740 10 client.0 _ll_register_callbacks cb 0x7f8248005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:19.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b97d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b97d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37998/0 (socket says 172.21.15.137:37998) 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 -- 172.21.15.137:0/2377645429 learned_addr learned my addr 172.21.15.137:0/2377645429 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f825a600640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b89d90 0x55e868b982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259c00640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 -- 172.21.15.137:0/2377645429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 msgr2=0x55e868b99700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b99700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 -- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b89d90 msgr2=0x55e868b982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b89d90 0x55e868b982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.970+0000 7f8259200640 1 -- 172.21.15.137:0/2377645429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e868b98a00 con 0x55e868b893d0 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f8259200640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b97d80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8240034dc0 tx=0x7f8240005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d576eff39143ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:19.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f823b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f823b400640 1 -- 172.21.15.137:0/2377645429 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240057070 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f823b400640 1 -- 172.21.15.137:0/2377645429 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f82400035f0 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f824809b850 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f823b400640 1 -- 172.21.15.137:0/2377645429 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f824004e040 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f825f376740 10 client.17358 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e868bfa030 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.971+0000 7f823b400640 1 -- 172.21.15.137:0/2377645429 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f824809b850 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f823b400640 1 -- 172.21.15.137:0/2377645429 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8240053440 con 0x55e868b893d0 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f825f376740 10 client.17358 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f825f376740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:19.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f825f376740 20 client.17358 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f825f376740 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868bfa840 0x55e868bfcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f825f376740 4 client.17358 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e868aebb50 con 0x55e868bfa840 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.972+0000 7f8259c00640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868bfa840 0x55e868bfcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.973+0000 7f8259c00640 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868bfa840 0x55e868bfcc60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ed9da8ac5e1d9c server_cookie=76ca680569bea42f in_seq=0 out_seq=0 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.973+0000 7f823b400640 10 client.17358 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f823b400640 1 -- 172.21.15.137:0/2377645429 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e868aebb50 con 0x55e868bfa840 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f823b400640 10 client.17358 handle_command_reply: tid=0 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 client.17358 shutdown 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 msgr2=0x55e868b97d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b97d80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8240034dc0 tx=0x7f8240005c00 comp rx=0 tx=0).stop 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 shutdown_connections 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e868bfa840 0x55e868bfcc60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e868b89d90 0x55e868b982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e868b893d0 0x55e868b97d80 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f823b400640 0 client.17358 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 --2- 172.21.15.137:0/2377645429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e868b88200 0x55e868b99700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 >> 172.21.15.137:0/2377645429 conn(0x55e868b7ade0 msgr2=0x55e868b798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 shutdown_connections 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 1 -- 172.21.15.137:0/2377645429 wait complete. 2024-03-26T14:10:19.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.974+0000 7f825f376740 20 client.17358 trim_cache size 0 max 16384 2024-03-26T14:10:19.982 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f823c0a2800 msgr2=0x7f823c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f823c0a2800 0x7f823c0a4cc0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x55e868b892d0 tx=0x7f824c009500 comp rx=0 tx=0).stop 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 msgr2=0x7f82541999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f82541999a0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f824800e9f0 tx=0x7f824800eeb0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 shutdown_connections 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f823c0a2800 0x7f823c0a4cc0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8254109430 0x7f82541a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8254075680 0x7f8254199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 --2- 172.21.15.137:0/3431170731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82540740a0 0x7f82541999a0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:19.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 >> 172.21.15.137:0/3431170731 conn(0x7f82540fcb10 msgr2=0x7f82541053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:19.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 shutdown_connections 2024-03-26T14:10:19.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:19.975+0000 7f825b000640 1 -- 172.21.15.137:0/3431170731 wait complete. 2024-03-26T14:10:20.993 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:10:20.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:10:20.994 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 -- 172.21.15.137:0/2244180839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b20103150 msgr2=0x7f3b201099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 --2- 172.21.15.137:0/2244180839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b20103150 0x7f3b201099e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009ca0 tx=0x7f3b1802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 -- 172.21.15.137:0/2244180839 shutdown_connections 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 --2- 172.21.15.137:0/2244180839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b20102790 0x7f3b20102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 --2- 172.21.15.137:0/2244180839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b20103150 0x7f3b201099e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 --2- 172.21.15.137:0/2244180839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b201015a0 0x7f3b201019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 -- 172.21.15.137:0/2244180839 >> 172.21.15.137:0/2244180839 conn(0x7f3b200fcb10 msgr2=0x7f3b200fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 -- 172.21.15.137:0/2244180839 shutdown_connections 2024-03-26T14:10:21.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.223+0000 7f3b25e00640 1 -- 172.21.15.137:0/2244180839 wait complete. 2024-03-26T14:10:21.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.224+0000 7f3b25e00640 1 Processor -- start 2024-03-26T14:10:21.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 -- start start 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 0x7f3b20197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b20102790 0x7f3b20197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b20103150 0x7f3b2019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2010d0e0 con 0x7f3b201015a0 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2010cf60 con 0x7f3b20102790 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b25e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2010d260 con 0x7f3b20103150 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 0x7f3b20197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 0x7f3b20197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38014/0 (socket says 172.21.15.137:38014) 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b24a00640 1 -- 172.21.15.137:0/870384458 learned_addr learned my addr 172.21.15.137:0/870384458 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.225+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b20102790 0x7f3b20197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b25400640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b20103150 0x7f3b2019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b24a00640 1 -- 172.21.15.137:0/870384458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b20103150 msgr2=0x7f3b2019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b24a00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b20103150 0x7f3b2019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b24a00640 1 -- 172.21.15.137:0/870384458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b20102790 msgr2=0x7f3b20197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b24a00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b20102790 0x7f3b20197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b24a00640 1 -- 172.21.15.137:0/870384458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2019f350 con 0x7f3b201015a0 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b24a00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 0x7f3b20197730 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400d950 tx=0x7f3b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.226+0000 7f3b1d600640 1 -- 172.21.15.137:0/870384458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14017070 con 0x7f3b201015a0 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.227+0000 7f3b1d600640 1 -- 172.21.15.137:0/870384458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b14009c30 con 0x7f3b201015a0 2024-03-26T14:10:21.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.227+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2019f640 con 0x7f3b201015a0 2024-03-26T14:10:21.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.227+0000 7f3b1d600640 1 -- 172.21.15.137:0/870384458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400f620 con 0x7f3b201015a0 2024-03-26T14:10:21.231 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.227+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2019fb90 con 0x7f3b201015a0 2024-03-26T14:10:21.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 Processor -- start 2024-03-26T14:10:21.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 -- start start 2024-03-26T14:10:21.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b1d600640 1 -- 172.21.15.137:0/870384458 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3b1401a070 con 0x7f3b201015a0 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6acae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 0x55759c6b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6b2220 0x55759c6b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759c60deb0 con 0x55759c6ad0b0 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759c5b6a60 con 0x55759c6b2220 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.231+0000 7f3b2a179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759c609f50 con 0x55759c6ac6c0 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 0x55759c6b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 0x55759c6b1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38030/0 (socket says 172.21.15.137:38030) 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 -- 172.21.15.137:0/946675817 learned_addr learned my addr 172.21.15.137:0/946675817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.235 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6b2220 0x55759c6b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b25400640 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 -- 172.21.15.137:0/946675817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 msgr2=0x55759c6acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 -- 172.21.15.137:0/946675817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6b2220 msgr2=0x55759c6b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6b2220 0x55759c6b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 -- 172.21.15.137:0/946675817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55759c3ea7e0 con 0x55759c6ad0b0 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6b2220 0x55759c6b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.232+0000 7f3b24a00640 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 0x55759c6b1bb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3b140d4c20 tx=0x7f3b140d6670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fc4fe0edcff3b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.233+0000 7f3b11c00640 1 -- 172.21.15.137:0/946675817 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140dd070 con 0x55759c6ad0b0 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.233+0000 7f3b1d600640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3b080a2880 0x7f3b080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.233+0000 7f3b11c00640 1 -- 172.21.15.137:0/946675817 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b140d0780 con 0x55759c6ad0b0 2024-03-26T14:10:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.233+0000 7f3b11c00640 1 -- 172.21.15.137:0/946675817 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14096e80 con 0x55759c6ad0b0 2024-03-26T14:10:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.233+0000 7f3b1d600640 1 -- 172.21.15.137:0/870384458 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3b140ce820 con 0x7f3b201015a0 2024-03-26T14:10:21.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.233+0000 7f3b25400640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3b080a2880 0x7f3b080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 -- 172.21.15.137:0/946675817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 msgr2=0x55759c6b1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 0x55759c6b1bb0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3b140d4c20 tx=0x7f3b140d6670 comp rx=0 tx=0).stop 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b25400640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3b080a2880 0x7f3b080a4d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006010 tx=0x7f3b1802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 -- 172.21.15.137:0/946675817 shutdown_connections 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6b2220 0x55759c6b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ad0b0 0x55759c6b1bb0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 --2- 172.21.15.137:0/946675817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 -- 172.21.15.137:0/946675817 >> 172.21.15.137:0/946675817 conn(0x55759c6a3820 msgr2=0x55759c6a3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 -- 172.21.15.137:0/946675817 shutdown_connections 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.234+0000 7f3b2a179740 1 -- 172.21.15.137:0/946675817 wait complete. 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.235+0000 7f3b2a179740 1 -- start start 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.235+0000 7f3b2a179740 1 Processor -- start 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.235+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6b6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.235+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 0x55759c6bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b2a179740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759c6b5140 con 0x55759c6b2220 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b2a179740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759c5c9eb0 con 0x55759c6ad0b0 2024-03-26T14:10:21.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b2a179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55759c6b52c0 con 0x55759c6ac6c0 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b2a179740 10 client.0 _ll_register_callbacks cb 0x55759c5be150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 0x55759c6bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 0x55759c6bdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38046/0 (socket says 172.21.15.137:38046) 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 -- 172.21.15.137:0/3069385632 learned_addr learned my addr 172.21.15.137:0/3069385632 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b24a00640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 -- 172.21.15.137:0/3069385632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 msgr2=0x55759c6b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6b6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 -- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 msgr2=0x55759c6b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.236+0000 7f3b25400640 1 -- 172.21.15.137:0/3069385632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55759c44af70 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.237+0000 7f3b25400640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 0x55759c6bdd90 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009ca0 tx=0x7f3b1802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ead70d8deeba8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.237+0000 7f3b06000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.237+0000 7f3b06000640 1 -- 172.21.15.137:0/3069385632 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18045070 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.237+0000 7f3b06000640 1 -- 172.21.15.137:0/3069385632 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3b1804c670 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.237+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b140d0780 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.237+0000 7f3b06000640 1 -- 172.21.15.137:0/3069385632 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b1805a050 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 10 client.17370 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55759c2e2170 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b06000640 1 -- 172.21.15.137:0/3069385632 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b140d0780 con 0x55759c6b2220 2024-03-26T14:10:21.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b06000640 1 -- 172.21.15.137:0/3069385632 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3b180349e0 con 0x55759c6b2220 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 10 client.17370 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 20 client.17370 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c6c26e0 0x55759c6c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 4 client.17370 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.238+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55759c61b3f0 con 0x55759c6c26e0 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.239+0000 7f3b24a00640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c6c26e0 0x55759c6c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.240+0000 7f3b24a00640 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c6c26e0 0x55759c6c4b20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=364889704f4d0700 server_cookie=4bf8371bf9f4969e in_seq=0 out_seq=0 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.240+0000 7f3b06000640 10 client.17370 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b06000640 1 -- 172.21.15.137:0/3069385632 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55759c61b3f0 con 0x55759c6c26e0 2024-03-26T14:10:21.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b06000640 10 client.17370 handle_command_reply: tid=0 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 client.17370 shutdown 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 msgr2=0x55759c6bdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 0x55759c6bdd90 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009ca0 tx=0x7f3b1802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 shutdown_connections 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c6c26e0 0x55759c6c4b20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6b2220 0x55759c6bdd90 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 --2- 172.21.15.137:0/3069385632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6ac6c0 0x55759c6b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 >> 172.21.15.137:0/3069385632 conn(0x55759c6a3820 msgr2=0x55759c6a2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.241+0000 7f3b06000640 0 client.17370 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.242+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 shutdown_connections 2024-03-26T14:10:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.242+0000 7f3b2a179740 1 -- 172.21.15.137:0/3069385632 wait complete. 2024-03-26T14:10:21.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.242+0000 7f3b2a179740 20 client.17370 trim_cache size 0 max 16384 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 Processor -- start 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 -- start start 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6a2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201aa330 con 0x55759c6ac6c0 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a7e30 con 0x55759c6ad0b0 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.261+0000 7f3b2a179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a5eb0 con 0x55759c6b2220 2024-03-26T14:10:21.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38060/0 (socket says 172.21.15.137:38060) 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 -- 172.21.15.137:0/159098175 learned_addr learned my addr 172.21.15.137:0/159098175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b25400640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 -- 172.21.15.137:0/159098175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 msgr2=0x55759c6c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 -- 172.21.15.137:0/159098175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 msgr2=0x55759c6bdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 -- 172.21.15.137:0/159098175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55759c6b53f0 con 0x55759c6ac6c0 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b25400640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:21.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.262+0000 7f3b24a00640 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6a2900 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3b14097780 tx=0x7f3b140d46d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12eba7300ed70c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.263+0000 7f3b11200640 1 -- 172.21.15.137:0/159098175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1409a200 con 0x55759c6ac6c0 2024-03-26T14:10:21.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.263+0000 7f3b11200640 1 -- 172.21.15.137:0/159098175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b1409e020 con 0x55759c6ac6c0 2024-03-26T14:10:21.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.263+0000 7f3b11200640 1 -- 172.21.15.137:0/159098175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1409a530 con 0x55759c6ac6c0 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 -- 172.21.15.137:0/159098175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 msgr2=0x55759c6a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6a2900 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3b14097780 tx=0x7f3b140d46d0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 -- 172.21.15.137:0/159098175 shutdown_connections 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 --2- 172.21.15.137:0/159098175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6a2900 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 -- 172.21.15.137:0/159098175 >> 172.21.15.137:0/159098175 conn(0x55759c6a3820 msgr2=0x55759c6a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 -- 172.21.15.137:0/159098175 shutdown_connections 2024-03-26T14:10:21.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 -- 172.21.15.137:0/159098175 wait complete. 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.264+0000 7f3b2a179740 1 -- start start 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 Processor -- start 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6bdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6bc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a64b0 con 0x55759c6ac6c0 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a77d0 con 0x55759c6ad0b0 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a6b10 con 0x55759c6b2220 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b2a179740 10 client.0 _ll_register_callbacks cb 0x7f3b18037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:21.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6bdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b24a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6bdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38068/0 (socket says 172.21.15.137:38068) 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.265+0000 7f3b24a00640 1 -- 172.21.15.137:0/484179318 learned_addr learned my addr 172.21.15.137:0/484179318 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b24a00640 1 -- 172.21.15.137:0/484179318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 msgr2=0x55759c6bc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b24a00640 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6bc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b24a00640 1 -- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 msgr2=0x55759c6bc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b24a00640 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b24a00640 1 -- 172.21.15.137:0/484179318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55759c6aeb60 con 0x55759c6ac6c0 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b24a00640 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6bdaa0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3b1409aab0 tx=0x7f3b140d6fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b1280b74b4c803 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b06000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:21.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b06000640 1 -- 172.21.15.137:0/484179318 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14021070 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b06000640 1 -- 172.21.15.137:0/484179318 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3b140cc020 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.266+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b1409e020 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b06000640 1 -- 172.21.15.137:0/484179318 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b1409a710 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 10 client.17376 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55759c71ec30 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b06000640 1 -- 172.21.15.137:0/484179318 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b1409e020 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b06000640 1 -- 172.21.15.137:0/484179318 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3b14097420 con 0x55759c6ac6c0 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 10 client.17376 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 20 client.17376 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:21.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c71efb0 0x55759c7213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 4 client.17376 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:10:21.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.267+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55759c615740 con 0x55759c71efb0 2024-03-26T14:10:21.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.268+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c71efb0 0x55759c7213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.269+0000 7f3b1fe00640 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c71efb0 0x55759c7213d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2239f2c58c882a server_cookie=30a9ce71f5bab82b in_seq=0 out_seq=0 2024-03-26T14:10:21.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.269+0000 7f3b06000640 10 client.17376 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.271+0000 7f3b06000640 1 -- 172.21.15.137:0/484179318 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55759c615740 con 0x55759c71efb0 2024-03-26T14:10:21.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.271+0000 7f3b06000640 10 client.17376 handle_command_reply: tid=0 2024-03-26T14:10:21.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.271+0000 7f3b2a179740 1 client.17376 shutdown 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.271+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 msgr2=0x55759c6bdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.271+0000 7f3b2a179740 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6bdaa0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3b1409aab0 tx=0x7f3b140d6fa0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 shutdown_connections 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55759c71efb0 0x55759c7213d0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55759c6ad0b0 0x55759c6bc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55759c6b2220 0x55759c6bc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 --2- 172.21.15.137:0/484179318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55759c6ac6c0 0x55759c6bdaa0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b06000640 0 client.17376 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 >> 172.21.15.137:0/484179318 conn(0x55759c6a3820 msgr2=0x55759c6ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 shutdown_connections 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 1 -- 172.21.15.137:0/484179318 wait complete. 2024-03-26T14:10:21.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.272+0000 7f3b2a179740 20 client.17376 trim_cache size 0 max 16384 2024-03-26T14:10:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:10:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:10:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "2c0ce644-ddc4-4550-bfc0-88d3f67a9f1f", 2024-03-26T14:10:21.277 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:10:21.277 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:10:21.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3b080a2880 msgr2=0x7f3b080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3b080a2880 0x7f3b080a4d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006010 tx=0x7f3b1802e040 comp rx=0 tx=0).stop 2024-03-26T14:10:21.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 msgr2=0x7f3b20197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 0x7f3b20197730 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400d950 tx=0x7f3b1400de10 comp rx=0 tx=0).stop 2024-03-26T14:10:21.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 shutdown_connections 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3b080a2880 0x7f3b080a4d40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b20102790 0x7f3b20197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3b20103150 0x7f3b2019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 --2- 172.21.15.137:0/870384458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b201015a0 0x7f3b20197730 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 >> 172.21.15.137:0/870384458 conn(0x7f3b200fcb10 msgr2=0x7f3b20105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 shutdown_connections 2024-03-26T14:10:21.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.273+0000 7f3b25e00640 1 -- 172.21.15.137:0/870384458 wait complete. 2024-03-26T14:10:21.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:10:21.294 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:10:21.366 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:21 smithi137 ceph-mon[31539]: pgmap v697: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 -- 172.21.15.137:0/1733685745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 msgr2=0x7f8e541096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1733685745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e541096f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009ca0 tx=0x7f8e4c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 -- 172.21.15.137:0/1733685745 shutdown_connections 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1733685745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e541049d0 0x7f8e54106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1733685745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e541096f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1733685745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e541020a0 0x7f8e54104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 -- 172.21.15.137:0/1733685745 >> 172.21.15.137:0/1733685745 conn(0x7f8e540fba10 msgr2=0x7f8e540fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.508+0000 7f8e53e00640 1 -- 172.21.15.137:0/1733685745 shutdown_connections 2024-03-26T14:10:21.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.509+0000 7f8e53e00640 1 -- 172.21.15.137:0/1733685745 wait complete. 2024-03-26T14:10:21.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 Processor -- start 2024-03-26T14:10:21.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 -- start start 2024-03-26T14:10:21.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e541020a0 0x7f8e54101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e541049d0 0x7f8e54100050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e54100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5410d0e0 con 0x7f8e54107300 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5410cf60 con 0x7f8e541020a0 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5410d260 con 0x7f8e541049d0 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.510+0000 7f8e53400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e54100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e54100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38092/0 (socket says 172.21.15.137:38092) 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 -- 172.21.15.137:0/1291886188 learned_addr learned my addr 172.21.15.137:0/1291886188 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e541020a0 0x7f8e54101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e52000640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e541049d0 0x7f8e54100050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 -- 172.21.15.137:0/1291886188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e541049d0 msgr2=0x7f8e54100050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e541049d0 0x7f8e54100050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 -- 172.21.15.137:0/1291886188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e541020a0 msgr2=0x7f8e54101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e541020a0 0x7f8e54101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 -- 172.21.15.137:0/1291886188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e54100e20 con 0x7f8e54107300 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.511+0000 7f8e53400640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e54100590 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c032380 tx=0x7f8e4c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.512+0000 7f8e4b400640 1 -- 172.21.15.137:0/1291886188 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c041020 con 0x7f8e54107300 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.512+0000 7f8e4b400640 1 -- 172.21.15.137:0/1291886188 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e4c003e20 con 0x7f8e54107300 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.512+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e541a1830 con 0x7f8e54107300 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.512+0000 7f8e4b400640 1 -- 172.21.15.137:0/1291886188 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c039040 con 0x7f8e54107300 2024-03-26T14:10:21.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.512+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e541a1dd0 con 0x7f8e54107300 2024-03-26T14:10:21.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.516+0000 7f8e5c60a740 1 Processor -- start 2024-03-26T14:10:21.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.516+0000 7f8e4b400640 1 -- 172.21.15.137:0/1291886188 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8e4c044020 con 0x7f8e54107300 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 -- start start 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228b050 0x56341228b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 0x563412290540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x563412293040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634121f0250 con 0x56341228ba40 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563412198a30 con 0x56341228b050 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634121ec2f0 con 0x563412290bb0 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 0x563412290540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 0x563412290540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38102/0 (socket says 172.21.15.137:38102) 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.517+0000 7f8e52a00640 1 -- 172.21.15.137:0/1987712085 learned_addr learned my addr 172.21.15.137:0/1987712085 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52000640 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228b050 0x56341228b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52000640 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x563412293040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52a00640 1 -- 172.21.15.137:0/1987712085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 msgr2=0x563412293040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x563412293040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52a00640 1 -- 172.21.15.137:0/1987712085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228b050 msgr2=0x56341228b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228b050 0x56341228b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52a00640 1 -- 172.21.15.137:0/1987712085 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563412183690 con 0x56341228ba40 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52000640 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228b050 0x56341228b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e4b400640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e380a2880 0x7f8e380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e4b400640 1 -- 172.21.15.137:0/1291886188 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8e4c0f0f70 con 0x7f8e54107300 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52000640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e380a2880 0x7f8e380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.518+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 0x563412290540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400bd10 tx=0x7f8e440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476e2a718591a92d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.519+0000 7f8e3fe00640 1 -- 172.21.15.137:0/1987712085 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44004030 con 0x56341228ba40 2024-03-26T14:10:21.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.519+0000 7f8e3fe00640 1 -- 172.21.15.137:0/1987712085 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e440035f0 con 0x56341228ba40 2024-03-26T14:10:21.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.519+0000 7f8e3fe00640 1 -- 172.21.15.137:0/1987712085 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440117b0 con 0x56341228ba40 2024-03-26T14:10:21.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1987712085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 msgr2=0x563412290540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 0x563412290540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400bd10 tx=0x7f8e440107a0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e52000640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e380a2880 0x7f8e380a4d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x56341228c2a0 tx=0x7f8e40014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1987712085 shutdown_connections 2024-03-26T14:10:21.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x563412293040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228ba40 0x563412290540 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1987712085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228b050 0x56341228b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1987712085 >> 172.21.15.137:0/1987712085 conn(0x563412253200 msgr2=0x5634122869c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1987712085 shutdown_connections 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1987712085 wait complete. 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.520+0000 7f8e5c60a740 1 -- start start 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 Processor -- start 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412295000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x563412295540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x56341229c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563412293b60 con 0x56341228b050 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634121abe80 con 0x563412290bb0 2024-03-26T14:10:21.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563412293ce0 con 0x56341228ba40 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e5c60a740 10 client.0 _ll_register_callbacks cb 0x5634121be130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412295000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412295000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38118/0 (socket says 172.21.15.137:38118) 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.521+0000 7f8e52a00640 1 -- 172.21.15.137:0/1849698655 learned_addr learned my addr 172.21.15.137:0/1849698655 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e53400640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x56341229c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52000640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x563412295540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52a00640 1 -- 172.21.15.137:0/1849698655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 msgr2=0x563412295540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x563412295540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52a00640 1 -- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 msgr2=0x56341229c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x56341229c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52a00640 1 -- 172.21.15.137:0/1849698655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563411fce7d0 con 0x56341228b050 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e52a00640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412295000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8e44000c00 tx=0x7f8e4400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59b83431323086f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.522+0000 7f8e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e3cc00640 1 -- 172.21.15.137:0/1849698655 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44004200 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e3cc00640 1 -- 172.21.15.137:0/1849698655 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8e440043a0 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e440035f0 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e3cc00640 1 -- 172.21.15.137:0/1849698655 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e44031050 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e5c60a740 10 client.17388 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563411f043f0 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.523+0000 7f8e3cc00640 1 -- 172.21.15.137:0/1849698655 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e440035f0 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e3cc00640 1 -- 172.21.15.137:0/1849698655 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8e4402c030 con 0x56341228b050 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e5c60a740 10 client.17388 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e5c60a740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e5c60a740 20 client.17388 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122a0b20 0x5634122a2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e5c60a740 4 client.17388 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:21.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634121fd790 con 0x5634122a0b20 2024-03-26T14:10:21.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.524+0000 7f8e52000640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122a0b20 0x5634122a2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.525+0000 7f8e52000640 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122a0b20 0x5634122a2f10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed87f846bd5d9da9 server_cookie=6f9f72c9dbd64722 in_seq=0 out_seq=0 2024-03-26T14:10:21.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.525+0000 7f8e3cc00640 10 client.17388 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.526+0000 7f8e3cc00640 1 -- 172.21.15.137:0/1849698655 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5634121fd790 con 0x5634122a0b20 2024-03-26T14:10:21.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.526+0000 7f8e3cc00640 10 client.17388 handle_command_reply: tid=0 2024-03-26T14:10:21.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.526+0000 7f8e5c60a740 1 client.17388 shutdown 2024-03-26T14:10:21.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 msgr2=0x563412295000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412295000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8e44000c00 tx=0x7f8e4400ba40 comp rx=0 tx=0).stop 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 shutdown_connections 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122a0b20 0x5634122a2f10 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x56341229c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x563412295540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/1849698655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412295000 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 >> 172.21.15.137:0/1849698655 conn(0x563412253200 msgr2=0x563412285bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e3cc00640 0 client.17388 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 shutdown_connections 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 1 -- 172.21.15.137:0/1849698655 wait complete. 2024-03-26T14:10:21.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.527+0000 7f8e5c60a740 20 client.17388 trim_cache size 0 max 16384 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.553+0000 7f8e5c60a740 1 Processor -- start 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 -- start start 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412285e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x56341229c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x5634122a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541ae790 con 0x56341228b050 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541ac290 con 0x563412290bb0 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541aa310 con 0x56341228ba40 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412285e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412285e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38128/0 (socket says 172.21.15.137:38128) 2024-03-26T14:10:21.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.554+0000 7f8e52a00640 1 -- 172.21.15.137:0/3428500804 learned_addr learned my addr 172.21.15.137:0/3428500804 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e53400640 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x5634122a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52000640 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x56341229c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52a00640 1 -- 172.21.15.137:0/3428500804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 msgr2=0x56341229c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52a00640 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x56341229c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52a00640 1 -- 172.21.15.137:0/3428500804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 msgr2=0x5634122a76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52a00640 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x5634122a76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52a00640 1 -- 172.21.15.137:0/3428500804 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563411ec40f0 con 0x56341228b050 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e53400640 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x5634122a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:10:21.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.555+0000 7f8e52a00640 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412285e00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400c010 tx=0x7f8e44010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c479727ef0fb31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.556+0000 7f8e3f400640 1 -- 172.21.15.137:0/3428500804 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44011670 con 0x56341228b050 2024-03-26T14:10:21.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.556+0000 7f8e3f400640 1 -- 172.21.15.137:0/3428500804 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e44011cc0 con 0x56341228b050 2024-03-26T14:10:21.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.556+0000 7f8e3f400640 1 -- 172.21.15.137:0/3428500804 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44005380 con 0x56341228b050 2024-03-26T14:10:21.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.556+0000 7f8e5c60a740 1 -- 172.21.15.137:0/3428500804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 msgr2=0x563412285e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.556+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412285e00 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400c010 tx=0x7f8e44010d50 comp rx=0 tx=0).stop 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 -- 172.21.15.137:0/3428500804 shutdown_connections 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563412290bb0 0x5634122a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56341228ba40 0x56341229c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/3428500804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412285e00 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 -- 172.21.15.137:0/3428500804 >> 172.21.15.137:0/3428500804 conn(0x563412253200 msgr2=0x56341228a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 -- 172.21.15.137:0/3428500804 shutdown_connections 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 -- 172.21.15.137:0/3428500804 wait complete. 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 -- start start 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.557+0000 7f8e5c60a740 1 Processor -- start 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412299a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228ba40 0x563412299fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x56341229afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541aa910 con 0x56341228b050 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541abc30 con 0x56341228ba40 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e541aaf70 con 0x563412290bb0 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e5c60a740 10 client.0 _ll_register_callbacks cb 0x7f8e4400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412299a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412299a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38144/0 (socket says 172.21.15.137:38144) 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.558+0000 7f8e52a00640 1 -- 172.21.15.137:0/459946632 learned_addr learned my addr 172.21.15.137:0/459946632 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52000640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228ba40 0x563412299fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e53400640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x56341229afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52a00640 1 -- 172.21.15.137:0/459946632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 msgr2=0x56341229afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52a00640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x56341229afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52a00640 1 -- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228ba40 msgr2=0x563412299fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52a00640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228ba40 0x563412299fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52a00640 1 -- 172.21.15.137:0/459946632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56341229b5b0 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e52a00640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412299a70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8e44010ca0 tx=0x7f8e4400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71827296908eb35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e3cc00640 1 -- 172.21.15.137:0/459946632 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44011a80 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e3cc00640 1 -- 172.21.15.137:0/459946632 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8e44005080 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.559+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e44011cc0 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e3cc00640 1 -- 172.21.15.137:0/459946632 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e4404f050 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e5c60a740 10 client.17394 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5634122fc2a0 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e3cc00640 1 -- 172.21.15.137:0/459946632 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e44011cc0 con 0x56341228b050 2024-03-26T14:10:21.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e3cc00640 1 -- 172.21.15.137:0/459946632 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8e4402c040 con 0x56341228b050 2024-03-26T14:10:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e5c60a740 10 client.17394 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e5c60a740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.560+0000 7f8e5c60a740 20 client.17394 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.561+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122fca90 0x5634122feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.561+0000 7f8e5c60a740 4 client.17394 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:10:21.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.561+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5634121f7ae0 con 0x5634122fca90 2024-03-26T14:10:21.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.561+0000 7f8e52000640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122fca90 0x5634122feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:21.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.562+0000 7f8e52000640 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122fca90 0x5634122feeb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e35bc781b2547e server_cookie=13d0f823a9623abd in_seq=0 out_seq=0 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.562+0000 7f8e3cc00640 10 client.17394 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.562+0000 7f8e3cc00640 1 -- 172.21.15.137:0/459946632 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5634121f7ae0 con 0x5634122fca90 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.562+0000 7f8e3cc00640 10 client.17394 handle_command_reply: tid=0 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.562+0000 7f8e5c60a740 1 client.17394 shutdown 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 msgr2=0x563412299a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412299a70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8e44010ca0 tx=0x7f8e4400b980 comp rx=0 tx=0).stop 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 shutdown_connections 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5634122fca90 0x5634122feeb0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56341228ba40 0x563412299fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563412290bb0 0x56341229afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 --2- 172.21.15.137:0/459946632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56341228b050 0x563412299a70 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 >> 172.21.15.137:0/459946632 conn(0x563412253200 msgr2=0x563412289c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e3cc00640 0 client.17394 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 shutdown_connections 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 1 -- 172.21.15.137:0/459946632 wait complete. 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.563+0000 7f8e5c60a740 20 client.17394 trim_cache size 0 max 16384 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout: "2c0ce644-ddc4-4550-bfc0-88d3f67a9f1f": { 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "2c0ce644-ddc4-4550-bfc0-88d3f67a9f1f", 2024-03-26T14:10:21.568 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:10:21.569 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:10:21.569 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:10:21.569 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:10:21.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e380a2880 msgr2=0x7f8e380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e380a2880 0x7f8e380a4d40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x56341228c2a0 tx=0x7f8e40014040 comp rx=0 tx=0).stop 2024-03-26T14:10:21.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 msgr2=0x7f8e54100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e54100590 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c032380 tx=0x7f8e4c032820 comp rx=0 tx=0).stop 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 shutdown_connections 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e380a2880 0x7f8e380a4d40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8e54107300 0x7f8e54100590 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e541049d0 0x7f8e54100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 --2- 172.21.15.137:0/1291886188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e541020a0 0x7f8e54101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 >> 172.21.15.137:0/1291886188 conn(0x7f8e540fba10 msgr2=0x7f8e54102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 shutdown_connections 2024-03-26T14:10:21.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:21.564+0000 7f8e53e00640 1 -- 172.21.15.137:0/1291886188 wait complete. 2024-03-26T14:10:21.583 INFO:tasks.cephfs.filesystem:scrub status for tag:2c0ce644-ddc4-4550-bfc0-88d3f67a9f1f - {'path': '/', 'tag': '2c0ce644-ddc4-4550-bfc0-88d3f67a9f1f', 'options': 'recursive,force'} 2024-03-26T14:10:21.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:21 smithi179 ceph-mon[35472]: pgmap v697: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:21.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:21 smithi159 ceph-mon[35501]: pgmap v697: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:22.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:22 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:10:22.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:22 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:22.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:22 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:10:22.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:22 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:22.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:22 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:10:22.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:22 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:10:22.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:22 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:22.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:22 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:10:22.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:22 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:22.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:22 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:10:22.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:22 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:10:22.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:22 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:22.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:22 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:10:22.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:22 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:22.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:22 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:10:23.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:23 smithi179 ceph-mon[35472]: pgmap v698: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:10:23.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:23 smithi159 ceph-mon[35501]: pgmap v698: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:10:23.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:23 smithi137 ceph-mon[31539]: pgmap v698: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:10:25.587 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:25 smithi137 ceph-mon[31539]: pgmap v699: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:10:25.673 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:25 smithi179 ceph-mon[35472]: pgmap v699: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:10:25.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:25 smithi159 ceph-mon[35501]: pgmap v699: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:10:27.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:27 smithi179 ceph-mon[35472]: pgmap v700: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:10:27.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:27 smithi159 ceph-mon[35501]: pgmap v700: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:10:27.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:27 smithi137 ceph-mon[31539]: pgmap v700: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:10:29.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:29 smithi179 ceph-mon[35472]: pgmap v701: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:10:29.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:29 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:10:29.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:29 smithi159 ceph-mon[35501]: pgmap v701: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:10:29.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:29 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:10:29.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:29 smithi137 ceph-mon[31539]: pgmap v701: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:10:29.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:29 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:10:30.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:30.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:30 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:30.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:30.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:30 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:30.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:30.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:30 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:31 smithi179 ceph-mon[35472]: pgmap v702: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:10:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:10:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:10:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:31 smithi159 ceph-mon[35501]: pgmap v702: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:10:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:10:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:10:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:31.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:31 smithi137 ceph-mon[31539]: pgmap v702: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:10:31.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:10:31.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:10:31.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:10:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:32 smithi179 ceph-mon[35472]: pgmap v703: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:10:32.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:32 smithi159 ceph-mon[35501]: pgmap v703: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:10:32.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:32 smithi137 ceph-mon[31539]: pgmap v703: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:10:34.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:34 smithi137 ceph-mon[31539]: pgmap v704: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:10:34.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:34 smithi179 ceph-mon[35472]: pgmap v704: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:10:34.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:34 smithi159 ceph-mon[35501]: pgmap v704: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:10:36.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:36 smithi137 ceph-mon[31539]: pgmap v705: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:10:36.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:36 smithi179 ceph-mon[35472]: pgmap v705: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:10:36.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:36 smithi159 ceph-mon[35501]: pgmap v705: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:10:38.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:38 smithi137 ceph-mon[31539]: pgmap v706: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:10:38.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:38 smithi179 ceph-mon[35472]: pgmap v706: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:10:38.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:38 smithi159 ceph-mon[35501]: pgmap v706: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:10:40.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:40 smithi137 ceph-mon[31539]: pgmap v707: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:40.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:40 smithi179 ceph-mon[35472]: pgmap v707: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:40.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:40 smithi159 ceph-mon[35501]: pgmap v707: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:42.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:42 smithi137 ceph-mon[31539]: pgmap v708: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:10:42.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:42 smithi179 ceph-mon[35472]: pgmap v708: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:10:42.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:42 smithi159 ceph-mon[35501]: pgmap v708: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:10:44.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:44 smithi137 ceph-mon[31539]: pgmap v709: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:44.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:44 smithi179 ceph-mon[35472]: pgmap v709: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:44.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:44 smithi159 ceph-mon[35501]: pgmap v709: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:10:46.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:46 smithi137 ceph-mon[31539]: pgmap v710: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:46.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:46 smithi179 ceph-mon[35472]: pgmap v710: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:46.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:46 smithi159 ceph-mon[35501]: pgmap v710: 129 pgs: 129 active+clean; 788 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:10:48.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:48 smithi179 ceph-mon[35472]: pgmap v711: 129 pgs: 129 active+clean; 788 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:48.952 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:48 smithi159 ceph-mon[35501]: pgmap v711: 129 pgs: 129 active+clean; 788 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:49.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:48 smithi137 ceph-mon[31539]: pgmap v711: 129 pgs: 129 active+clean; 788 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:10:51.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:51 smithi137 ceph-mon[31539]: pgmap v712: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:51.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:51 smithi179 ceph-mon[35472]: pgmap v712: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:51.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:51 smithi159 ceph-mon[35501]: pgmap v712: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:51.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:10:51.584 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:10:51.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.810+0000 7fae1ca00640 1 -- 172.21.15.137:0/3400492651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae181005a0 msgr2=0x7fae181009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.810+0000 7fae1ca00640 1 --2- 172.21.15.137:0/3400492651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae181005a0 0x7fae181009c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fae08009900 tx=0x7fae0802bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:51.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 -- 172.21.15.137:0/3400492651 shutdown_connections 2024-03-26T14:10:51.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 --2- 172.21.15.137:0/3400492651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae18102150 0x7fae181089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 --2- 172.21.15.137:0/3400492651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18101790 0x7fae18101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 --2- 172.21.15.137:0/3400492651 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae181005a0 0x7fae181009c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 -- 172.21.15.137:0/3400492651 >> 172.21.15.137:0/3400492651 conn(0x7fae180fbb30 msgr2=0x7fae180fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:51.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 -- 172.21.15.137:0/3400492651 shutdown_connections 2024-03-26T14:10:51.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.811+0000 7fae1ca00640 1 -- 172.21.15.137:0/3400492651 wait complete. 2024-03-26T14:10:51.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 Processor -- start 2024-03-26T14:10:51.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 -- start start 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae181005a0 0x7fae18197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 0x7fae18197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18102150 0x7fae1819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1810c110 con 0x7fae18101790 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1810bf90 con 0x7fae181005a0 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae1ca00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1810c290 con 0x7fae18102150 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.812+0000 7fae15400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 0x7fae18197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 0x7fae18197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48026/0 (socket says 172.21.15.137:48026) 2024-03-26T14:10:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 -- 172.21.15.137:0/1866247343 learned_addr learned my addr 172.21.15.137:0/1866247343 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15e00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae181005a0 0x7fae18197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae16800640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18102150 0x7fae1819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 -- 172.21.15.137:0/1866247343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18102150 msgr2=0x7fae1819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18102150 0x7fae1819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 -- 172.21.15.137:0/1866247343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae181005a0 msgr2=0x7fae18197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae181005a0 0x7fae18197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 -- 172.21.15.137:0/1866247343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1819f5b0 con 0x7fae18101790 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae16800640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18102150 0x7fae1819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:51.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.813+0000 7fae15400640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 0x7fae18197ed0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fae0400d770 tx=0x7fae0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.814+0000 7fae0ea00640 1 -- 172.21.15.137:0/1866247343 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04017070 con 0x7fae18101790 2024-03-26T14:10:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.814+0000 7fae0ea00640 1 -- 172.21.15.137:0/1866247343 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae040054d0 con 0x7fae18101790 2024-03-26T14:10:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.814+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1819f8a0 con 0x7fae18101790 2024-03-26T14:10:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.814+0000 7fae0ea00640 1 -- 172.21.15.137:0/1866247343 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04004cf0 con 0x7fae18101790 2024-03-26T14:10:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.814+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1819fda0 con 0x7fae18101790 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.817+0000 7fae0ea00640 1 -- 172.21.15.137:0/1866247343 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fae04004e90 con 0x7fae18101790 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.817+0000 7fae1f7c6740 1 Processor -- start 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.817+0000 7fae1f7c6740 1 -- start start 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cb22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb2870 0x562b62cb7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cb9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62c17250 con 0x562b62cb1e80 2024-03-26T14:10:51.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62bbfa30 con 0x562b62cb79e0 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62c132f0 con 0x562b62cb2870 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae16800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cb22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae16800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cb22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48040/0 (socket says 172.21.15.137:48040) 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae16800640 1 -- 172.21.15.137:0/816269025 learned_addr learned my addr 172.21.15.137:0/816269025 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae15e00640 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb2870 0x562b62cb7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae15400640 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cb9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae16800640 1 -- 172.21.15.137:0/816269025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb2870 msgr2=0x562b62cb7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.818+0000 7fae16800640 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb2870 0x562b62cb7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae16800640 1 -- 172.21.15.137:0/816269025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 msgr2=0x562b62cb9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae16800640 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cb9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae16800640 1 -- 172.21.15.137:0/816269025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62baa690 con 0x562b62cb1e80 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae0ea00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadfc0a2800 0x7fadfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae16800640 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cb22a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fae10009ea0 tx=0x7fae10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8704d27682e11a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae0ea00640 1 -- 172.21.15.137:0/1866247343 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae040ce980 con 0x7fae18101790 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae03000640 1 -- 172.21.15.137:0/816269025 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10004d20 con 0x562b62cb1e80 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae15e00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadfc0a2800 0x7fadfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae03000640 1 -- 172.21.15.137:0/816269025 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae100054d0 con 0x562b62cb1e80 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.819+0000 7fae03000640 1 -- 172.21.15.137:0/816269025 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae10011820 con 0x562b62cb1e80 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 -- 172.21.15.137:0/816269025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 msgr2=0x562b62cb22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae15e00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadfc0a2800 0x7fadfc0a4cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae0802c240 tx=0x7fae0802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cb22a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fae10009ea0 tx=0x7fae10009f90 comp rx=0 tx=0).stop 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 -- 172.21.15.137:0/816269025 shutdown_connections 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cb9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb2870 0x562b62cb7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/816269025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cb22a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.820+0000 7fae1f7c6740 1 -- 172.21.15.137:0/816269025 >> 172.21.15.137:0/816269025 conn(0x562b62c7a200 msgr2=0x562b62cad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.821+0000 7fae1f7c6740 1 -- 172.21.15.137:0/816269025 shutdown_connections 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.821+0000 7fae1f7c6740 1 -- 172.21.15.137:0/816269025 wait complete. 2024-03-26T14:10:51.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.821+0000 7fae1f7c6740 1 -- start start 2024-03-26T14:10:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.821+0000 7fae1f7c6740 1 Processor -- start 2024-03-26T14:10:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb1e80 0x562b62cbbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cbc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62cba990 con 0x562b62cb2870 2024-03-26T14:10:51.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62bd2e80 con 0x562b62cb1e80 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62cbab10 con 0x562b62cb79e0 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae1f7c6740 10 client.0 _ll_register_callbacks cb 0x562b62be5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae15400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cbc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae15400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cbc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48048/0 (socket says 172.21.15.137:48048) 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.822+0000 7fae15400640 1 -- 172.21.15.137:0/2312619584 learned_addr learned my addr 172.21.15.137:0/2312619584 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15e00640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb1e80 0x562b62cbbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15400640 1 -- 172.21.15.137:0/2312619584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 msgr2=0x562b62cc3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae16800640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15400640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15400640 1 -- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb1e80 msgr2=0x562b62cbbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15400640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb1e80 0x562b62cbbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15400640 1 -- 172.21.15.137:0/2312619584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b629f57d0 con 0x562b62cb2870 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15e00640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb1e80 0x562b62cbbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae15400640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cbc3c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fae04009610 tx=0x7fae0409add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f0cd244c94edaeb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fadf7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fadf7400640 1 -- 172.21.15.137:0/2312619584 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04020070 con 0x562b62cb2870 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fadf7400640 1 -- 172.21.15.137:0/2312619584 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae0409a360 con 0x562b62cb2870 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.823+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae100054d0 con 0x562b62cb2870 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fadf7400640 1 -- 172.21.15.137:0/2312619584 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae040b8050 con 0x562b62cb2870 2024-03-26T14:10:51.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 10 client.17406 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562b6292b3f0 con 0x562b62cb2870 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fadf7400640 1 -- 172.21.15.137:0/2312619584 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae100054d0 con 0x562b62cb2870 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fadf7400640 1 -- 172.21.15.137:0/2312619584 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae040b0020 con 0x562b62cb2870 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 10 client.17406 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 20 client.17406 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62cc7c30 0x562b62cca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 4 client.17406 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:51.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.824+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b62c24790 con 0x562b62cc7c30 2024-03-26T14:10:51.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.825+0000 7fae15e00640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62cc7c30 0x562b62cca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.826+0000 7fadf7400640 10 client.17406 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:51.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.826+0000 7fae15e00640 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62cc7c30 0x562b62cca0d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7e952e02eddf24c server_cookie=e38fb01d4eab7af0 in_seq=0 out_seq=0 2024-03-26T14:10:51.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.827+0000 7fadf7400640 1 -- 172.21.15.137:0/2312619584 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562b62c24790 con 0x562b62cc7c30 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.827+0000 7fadf7400640 10 client.17406 handle_command_reply: tid=0 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.827+0000 7fae1f7c6740 1 client.17406 shutdown 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 msgr2=0x562b62cbc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cbc3c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fae04009610 tx=0x7fae0409add0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 shutdown_connections 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62cc7c30 0x562b62cca0d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cbc3c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2312619584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb1e80 0x562b62cbbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 >> 172.21.15.137:0/2312619584 conn(0x562b62c7a200 msgr2=0x562b62caca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:51.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fadf7400640 0 client.17406 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:51.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 shutdown_connections 2024-03-26T14:10:51.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2312619584 wait complete. 2024-03-26T14:10:51.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.828+0000 7fae1f7c6740 20 client.17406 trim_cache size 0 max 16384 2024-03-26T14:10:51.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 Processor -- start 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 -- start start 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb1e80 0x562b62cac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cc33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181ac7c0 con 0x562b62cb2870 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181aa2c0 con 0x562b62cb79e0 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a8340 con 0x562b62cb1e80 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae15400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cc33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae15400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cc33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48062/0 (socket says 172.21.15.137:48062) 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.848+0000 7fae15400640 1 -- 172.21.15.137:0/2897453182 learned_addr learned my addr 172.21.15.137:0/2897453182 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae16800640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15400640 1 -- 172.21.15.137:0/2897453182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb1e80 msgr2=0x562b62cac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15e00640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb1e80 0x562b62cac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15400640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb1e80 0x562b62cac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15400640 1 -- 172.21.15.137:0/2897453182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 msgr2=0x562b62cac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15400640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15400640 1 -- 172.21.15.137:0/2897453182 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b628eb0f0 con 0x562b62cb2870 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae16800640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15e00640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb1e80 0x562b62cac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:51.851 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.849+0000 7fae15400640 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cc33c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fae0409aea0 tx=0x7fae040a37d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0861106e17ae768 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae02600640 1 -- 172.21.15.137:0/2897453182 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04020020 con 0x562b62cb2870 2024-03-26T14:10:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae02600640 1 -- 172.21.15.137:0/2897453182 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae0409a650 con 0x562b62cb2870 2024-03-26T14:10:51.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae02600640 1 -- 172.21.15.137:0/2897453182 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040d0e60 con 0x562b62cb2870 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2897453182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 msgr2=0x562b62cc33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cc33c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fae0409aea0 tx=0x7fae040a37d0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2897453182 shutdown_connections 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb79e0 0x562b62cac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb2870 0x562b62cc33c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/2897453182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb1e80 0x562b62cac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.850+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2897453182 >> 172.21.15.137:0/2897453182 conn(0x562b62c7a200 msgr2=0x562b62cb12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.851+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2897453182 shutdown_connections 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.851+0000 7fae1f7c6740 1 -- 172.21.15.137:0/2897453182 wait complete. 2024-03-26T14:10:51.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.851+0000 7fae1f7c6740 1 -- start start 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.851+0000 7fae1f7c6740 1 Processor -- start 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cc1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb2870 0x562b62cc1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a8940 con 0x562b62cb1e80 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a9c60 con 0x562b62cb2870 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181a8fa0 con 0x562b62cb79e0 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae1f7c6740 10 client.0 _ll_register_callbacks cb 0x7fae04009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cc1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cc1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48078/0 (socket says 172.21.15.137:48078) 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 -- 172.21.15.137:0/183162482 learned_addr learned my addr 172.21.15.137:0/183162482 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15400640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb2870 0x562b62cc1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 -- 172.21.15.137:0/183162482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 msgr2=0x562b62cc26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 -- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb2870 msgr2=0x562b62cc1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb2870 0x562b62cc1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15e00640 1 -- 172.21.15.137:0/183162482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b62cc2c90 con 0x562b62cb1e80 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.852+0000 7fae15400640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb2870 0x562b62cc1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fae15e00640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cc1150 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fae08006010 tx=0x7fae0802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80383a59d651bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fadf7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fadf7400640 1 -- 172.21.15.137:0/183162482 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08050070 con 0x562b62cb1e80 2024-03-26T14:10:51.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fadf7400640 1 -- 172.21.15.137:0/183162482 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae08050a40 con 0x562b62cb1e80 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae0409a650 con 0x562b62cb1e80 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fadf7400640 1 -- 172.21.15.137:0/183162482 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae0805e050 con 0x562b62cb1e80 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fae1f7c6740 10 client.17412 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.853+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562b62d2c380 con 0x562b62cb1e80 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fadf7400640 1 -- 172.21.15.137:0/183162482 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae0409a650 con 0x562b62cb1e80 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fadf7400640 1 -- 172.21.15.137:0/183162482 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae0804a040 con 0x562b62cb1e80 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae1f7c6740 10 client.17412 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae1f7c6740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae1f7c6740 20 client.17412 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62d2cd60 0x562b62d2f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:51.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae1f7c6740 4 client.17412 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b62c1eae0 con 0x562b62d2cd60 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.854+0000 7fae15400640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62d2cd60 0x562b62d2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.855+0000 7fae15400640 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62d2cd60 0x562b62d2f180 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223c49373ee4f550 server_cookie=33bf4a10d832d44b in_seq=0 out_seq=0 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.855+0000 7fadf7400640 10 client.17412 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fadf7400640 1 -- 172.21.15.137:0/183162482 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b62c1eae0 con 0x562b62d2cd60 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fadf7400640 10 client.17412 handle_command_reply: tid=0 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 client.17412 shutdown 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 msgr2=0x562b62cc1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cc1150 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fae08006010 tx=0x7fae0802bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 shutdown_connections 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562b62d2cd60 0x562b62d2f180 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b62cb2870 0x562b62cc1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b62cb79e0 0x562b62cc26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 --2- 172.21.15.137:0/183162482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562b62cb1e80 0x562b62cc1150 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fadf7400640 0 client.17412 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 >> 172.21.15.137:0/183162482 conn(0x562b62c7a200 msgr2=0x562b62cac200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 shutdown_connections 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 1 -- 172.21.15.137:0/183162482 wait complete. 2024-03-26T14:10:51.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.856+0000 7fae1f7c6740 20 client.17412 trim_cache size 0 max 16384 2024-03-26T14:10:51.863 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:10:51.863 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:10:51.863 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:10:51.863 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:10:51.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadfc0a2800 msgr2=0x7fadfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadfc0a2800 0x7fadfc0a4cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae0802c240 tx=0x7fae0802c7e0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 msgr2=0x7fae18197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:51.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 0x7fae18197ed0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fae0400d770 tx=0x7fae0400dc30 comp rx=0 tx=0).stop 2024-03-26T14:10:51.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 shutdown_connections 2024-03-26T14:10:51.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fadfc0a2800 0x7fadfc0a4cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae18102150 0x7fae1819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae18101790 0x7fae18197ed0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 --2- 172.21.15.137:0/1866247343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae181005a0 0x7fae18197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:51.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 >> 172.21.15.137:0/1866247343 conn(0x7fae180fbb30 msgr2=0x7fae18068870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:51.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 shutdown_connections 2024-03-26T14:10:51.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:51.857+0000 7fae1ca00640 1 -- 172.21.15.137:0/1866247343 wait complete. 2024-03-26T14:10:51.875 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:10:51.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 -- 172.21.15.137:0/648627375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 msgr2=0x7f282c074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 --2- 172.21.15.137:0/648627375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c074500 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2818009900 tx=0x7f281802bcd0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 -- 172.21.15.137:0/648627375 shutdown_connections 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 --2- 172.21.15.137:0/648627375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f282c109390 0x7f282c10b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 --2- 172.21.15.137:0/648627375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c074500 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 --2- 172.21.15.137:0/648627375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 0x7f282c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 -- 172.21.15.137:0/648627375 >> 172.21.15.137:0/648627375 conn(0x7f282c0fca70 msgr2=0x7f282c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 -- 172.21.15.137:0/648627375 shutdown_connections 2024-03-26T14:10:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.099+0000 7f2831c00640 1 -- 172.21.15.137:0/648627375 wait complete. 2024-03-26T14:10:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.100+0000 7f2831c00640 1 Processor -- start 2024-03-26T14:10:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.100+0000 7f2831c00640 1 -- start start 2024-03-26T14:10:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.100+0000 7f2831c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.100+0000 7f2831c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 0x7f282c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.100+0000 7f2831c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f282c109390 0x7f282c1a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f2831c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c10f260 con 0x7f282c0740a0 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f2831c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c10f0e0 con 0x7f282c109390 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f2831c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c10f3e0 con 0x7f282c075680 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48106/0 (socket says 172.21.15.137:48106) 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 -- 172.21.15.137:0/247691675 learned_addr learned my addr 172.21.15.137:0/247691675 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282b400640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 0x7f282c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f2831200640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f282c109390 0x7f282c1a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 -- 172.21.15.137:0/247691675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 msgr2=0x7f282c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 0x7f282c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 -- 172.21.15.137:0/247691675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f282c109390 msgr2=0x7f282c1a0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f282c109390 0x7f282c1a0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282be00640 1 -- 172.21.15.137:0/247691675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c1a1580 con 0x7f282c0740a0 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.101+0000 7f282b400640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 0x7f282c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.102+0000 7f282be00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c1999b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d950 tx=0x7f281c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.102+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c017070 con 0x7f282c0740a0 2024-03-26T14:10:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.102+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282c1a1870 con 0x7f282c0740a0 2024-03-26T14:10:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.102+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f281c009c30 con 0x7f282c0740a0 2024-03-26T14:10:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.102+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c020060 con 0x7f282c0740a0 2024-03-26T14:10:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.102+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c1a1e10 con 0x7f282c0740a0 2024-03-26T14:10:52.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.104+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c108150 con 0x7f282c0740a0 2024-03-26T14:10:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.106+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f281c0109a0 con 0x7f282c0740a0 2024-03-26T14:10:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.107+0000 7f2828c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28100a2880 0x7f28100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.107+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f281c0cead0 con 0x7f282c0740a0 2024-03-26T14:10:52.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.107+0000 7f282b400640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28100a2880 0x7f28100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.108+0000 7f282b400640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28100a2880 0x7f28100a4d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2818036040 tx=0x7f2818005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.112+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f281c009de0 con 0x7f282c0740a0 2024-03-26T14:10:52.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.369+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f282c0622a0 con 0x7f282c0740a0 2024-03-26T14:10:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.370+0000 7f2828c00640 1 -- 172.21.15.137:0/247691675 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f281c09b7a0 con 0x7f282c0740a0 2024-03-26T14:10:52.371 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:10:52.371 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:10:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:10:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28100a2880 msgr2=0x7f28100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28100a2880 0x7f28100a4d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2818036040 tx=0x7f2818005fd0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 msgr2=0x7f282c1999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c1999b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f281c00d950 tx=0x7f281c00de10 comp rx=0 tx=0).stop 2024-03-26T14:10:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 shutdown_connections 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f28100a2880 0x7f28100a4d40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f282c109390 0x7f282c1a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f282c075680 0x7f282c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 --2- 172.21.15.137:0/247691675 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f282c0740a0 0x7f282c1999b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 >> 172.21.15.137:0/247691675 conn(0x7f282c0fca70 msgr2=0x7f282c105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 shutdown_connections 2024-03-26T14:10:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.373+0000 7f2831c00640 1 -- 172.21.15.137:0/247691675 wait complete. 2024-03-26T14:10:52.391 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:10:52.391 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:10:52.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 -- 172.21.15.137:0/3970475656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 msgr2=0x7fbe44104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 --2- 172.21.15.137:0/3970475656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe44104450 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009900 tx=0x7fbe3402bc40 comp rx=0 tx=0).stop 2024-03-26T14:10:52.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 -- 172.21.15.137:0/3970475656 shutdown_connections 2024-03-26T14:10:52.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 --2- 172.21.15.137:0/3970475656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 0x7fbe441096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 --2- 172.21.15.137:0/3970475656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe44104990 0x7fbe44106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 --2- 172.21.15.137:0/3970475656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe44104450 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 -- 172.21.15.137:0/3970475656 >> 172.21.15.137:0/3970475656 conn(0x7fbe440fba10 msgr2=0x7fbe440fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 -- 172.21.15.137:0/3970475656 shutdown_connections 2024-03-26T14:10:52.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.614+0000 7fbe43e00640 1 -- 172.21.15.137:0/3970475656 wait complete. 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.615+0000 7fbe43e00640 1 Processor -- start 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.615+0000 7fbe43e00640 1 -- start start 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe441999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe44104990 0x7fbe44199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 0x7fbe441a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4410d110 con 0x7fbe44102060 2024-03-26T14:10:52.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4410cf90 con 0x7fbe441072c0 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4410d290 con 0x7fbe44104990 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe42a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe441999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe42a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe441999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48130/0 (socket says 172.21.15.137:48130) 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe42a00640 1 -- 172.21.15.137:0/637791112 learned_addr learned my addr 172.21.15.137:0/637791112 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe42000640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe44104990 0x7fbe44199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe43400640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 0x7fbe441a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.616+0000 7fbe42a00640 1 -- 172.21.15.137:0/637791112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe44104990 msgr2=0x7fbe44199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe42a00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe44104990 0x7fbe44199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe42a00640 1 -- 172.21.15.137:0/637791112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 msgr2=0x7fbe441a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe42a00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 0x7fbe441a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe42a00640 1 -- 172.21.15.137:0/637791112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe441a1610 con 0x7fbe44102060 2024-03-26T14:10:52.619 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe43400640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 0x7fbe441a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe42a00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe441999f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbe34000c00 tx=0x7fbe340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe3b400640 1 -- 172.21.15.137:0/637791112 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34043020 con 0x7fbe44102060 2024-03-26T14:10:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe3b400640 1 -- 172.21.15.137:0/637791112 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe340054c0 con 0x7fbe44102060 2024-03-26T14:10:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe441a18a0 con 0x7fbe44102060 2024-03-26T14:10:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe3b400640 1 -- 172.21.15.137:0/637791112 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34003d40 con 0x7fbe44102060 2024-03-26T14:10:52.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.617+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe441a1df0 con 0x7fbe44102060 2024-03-26T14:10:52.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.621+0000 7fbe4c5a7740 1 Processor -- start 2024-03-26T14:10:52.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 -- start start 2024-03-26T14:10:52.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x559706842e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x559706844040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970684ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597067a72c0 con 0x559706844580 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970674fc90 con 0x559706843bc0 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597067a3360 con 0x5597068429f0 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe3b400640 1 -- 172.21.15.137:0/637791112 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbe34046020 con 0x7fbe44102060 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe42000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970684ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe42000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970684ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48146/0 (socket says 172.21.15.137:48146) 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.622+0000 7fbe42000640 1 -- 172.21.15.137:0/78099690 learned_addr learned my addr 172.21.15.137:0/78099690 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe43400640 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x559706842e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42a00640 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x559706844040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42000640 1 -- 172.21.15.137:0/78099690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 msgr2=0x559706842e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42000640 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x559706842e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42000640 1 -- 172.21.15.137:0/78099690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 msgr2=0x559706844040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42000640 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x559706844040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42000640 1 -- 172.21.15.137:0/78099690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559706774080 con 0x559706844580 2024-03-26T14:10:52.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.623+0000 7fbe42000640 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970684ae10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000bc40 tx=0x7fbe3000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7758676abf5601 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.624+0000 7fbe2fe00640 1 -- 172.21.15.137:0/78099690 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30015070 con 0x559706844580 2024-03-26T14:10:52.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.624+0000 7fbe2fe00640 1 -- 172.21.15.137:0/78099690 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe3001a910 con 0x559706844580 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.624+0000 7fbe2fe00640 1 -- 172.21.15.137:0/78099690 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30011900 con 0x559706844580 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.624+0000 7fbe3b400640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbe280a2800 0x7fbe280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.624+0000 7fbe3b400640 1 -- 172.21.15.137:0/637791112 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbe340f1310 con 0x7fbe44102060 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.624+0000 7fbe43400640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbe280a2800 0x7fbe280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/78099690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 msgr2=0x55970684ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970684ae10 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000bc40 tx=0x7fbe3000f730 comp rx=0 tx=0).stop 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/78099690 shutdown_connections 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x559706844040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970684ae10 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/78099690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x559706842e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/78099690 >> 172.21.15.137:0/78099690 conn(0x55970683dde0 msgr2=0x55970683e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/78099690 shutdown_connections 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe43400640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbe280a2800 0x7fbe280a4cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x559706843ac0 tx=0x7fbe3c009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.625+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/78099690 wait complete. 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 -- start start 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 Processor -- start 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x55970684cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706843bc0 0x55970684d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x5597068541e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970684b590 con 0x559706844580 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597067630d0 con 0x5597068429f0 2024-03-26T14:10:52.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.626+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55970684b710 con 0x559706843bc0 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe4c5a7740 10 client.0 _ll_register_callbacks cb 0x559706767e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x5597068541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x5597068541e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48150/0 (socket says 172.21.15.137:48150) 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 -- 172.21.15.137:0/1342895230 learned_addr learned my addr 172.21.15.137:0/1342895230 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe42000640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706843bc0 0x55970684d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe42a00640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x55970684cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 -- 172.21.15.137:0/1342895230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706843bc0 msgr2=0x55970684d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706843bc0 0x55970684d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 -- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 msgr2=0x55970684cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x55970684cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe43400640 1 -- 172.21.15.137:0/1342895230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597064bb3f0 con 0x559706844580 2024-03-26T14:10:52.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe42000640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706843bc0 0x55970684d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.627+0000 7fbe42a00640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x55970684cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe43400640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x5597068541e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c010940 tx=0x7fbe3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae6fe767affa2c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe2cc00640 1 -- 172.21.15.137:0/1342895230 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00fd50 con 0x559706844580 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe2cc00640 1 -- 172.21.15.137:0/1342895230 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbe3c027400 con 0x559706844580 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe3001a910 con 0x559706844580 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe2cc00640 1 -- 172.21.15.137:0/1342895230 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3c030050 con 0x559706844580 2024-03-26T14:10:52.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe4c5a7740 10 client.17430 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5597065857d0 con 0x559706844580 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.628+0000 7fbe2cc00640 1 -- 172.21.15.137:0/1342895230 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe3001a910 con 0x559706844580 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe2cc00640 1 -- 172.21.15.137:0/1342895230 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbe3c01f510 con 0x559706844580 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe4c5a7740 10 client.17430 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe4c5a7740 10 client.17430 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe4c5a7740 20 client.17430 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559706858810 0x55970685acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe4c5a7740 4 client.17430 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.629+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597067b4800 con 0x559706858810 2024-03-26T14:10:52.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.630+0000 7fbe42a00640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559706858810 0x55970685acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.631+0000 7fbe42a00640 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559706858810 0x55970685acb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=556073d409285574 server_cookie=681acfba500deccb in_seq=0 out_seq=0 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.631+0000 7fbe2cc00640 10 client.17430 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.632+0000 7fbe2cc00640 1 -- 172.21.15.137:0/1342895230 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5597067b4800 con 0x559706858810 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.632+0000 7fbe2cc00640 10 client.17430 handle_command_reply: tid=0 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.632+0000 7fbe4c5a7740 1 client.17430 shutdown 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 msgr2=0x5597068541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x5597068541e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c010940 tx=0x7fbe3c004a40 comp rx=0 tx=0).stop 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 shutdown_connections 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559706858810 0x55970685acb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x5597068541e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706843bc0 0x55970684d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/1342895230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x55970684cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 >> 172.21.15.137:0/1342895230 conn(0x55970683dde0 msgr2=0x55970683d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe2cc00640 0 client.17430 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:52.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 shutdown_connections 2024-03-26T14:10:52.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/1342895230 wait complete. 2024-03-26T14:10:52.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.633+0000 7fbe4c5a7740 20 client.17430 trim_cache size 0 max 16384 2024-03-26T14:10:52.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 Processor -- start 2024-03-26T14:10:52.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 -- start start 2024-03-26T14:10:52.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x55970683cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x5597068541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970685f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441ae700 con 0x559706844580 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441ac200 con 0x559706843bc0 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.650+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441aa280 con 0x5597068429f0 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970685f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970685f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48166/0 (socket says 172.21.15.137:48166) 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 -- 172.21.15.137:0/3445086515 learned_addr learned my addr 172.21.15.137:0/3445086515 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe42000640 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x5597068541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe42a00640 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x55970683cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 -- 172.21.15.137:0/3445086515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 msgr2=0x55970683cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x55970683cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 -- 172.21.15.137:0/3445086515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 msgr2=0x5597068541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x5597068541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe43400640 1 -- 172.21.15.137:0/3445086515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55970683e850 con 0x559706844580 2024-03-26T14:10:52.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.651+0000 7fbe42000640 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x5597068541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.652+0000 7fbe43400640 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970685f5b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c000f80 tx=0x7fbe3c012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fadb90badea1a9f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.652+0000 7fbe2f400640 1 -- 172.21.15.137:0/3445086515 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c038070 con 0x559706844580 2024-03-26T14:10:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.652+0000 7fbe2f400640 1 -- 172.21.15.137:0/3445086515 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe3c01fe50 con 0x559706844580 2024-03-26T14:10:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.652+0000 7fbe2f400640 1 -- 172.21.15.137:0/3445086515 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c02d040 con 0x559706844580 2024-03-26T14:10:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3445086515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 msgr2=0x55970685f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970685f5b0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c000f80 tx=0x7fbe3c012f90 comp rx=0 tx=0).stop 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3445086515 shutdown_connections 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559706843bc0 0x5597068541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706844580 0x55970685f5b0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3445086515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5597068429f0 0x55970683cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3445086515 >> 172.21.15.137:0/3445086515 conn(0x55970683dde0 msgr2=0x55970683c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3445086515 shutdown_connections 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3445086515 wait complete. 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.653+0000 7fbe4c5a7740 1 -- start start 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 Processor -- start 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x559706853b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 0x559706854060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706844580 0x559706852130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441aa880 con 0x559706843bc0 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441abba0 con 0x5597068429f0 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441aaee0 con 0x559706844580 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe4c5a7740 10 client.0 _ll_register_callbacks cb 0x7fbe3c009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe42000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 0x559706854060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe42000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 0x559706854060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48174/0 (socket says 172.21.15.137:48174) 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe42000640 1 -- 172.21.15.137:0/3436587908 learned_addr learned my addr 172.21.15.137:0/3436587908 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.654+0000 7fbe42a00640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x559706853b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe43400640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706844580 0x559706852130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42000640 1 -- 172.21.15.137:0/3436587908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706844580 msgr2=0x559706852130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42000640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706844580 0x559706852130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42000640 1 -- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 msgr2=0x559706853b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42000640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x559706853b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42000640 1 -- 172.21.15.137:0/3436587908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5597068528c0 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42a00640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x559706853b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe42000640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 0x559706854060 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000fc30 tx=0x7fbe3000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f9fb26bbcb5bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe2cc00640 1 -- 172.21.15.137:0/3436587908 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30015040 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe2cc00640 1 -- 172.21.15.137:0/3436587908 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbe30004450 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.655+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe3c01fe50 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe2cc00640 1 -- 172.21.15.137:0/3436587908 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe30005030 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 10 client.17436 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5597068b4ca0 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe2cc00640 1 -- 172.21.15.137:0/3436587908 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbe3c01fe50 con 0x559706843bc0 2024-03-26T14:10:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe2cc00640 1 -- 172.21.15.137:0/3436587908 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbe300035f0 con 0x559706843bc0 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 10 client.17436 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 10 client.17436 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 20 client.17436 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5597068b5410 0x5597068b7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 4 client.17436 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.656+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5597067aeb50 con 0x5597068b5410 2024-03-26T14:10:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.657+0000 7fbe42a00640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5597068b5410 0x5597068b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.658+0000 7fbe42a00640 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5597068b5410 0x5597068b7830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f3b3385127bffd8 server_cookie=40b5bb7db37ab3c0 in_seq=0 out_seq=0 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.658+0000 7fbe2cc00640 10 client.17436 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.658+0000 7fbe2cc00640 1 -- 172.21.15.137:0/3436587908 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5597067aeb50 con 0x5597068b5410 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.658+0000 7fbe2cc00640 10 client.17436 handle_command_reply: tid=0 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.658+0000 7fbe4c5a7740 1 client.17436 shutdown 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 msgr2=0x559706854060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 0x559706854060 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000fc30 tx=0x7fbe3000b6f0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 shutdown_connections 2024-03-26T14:10:52.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5597068b5410 0x5597068b7830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559706844580 0x559706852130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559706843bc0 0x559706854060 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 --2- 172.21.15.137:0/3436587908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5597068429f0 0x559706853b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe2cc00640 0 client.17436 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 >> 172.21.15.137:0/3436587908 conn(0x55970683dde0 msgr2=0x55970683d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 shutdown_connections 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 1 -- 172.21.15.137:0/3436587908 wait complete. 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.659+0000 7fbe4c5a7740 20 client.17436 trim_cache size 0 max 16384 2024-03-26T14:10:52.664 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:10:52.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbe280a2800 msgr2=0x7fbe280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbe280a2800 0x7fbe280a4cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x559706843ac0 tx=0x7fbe3c009700 comp rx=0 tx=0).stop 2024-03-26T14:10:52.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 msgr2=0x7fbe441999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe441999f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fbe34000c00 tx=0x7fbe340042f0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 shutdown_connections 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbe280a2800 0x7fbe280a4cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe441072c0 0x7fbe441a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbe44104990 0x7fbe44199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 --2- 172.21.15.137:0/637791112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe44102060 0x7fbe441999f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 >> 172.21.15.137:0/637791112 conn(0x7fbe440fba10 msgr2=0x7fbe44101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 shutdown_connections 2024-03-26T14:10:52.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.660+0000 7fbe43e00640 1 -- 172.21.15.137:0/637791112 wait complete. 2024-03-26T14:10:52.677 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:10:52.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:10:52.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 -- 172.21.15.137:0/1820587521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b440fff50 msgr2=0x7f7b44100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 --2- 172.21.15.137:0/1820587521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b440fff50 0x7f7b44100370 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b3402bc80 comp rx=0 tx=0).stop 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 -- 172.21.15.137:0/1820587521 shutdown_connections 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 --2- 172.21.15.137:0/1820587521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b441008b0 0x7f7b44100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 --2- 172.21.15.137:0/1820587521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b44101250 0x7f7b441016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 --2- 172.21.15.137:0/1820587521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b440fff50 0x7f7b44100370 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 -- 172.21.15.137:0/1820587521 >> 172.21.15.137:0/1820587521 conn(0x7f7b440fbb30 msgr2=0x7f7b440fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 -- 172.21.15.137:0/1820587521 shutdown_connections 2024-03-26T14:10:52.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.892+0000 7f7b49200640 1 -- 172.21.15.137:0/1820587521 wait complete. 2024-03-26T14:10:52.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.893+0000 7f7b49200640 1 Processor -- start 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 -- start start 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b440fff50 0x7f7b44199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 0x7f7b4419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b44101250 0x7f7b441a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4410f3c0 con 0x7f7b441008b0 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4410f240 con 0x7f7b44101250 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b49200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4410f540 con 0x7f7b440fff50 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b41400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 0x7f7b4419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b41400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 0x7f7b4419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48188/0 (socket says 172.21.15.137:48188) 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.894+0000 7f7b41400640 1 -- 172.21.15.137:0/3937215002 learned_addr learned my addr 172.21.15.137:0/3937215002 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b42800640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b44101250 0x7f7b441a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41e00640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b440fff50 0x7f7b44199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41400640 1 -- 172.21.15.137:0/3937215002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b440fff50 msgr2=0x7f7b44199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41400640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b440fff50 0x7f7b44199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41400640 1 -- 172.21.15.137:0/3937215002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b44101250 msgr2=0x7f7b441a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41400640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b44101250 0x7f7b441a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41400640 1 -- 172.21.15.137:0/3937215002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b441a1760 con 0x7f7b441008b0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41e00640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b440fff50 0x7f7b44199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b41400640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 0x7f7b4419a080 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000e9f0 tx=0x7f7b3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.895+0000 7f7b3aa00640 1 -- 172.21.15.137:0/3937215002 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3000cd80 con 0x7f7b441008b0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.896+0000 7f7b3aa00640 1 -- 172.21.15.137:0/3937215002 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b300108e0 con 0x7f7b441008b0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.896+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b441a1a50 con 0x7f7b441008b0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.896+0000 7f7b3aa00640 1 -- 172.21.15.137:0/3937215002 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30018920 con 0x7f7b441008b0 2024-03-26T14:10:52.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.896+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b441a1fa0 con 0x7f7b441008b0 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.899+0000 7f7b4be33740 1 Processor -- start 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 -- start start 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b3aa00640 1 -- 172.21.15.137:0/3937215002 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7b30010a90 con 0x7f7b441008b0 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd23ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd23fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd246c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfcd1a32c0 con 0x55bfcd23e810 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfcd14bc90 con 0x55bfcd23f9e0 2024-03-26T14:10:52.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b4be33740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfcd19f360 con 0x55bfcd2403a0 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b42800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd23ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b42800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd23ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48200/0 (socket says 172.21.15.137:48200) 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.900+0000 7f7b42800640 1 -- 172.21.15.137:0/2582288270 learned_addr learned my addr 172.21.15.137:0/2582288270 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd23fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b41400640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd246c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b42800640 1 -- 172.21.15.137:0/2582288270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 msgr2=0x55bfcd246c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b42800640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd246c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b42800640 1 -- 172.21.15.137:0/2582288270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 msgr2=0x55bfcd23fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b42800640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd23fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b42800640 1 -- 172.21.15.137:0/2582288270 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfcd170080 con 0x55bfcd23e810 2024-03-26T14:10:52.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd23fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b41400640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd246c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.901+0000 7f7b42800640 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd23ec30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009fd0 tx=0x7f7b3c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facebbffc9fa2b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.902+0000 7f7b3aa00640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b280a2800 0x7f7b280a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.902+0000 7f7b2f400640 1 -- 172.21.15.137:0/2582288270 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c004d20 con 0x55bfcd23e810 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.902+0000 7f7b2f400640 1 -- 172.21.15.137:0/2582288270 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3c004050 con 0x55bfcd23e810 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.902+0000 7f7b2f400640 1 -- 172.21.15.137:0/2582288270 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c005280 con 0x55bfcd23e810 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.902+0000 7f7b3aa00640 1 -- 172.21.15.137:0/3937215002 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b300cf1d0 con 0x7f7b441008b0 2024-03-26T14:10:52.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.902+0000 7f7b41e00640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b280a2800 0x7f7b280a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 -- 172.21.15.137:0/2582288270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 msgr2=0x55bfcd23ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd23ec30 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009fd0 tx=0x7f7b3c0118d0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 -- 172.21.15.137:0/2582288270 shutdown_connections 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd23fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd246c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2582288270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd23ec30 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 -- 172.21.15.137:0/2582288270 >> 172.21.15.137:0/2582288270 conn(0x55bfcd239c00 msgr2=0x55bfcd23a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b41e00640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b280a2800 0x7f7b280a4cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7b34000c00 tx=0x7f7b3402c7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 -- 172.21.15.137:0/2582288270 shutdown_connections 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 -- 172.21.15.137:0/2582288270 wait complete. 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.903+0000 7f7b4be33740 1 -- start start 2024-03-26T14:10:52.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 Processor -- start 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd248a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd248fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd24ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfcd2473b0 con 0x55bfcd23e810 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfcd15f0d0 con 0x55bfcd23f9e0 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfcd247530 con 0x55bfcd2403a0 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b4be33740 10 client.0 _ll_register_callbacks cb 0x55bfcd163e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b41e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd248a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b41e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd248a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48202/0 (socket says 172.21.15.137:48202) 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.904+0000 7f7b41e00640 1 -- 172.21.15.137:0/2023244024 learned_addr learned my addr 172.21.15.137:0/2023244024 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41400640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd248fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b42800640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd24ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41e00640 1 -- 172.21.15.137:0/2023244024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 msgr2=0x55bfcd24ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd24ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41e00640 1 -- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 msgr2=0x55bfcd248fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd248fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41e00640 1 -- 172.21.15.137:0/2023244024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfcceb73f0 con 0x55bfcd23e810 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b42800640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd24ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd248a80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7b34035000 tx=0x7f7b3402bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61940e92f844642 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b23e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.905+0000 7f7b23e00640 1 -- 172.21.15.137:0/2023244024 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3404e030 con 0x55bfcd23e810 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.906+0000 7f7b23e00640 1 -- 172.21.15.137:0/2023244024 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b340046c0 con 0x55bfcd23e810 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.906+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b3c004050 con 0x55bfcd23e810 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.906+0000 7f7b23e00640 1 -- 172.21.15.137:0/2023244024 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3405b050 con 0x55bfcd23e810 2024-03-26T14:10:52.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.906+0000 7f7b4be33740 10 client.17448 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.906+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bfccf817d0 con 0x55bfcd23e810 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.906+0000 7f7b23e00640 1 -- 172.21.15.137:0/2023244024 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3c004050 con 0x55bfcd23e810 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b23e00640 1 -- 172.21.15.137:0/2023244024 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7b340039d0 con 0x55bfcd23e810 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b4be33740 10 client.17448 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b4be33740 10 client.17448 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b4be33740 20 client.17448 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2544a0 0x55bfcd256910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b4be33740 4 client.17448 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfcd1b0800 con 0x55bfcd2544a0 2024-03-26T14:10:52.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.907+0000 7f7b41400640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2544a0 0x55bfcd256910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.908+0000 7f7b41400640 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2544a0 0x55bfcd256910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0f71a6819f39cc6 server_cookie=cd61053403b88030 in_seq=0 out_seq=0 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.908+0000 7f7b23e00640 10 client.17448 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.910+0000 7f7b23e00640 1 -- 172.21.15.137:0/2023244024 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55bfcd1b0800 con 0x55bfcd2544a0 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.910+0000 7f7b23e00640 10 client.17448 handle_command_reply: tid=0 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.910+0000 7f7b4be33740 1 client.17448 shutdown 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.910+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 msgr2=0x55bfcd248a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.910+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd248a80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7b34035000 tx=0x7f7b3402bc80 comp rx=0 tx=0).stop 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 shutdown_connections 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2544a0 0x55bfcd256910 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23f9e0 0x55bfcd248fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd24ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b23e00640 0 client.17448 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:52.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2023244024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23e810 0x55bfcd248a80 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 >> 172.21.15.137:0/2023244024 conn(0x55bfcd239c00 msgr2=0x55bfcd238eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 shutdown_connections 2024-03-26T14:10:52.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 1 -- 172.21.15.137:0/2023244024 wait complete. 2024-03-26T14:10:52.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.911+0000 7f7b4be33740 20 client.17448 trim_cache size 0 max 16384 2024-03-26T14:10:52.928 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.928+0000 7f7b4be33740 1 Processor -- start 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 -- start start 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd242c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd243150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441ae8d0 con 0x55bfcd23f9e0 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441ac3d0 con 0x55bfcd23e810 2024-03-26T14:10:52.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b4be33740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441aa450 con 0x55bfcd2403a0 2024-03-26T14:10:52.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b41400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b41400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48210/0 (socket says 172.21.15.137:48210) 2024-03-26T14:10:52.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b41400640 1 -- 172.21.15.137:0/2266069909 learned_addr learned my addr 172.21.15.137:0/2266069909 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd242c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.929+0000 7f7b42800640 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd243150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b41400640 1 -- 172.21.15.137:0/2266069909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 msgr2=0x55bfcd243150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b41400640 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd243150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b41400640 1 -- 172.21.15.137:0/2266069909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 msgr2=0x55bfcd242c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b41400640 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd242c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b41400640 1 -- 172.21.15.137:0/2266069909 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfcd235510 con 0x55bfcd23f9e0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b41400640 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24ff50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f7b30097c00 tx=0x7f7b30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d424325fda5e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b2ea00640 1 -- 172.21.15.137:0/2266069909 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3009b7a0 con 0x55bfcd23f9e0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b2ea00640 1 -- 172.21.15.137:0/2266069909 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3009f8f0 con 0x55bfcd23f9e0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.930+0000 7f7b2ea00640 1 -- 172.21.15.137:0/2266069909 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30097e10 con 0x55bfcd23f9e0 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 -- 172.21.15.137:0/2266069909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 msgr2=0x55bfcd24ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24ff50 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f7b30097c00 tx=0x7f7b30000bc0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 -- 172.21.15.137:0/2266069909 shutdown_connections 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd2403a0 0x55bfcd243150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24ff50 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 --2- 172.21.15.137:0/2266069909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd242c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 -- 172.21.15.137:0/2266069909 >> 172.21.15.137:0/2266069909 conn(0x55bfcd239c00 msgr2=0x55bfcd23d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 -- 172.21.15.137:0/2266069909 shutdown_connections 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.931+0000 7f7b4be33740 1 -- 172.21.15.137:0/2266069909 wait complete. 2024-03-26T14:10:52.933 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.932+0000 7f7b4be33740 1 -- start start 2024-03-26T14:10:52.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.932+0000 7f7b4be33740 1 Processor -- start 2024-03-26T14:10:52.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.932+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd24dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.932+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.932+0000 7f7b4be33740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 0x55bfcd24f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b4be33740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441aaa50 con 0x55bfcd2403a0 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b4be33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441abd70 con 0x55bfcd23e810 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b4be33740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b441ab0b0 con 0x55bfcd23f9e0 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b4be33740 10 client.0 _ll_register_callbacks cb 0x7f7b34042050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 0x55bfcd24f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 0x55bfcd24f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48218/0 (socket says 172.21.15.137:48218) 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 -- 172.21.15.137:0/371048019 learned_addr learned my addr 172.21.15.137:0/371048019 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b41e00640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd24dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b41400640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 -- 172.21.15.137:0/371048019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd23f9e0 msgr2=0x55bfcd24e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 -- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 msgr2=0x55bfcd24dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd24dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b42800640 1 -- 172.21.15.137:0/371048019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfcd24f870 con 0x55bfcd2403a0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.933+0000 7f7b41e00640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd24dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b42800640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 0x55bfcd24f280 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c005c10 tx=0x7f7b3c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ab3ac8ec96a1aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b23e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b23e00640 1 -- 172.21.15.137:0/371048019 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c028070 con 0x55bfcd2403a0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b23e00640 1 -- 172.21.15.137:0/371048019 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b3c0054d0 con 0x55bfcd2403a0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b3009f8f0 con 0x55bfcd2403a0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b23e00640 1 -- 172.21.15.137:0/371048019 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3c0056d0 con 0x55bfcd2403a0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b4be33740 10 client.17454 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.934+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55bfcd2b8d00 con 0x55bfcd2403a0 2024-03-26T14:10:52.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b23e00640 1 -- 172.21.15.137:0/371048019 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3009f8f0 con 0x55bfcd2403a0 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b23e00640 1 -- 172.21.15.137:0/371048019 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7b3c024440 con 0x55bfcd2403a0 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b4be33740 10 client.17454 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b4be33740 10 client.17454 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b4be33740 20 client.17454 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b4be33740 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2b94b0 0x55bfcd2bb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b4be33740 4 client.17454 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfcd1aab50 con 0x55bfcd2b94b0 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.935+0000 7f7b41e00640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2b94b0 0x55bfcd2bb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.936+0000 7f7b23e00640 10 client.17454 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:52.939 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.936+0000 7f7b41e00640 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2b94b0 0x55bfcd2bb8a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e8b18f101f9f03d server_cookie=bdae334378992bc4 in_seq=0 out_seq=0 2024-03-26T14:10:52.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b23e00640 1 -- 172.21.15.137:0/371048019 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfcd1aab50 con 0x55bfcd2b94b0 2024-03-26T14:10:52.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b23e00640 10 client.17454 handle_command_reply: tid=0 2024-03-26T14:10:52.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 client.17454 shutdown 2024-03-26T14:10:52.940 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 msgr2=0x55bfcd24f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 0x55bfcd24f280 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c005c10 tx=0x7f7b3c003ce0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 shutdown_connections 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55bfcd2b94b0 0x55bfcd2bb8a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55bfcd2403a0 0x55bfcd24f280 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfcd23f9e0 0x55bfcd24e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 --2- 172.21.15.137:0/371048019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfcd23e810 0x55bfcd24dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 >> 172.21.15.137:0/371048019 conn(0x55bfcd239c00 msgr2=0x55bfcd238620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b23e00640 0 client.17454 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 shutdown_connections 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.937+0000 7f7b4be33740 1 -- 172.21.15.137:0/371048019 wait complete. 2024-03-26T14:10:52.941 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b4be33740 20 client.17454 trim_cache size 0 max 16384 2024-03-26T14:10:52.942 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:10:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b280a2800 msgr2=0x7f7b280a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b280a2800 0x7f7b280a4cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7b34000c00 tx=0x7f7b3402c7d0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 msgr2=0x7f7b4419a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:52.942 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 0x7f7b4419a080 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000e9f0 tx=0x7f7b3000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 shutdown_connections 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b280a2800 0x7f7b280a4cc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b44101250 0x7f7b441a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b441008b0 0x7f7b4419a080 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 --2- 172.21.15.137:0/3937215002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b440fff50 0x7f7b44199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.938+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 >> 172.21.15.137:0/3937215002 conn(0x7f7b440fbb30 msgr2=0x7f7b440fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.939+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 shutdown_connections 2024-03-26T14:10:52.943 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:52.939+0000 7f7b49200640 1 -- 172.21.15.137:0/3937215002 wait complete. 2024-03-26T14:10:52.958 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:10:52.959 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 -- 172.21.15.137:0/3203472460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db00740a0 msgr2=0x7f6db0074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 --2- 172.21.15.137:0/3203472460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db00740a0 0x7f6db0074500 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c009900 tx=0x7f6d9c02bcd0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 -- 172.21.15.137:0/3203472460 shutdown_connections 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 --2- 172.21.15.137:0/3203472460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6db0109410 0x7f6db010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 --2- 172.21.15.137:0/3203472460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db00740a0 0x7f6db0074500 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 --2- 172.21.15.137:0/3203472460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db0075680 0x7f6db0073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 -- 172.21.15.137:0/3203472460 >> 172.21.15.137:0/3203472460 conn(0x7f6db00fcb10 msgr2=0x7f6db00fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 -- 172.21.15.137:0/3203472460 shutdown_connections 2024-03-26T14:10:53.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.161+0000 7f6db5400640 1 -- 172.21.15.137:0/3203472460 wait complete. 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 Processor -- start 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 -- start start 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db00740a0 0x7f6db01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 0x7f6db0199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6db0109410 0x7f6db01a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db010f300 con 0x7f6db0075680 2024-03-26T14:10:53.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db010f180 con 0x7f6db0109410 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.162+0000 7f6db5400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db010f480 con 0x7f6db00740a0 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 0x7f6db0199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 0x7f6db0199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48238/0 (socket says 172.21.15.137:48238) 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 -- 172.21.15.137:0/2178406837 learned_addr learned my addr 172.21.15.137:0/2178406837 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6db4a00640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6db0109410 0x7f6db01a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6dafe00640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db00740a0 0x7f6db01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 -- 172.21.15.137:0/2178406837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db00740a0 msgr2=0x7f6db01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db00740a0 0x7f6db01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 -- 172.21.15.137:0/2178406837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6db0109410 msgr2=0x7f6db01a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6db0109410 0x7f6db01a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 -- 172.21.15.137:0/2178406837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db01a15c0 con 0x7f6db0075680 2024-03-26T14:10:53.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.163+0000 7f6daf400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 0x7f6db0199ee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c02dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.164+0000 7f6dacc00640 1 -- 172.21.15.137:0/2178406837 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c043020 con 0x7f6db0075680 2024-03-26T14:10:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.164+0000 7f6dacc00640 1 -- 172.21.15.137:0/2178406837 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d9c02de30 con 0x7f6db0075680 2024-03-26T14:10:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.164+0000 7f6dacc00640 1 -- 172.21.15.137:0/2178406837 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c003c40 con 0x7f6db0075680 2024-03-26T14:10:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.164+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db01a1850 con 0x7f6db0075680 2024-03-26T14:10:53.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.164+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db01a1da0 con 0x7f6db0075680 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.167+0000 7f6db9603740 1 Processor -- start 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 -- start start 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6dacc00640 1 -- 172.21.15.137:0/2178406837 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6d9c046020 con 0x7f6db0075680 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd026679f0 0x55dd02667e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd02668bc0 0x55dd02669040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd025cc2c0 con 0x55dd02669580 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd02574c90 con 0x55dd02668bc0 2024-03-26T14:10:53.168 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6db9603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd025c8360 con 0x55dd026679f0 2024-03-26T14:10:53.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6daf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6daf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48252/0 (socket says 172.21.15.137:48252) 2024-03-26T14:10:53.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.168+0000 7f6daf400640 1 -- 172.21.15.137:0/665046226 learned_addr learned my addr 172.21.15.137:0/665046226 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6dafe00640 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd02668bc0 0x55dd02669040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6db4a00640 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd026679f0 0x55dd02667e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6daf400640 1 -- 172.21.15.137:0/665046226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd026679f0 msgr2=0x55dd02667e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6daf400640 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd026679f0 0x55dd02667e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6daf400640 1 -- 172.21.15.137:0/665046226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd02668bc0 msgr2=0x55dd02669040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6daf400640 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd02668bc0 0x55dd02669040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6daf400640 1 -- 172.21.15.137:0/665046226 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd02599080 con 0x55dd02669580 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.169+0000 7f6daf400640 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266fe10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0f9210 tx=0x7f6d9c0f9300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8130e8d963546 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.170+0000 7f6da5200640 1 -- 172.21.15.137:0/665046226 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0f30d0 con 0x55dd02669580 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.170+0000 7f6da5200640 1 -- 172.21.15.137:0/665046226 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d9c0fe030 con 0x55dd02669580 2024-03-26T14:10:53.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.170+0000 7f6da5200640 1 -- 172.21.15.137:0/665046226 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c102630 con 0x55dd02669580 2024-03-26T14:10:53.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.170+0000 7f6dacc00640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d940a2800 0x7f6d940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.170+0000 7f6dacc00640 1 -- 172.21.15.137:0/2178406837 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d9c0f1220 con 0x7f6db0075680 2024-03-26T14:10:53.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.170+0000 7f6db4a00640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d940a2800 0x7f6d940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.173 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 -- 172.21.15.137:0/665046226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 msgr2=0x55dd0266fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266fe10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0f9210 tx=0x7f6d9c0f9300 comp rx=0 tx=0).stop 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 -- 172.21.15.137:0/665046226 shutdown_connections 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd02668bc0 0x55dd02669040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266fe10 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 --2- 172.21.15.137:0/665046226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd026679f0 0x55dd02667e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 -- 172.21.15.137:0/665046226 >> 172.21.15.137:0/665046226 conn(0x55dd02662de0 msgr2=0x55dd026631e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 -- 172.21.15.137:0/665046226 shutdown_connections 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 -- 172.21.15.137:0/665046226 wait complete. 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db4a00640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d940a2800 0x7f6d940a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55dd02668ac0 tx=0x7f6da8009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.171+0000 7f6db9603740 1 -- start start 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 Processor -- start 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02671980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02671ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.174 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd02678f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd02670350 con 0x55dd02669580 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd025880d0 con 0x55dd026679f0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd026704d0 con 0x55dd02668bc0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db9603740 10 client.0 _ll_register_callbacks cb 0x55dd0258ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd02678f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd02678f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48268/0 (socket says 172.21.15.137:48268) 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.172+0000 7f6db4a00640 1 -- 172.21.15.137:0/3301200022 learned_addr learned my addr 172.21.15.137:0/3301200022 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02671980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6daf400640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02671ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6db4a00640 1 -- 172.21.15.137:0/3301200022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 msgr2=0x55dd02671ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6db4a00640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02671ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6db4a00640 1 -- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 msgr2=0x55dd02671980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6db4a00640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02671980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6db4a00640 1 -- 172.21.15.137:0/3301200022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd022e03f0 con 0x55dd02669580 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6db4a00640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd02678f00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009700 tx=0x7f6da8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5123771a9a5e23b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6d99600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:53.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.173+0000 7f6d99600640 1 -- 172.21.15.137:0/3301200022 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8004d20 con 0x55dd02669580 2024-03-26T14:10:53.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6d99600640 1 -- 172.21.15.137:0/3301200022 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6da800e8a0 con 0x55dd02669580 2024-03-26T14:10:53.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d9c0fe030 con 0x55dd02669580 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6d99600640 1 -- 172.21.15.137:0/3301200022 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6da802f050 con 0x55dd02669580 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6db9603740 10 client.17466 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dd023aa7d0 con 0x55dd02669580 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6d99600640 1 -- 172.21.15.137:0/3301200022 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d9c0fe030 con 0x55dd02669580 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.174+0000 7f6d99600640 1 -- 172.21.15.137:0/3301200022 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6da800f040 con 0x55dd02669580 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6db9603740 10 client.17466 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6db9603740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6db9603740 20 client.17466 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6db9603740 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd0267d530 0x55dd0267f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6db9603740 4 client.17466 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd025d9800 con 0x55dd0267d530 2024-03-26T14:10:53.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.175+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd0267d530 0x55dd0267f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.176+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd0267d530 0x55dd0267f9a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6c42c61377a122 server_cookie=bce19a270b1b5335 in_seq=0 out_seq=0 2024-03-26T14:10:53.178 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.176+0000 7f6d99600640 10 client.17466 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6d99600640 1 -- 172.21.15.137:0/3301200022 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dd025d9800 con 0x55dd0267d530 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6d99600640 10 client.17466 handle_command_reply: tid=0 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 client.17466 shutdown 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 msgr2=0x55dd02678f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd02678f00 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009700 tx=0x7f6da8004a40 comp rx=0 tx=0).stop 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 shutdown_connections 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd0267d530 0x55dd0267f9a0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd02678f00 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02671ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 --2- 172.21.15.137:0/3301200022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02671980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6d99600640 0 client.17466 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:53.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 >> 172.21.15.137:0/3301200022 conn(0x55dd02662de0 msgr2=0x55dd02662160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 shutdown_connections 2024-03-26T14:10:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.178+0000 7f6db9603740 1 -- 172.21.15.137:0/3301200022 wait complete. 2024-03-26T14:10:53.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.179+0000 7f6db9603740 20 client.17466 trim_cache size 0 max 16384 2024-03-26T14:10:53.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.193+0000 7f6db9603740 1 Processor -- start 2024-03-26T14:10:53.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.193+0000 7f6db9603740 1 -- start start 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db9603740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd0265dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db9603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02678ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db9603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db9603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01ae710 con 0x55dd02669580 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db9603740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01ac210 con 0x55dd026679f0 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db9603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01aa290 con 0x55dd02668bc0 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48284/0 (socket says 172.21.15.137:48284) 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6db4a00640 1 -- 172.21.15.137:0/496368278 learned_addr learned my addr 172.21.15.137:0/496368278 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6dafe00640 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd0265dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.194+0000 7f6daf400640 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02678ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6db4a00640 1 -- 172.21.15.137:0/496368278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 msgr2=0x55dd02678ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6db4a00640 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02678ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6db4a00640 1 -- 172.21.15.137:0/496368278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 msgr2=0x55dd0265dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6db4a00640 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd0265dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6db4a00640 1 -- 172.21.15.137:0/496368278 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd02663b50 con 0x55dd02669580 2024-03-26T14:10:53.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6daf400640 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02678ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:53.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6db4a00640 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266bdf0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6da80093c0 tx=0x7f6da80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dc113fe27419ff server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6d9be00640 1 -- 172.21.15.137:0/496368278 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da8004d70 con 0x55dd02669580 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6d9be00640 1 -- 172.21.15.137:0/496368278 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6da801f8d0 con 0x55dd02669580 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.195+0000 7f6d9be00640 1 -- 172.21.15.137:0/496368278 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800eda0 con 0x55dd02669580 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 -- 172.21.15.137:0/496368278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 msgr2=0x55dd0266bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266bdf0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6da80093c0 tx=0x7f6da80093f0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 -- 172.21.15.137:0/496368278 shutdown_connections 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02669580 0x55dd0266bdf0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02668bc0 0x55dd02678ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 --2- 172.21.15.137:0/496368278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd0265dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 -- 172.21.15.137:0/496368278 >> 172.21.15.137:0/496368278 conn(0x55dd02662de0 msgr2=0x55dd026668d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 -- 172.21.15.137:0/496368278 shutdown_connections 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.196+0000 7f6db9603740 1 -- 172.21.15.137:0/496368278 wait complete. 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 -- start start 2024-03-26T14:10:53.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 Processor -- start 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02678c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 0x55dd02677290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02669580 0x55dd026777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01aa890 con 0x55dd02668bc0 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01abbb0 con 0x55dd026679f0 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db01aaef0 con 0x55dd02669580 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.197+0000 7f6db9603740 10 client.0 _ll_register_callbacks cb 0x7f6da8009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 0x55dd02677290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 0x55dd02677290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48298/0 (socket says 172.21.15.137:48298) 2024-03-26T14:10:53.200 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 -- 172.21.15.137:0/3967598912 learned_addr learned my addr 172.21.15.137:0/3967598912 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02678c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6db4a00640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02669580 0x55dd026777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 -- 172.21.15.137:0/3967598912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02669580 msgr2=0x55dd026777d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02669580 0x55dd026777d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 -- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 msgr2=0x55dd02678c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02678c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 -- 172.21.15.137:0/3967598912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd02670330 con 0x55dd02668bc0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02678c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6daf400640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 0x55dd02677290 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0f91b0 tx=0x7f6d9c0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc27bd2039471b16 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.198+0000 7f6d99600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6d99600640 1 -- 172.21.15.137:0/3967598912 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0b9b70 con 0x55dd02668bc0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6d99600640 1 -- 172.21.15.137:0/3967598912 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d9c0bb970 con 0x55dd02668bc0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6da801f8d0 con 0x55dd02668bc0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6d99600640 1 -- 172.21.15.137:0/3967598912 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d9c0bd230 con 0x55dd02668bc0 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6db9603740 10 client.17472 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:53.201 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dd026d9b10 con 0x55dd02668bc0 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6d99600640 1 -- 172.21.15.137:0/3967598912 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6da801f8d0 con 0x55dd02668bc0 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6d99600640 1 -- 172.21.15.137:0/3967598912 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6d9c0b9f40 con 0x55dd02668bc0 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6db9603740 10 client.17472 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6db9603740 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.199+0000 7f6db9603740 20 client.17472 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.200+0000 7f6db9603740 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd026da1d0 0x55dd026dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.200+0000 7f6db9603740 4 client.17472 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.200+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd025d3b50 con 0x55dd026da1d0 2024-03-26T14:10:53.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.200+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd026da1d0 0x55dd026dc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:53.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.201+0000 7f6dafe00640 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd026da1d0 0x55dd026dc5c0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15259c365bcf6299 server_cookie=5245cfab81742510 in_seq=0 out_seq=0 2024-03-26T14:10:53.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.201+0000 7f6d99600640 10 client.17472 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:53.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6d99600640 1 -- 172.21.15.137:0/3967598912 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd025d3b50 con 0x55dd026da1d0 2024-03-26T14:10:53.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6d99600640 10 client.17472 handle_command_reply: tid=0 2024-03-26T14:10:53.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 client.17472 shutdown 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 msgr2=0x55dd02677290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 0x55dd02677290 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0f91b0 tx=0x7f6d9c0044f0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 shutdown_connections 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55dd026da1d0 0x55dd026dc5c0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dd02669580 0x55dd026777d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dd02668bc0 0x55dd02677290 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 --2- 172.21.15.137:0/3967598912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd026679f0 0x55dd02678c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 >> 172.21.15.137:0/3967598912 conn(0x55dd02662de0 msgr2=0x55dd026618d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6d99600640 0 client.17472 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 shutdown_connections 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 1 -- 172.21.15.137:0/3967598912 wait complete. 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.202+0000 7f6db9603740 20 client.17472 trim_cache size 0 max 16384 2024-03-26T14:10:53.204 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d940a2800 msgr2=0x7f6d940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d940a2800 0x7f6d940a4cc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55dd02668ac0 tx=0x7f6da8009580 comp rx=0 tx=0).stop 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 msgr2=0x7f6db0199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 0x7f6db0199ee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c000c00 tx=0x7f6d9c02dc40 comp rx=0 tx=0).stop 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 shutdown_connections 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d940a2800 0x7f6d940a4cc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6db0109410 0x7f6db01a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6db0075680 0x7f6db0199ee0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 --2- 172.21.15.137:0/2178406837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6db00740a0 0x7f6db01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:53.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 >> 172.21.15.137:0/2178406837 conn(0x7f6db00fcb10 msgr2=0x7f6db01053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:53.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 shutdown_connections 2024-03-26T14:10:53.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:53.203+0000 7f6db5400640 1 -- 172.21.15.137:0/2178406837 wait complete. 2024-03-26T14:10:53.218 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:53 smithi137 ceph-mon[31539]: pgmap v713: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 KiB/s wr, 6 op/s 2024-03-26T14:10:53.219 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:53 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/247691675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:10:53.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:53 smithi179 ceph-mon[35472]: pgmap v713: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 KiB/s wr, 6 op/s 2024-03-26T14:10:53.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:53 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/247691675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:10:53.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:53 smithi159 ceph-mon[35501]: pgmap v713: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 KiB/s wr, 6 op/s 2024-03-26T14:10:53.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:53 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/247691675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:10:54.222 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:10:54.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:10:54.223 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:10:54.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 -- 172.21.15.137:0/3412512642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320107bc0 msgr2=0x7ff320107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 --2- 172.21.15.137:0/3412512642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320107bc0 0x7ff320107fe0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff310009900 tx=0x7ff31002bc30 comp rx=0 tx=0).stop 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 -- 172.21.15.137:0/3412512642 shutdown_connections 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 --2- 172.21.15.137:0/3412512642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff320109770 0x7ff320110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 --2- 172.21.15.137:0/3412512642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff320108db0 0x7ff320109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 --2- 172.21.15.137:0/3412512642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320107bc0 0x7ff320107fe0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 -- 172.21.15.137:0/3412512642 >> 172.21.15.137:0/3412512642 conn(0x7ff320076b70 msgr2=0x7ff320078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 -- 172.21.15.137:0/3412512642 shutdown_connections 2024-03-26T14:10:54.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.462+0000 7ff325800640 1 -- 172.21.15.137:0/3412512642 wait complete. 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.463+0000 7ff325800640 1 Processor -- start 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.463+0000 7ff325800640 1 -- start start 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff325800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff320107bc0 0x7ff32019de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff325800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff320108db0 0x7ff32019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff325800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 0x7ff3201a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff325800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320113710 con 0x7ff320109770 2024-03-26T14:10:54.465 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff325800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320113590 con 0x7ff320108db0 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff325800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff320113890 con 0x7ff320107bc0 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff324e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 0x7ff3201a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff324e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 0x7ff3201a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48326/0 (socket says 172.21.15.137:48326) 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff324e00640 1 -- 172.21.15.137:0/40703943 learned_addr learned my addr 172.21.15.137:0/40703943 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff31f400640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff320108db0 0x7ff32019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.464+0000 7ff31fe00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff320107bc0 0x7ff32019de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff324e00640 1 -- 172.21.15.137:0/40703943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff320107bc0 msgr2=0x7ff32019de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff324e00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff320107bc0 0x7ff32019de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff324e00640 1 -- 172.21.15.137:0/40703943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff320108db0 msgr2=0x7ff32019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff324e00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff320108db0 0x7ff32019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff324e00640 1 -- 172.21.15.137:0/40703943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3201a5a60 con 0x7ff320109770 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff324e00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 0x7ff3201a53f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff31800cad0 tx=0x7ff31800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff31cc00640 1 -- 172.21.15.137:0/40703943 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800ae20 con 0x7ff320109770 2024-03-26T14:10:54.466 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff31cc00640 1 -- 172.21.15.137:0/40703943 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff318005020 con 0x7ff320109770 2024-03-26T14:10:54.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3201a5d50 con 0x7ff320109770 2024-03-26T14:10:54.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff31cc00640 1 -- 172.21.15.137:0/40703943 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800f6b0 con 0x7ff320109770 2024-03-26T14:10:54.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.465+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3201a6240 con 0x7ff320109770 2024-03-26T14:10:54.468 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.469+0000 7ff31cc00640 1 -- 172.21.15.137:0/40703943 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff3180051d0 con 0x7ff320109770 2024-03-26T14:10:54.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.469+0000 7ff3297fd740 1 Processor -- start 2024-03-26T14:10:54.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.469+0000 7ff3297fd740 1 -- start start 2024-03-26T14:10:54.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03aca0 0x56296c03f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c0422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff3297fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf9feb0 con 0x56296c03a2b0 2024-03-26T14:10:54.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff3297fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf48a60 con 0x56296c03fe10 2024-03-26T14:10:54.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff3297fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf9bf50 con 0x56296c03aca0 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff31f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff31f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48340/0 (socket says 172.21.15.137:48340) 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff31f400640 1 -- 172.21.15.137:0/901199609 learned_addr learned my addr 172.21.15.137:0/901199609 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff31fe00640 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03aca0 0x56296c03f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.470+0000 7ff31f400640 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c0422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31f400640 1 -- 172.21.15.137:0/901199609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03aca0 msgr2=0x56296c03f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31f400640 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03aca0 0x56296c03f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31f400640 1 -- 172.21.15.137:0/901199609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 msgr2=0x56296c0422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31f400640 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c0422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31cc00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3040a2880 0x7ff3040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31f400640 1 -- 172.21.15.137:0/901199609 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296bd7c7e0 con 0x56296c03a2b0 2024-03-26T14:10:54.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31cc00640 1 -- 172.21.15.137:0/40703943 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff3180cef20 con 0x7ff320109770 2024-03-26T14:10:54.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff324e00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3040a2880 0x7ff3040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.472 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.471+0000 7ff31f400640 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03a6d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00bfd0 tx=0x7ff30c00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f388661737ac6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.472+0000 7ff315600640 1 -- 172.21.15.137:0/901199609 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c019030 con 0x56296c03a2b0 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.472+0000 7ff315600640 1 -- 172.21.15.137:0/901199609 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff30c0035f0 con 0x56296c03a2b0 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.472+0000 7ff315600640 1 -- 172.21.15.137:0/901199609 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c011700 con 0x56296c03a2b0 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.472+0000 7ff3297fd740 1 -- 172.21.15.137:0/901199609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 msgr2=0x56296c03a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.472+0000 7ff324e00640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3040a2880 0x7ff3040a4d40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff3180058e0 tx=0x7ff318005830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.472+0000 7ff3297fd740 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03a6d0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00bfd0 tx=0x7ff30c00b6a0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 -- 172.21.15.137:0/901199609 shutdown_connections 2024-03-26T14:10:54.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c0422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03aca0 0x56296c03f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 --2- 172.21.15.137:0/901199609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03a6d0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 -- 172.21.15.137:0/901199609 >> 172.21.15.137:0/901199609 conn(0x56296c035820 msgr2=0x56296c035c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 -- 172.21.15.137:0/901199609 shutdown_connections 2024-03-26T14:10:54.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 -- 172.21.15.137:0/901199609 wait complete. 2024-03-26T14:10:54.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.473+0000 7ff3297fd740 1 -- start start 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 Processor -- start 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c044490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c0449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 0x56296c04ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296c042dc0 con 0x56296c03a2b0 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf5beb0 con 0x56296c03aca0 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296c042f40 con 0x56296c03fe10 2024-03-26T14:10:54.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.474+0000 7ff3297fd740 10 client.0 _ll_register_callbacks cb 0x56296bf50150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c044490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c044490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48354/0 (socket says 172.21.15.137:48354) 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 -- 172.21.15.137:0/782606917 learned_addr learned my addr 172.21.15.137:0/782606917 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31fe00640 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c0449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 -- 172.21.15.137:0/782606917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 msgr2=0x56296c04ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 0x56296c04ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 -- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 msgr2=0x56296c0449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c0449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 -- 172.21.15.137:0/782606917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56296bddcf70 con 0x56296c03a2b0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff31f400640 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c044490 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff30c016040 tx=0x7ff30c00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad6b8d9c2aa7134 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.475+0000 7ff30a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff30a000640 1 -- 172.21.15.137:0/782606917 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c019040 con 0x56296c03a2b0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff30a000640 1 -- 172.21.15.137:0/782606917 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff30c004000 con 0x56296c03a2b0 2024-03-26T14:10:54.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff30c0035f0 con 0x56296c03a2b0 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff30a000640 1 -- 172.21.15.137:0/782606917 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff30c004200 con 0x56296c03a2b0 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff3297fd740 10 client.17484 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56296bc74170 con 0x56296c03a2b0 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.476+0000 7ff30a000640 1 -- 172.21.15.137:0/782606917 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff30c0035f0 con 0x56296c03a2b0 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff30a000640 1 -- 172.21.15.137:0/782606917 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff30c004910 con 0x56296c03a2b0 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff3297fd740 10 client.17484 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff3297fd740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff3297fd740 20 client.17484 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff3297fd740 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c04ff60 0x56296c052350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff3297fd740 4 client.17484 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56296bfad3f0 con 0x56296c04ff60 2024-03-26T14:10:54.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.477+0000 7ff31fe00640 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c04ff60 0x56296c052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.478+0000 7ff31fe00640 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c04ff60 0x56296c052350 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24eb2fd105a9c58 server_cookie=27e686baff954f48 in_seq=0 out_seq=0 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.478+0000 7ff30a000640 10 client.17484 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff30a000640 1 -- 172.21.15.137:0/782606917 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56296bfad3f0 con 0x56296c04ff60 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff30a000640 10 client.17484 handle_command_reply: tid=0 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff3297fd740 1 client.17484 shutdown 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 msgr2=0x56296c044490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff3297fd740 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c044490 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff30c016040 tx=0x7ff30c00b3a0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 shutdown_connections 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.480+0000 7ff3297fd740 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c04ff60 0x56296c052350 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c0449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 0x56296c04ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 1 --2- 172.21.15.137:0/782606917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c044490 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 >> 172.21.15.137:0/782606917 conn(0x56296c035820 msgr2=0x56296c034a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff30a000640 0 client.17484 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 shutdown_connections 2024-03-26T14:10:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 1 -- 172.21.15.137:0/782606917 wait complete. 2024-03-26T14:10:54.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.481+0000 7ff3297fd740 20 client.17484 trim_cache size 0 max 16384 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 Processor -- start 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 -- start start 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c04b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 0x56296c03c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201b2c80 con 0x56296c03a2b0 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201b0780 con 0x56296c03aca0 2024-03-26T14:10:54.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff3297fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201ae800 con 0x56296c03fe10 2024-03-26T14:10:54.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff31f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.507+0000 7ff31fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c04b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48358/0 (socket says 172.21.15.137:48358) 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 -- 172.21.15.137:0/4115096869 learned_addr learned my addr 172.21.15.137:0/4115096869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c04b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:40972/0 (socket says 172.21.15.137:40972) 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 -- 172.21.15.137:0/4115096869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 msgr2=0x56296c03c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 0x56296c03c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 -- 172.21.15.137:0/4115096869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 msgr2=0x56296c04b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c04b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 -- 172.21.15.137:0/4115096869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296c052910 con 0x56296c03a2b0 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.508+0000 7ff31f400640 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03be20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff30c005c20 tx=0x7ff30c00bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db874fcdb2f97dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.509+0000 7ff314c00640 1 -- 172.21.15.137:0/4115096869 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c019070 con 0x56296c03a2b0 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.509+0000 7ff314c00640 1 -- 172.21.15.137:0/4115096869 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff30c024030 con 0x56296c03a2b0 2024-03-26T14:10:54.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.509+0000 7ff314c00640 1 -- 172.21.15.137:0/4115096869 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004e80 con 0x56296c03a2b0 2024-03-26T14:10:54.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.509+0000 7ff3297fd740 1 -- 172.21.15.137:0/4115096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 msgr2=0x56296c03be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.509+0000 7ff3297fd740 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03be20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff30c005c20 tx=0x7ff30c00bb80 comp rx=0 tx=0).stop 2024-03-26T14:10:54.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 -- 172.21.15.137:0/4115096869 shutdown_connections 2024-03-26T14:10:54.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03aca0 0x56296c04b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03fe10 0x56296c03c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 --2- 172.21.15.137:0/4115096869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03a2b0 0x56296c03be20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 -- 172.21.15.137:0/4115096869 >> 172.21.15.137:0/4115096869 conn(0x56296c035820 msgr2=0x56296c03b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 -- 172.21.15.137:0/4115096869 shutdown_connections 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 -- 172.21.15.137:0/4115096869 wait complete. 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 -- start start 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.510+0000 7ff3297fd740 1 Processor -- start 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03a2b0 0x56296c04b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 0x56296c04ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c049bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201aee00 con 0x56296c03aca0 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201b0120 con 0x56296c03fe10 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201af460 con 0x56296c03a2b0 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff3297fd740 10 client.0 _ll_register_callbacks cb 0x7ff310009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:54.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff31fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 0x56296c04ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff31fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 0x56296c04ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48368/0 (socket says 172.21.15.137:48368) 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff31fe00640 1 -- 172.21.15.137:0/1164142636 learned_addr learned my addr 172.21.15.137:0/1164142636 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff31f400640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c049bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.511+0000 7ff31f400640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03a2b0 0x56296c04b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31fe00640 1 -- 172.21.15.137:0/1164142636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03a2b0 msgr2=0x56296c04b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31fe00640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03a2b0 0x56296c04b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31fe00640 1 -- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 msgr2=0x56296c049bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31fe00640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c049bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31fe00640 1 -- 172.21.15.137:0/1164142636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56296c043080 con 0x56296c03aca0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31f400640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03a2b0 0x56296c04b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff31fe00640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 0x56296c04ba20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff310009900 tx=0x7ff31002c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2338a6da2bc27071 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff30a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff30a000640 1 -- 172.21.15.137:0/1164142636 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff310005040 con 0x56296c03aca0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff30a000640 1 -- 172.21.15.137:0/1164142636 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff3100051e0 con 0x56296c03aca0 2024-03-26T14:10:54.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.512+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff30c024030 con 0x56296c03aca0 2024-03-26T14:10:54.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff30a000640 1 -- 172.21.15.137:0/1164142636 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff310058050 con 0x56296c03aca0 2024-03-26T14:10:54.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 10 client.17490 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:54.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56296c0ac1d0 con 0x56296c03aca0 2024-03-26T14:10:54.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff30a000640 1 -- 172.21.15.137:0/1164142636 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff30c024030 con 0x56296c03aca0 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff30a000640 1 -- 172.21.15.137:0/1164142636 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff3100035f0 con 0x56296c03aca0 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 10 client.17490 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 20 client.17490 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c0acb60 0x56296c0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 4 client.17490 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.513+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56296bfa7740 con 0x56296c0acb60 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.514+0000 7ff31f400640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c0acb60 0x56296c0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.514+0000 7ff31f400640 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c0acb60 0x56296c0aef80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8a04d976630e0c server_cookie=e378abd800d4efe7 in_seq=0 out_seq=0 2024-03-26T14:10:54.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.514+0000 7ff30a000640 10 client.17490 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.516+0000 7ff30a000640 1 -- 172.21.15.137:0/1164142636 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56296bfa7740 con 0x56296c0acb60 2024-03-26T14:10:54.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.516+0000 7ff30a000640 10 client.17490 handle_command_reply: tid=0 2024-03-26T14:10:54.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.516+0000 7ff3297fd740 1 client.17490 shutdown 2024-03-26T14:10:54.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.516+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 msgr2=0x56296c04ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.516+0000 7ff3297fd740 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 0x56296c04ba20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff310009900 tx=0x7ff31002c1f0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 shutdown_connections 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56296c0acb60 0x56296c0aef80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56296c03fe10 0x56296c049bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56296c03aca0 0x56296c04ba20 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 --2- 172.21.15.137:0/1164142636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56296c03a2b0 0x56296c04b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff30a000640 0 client.17490 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 >> 172.21.15.137:0/1164142636 conn(0x56296c035820 msgr2=0x56296c03bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 shutdown_connections 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 1 -- 172.21.15.137:0/1164142636 wait complete. 2024-03-26T14:10:54.518 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.517+0000 7ff3297fd740 20 client.17490 trim_cache size 0 max 16384 2024-03-26T14:10:54.519 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:10:54.519 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:10:54.519 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "e850e480-b9bb-4095-9bfe-922f24291f25", 2024-03-26T14:10:54.519 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:10:54.519 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:10:54.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3040a2880 msgr2=0x7ff3040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3040a2880 0x7ff3040a4d40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff3180058e0 tx=0x7ff318005830 comp rx=0 tx=0).stop 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 msgr2=0x7ff3201a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 0x7ff3201a53f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff31800cad0 tx=0x7ff31800cf90 comp rx=0 tx=0).stop 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 shutdown_connections 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff3040a2880 0x7ff3040a4d40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff320108db0 0x7ff32019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff320109770 0x7ff3201a53f0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 --2- 172.21.15.137:0/40703943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff320107bc0 0x7ff32019de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 >> 172.21.15.137:0/40703943 conn(0x7ff320076b70 msgr2=0x7ff32010c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 shutdown_connections 2024-03-26T14:10:54.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.518+0000 7ff325800640 1 -- 172.21.15.137:0/40703943 wait complete. 2024-03-26T14:10:54.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:10:54.536 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:10:54.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.757+0000 7f22afe00640 1 -- 172.21.15.137:0/3711633324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0102000 msgr2=0x7f22b01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.757+0000 7f22afe00640 1 --2- 172.21.15.137:0/3711633324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0102000 0x7f22b01043f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009ca0 tx=0x7f22a002c100 comp rx=0 tx=0).stop 2024-03-26T14:10:54.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 -- 172.21.15.137:0/3711633324 shutdown_connections 2024-03-26T14:10:54.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 --2- 172.21.15.137:0/3711633324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22b0107260 0x7f22b01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 --2- 172.21.15.137:0/3711633324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0104930 0x7f22b0106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 --2- 172.21.15.137:0/3711633324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0102000 0x7f22b01043f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 -- 172.21.15.137:0/3711633324 >> 172.21.15.137:0/3711633324 conn(0x7f22b00fba10 msgr2=0x7f22b00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 -- 172.21.15.137:0/3711633324 shutdown_connections 2024-03-26T14:10:54.759 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.758+0000 7f22afe00640 1 -- 172.21.15.137:0/3711633324 wait complete. 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 Processor -- start 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 -- start start 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22b0102000 0x7f22b0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0104930 0x7f22b0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 0x7f22b01a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0101960 con 0x7f22b0107260 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01017e0 con 0x7f22b0102000 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.759+0000 7f22afe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b0101ae0 con 0x7f22b0104930 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 0x7f22b01a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 0x7f22b01a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48372/0 (socket says 172.21.15.137:48372) 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22b0102000 0x7f22b0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 -- 172.21.15.137:0/3400708483 learned_addr learned my addr 172.21.15.137:0/3400708483 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22ae000640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0104930 0x7f22b0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 -- 172.21.15.137:0/3400708483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0104930 msgr2=0x7f22b0199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0104930 0x7f22b0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 -- 172.21.15.137:0/3400708483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22b0102000 msgr2=0x7f22b0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22b0102000 0x7f22b0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 -- 172.21.15.137:0/3400708483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b01a1620 con 0x7f22b0107260 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.760+0000 7f22af400640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 0x7f22b01a0fb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f22a800e970 tx=0x7f22a800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.761+0000 7f22a7400640 1 -- 172.21.15.137:0/3400708483 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a800ccc0 con 0x7f22b0107260 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.761+0000 7f22a7400640 1 -- 172.21.15.137:0/3400708483 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f22a800ce60 con 0x7f22b0107260 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.761+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b01a1910 con 0x7f22b0107260 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.761+0000 7f22a7400640 1 -- 172.21.15.137:0/3400708483 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8010430 con 0x7f22b0107260 2024-03-26T14:10:54.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.761+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b01a1e10 con 0x7f22b0107260 2024-03-26T14:10:54.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.764+0000 7f22b89a7740 1 Processor -- start 2024-03-26T14:10:54.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.765+0000 7f22a7400640 1 -- 172.21.15.137:0/3400708483 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f22a8004ff0 con 0x7f22b0107260 2024-03-26T14:10:54.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.765+0000 7f22b89a7740 1 -- start start 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x55555778c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x555557791540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557794040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22b89a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555576f1250 con 0x55555778ca40 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22b89a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555557699a30 con 0x555557791bb0 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22b89a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555576ed2f0 con 0x55555778c050 2024-03-26T14:10:54.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22aea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x555557791540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22aea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x555557791540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48376/0 (socket says 172.21.15.137:48376) 2024-03-26T14:10:54.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22aea00640 1 -- 172.21.15.137:0/1500178364 learned_addr learned my addr 172.21.15.137:0/1500178364 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22ae000640 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x55555778c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.766+0000 7f22ae000640 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557794040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22aea00640 1 -- 172.21.15.137:0/1500178364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 msgr2=0x55555778c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.768 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22aea00640 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x55555778c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22aea00640 1 -- 172.21.15.137:0/1500178364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 msgr2=0x555557794040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22aea00640 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557794040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22aea00640 1 -- 172.21.15.137:0/1500178364 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555557684690 con 0x55555778ca40 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22ae000640 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557794040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22aea00640 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x555557791540 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009ca0 tx=0x7f22a00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925bb472bbdb63ea server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f22a7400640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f22940a2800 0x7f22940a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f229fe00640 1 -- 172.21.15.137:0/1500178364 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0043070 con 0x55555778ca40 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.767+0000 7f229fe00640 1 -- 172.21.15.137:0/1500178364 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f22a0032dc0 con 0x55555778ca40 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.768+0000 7f22a7400640 1 -- 172.21.15.137:0/3400708483 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f22a80cf2f0 con 0x7f22b0107260 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.768+0000 7f229fe00640 1 -- 172.21.15.137:0/1500178364 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0039040 con 0x55555778ca40 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.768+0000 7f22ae000640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f22940a2800 0x7f22940a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.768+0000 7f22b89a7740 1 -- 172.21.15.137:0/1500178364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 msgr2=0x555557791540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.769 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.768+0000 7f22b89a7740 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x555557791540 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009ca0 tx=0x7f22a00093f0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.768+0000 7f22b89a7740 1 -- 172.21.15.137:0/1500178364 shutdown_connections 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557794040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x555557791540 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 --2- 172.21.15.137:0/1500178364 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x55555778c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 -- 172.21.15.137:0/1500178364 >> 172.21.15.137:0/1500178364 conn(0x555557754200 msgr2=0x5555577879c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22ae000640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f22940a2800 0x7f22940a4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2298005d00 tx=0x7f22980046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 -- 172.21.15.137:0/1500178364 shutdown_connections 2024-03-26T14:10:54.770 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 -- 172.21.15.137:0/1500178364 wait complete. 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 -- start start 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.769+0000 7f22b89a7740 1 Processor -- start 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778c050 0x555557795f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x5555577964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555557794b50 con 0x55555778ca40 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5555576ace80 con 0x55555778c050 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555557794cd0 con 0x555557791bb0 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22b89a7740 10 client.0 _ll_register_callbacks cb 0x5555576bf130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.770+0000 7f22ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x5555577964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x5555577964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48388/0 (socket says 172.21.15.137:48388) 2024-03-26T14:10:54.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 -- 172.21.15.137:0/3043379876 learned_addr learned my addr 172.21.15.137:0/3043379876 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22aea00640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778c050 0x555557795f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22af400640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 -- 172.21.15.137:0/3043379876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 msgr2=0x55555779d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 -- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778c050 msgr2=0x555557795f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778c050 0x555557795f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 -- 172.21.15.137:0/3043379876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555574cf7d0 con 0x55555778ca40 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f22ae000640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x5555577964d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f229800d490 tx=0x7f2298004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3be73edca8d105 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.771+0000 7f229cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f229cc00640 1 -- 172.21.15.137:0/3043379876 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298005220 con 0x55555778ca40 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f229cc00640 1 -- 172.21.15.137:0/3043379876 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f22980053c0 con 0x55555778ca40 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22a0032dc0 con 0x55555778ca40 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f229cc00640 1 -- 172.21.15.137:0/3043379876 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2298030050 con 0x55555778ca40 2024-03-26T14:10:54.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f22b89a7740 10 client.17502 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5555574053f0 con 0x55555778ca40 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.772+0000 7f229cc00640 1 -- 172.21.15.137:0/3043379876 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f22a0032dc0 con 0x55555778ca40 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f229cc00640 1 -- 172.21.15.137:0/3043379876 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f229800fa50 con 0x55555778ca40 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22b89a7740 10 client.17502 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22b89a7740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22b89a7740 20 client.17502 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22b89a7740 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577a1e30 0x5555577a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22b89a7740 4 client.17502 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:10:54.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5555576fe790 con 0x5555577a1e30 2024-03-26T14:10:54.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.773+0000 7f22aea00640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577a1e30 0x5555577a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.774+0000 7f22aea00640 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577a1e30 0x5555577a42d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0b138ca834d7097 server_cookie=bbf9eb54b9ad5394 in_seq=0 out_seq=0 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.774+0000 7f229cc00640 10 client.17502 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f229cc00640 1 -- 172.21.15.137:0/3043379876 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5555576fe790 con 0x5555577a1e30 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f229cc00640 10 client.17502 handle_command_reply: tid=0 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 client.17502 shutdown 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 msgr2=0x5555577964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x5555577964d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f229800d490 tx=0x7f2298004730 comp rx=0 tx=0).stop 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 shutdown_connections 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577a1e30 0x5555577a42d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x5555577964d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 --2- 172.21.15.137:0/3043379876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778c050 0x555557795f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 >> 172.21.15.137:0/3043379876 conn(0x555557754200 msgr2=0x555557786bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.776+0000 7f229cc00640 0 client.17502 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.777+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 shutdown_connections 2024-03-26T14:10:54.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.777+0000 7f22b89a7740 1 -- 172.21.15.137:0/3043379876 wait complete. 2024-03-26T14:10:54.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.777+0000 7f22b89a7740 20 client.17502 trim_cache size 0 max 16384 2024-03-26T14:10:54.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.798+0000 7f22b89a7740 1 Processor -- start 2024-03-26T14:10:54.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.798+0000 7f22b89a7740 1 -- start start 2024-03-26T14:10:54.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x555557787bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x55555779d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557786330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22b89a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01ae7b0 con 0x55555778ca40 2024-03-26T14:10:54.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22b89a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01ac2b0 con 0x555557791bb0 2024-03-26T14:10:54.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22b89a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01aa330 con 0x55555778c050 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x55555779d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x55555779d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48400/0 (socket says 172.21.15.137:48400) 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 -- 172.21.15.137:0/2696487049 learned_addr learned my addr 172.21.15.137:0/2696487049 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22af400640 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557786330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22aea00640 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x555557787bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 -- 172.21.15.137:0/2696487049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 msgr2=0x555557787bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x555557787bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 -- 172.21.15.137:0/2696487049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 msgr2=0x555557786330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557786330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.799+0000 7f22ae000640 1 -- 172.21.15.137:0/2696487049 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555577831f0 con 0x55555778ca40 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.800+0000 7f22ae000640 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x55555779d4d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f229800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e693f2aedbf73097 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.800+0000 7f229f400640 1 -- 172.21.15.137:0/2696487049 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800f990 con 0x55555778ca40 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.800+0000 7f229f400640 1 -- 172.21.15.137:0/2696487049 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f229800fb30 con 0x55555778ca40 2024-03-26T14:10:54.801 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.800+0000 7f229f400640 1 -- 172.21.15.137:0/2696487049 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298004c80 con 0x55555778ca40 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 -- 172.21.15.137:0/2696487049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 msgr2=0x55555779d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x55555779d4d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2298000c00 tx=0x7f229800df40 comp rx=0 tx=0).stop 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 -- 172.21.15.137:0/2696487049 shutdown_connections 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555557791bb0 0x555557786330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778ca40 0x55555779d4d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 --2- 172.21.15.137:0/2696487049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55555778c050 0x555557787bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 -- 172.21.15.137:0/2696487049 >> 172.21.15.137:0/2696487049 conn(0x555557754200 msgr2=0x5555577874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 -- 172.21.15.137:0/2696487049 shutdown_connections 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 -- 172.21.15.137:0/2696487049 wait complete. 2024-03-26T14:10:54.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.801+0000 7f22b89a7740 1 -- start start 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 Processor -- start 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 0x55555779af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778ca40 0x55555779b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01aa930 con 0x55555778c050 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01abc50 con 0x55555778ca40 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01aaf90 con 0x555557791bb0 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22b89a7740 10 client.0 _ll_register_callbacks cb 0x7f229800dad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22aea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 0x55555779af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.802+0000 7f22aea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 0x55555779af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48402/0 (socket says 172.21.15.137:48402) 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 -- 172.21.15.137:0/4076624330 learned_addr learned my addr 172.21.15.137:0/4076624330 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22ae000640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778ca40 0x55555779b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 -- 172.21.15.137:0/4076624330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 msgr2=0x55555779c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22af400640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 -- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778ca40 msgr2=0x55555779b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778ca40 0x55555779b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 -- 172.21.15.137:0/4076624330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555573c50f0 con 0x55555778c050 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f22aea00640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 0x55555779af90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009560 tx=0x7f22a0004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696406892643b2f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f229cc00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.803+0000 7f229cc00640 1 -- 172.21.15.137:0/4076624330 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0057060 con 0x55555778c050 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.804+0000 7f229cc00640 1 -- 172.21.15.137:0/4076624330 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f22a00325c0 con 0x55555778c050 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.804+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f229800fb30 con 0x55555778c050 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.804+0000 7f229cc00640 1 -- 172.21.15.137:0/4076624330 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22a005e050 con 0x55555778c050 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.804+0000 7f22b89a7740 10 client.17508 fetch_fsmap learned FSMap version 24 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.804+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5555577fd8e0 con 0x55555778c050 2024-03-26T14:10:54.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.804+0000 7f229cc00640 1 -- 172.21.15.137:0/4076624330 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f229800fb30 con 0x55555778c050 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f229cc00640 1 -- 172.21.15.137:0/4076624330 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f22a0048450 con 0x55555778c050 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22b89a7740 10 client.17508 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22b89a7740 10 client.17508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22b89a7740 20 client.17508 populate_metadata read hostname 'smithi137' 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22b89a7740 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577fdef0 0x5555578002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22b89a7740 4 client.17508 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5555576f8ae0 con 0x5555577fdef0 2024-03-26T14:10:54.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.805+0000 7f22ae000640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577fdef0 0x5555578002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.806+0000 7f229cc00640 10 client.17508 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.806+0000 7f22ae000640 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577fdef0 0x5555578002e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475f7f7c7fd8df4c server_cookie=72fed1a6a2f28bd4 in_seq=0 out_seq=0 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f229cc00640 1 -- 172.21.15.137:0/4076624330 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5555576f8ae0 con 0x5555577fdef0 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f229cc00640 10 client.17508 handle_command_reply: tid=0 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 client.17508 shutdown 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 msgr2=0x55555779af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 0x55555779af90 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f22a0009560 tx=0x7f22a0004a80 comp rx=0 tx=0).stop 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 shutdown_connections 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5555577fdef0 0x5555578002e0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55555778ca40 0x55555779b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555557791bb0 0x55555779c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f229cc00640 0 client.17508 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 --2- 172.21.15.137:0/4076624330 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55555778c050 0x55555779af90 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 >> 172.21.15.137:0/4076624330 conn(0x555557754200 msgr2=0x55555778b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 shutdown_connections 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 1 -- 172.21.15.137:0/4076624330 wait complete. 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.807+0000 7f22b89a7740 20 client.17508 trim_cache size 0 max 16384 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: "e850e480-b9bb-4095-9bfe-922f24291f25": { 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "e850e480-b9bb-4095-9bfe-922f24291f25", 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:10:54.810 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:10:54.811 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:10:54.811 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:10:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f22940a2800 msgr2=0x7f22940a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f22940a2800 0x7f22940a4cc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2298005d00 tx=0x7f22980046c0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 msgr2=0x7f22b01a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 0x7f22b01a0fb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f22a800e970 tx=0x7f22a800ed70 comp rx=0 tx=0).stop 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 shutdown_connections 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f22940a2800 0x7f22940a4cc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f22b0107260 0x7f22b01a0fb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22b0104930 0x7f22b0199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 --2- 172.21.15.137:0/3400708483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22b0102000 0x7f22b0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 >> 172.21.15.137:0/3400708483 conn(0x7f22b00fba10 msgr2=0x7f22b0102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 shutdown_connections 2024-03-26T14:10:54.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:10:54.808+0000 7f22afe00640 1 -- 172.21.15.137:0/3400708483 wait complete. 2024-03-26T14:10:54.828 INFO:tasks.cephfs.filesystem:scrub status for tag:e850e480-b9bb-4095-9bfe-922f24291f25 - {'path': '/', 'tag': 'e850e480-b9bb-4095-9bfe-922f24291f25', 'options': 'recursive,force'} 2024-03-26T14:10:55.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:55 smithi137 ceph-mon[31539]: pgmap v714: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:55.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:55 smithi179 ceph-mon[35472]: pgmap v714: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:55.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:55 smithi159 ceph-mon[35501]: pgmap v714: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:56.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:56 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:10:56.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:56 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:56.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:56 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:10:56.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:56 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:56.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:56 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:10:56.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:56 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:56.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:56 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:10:56.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:56 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:56.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:56 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:10:56.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:56 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:56.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:56 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:10:56.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:56 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:10:57.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:57 smithi179 ceph-mon[35472]: pgmap v715: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:57.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:57 smithi159 ceph-mon[35501]: pgmap v715: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:57.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:57 smithi137 ceph-mon[31539]: pgmap v715: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:10:58.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:58 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:10:58.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:58 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:10:58.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:58 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:10:59.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:10:59 smithi179 ceph-mon[35472]: pgmap v716: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 KiB/s wr, 7 op/s 2024-03-26T14:10:59.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:10:59 smithi159 ceph-mon[35501]: pgmap v716: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 KiB/s wr, 7 op/s 2024-03-26T14:10:59.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:10:59 smithi137 ceph-mon[31539]: pgmap v716: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 341 KiB/s wr, 7 op/s 2024-03-26T14:11:01.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:01 smithi179 ceph-mon[35472]: pgmap v717: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:11:01.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:01 smithi159 ceph-mon[35501]: pgmap v717: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:11:01.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:01 smithi137 ceph-mon[31539]: pgmap v717: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-26T14:11:03.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:03 smithi179 ceph-mon[35472]: pgmap v718: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:03.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:03 smithi159 ceph-mon[35501]: pgmap v718: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:03.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:03 smithi137 ceph-mon[31539]: pgmap v718: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:05.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:05 smithi179 ceph-mon[35472]: pgmap v719: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:11:05.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:05 smithi159 ceph-mon[35501]: pgmap v719: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:11:05.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:05 smithi137 ceph-mon[31539]: pgmap v719: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:11:07.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:07 smithi179 ceph-mon[35472]: pgmap v720: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:07.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:07 smithi159 ceph-mon[35501]: pgmap v720: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:07.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:07 smithi137 ceph-mon[31539]: pgmap v720: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:09.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:09 smithi179 ceph-mon[35472]: pgmap v721: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:11:09.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:09 smithi159 ceph-mon[35501]: pgmap v721: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:11:09.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:09 smithi137 ceph-mon[31539]: pgmap v721: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:11:11.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:11 smithi179 ceph-mon[35472]: pgmap v722: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:11.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:11 smithi159 ceph-mon[35501]: pgmap v722: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:11.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:11 smithi137 ceph-mon[31539]: pgmap v722: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:13.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:13 smithi179 ceph-mon[35472]: pgmap v723: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:13.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:13 smithi159 ceph-mon[35501]: pgmap v723: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:13.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:13 smithi137 ceph-mon[31539]: pgmap v723: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:15.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:15 smithi179 ceph-mon[35472]: pgmap v724: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:15.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:15 smithi159 ceph-mon[35501]: pgmap v724: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:15.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:15 smithi137 ceph-mon[31539]: pgmap v724: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:17.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:17 smithi179 ceph-mon[35472]: pgmap v725: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:17.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:17 smithi159 ceph-mon[35501]: pgmap v725: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:17.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:17 smithi137 ceph-mon[31539]: pgmap v725: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:19.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:19 smithi179 ceph-mon[35472]: pgmap v726: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:19.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:19 smithi159 ceph-mon[35501]: pgmap v726: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:19.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:19 smithi137 ceph-mon[31539]: pgmap v726: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:21.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:21 smithi179 ceph-mon[35472]: pgmap v727: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:21.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:21 smithi159 ceph-mon[35501]: pgmap v727: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:21.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:21 smithi137 ceph-mon[31539]: pgmap v727: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:23.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:23 smithi179 ceph-mon[35472]: pgmap v728: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:23.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:23 smithi159 ceph-mon[35501]: pgmap v728: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:23.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:23 smithi137 ceph-mon[31539]: pgmap v728: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:24.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:11:24.830 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.066+0000 7f3fb7800640 1 -- 172.21.15.137:0/985997464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0104a50 msgr2=0x7f3fb0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.066+0000 7f3fb7800640 1 --2- 172.21.15.137:0/985997464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0104a50 0x7f3fb0106e40 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009900 tx=0x7f3f9c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.066+0000 7f3fb7800640 1 -- 172.21.15.137:0/985997464 shutdown_connections 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.067+0000 7f3fb7800640 1 --2- 172.21.15.137:0/985997464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 0x7f3fb01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.067+0000 7f3fb7800640 1 --2- 172.21.15.137:0/985997464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0104a50 0x7f3fb0106e40 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.067+0000 7f3fb7800640 1 --2- 172.21.15.137:0/985997464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fb0102120 0x7f3fb0104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.067+0000 7f3fb7800640 1 -- 172.21.15.137:0/985997464 >> 172.21.15.137:0/985997464 conn(0x7f3fb00fbb30 msgr2=0x7f3fb00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.067+0000 7f3fb7800640 1 -- 172.21.15.137:0/985997464 shutdown_connections 2024-03-26T14:11:25.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.067+0000 7f3fb7800640 1 -- 172.21.15.137:0/985997464 wait complete. 2024-03-26T14:11:25.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 Processor -- start 2024-03-26T14:11:25.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 -- start start 2024-03-26T14:11:25.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 0x7f3fb0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fb0104a50 0x7f3fb019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 0x7f3fb01a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb010d210 con 0x7f3fb0102120 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb010d090 con 0x7f3fb0107380 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.068+0000 7f3fb7800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb010d390 con 0x7f3fb0104a50 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 0x7f3fb0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 0x7f3fb0199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42376/0 (socket says 172.21.15.137:42376) 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 -- 172.21.15.137:0/1960979963 learned_addr learned my addr 172.21.15.137:0/1960979963 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb5800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 0x7f3fb01a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fafe00640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fb0104a50 0x7f3fb019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 -- 172.21.15.137:0/1960979963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fb0104a50 msgr2=0x7f3fb019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fb0104a50 0x7f3fb019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 -- 172.21.15.137:0/1960979963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 msgr2=0x7f3fb01a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 0x7f3fb01a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb4e00640 1 -- 172.21.15.137:0/1960979963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb01a1720 con 0x7f3fb0102120 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.069+0000 7f3fb5800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 0x7f3fb01a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:25.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.070+0000 7f3fb4e00640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 0x7f3fb0199b00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d950 tx=0x7f3fa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.070+0000 7f3fad600640 1 -- 172.21.15.137:0/1960979963 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4017070 con 0x7f3fb0102120 2024-03-26T14:11:25.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.070+0000 7f3fad600640 1 -- 172.21.15.137:0/1960979963 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa4009c30 con 0x7f3fb0102120 2024-03-26T14:11:25.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.070+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb01a1a10 con 0x7f3fb0102120 2024-03-26T14:11:25.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.070+0000 7f3fad600640 1 -- 172.21.15.137:0/1960979963 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400f620 con 0x7f3fb0102120 2024-03-26T14:11:25.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.070+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb01a1f80 con 0x7f3fb0102120 2024-03-26T14:11:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 Processor -- start 2024-03-26T14:11:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 -- start start 2024-03-26T14:11:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fad600640 1 -- 172.21.15.137:0/1960979963 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3fa40109a0 con 0x7f3fb0102120 2024-03-26T14:11:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf36370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcae96250 con 0x55ebcaf30e80 2024-03-26T14:11:25.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcae3ea30 con 0x55ebcaf31870 2024-03-26T14:11:25.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.074+0000 7f3fba42e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcae922f0 con 0x55ebcaf369e0 2024-03-26T14:11:25.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf312a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42380/0 (socket says 172.21.15.137:42380) 2024-03-26T14:11:25.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 -- 172.21.15.137:0/3659846763 learned_addr learned my addr 172.21.15.137:0/3659846763 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb4e00640 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf36370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fafe00640 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 -- 172.21.15.137:0/3659846763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 msgr2=0x55ebcaf38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 -- 172.21.15.137:0/3659846763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 msgr2=0x55ebcaf36370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf36370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fb5800640 1 -- 172.21.15.137:0/3659846763 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebcae29690 con 0x55ebcaf30e80 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.075+0000 7f3fafe00640 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fb5800640 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf312a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009fd0 tx=0x7f3fa80108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77c7d565e2299f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fad600640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3f980a2880 0x7f3f980a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fb4e00640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3f980a2880 0x7f3f980a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fad600640 1 -- 172.21.15.137:0/1960979963 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3fa40ceaf0 con 0x7f3fb0102120 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fa2000640 1 -- 172.21.15.137:0/3659846763 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8011760 con 0x55ebcaf30e80 2024-03-26T14:11:25.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fa2000640 1 -- 172.21.15.137:0/3659846763 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa8011db0 con 0x55ebcaf30e80 2024-03-26T14:11:25.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.076+0000 7f3fa2000640 1 -- 172.21.15.137:0/3659846763 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800e3c0 con 0x55ebcaf30e80 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 -- 172.21.15.137:0/3659846763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 msgr2=0x55ebcaf312a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fb4e00640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3f980a2880 0x7f3f980a4d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40974e0 tx=0x7f3fa40d2530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf312a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009fd0 tx=0x7f3fa80108e0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 -- 172.21.15.137:0/3659846763 shutdown_connections 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf36370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3659846763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf312a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 -- 172.21.15.137:0/3659846763 >> 172.21.15.137:0/3659846763 conn(0x55ebcaef9200 msgr2=0x55ebcaf2c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.077+0000 7f3fba42e740 1 -- 172.21.15.137:0/3659846763 shutdown_connections 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.078+0000 7f3fba42e740 1 -- 172.21.15.137:0/3659846763 wait complete. 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.078+0000 7f3fba42e740 1 -- start start 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.078+0000 7f3fba42e740 1 Processor -- start 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf3b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcaf39990 con 0x55ebcaf30e80 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcae51e80 con 0x55ebcaf31870 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebcaf39b10 con 0x55ebcaf369e0 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fba42e740 10 client.0 _ll_register_callbacks cb 0x55ebcae64130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:25.082 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42396/0 (socket says 172.21.15.137:42396) 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 -- 172.21.15.137:0/3320029815 learned_addr learned my addr 172.21.15.137:0/3320029815 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fafe00640 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf3b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 -- 172.21.15.137:0/3320029815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 msgr2=0x55ebcaf42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 -- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 msgr2=0x55ebcaf3b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf3b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.079+0000 7f3fb5800640 1 -- 172.21.15.137:0/3320029815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebcac747d0 con 0x55ebcaf30e80 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3fb5800640 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3aeb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f3fa801b9f0 tx=0x7f3fa8009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd1574cca321f0b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3f96a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3f96a00640 1 -- 172.21.15.137:0/3320029815 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa802b060 con 0x55ebcaf30e80 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3f96a00640 1 -- 172.21.15.137:0/3320029815 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3fa8004c50 con 0x55ebcaf30e80 2024-03-26T14:11:25.083 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fa8011db0 con 0x55ebcaf30e80 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3f96a00640 1 -- 172.21.15.137:0/3320029815 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fa8032050 con 0x55ebcaf30e80 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3fba42e740 10 client.17520 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.080+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ebcabaa3f0 con 0x55ebcaf30e80 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3f96a00640 1 -- 172.21.15.137:0/3320029815 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3fa8011db0 con 0x55ebcaf30e80 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3f96a00640 1 -- 172.21.15.137:0/3320029815 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3fa8005330 con 0x55ebcaf30e80 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3fba42e740 10 client.17520 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3fba42e740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3fba42e740 20 client.17520 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcaf46920 0x55ebcaf48d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3fba42e740 4 client.17520 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:25.084 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.081+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebcaea3790 con 0x55ebcaf46920 2024-03-26T14:11:25.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.082+0000 7f3fafe00640 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcaf46920 0x55ebcaf48d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.085 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.082+0000 7f3fafe00640 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcaf46920 0x55ebcaf48d10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ddc83f8f947f7da server_cookie=85906514ff794b9e in_seq=0 out_seq=0 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.082+0000 7f3f96a00640 10 client.17520 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3f96a00640 1 -- 172.21.15.137:0/3320029815 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ebcaea3790 con 0x55ebcaf46920 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3f96a00640 10 client.17520 handle_command_reply: tid=0 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 client.17520 shutdown 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 msgr2=0x55ebcaf3aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3aeb0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f3fa801b9f0 tx=0x7f3fa8009d90 comp rx=0 tx=0).stop 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 shutdown_connections 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcaf46920 0x55ebcaf48d10 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf31870 0x55ebcaf3b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf369e0 0x55ebcaf42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 --2- 172.21.15.137:0/3320029815 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3aeb0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 >> 172.21.15.137:0/3320029815 conn(0x55ebcaef9200 msgr2=0x55ebcaf2ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.084+0000 7f3f96a00640 0 client.17520 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:25.086 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.085+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 shutdown_connections 2024-03-26T14:11:25.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.085+0000 7f3fba42e740 1 -- 172.21.15.137:0/3320029815 wait complete. 2024-03-26T14:11:25.087 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.085+0000 7f3fba42e740 20 client.17520 trim_cache size 0 max 16384 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 Processor -- start 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 -- start start 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 0x55ebcaf423f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf32a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01aea40 con 0x55ebcaf31870 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01ac540 con 0x55ebcaf369e0 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fba42e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01aa5c0 con 0x55ebcaf30e80 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 0x55ebcaf423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fafe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 0x55ebcaf423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42406/0 (socket says 172.21.15.137:42406) 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.107+0000 7f3fafe00640 1 -- 172.21.15.137:0/2287679000 learned_addr learned my addr 172.21.15.137:0/2287679000 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fb5800640 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf32a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fb5800640 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fafe00640 1 -- 172.21.15.137:0/2287679000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf30e80 msgr2=0x55ebcaf3b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fafe00640 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fafe00640 1 -- 172.21.15.137:0/2287679000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 msgr2=0x55ebcaf32a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fafe00640 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf32a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fafe00640 1 -- 172.21.15.137:0/2287679000 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebcab6a0f0 con 0x55ebcaf31870 2024-03-26T14:11:25.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fb5800640 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf32a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:25.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fafe00640 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 0x55ebcaf423f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0069360 tx=0x7f3f9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1a1db406c387c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.108+0000 7f3fa1600640 1 -- 172.21.15.137:0/2287679000 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c004040 con 0x55ebcaf31870 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fa1600640 1 -- 172.21.15.137:0/2287679000 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3f9c0054f0 con 0x55ebcaf31870 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fa1600640 1 -- 172.21.15.137:0/2287679000 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c004a40 con 0x55ebcaf31870 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fba42e740 1 -- 172.21.15.137:0/2287679000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 msgr2=0x55ebcaf423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fba42e740 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 0x55ebcaf423f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0069360 tx=0x7f3f9c000bc0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fba42e740 1 -- 172.21.15.137:0/2287679000 shutdown_connections 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fba42e740 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf32a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.109+0000 7f3fba42e740 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf31870 0x55ebcaf423f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.110+0000 7f3fba42e740 1 --2- 172.21.15.137:0/2287679000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.110+0000 7f3fba42e740 1 -- 172.21.15.137:0/2287679000 >> 172.21.15.137:0/2287679000 conn(0x55ebcaef9200 msgr2=0x55ebcaf32310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.110+0000 7f3fba42e740 1 -- 172.21.15.137:0/2287679000 shutdown_connections 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.110+0000 7f3fba42e740 1 -- 172.21.15.137:0/2287679000 wait complete. 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.110+0000 7f3fba42e740 1 -- start start 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.110+0000 7f3fba42e740 1 Processor -- start 2024-03-26T14:11:25.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf31870 0x55ebcaf3fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf40e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01aabc0 con 0x55ebcaf30e80 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01abee0 con 0x55ebcaf369e0 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01ab220 con 0x55ebcaf31870 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fba42e740 10 client.0 _ll_register_callbacks cb 0x7f3fa40d2400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fb5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fb5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42416/0 (socket says 172.21.15.137:42416) 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fb5800640 1 -- 172.21.15.137:0/682380536 learned_addr learned my addr 172.21.15.137:0/682380536 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fafe00640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf31870 0x55ebcaf3fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.111+0000 7f3fb5800640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf40e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fb5800640 1 -- 172.21.15.137:0/682380536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf31870 msgr2=0x55ebcaf3fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fb5800640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf31870 0x55ebcaf3fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fb5800640 1 -- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 msgr2=0x55ebcaf40e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fb5800640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf40e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fb5800640 1 -- 172.21.15.137:0/682380536 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebcaf41470 con 0x55ebcaf30e80 2024-03-26T14:11:25.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fafe00640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf31870 0x55ebcaf3fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:25.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fb5800640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3f930 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009fd0 tx=0x7f3fa8009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf68c2756fcd27fe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3f96a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:25.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3f96a00640 1 -- 172.21.15.137:0/682380536 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa802b040 con 0x55ebcaf30e80 2024-03-26T14:11:25.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3f96a00640 1 -- 172.21.15.137:0/682380536 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3fa8023040 con 0x55ebcaf30e80 2024-03-26T14:11:25.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f9c0054f0 con 0x55ebcaf30e80 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3f96a00640 1 -- 172.21.15.137:0/682380536 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fa8046050 con 0x55ebcaf30e80 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fba42e740 10 client.17526 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.112+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ebcafa1fc0 con 0x55ebcaf30e80 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3f96a00640 1 -- 172.21.15.137:0/682380536 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3f9c0054f0 con 0x55ebcaf30e80 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3f96a00640 1 -- 172.21.15.137:0/682380536 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3fa8011560 con 0x55ebcaf30e80 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3fba42e740 10 client.17526 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3fba42e740 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3fba42e740 20 client.17526 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3fba42e740 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcafa26e0 0x55ebcafa4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3fba42e740 4 client.17526 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.113+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ebcae9dae0 con 0x55ebcafa26e0 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.114+0000 7f3fafe00640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcafa26e0 0x55ebcafa4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.114+0000 7f3fafe00640 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcafa26e0 0x55ebcafa4b80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f51ba7e6015f26 server_cookie=54c19c6212feb7dd in_seq=0 out_seq=0 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.114+0000 7f3f96a00640 10 client.17526 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3f96a00640 1 -- 172.21.15.137:0/682380536 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ebcae9dae0 con 0x55ebcafa26e0 2024-03-26T14:11:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3f96a00640 10 client.17526 handle_command_reply: tid=0 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 client.17526 shutdown 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 msgr2=0x55ebcaf3f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3f930 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009fd0 tx=0x7f3fa8009d90 comp rx=0 tx=0).stop 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 shutdown_connections 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ebcafa26e0 0x55ebcafa4b80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ebcaf369e0 0x55ebcaf40e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ebcaf31870 0x55ebcaf3fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 --2- 172.21.15.137:0/682380536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ebcaf30e80 0x55ebcaf3f930 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 >> 172.21.15.137:0/682380536 conn(0x55ebcaef9200 msgr2=0x55ebcaf3b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3f96a00640 0 client.17526 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 shutdown_connections 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 1 -- 172.21.15.137:0/682380536 wait complete. 2024-03-26T14:11:25.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.115+0000 7f3fba42e740 20 client.17526 trim_cache size 0 max 16384 2024-03-26T14:11:25.119 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:11:25.119 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:11:25.119 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:11:25.119 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:11:25.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3f980a2880 msgr2=0x7f3f980a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3f980a2880 0x7f3f980a4d40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40974e0 tx=0x7f3fa40d2530 comp rx=0 tx=0).stop 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 msgr2=0x7f3fb0199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 0x7f3fb0199b00 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400d950 tx=0x7f3fa400de10 comp rx=0 tx=0).stop 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 shutdown_connections 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3f980a2880 0x7f3f980a4d40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb0107380 0x7f3fb01a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fb0104a50 0x7f3fb019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 --2- 172.21.15.137:0/1960979963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3fb0102120 0x7f3fb0199b00 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 >> 172.21.15.137:0/1960979963 conn(0x7f3fb00fbb30 msgr2=0x7f3fb0101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 shutdown_connections 2024-03-26T14:11:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.116+0000 7f3fb7800640 1 -- 172.21.15.137:0/1960979963 wait complete. 2024-03-26T14:11:25.133 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:11:25.133 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:11:25.195 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:25 smithi137 ceph-mon[31539]: pgmap v729: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 -- 172.21.15.137:0/1723160163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41036f0 msgr2=0x7fa5d4103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/1723160163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41036f0 0x7fa5d4103b10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009900 tx=0x7fa5c402bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 -- 172.21.15.137:0/1723160163 shutdown_connections 2024-03-26T14:11:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/1723160163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d41048e0 0x7fa5d4104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/1723160163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5d41052a0 0x7fa5d410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/1723160163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41036f0 0x7fa5d4103b10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 -- 172.21.15.137:0/1723160163 >> 172.21.15.137:0/1723160163 conn(0x7fa5d40fec80 msgr2=0x7fa5d41010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 -- 172.21.15.137:0/1723160163 shutdown_connections 2024-03-26T14:11:25.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.330+0000 7fa5d8a00640 1 -- 172.21.15.137:0/1723160163 wait complete. 2024-03-26T14:11:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.331+0000 7fa5d8a00640 1 Processor -- start 2024-03-26T14:11:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.331+0000 7fa5d8a00640 1 -- start start 2024-03-26T14:11:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5d41048e0 0x7fa5d4199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 0x7fa5d419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d419f0b0 0x7fa5d41a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d8a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d410f220 con 0x7fa5d41052a0 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d410f0a0 con 0x7fa5d419f0b0 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d8a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d410f3a0 con 0x7fa5d41048e0 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 0x7fa5d419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5cbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 0x7fa5d419a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42438/0 (socket says 172.21.15.137:42438) 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5cbe00640 1 -- 172.21.15.137:0/4059699750 learned_addr learned my addr 172.21.15.137:0/4059699750 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d3200640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d419f0b0 0x7fa5d41a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.332+0000 7fa5d2800640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5d41048e0 0x7fa5d4199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cbe00640 1 -- 172.21.15.137:0/4059699750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5d41048e0 msgr2=0x7fa5d4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cbe00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5d41048e0 0x7fa5d4199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cbe00640 1 -- 172.21.15.137:0/4059699750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d419f0b0 msgr2=0x7fa5d41a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cbe00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d419f0b0 0x7fa5d41a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cbe00640 1 -- 172.21.15.137:0/4059699750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d41a1b10 con 0x7fa5d41052a0 2024-03-26T14:11:25.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cbe00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 0x7fa5d419a050 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9f0 tx=0x7fa5bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00cd80 con 0x7fa5d41052a0 2024-03-26T14:11:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa5bc0108e0 con 0x7fa5d41052a0 2024-03-26T14:11:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc018920 con 0x7fa5d41052a0 2024-03-26T14:11:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.333+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d41a1e00 con 0x7fa5d41052a0 2024-03-26T14:11:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.334+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41a2270 con 0x7fa5d41052a0 2024-03-26T14:11:25.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.335+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d410ab80 con 0x7fa5d41052a0 2024-03-26T14:11:25.337 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.337+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa5bc010a90 con 0x7fa5d41052a0 2024-03-26T14:11:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.339+0000 7fa5cb400640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5b80a2800 0x7fa5b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.339+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa5bc0cf230 con 0x7fa5d41052a0 2024-03-26T14:11:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.339+0000 7fa5d2800640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5b80a2800 0x7fa5b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.340+0000 7fa5d2800640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5b80a2800 0x7fa5b80a4cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4036040 tx=0x7fa5c4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.344+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa5bc09bec0 con 0x7fa5d41052a0 2024-03-26T14:11:25.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:25 smithi179 ceph-mon[35472]: pgmap v729: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:25.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:25 smithi159 ceph-mon[35501]: pgmap v729: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:25.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.617+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5d4062340 con 0x7fa5d41052a0 2024-03-26T14:11:25.618 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.618+0000 7fa5cb400640 1 -- 172.21.15.137:0/4059699750 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fa5bc09b7c0 con 0x7fa5d41052a0 2024-03-26T14:11:25.619 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:11:25.619 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:11:25.619 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:11:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5b80a2800 msgr2=0x7fa5b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5b80a2800 0x7fa5b80a4cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4036040 tx=0x7fa5c4005e30 comp rx=0 tx=0).stop 2024-03-26T14:11:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 msgr2=0x7fa5d419a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 0x7fa5d419a050 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9f0 tx=0x7fa5bc00eeb0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 shutdown_connections 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5b80a2800 0x7fa5b80a4cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d419f0b0 0x7fa5d41a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5d41052a0 0x7fa5d419a050 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 --2- 172.21.15.137:0/4059699750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5d41048e0 0x7fa5d4199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 >> 172.21.15.137:0/4059699750 conn(0x7fa5d40fec80 msgr2=0x7fa5d4107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 shutdown_connections 2024-03-26T14:11:25.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.621+0000 7fa5d8a00640 1 -- 172.21.15.137:0/4059699750 wait complete. 2024-03-26T14:11:25.640 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:11:25.640 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:11:25.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.860+0000 7f55a3400640 1 -- 172.21.15.137:0/2611678515 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559802cbf0 con 0x7f55a41037a0 2024-03-26T14:11:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 -- 172.21.15.137:0/2611678515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a41037a0 msgr2=0x7f55a4103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 --2- 172.21.15.137:0/2611678515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a41037a0 0x7f55a4103bc0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5598009900 tx=0x7f559802bc40 comp rx=0 tx=0).stop 2024-03-26T14:11:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 -- 172.21.15.137:0/2611678515 shutdown_connections 2024-03-26T14:11:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 --2- 172.21.15.137:0/2611678515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55a4105350 0x7f55a410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 --2- 172.21.15.137:0/2611678515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a4104990 0x7f55a4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 --2- 172.21.15.137:0/2611678515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a41037a0 0x7f55a4103bc0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 -- 172.21.15.137:0/2611678515 >> 172.21.15.137:0/2611678515 conn(0x7f55a40fed10 msgr2=0x7f55a4101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 -- 172.21.15.137:0/2611678515 shutdown_connections 2024-03-26T14:11:25.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.861+0000 7f55aa600640 1 -- 172.21.15.137:0/2611678515 wait complete. 2024-03-26T14:11:25.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.863+0000 7f55aa600640 1 Processor -- start 2024-03-26T14:11:25.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.863+0000 7f55aa600640 1 -- start start 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55aa600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55a4104990 0x7f55a4199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55aa600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a4105350 0x7f55a419a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55aa600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 0x7f55a41a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55aa600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a410f2a0 con 0x7f55a419f180 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55aa600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a410f120 con 0x7f55a4104990 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55aa600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a410f420 con 0x7f55a4105350 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55a9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 0x7f55a41a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55a9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55a4104990 0x7f55a4199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55a9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 0x7f55a41a1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42452/0 (socket says 172.21.15.137:42452) 2024-03-26T14:11:25.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.864+0000 7f55a9c00640 1 -- 172.21.15.137:0/2227879251 learned_addr learned my addr 172.21.15.137:0/2227879251 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a3e00640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a4105350 0x7f55a419a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a9c00640 1 -- 172.21.15.137:0/2227879251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a4105350 msgr2=0x7f55a419a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a9c00640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a4105350 0x7f55a419a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a9c00640 1 -- 172.21.15.137:0/2227879251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55a4104990 msgr2=0x7f55a4199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a9c00640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55a4104990 0x7f55a4199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a9c00640 1 -- 172.21.15.137:0/2227879251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a41a1be0 con 0x7f55a419f180 2024-03-26T14:11:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a9c00640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 0x7f55a41a1570 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f559c00eaa0 tx=0x7f559c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.865+0000 7f55a1600640 1 -- 172.21.15.137:0/2227879251 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c00cdb0 con 0x7f55a419f180 2024-03-26T14:11:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.866+0000 7f55a1600640 1 -- 172.21.15.137:0/2227879251 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f559c005020 con 0x7f55a419f180 2024-03-26T14:11:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.866+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a41a1ed0 con 0x7f55a419f180 2024-03-26T14:11:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.866+0000 7f55a1600640 1 -- 172.21.15.137:0/2227879251 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c0116d0 con 0x7f55a419f180 2024-03-26T14:11:25.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.866+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a41a2390 con 0x7f55a419f180 2024-03-26T14:11:25.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.870+0000 7f55ae9e1740 1 Processor -- start 2024-03-26T14:11:25.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.870+0000 7f55a1600640 1 -- 172.21.15.137:0/2227879251 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f559c0051d0 con 0x7f55a419f180 2024-03-26T14:11:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 -- start start 2024-03-26T14:11:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x55700858ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x557008590040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x557008596e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570084f32c0 con 0x55700858e9f0 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700849bc90 con 0x557008590580 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570084ef360 con 0x55700858fbc0 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x55700858ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x55700858ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42454/0 (socket says 172.21.15.137:42454) 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.871+0000 7f55a3e00640 1 -- 172.21.15.137:0/605182102 learned_addr learned my addr 172.21.15.137:0/605182102 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x557008596e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a9200640 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x557008590040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 -- 172.21.15.137:0/605182102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 msgr2=0x557008590040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x557008590040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 -- 172.21.15.137:0/605182102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 msgr2=0x557008596e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x557008596e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 -- 172.21.15.137:0/605182102 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570084c0080 con 0x55700858e9f0 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a1600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f558c0a2880 0x7f558c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a3e00640 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x55700858ee10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f55900124c0 tx=0x7f559000c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7256b4d96d7a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a1600640 1 -- 172.21.15.137:0/2227879251 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f559c0cec50 con 0x7f55a419f180 2024-03-26T14:11:25.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.872+0000 7f55a9200640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f558c0a2880 0x7f558c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.873+0000 7f5596000640 1 -- 172.21.15.137:0/605182102 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590018030 con 0x55700858e9f0 2024-03-26T14:11:25.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.873+0000 7f5596000640 1 -- 172.21.15.137:0/605182102 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55900054d0 con 0x55700858e9f0 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.873+0000 7f5596000640 1 -- 172.21.15.137:0/605182102 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559000d550 con 0x55700858e9f0 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 -- 172.21.15.137:0/605182102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 msgr2=0x55700858ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x55700858ee10 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f55900124c0 tx=0x7f559000c5e0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55a9200640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f558c0a2880 0x7f558c0a4d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5598036040 tx=0x7f5598000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 -- 172.21.15.137:0/605182102 shutdown_connections 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x557008596e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x557008590040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/605182102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x55700858ee10 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 -- 172.21.15.137:0/605182102 >> 172.21.15.137:0/605182102 conn(0x557008589de0 msgr2=0x55700858a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 -- 172.21.15.137:0/605182102 shutdown_connections 2024-03-26T14:11:25.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 -- 172.21.15.137:0/605182102 wait complete. 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.874+0000 7f55ae9e1740 1 -- start start 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 Processor -- start 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x557008598ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858fbc0 0x5570085991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557008590580 0x5570085a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557008597590 con 0x55700858e9f0 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570084af0d0 con 0x55700858fbc0 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557008597710 con 0x557008590580 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55ae9e1740 10 client.0 _ll_register_callbacks cb 0x5570084b3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.875+0000 7f55a9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x557008598ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x557008598ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42468/0 (socket says 172.21.15.137:42468) 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 -- 172.21.15.137:0/3510589299 learned_addr learned my addr 172.21.15.137:0/3510589299 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a3e00640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858fbc0 0x5570085991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9c00640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557008590580 0x5570085a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 -- 172.21.15.137:0/3510589299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557008590580 msgr2=0x5570085a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557008590580 0x5570085a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 -- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858fbc0 msgr2=0x5570085991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858fbc0 0x5570085991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 -- 172.21.15.137:0/3510589299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570082073f0 con 0x55700858e9f0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f55a9200640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x557008598ca0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5598005ab0 tx=0x7f559802c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1102357e8c03852a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.876+0000 7f558aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.877+0000 7f558aa00640 1 -- 172.21.15.137:0/3510589299 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559804f030 con 0x55700858e9f0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.877+0000 7f558aa00640 1 -- 172.21.15.137:0/3510589299 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f559803a040 con 0x55700858e9f0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.877+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55900054d0 con 0x55700858e9f0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.877+0000 7f558aa00640 1 -- 172.21.15.137:0/3510589299 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f559805c050 con 0x55700858e9f0 2024-03-26T14:11:25.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.877+0000 7f55ae9e1740 10 client.17544 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.877+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5570082d17d0 con 0x55700858e9f0 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f558aa00640 1 -- 172.21.15.137:0/3510589299 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f55900054d0 con 0x55700858e9f0 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f558aa00640 1 -- 172.21.15.137:0/3510589299 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f55980035f0 con 0x55700858e9f0 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55ae9e1740 10 client.17544 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55ae9e1740 10 client.17544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55ae9e1740 20 client.17544 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5570085a4770 0x5570085a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55ae9e1740 4 client.17544 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557008500800 con 0x5570085a4770 2024-03-26T14:11:25.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.878+0000 7f55a3e00640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5570085a4770 0x5570085a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.879+0000 7f55a3e00640 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5570085a4770 0x5570085a6bf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6560d8835ccb4f19 server_cookie=7d0f40e3f49f366e in_seq=0 out_seq=0 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.879+0000 7f558aa00640 10 client.17544 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f558aa00640 1 -- 172.21.15.137:0/3510589299 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557008500800 con 0x5570085a4770 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f558aa00640 10 client.17544 handle_command_reply: tid=0 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 client.17544 shutdown 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 msgr2=0x557008598ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x557008598ca0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f5598005ab0 tx=0x7f559802c810 comp rx=0 tx=0).stop 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 shutdown_connections 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5570085a4770 0x5570085a6bf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858fbc0 0x5570085991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557008590580 0x5570085a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/3510589299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858e9f0 0x557008598ca0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 >> 172.21.15.137:0/3510589299 conn(0x557008589de0 msgr2=0x557008589160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f558aa00640 0 client.17544 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:25.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 shutdown_connections 2024-03-26T14:11:25.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.881+0000 7f55ae9e1740 1 -- 172.21.15.137:0/3510589299 wait complete. 2024-03-26T14:11:25.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.882+0000 7f55ae9e1740 20 client.17544 trim_cache size 0 max 16384 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.898+0000 7f55ae9e1740 1 Processor -- start 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.898+0000 7f55ae9e1740 1 -- start start 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858e9f0 0x557008599140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x5570085a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 0x55700858d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41aed50 con 0x557008590580 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41ac850 con 0x55700858e9f0 2024-03-26T14:11:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41aa8d0 con 0x55700858fbc0 2024-03-26T14:11:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55a9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 0x55700858d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55a9c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 0x55700858d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42482/0 (socket says 172.21.15.137:42482) 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55a9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858e9f0 0x557008599140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.899+0000 7f55a9c00640 1 -- 172.21.15.137:0/547212818 learned_addr learned my addr 172.21.15.137:0/547212818 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f55a9c00640 1 -- 172.21.15.137:0/547212818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 msgr2=0x5570085a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f55a9c00640 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x5570085a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f55a9c00640 1 -- 172.21.15.137:0/547212818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858e9f0 msgr2=0x557008599140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f55a9c00640 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858e9f0 0x557008599140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f55a9c00640 1 -- 172.21.15.137:0/547212818 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570085858b0 con 0x557008590580 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f55a9c00640 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 0x55700858d750 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f559c00efd0 tx=0x7f559c09ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da416a5b1058bfab server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f5595600640 1 -- 172.21.15.137:0/547212818 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c01f040 con 0x557008590580 2024-03-26T14:11:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f5595600640 1 -- 172.21.15.137:0/547212818 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f559c0a3980 con 0x557008590580 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.900+0000 7f5595600640 1 -- 172.21.15.137:0/547212818 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c096f30 con 0x557008590580 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 -- 172.21.15.137:0/547212818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 msgr2=0x55700858d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 0x55700858d750 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f559c00efd0 tx=0x7f559c09ae50 comp rx=0 tx=0).stop 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 -- 172.21.15.137:0/547212818 shutdown_connections 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557008590580 0x55700858d750 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858fbc0 0x5570085a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/547212818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55700858e9f0 0x557008599140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 -- 172.21.15.137:0/547212818 >> 172.21.15.137:0/547212818 conn(0x557008589de0 msgr2=0x55700858d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 -- 172.21.15.137:0/547212818 shutdown_connections 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.901+0000 7f55ae9e1740 1 -- 172.21.15.137:0/547212818 wait complete. 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 -- start start 2024-03-26T14:11:25.903 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 Processor -- start 2024-03-26T14:11:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858e9f0 0x55700859bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 0x55700859c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x55700859f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41aaed0 con 0x55700858fbc0 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41ac1f0 con 0x557008590580 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41ab530 con 0x55700858e9f0 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55ae9e1740 10 client.0 _ll_register_callbacks cb 0x7f5598043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.902+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 0x55700859c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 0x55700859c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42490/0 (socket says 172.21.15.137:42490) 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 -- 172.21.15.137:0/2309978512 learned_addr learned my addr 172.21.15.137:0/2309978512 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a9c00640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x55700859f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 -- 172.21.15.137:0/2309978512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858e9f0 msgr2=0x55700859bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a9200640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858e9f0 0x55700859bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858e9f0 0x55700859bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 -- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 msgr2=0x55700859f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x55700859f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 -- 172.21.15.137:0/2309978512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55700859f880 con 0x55700858fbc0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a9200640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858e9f0 0x55700859bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f55a3e00640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 0x55700859c310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x55700858fac0 tx=0x7f559000c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5e747786e88b86 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.903+0000 7f558aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f558aa00640 1 -- 172.21.15.137:0/2309978512 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559002a020 con 0x55700858fbc0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f558aa00640 1 -- 172.21.15.137:0/2309978512 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f559002a1c0 con 0x55700858fbc0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f559c0a3980 con 0x55700858fbc0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f558aa00640 1 -- 172.21.15.137:0/2309978512 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5590033050 con 0x55700858fbc0 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f55ae9e1740 10 client.17550 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557008600590 con 0x55700858fbc0 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f558aa00640 1 -- 172.21.15.137:0/2309978512 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f559c0a3980 con 0x55700858fbc0 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f558aa00640 1 -- 172.21.15.137:0/2309978512 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f55900035f0 con 0x55700858fbc0 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f55ae9e1740 10 client.17550 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f55ae9e1740 10 client.17550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.904+0000 7f55ae9e1740 20 client.17550 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.905+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x557008600c90 0x5570086030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.905+0000 7f55ae9e1740 4 client.17550 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.905+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5570084fab50 con 0x557008600c90 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.905+0000 7f55a9200640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x557008600c90 0x5570086030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55a9200640 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x557008600c90 0x5570086030b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=84397636a01c626d server_cookie=f4cd46b7ce8a666e in_seq=0 out_seq=0 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f558aa00640 10 client.17550 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f558aa00640 1 -- 172.21.15.137:0/2309978512 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f5598059020 con 0x557008600c90 2024-03-26T14:11:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f558aa00640 10 client.17550 handle_command_reply: tid=0 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 client.17550 shutdown 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 msgr2=0x55700859c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 0x55700859c310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x55700858fac0 tx=0x7f559000c380 comp rx=0 tx=0).stop 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 shutdown_connections 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x557008600c90 0x5570086030b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557008590580 0x55700859f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55700858fbc0 0x55700859c310 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 --2- 172.21.15.137:0/2309978512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55700858e9f0 0x55700859bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f558aa00640 0 client.17550 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.906+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 >> 172.21.15.137:0/2309978512 conn(0x557008589de0 msgr2=0x55700858d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.907+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 shutdown_connections 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.907+0000 7f55ae9e1740 1 -- 172.21.15.137:0/2309978512 wait complete. 2024-03-26T14:11:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.907+0000 7f55ae9e1740 20 client.17550 trim_cache size 0 max 16384 2024-03-26T14:11:25.912 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:11:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f558c0a2880 msgr2=0x7f558c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f558c0a2880 0x7f558c0a4d40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5598036040 tx=0x7f5598000bc0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 msgr2=0x7f55a41a1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 0x7f55a41a1570 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f559c00eaa0 tx=0x7f559c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 shutdown_connections 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f558c0a2880 0x7f558c0a4d40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55a419f180 0x7f55a41a1570 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f55a4105350 0x7f55a419a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 --2- 172.21.15.137:0/2227879251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55a4104990 0x7f55a4199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 >> 172.21.15.137:0/2227879251 conn(0x7f55a40fed10 msgr2=0x7f55a4107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 shutdown_connections 2024-03-26T14:11:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:25.908+0000 7f55aa600640 1 -- 172.21.15.137:0/2227879251 wait complete. 2024-03-26T14:11:25.929 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:11:25.929 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:11:26.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 -- 172.21.15.137:0/3481565347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 msgr2=0x7fc8c0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/3481565347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0106e50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009900 tx=0x7fc8ac02bc40 comp rx=0 tx=0).stop 2024-03-26T14:11:26.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 -- 172.21.15.137:0/3481565347 shutdown_connections 2024-03-26T14:11:26.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/3481565347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0107390 0x7fc8c01097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/3481565347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0106e50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/3481565347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0102130 0x7fc8c0104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 -- 172.21.15.137:0/3481565347 >> 172.21.15.137:0/3481565347 conn(0x7fc8c00fbb40 msgr2=0x7fc8c00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 -- 172.21.15.137:0/3481565347 shutdown_connections 2024-03-26T14:11:26.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.144+0000 7fc8c4a00640 1 -- 172.21.15.137:0/3481565347 wait complete. 2024-03-26T14:11:26.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.145+0000 7fc8c4a00640 1 Processor -- start 2024-03-26T14:11:26.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.145+0000 7fc8c4a00640 1 -- start start 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0102130 0x7fc8c0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8c4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0107390 0x7fc8c019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8c4a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c010d230 con 0x7fc8c0104a60 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8c4a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c010d0b0 con 0x7fc8c0107390 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8c4a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c010d3b0 con 0x7fc8c0102130 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8bd400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8bd400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42504/0 (socket says 172.21.15.137:42504) 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8bd400640 1 -- 172.21.15.137:0/1017392726 learned_addr learned my addr 172.21.15.137:0/1017392726 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8be800640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0107390 0x7fc8c019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.146+0000 7fc8bde00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0102130 0x7fc8c0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bd400640 1 -- 172.21.15.137:0/1017392726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0102130 msgr2=0x7fc8c0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bd400640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0102130 0x7fc8c0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bd400640 1 -- 172.21.15.137:0/1017392726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0107390 msgr2=0x7fc8c019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bd400640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0107390 0x7fc8c019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bd400640 1 -- 172.21.15.137:0/1017392726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c019f590 con 0x7fc8c0104a60 2024-03-26T14:11:26.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bde00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0102130 0x7fc8c0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:26.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.147+0000 7fc8bd400640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0197eb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005ab0 tx=0x7fc8ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.151+0000 7fc8b6a00640 1 -- 172.21.15.137:0/1017392726 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac043020 con 0x7fc8c0104a60 2024-03-26T14:11:26.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.151+0000 7fc8b6a00640 1 -- 172.21.15.137:0/1017392726 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8ac0054c0 con 0x7fc8c0104a60 2024-03-26T14:11:26.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.151+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c019f820 con 0x7fc8c0104a60 2024-03-26T14:11:26.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.151+0000 7fc8b6a00640 1 -- 172.21.15.137:0/1017392726 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac003d10 con 0x7fc8c0104a60 2024-03-26T14:11:26.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.151+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c019fdc0 con 0x7fc8c0104a60 2024-03-26T14:11:26.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.155+0000 7fc8b6a00640 1 -- 172.21.15.137:0/1017392726 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc8ac046020 con 0x7fc8c0104a60 2024-03-26T14:11:26.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.155+0000 7fc8c77ed740 1 Processor -- start 2024-03-26T14:11:26.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 -- start start 2024-03-26T14:11:26.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c022983810 0x55c022983c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c022984e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0229853a0 0x55c02298bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0228e82c0 con 0x55c0229853a0 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c022890c90 con 0x55c0229849e0 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0228e4360 con 0x55c022983810 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8bd400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0229853a0 0x55c02298bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.156+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c022984e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c022984e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:47222/0 (socket says 172.21.15.137:47222) 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 -- 172.21.15.137:0/30994231 learned_addr learned my addr 172.21.15.137:0/30994231 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8be800640 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c022983810 0x55c022983c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 -- 172.21.15.137:0/30994231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c022983810 msgr2=0x55c022983c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c022983810 0x55c022983c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 -- 172.21.15.137:0/30994231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0229853a0 msgr2=0x55c02298bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0229853a0 0x55c02298bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bde00640 1 -- 172.21.15.137:0/30994231 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0228b5080 con 0x55c0229849e0 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.157+0000 7fc8bd400640 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0229853a0 0x55c02298bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.158+0000 7fc8b6a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc8a40a2880 0x7fc8a40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.158+0000 7fc8bde00640 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c022984e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bd40 tx=0x7fc8b000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809d2bfa1af1f3bc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.158+0000 7fc8b6a00640 1 -- 172.21.15.137:0/1017392726 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc8ac0f10d0 con 0x7fc8c0104a60 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.158+0000 7fc8be800640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc8a40a2880 0x7fc8a40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.159+0000 7fc8be800640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc8a40a2880 0x7fc8a40a4d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55c0229848e0 tx=0x7fc8b8009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.162+0000 7fc8ab000640 1 -- 172.21.15.137:0/30994231 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0015070 con 0x55c0229849e0 2024-03-26T14:11:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.162+0000 7fc8ab000640 1 -- 172.21.15.137:0/30994231 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8b000be30 con 0x55c0229849e0 2024-03-26T14:11:26.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.162+0000 7fc8ab000640 1 -- 172.21.15.137:0/30994231 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0011990 con 0x55c0229849e0 2024-03-26T14:11:26.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.163+0000 7fc8c77ed740 1 -- 172.21.15.137:0/30994231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 msgr2=0x55c022984e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.163+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c022984e60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000bd40 tx=0x7fc8b000f730 comp rx=0 tx=0).stop 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 -- 172.21.15.137:0/30994231 shutdown_connections 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c022984e60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c0229853a0 0x55c02298bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/30994231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c022983810 0x55c022983c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 -- 172.21.15.137:0/30994231 >> 172.21.15.137:0/30994231 conn(0x55c02297ec00 msgr2=0x55c02297f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 -- 172.21.15.137:0/30994231 shutdown_connections 2024-03-26T14:11:26.164 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 -- 172.21.15.137:0/30994231 wait complete. 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 -- start start 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.164+0000 7fc8c77ed740 1 Processor -- start 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c02298dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022995010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c02298c3b0 con 0x55c022983810 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0228a40d0 con 0x55c0229849e0 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c02298c530 con 0x55c0229853a0 2024-03-26T14:11:26.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8c77ed740 10 client.0 _ll_register_callbacks cb 0x55c0228a8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42516/0 (socket says 172.21.15.137:42516) 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8bde00640 1 -- 172.21.15.137:0/2984425979 learned_addr learned my addr 172.21.15.137:0/2984425979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.165+0000 7fc8bd400640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c02298dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8be800640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022995010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8bde00640 1 -- 172.21.15.137:0/2984425979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 msgr2=0x55c022995010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8bde00640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022995010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8bde00640 1 -- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 msgr2=0x55c02298dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8bde00640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c02298dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8bde00640 1 -- 172.21.15.137:0/2984425979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0225fc3f0 con 0x55c022983810 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc8bde00640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298da90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c010 tx=0x7fc8b000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05a81d6ca9e55d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.167 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.166+0000 7fc89f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc89f400640 1 -- 172.21.15.137:0/2984425979 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0028060 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc89f400640 1 -- 172.21.15.137:0/2984425979 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc8b00052d0 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b000be30 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc89f400640 1 -- 172.21.15.137:0/2984425979 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b002f050 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc8c77ed740 10 client.17562 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c0226c67d0 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc89f400640 1 -- 172.21.15.137:0/2984425979 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8b000be30 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.167+0000 7fc89f400640 1 -- 172.21.15.137:0/2984425979 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc8b00048f0 con 0x55c022983810 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8c77ed740 10 client.17562 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:26.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8c77ed740 10 client.17562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8c77ed740 20 client.17562 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c022999220 0x55c02299b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8c77ed740 4 client.17562 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0228f5800 con 0x55c022999220 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.168+0000 7fc8bd400640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c022999220 0x55c02299b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.169+0000 7fc8bd400640 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c022999220 0x55c02299b610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43596b5639c7548a server_cookie=4cdbdeaaba80f238 in_seq=0 out_seq=0 2024-03-26T14:11:26.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.169+0000 7fc89f400640 10 client.17562 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc89f400640 1 -- 172.21.15.137:0/2984425979 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c0228f5800 con 0x55c022999220 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc89f400640 10 client.17562 handle_command_reply: tid=0 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 client.17562 shutdown 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 msgr2=0x55c02298da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298da90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c010 tx=0x7fc8b000bab0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 shutdown_connections 2024-03-26T14:11:26.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c022999220 0x55c02299b610 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c02298dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022995010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2984425979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298da90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc89f400640 0 client.17562 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 >> 172.21.15.137:0/2984425979 conn(0x55c02297ec00 msgr2=0x55c02297deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 shutdown_connections 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2984425979 wait complete. 2024-03-26T14:11:26.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.171+0000 7fc8c77ed740 20 client.17562 trim_cache size 0 max 16384 2024-03-26T14:11:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 Processor -- start 2024-03-26T14:11:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 -- start start 2024-03-26T14:11:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c022987a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229849e0 0x55c022987fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229853a0 0x55c022988500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01aa5a0 con 0x55c022983810 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a80a0 con 0x55c0229853a0 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.185+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a6120 con 0x55c0229849e0 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c022987a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c022987a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42524/0 (socket says 172.21.15.137:42524) 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 -- 172.21.15.137:0/2496893210 learned_addr learned my addr 172.21.15.137:0/2496893210 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8be800640 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229853a0 0x55c022988500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 -- 172.21.15.137:0/2496893210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229849e0 msgr2=0x55c022987fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229849e0 0x55c022987fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 -- 172.21.15.137:0/2496893210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229853a0 msgr2=0x55c022988500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229853a0 0x55c022988500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 -- 172.21.15.137:0/2496893210 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c02297f970 con 0x55c022983810 2024-03-26T14:11:26.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.186+0000 7fc8bde00640 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c022987a80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0010010 tx=0x7fc8b000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6fd375d18e333 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.187+0000 7fc8aa600640 1 -- 172.21.15.137:0/2496893210 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0038070 con 0x55c022983810 2024-03-26T14:11:26.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.187+0000 7fc8aa600640 1 -- 172.21.15.137:0/2496893210 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8b000b3a0 con 0x55c022983810 2024-03-26T14:11:26.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.187+0000 7fc8aa600640 1 -- 172.21.15.137:0/2496893210 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0028070 con 0x55c022983810 2024-03-26T14:11:26.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.187+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2496893210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 msgr2=0x55c022987a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.187+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c022987a80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0010010 tx=0x7fc8b000bbc0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2496893210 shutdown_connections 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229853a0 0x55c022988500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229849e0 0x55c022987fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/2496893210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c022987a80 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2496893210 >> 172.21.15.137:0/2496893210 conn(0x55c02297ec00 msgr2=0x55c0229826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2496893210 shutdown_connections 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 -- 172.21.15.137:0/2496893210 wait complete. 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 -- start start 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.188+0000 7fc8c77ed740 1 Processor -- start 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c0229902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022993300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a6720 con 0x55c022983810 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a7a40 con 0x55c0229849e0 2024-03-26T14:11:26.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c01a6d80 con 0x55c0229853a0 2024-03-26T14:11:26.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8c77ed740 10 client.0 _ll_register_callbacks cb 0x7fc8ac0f4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:26.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8bde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42540/0 (socket says 172.21.15.137:42540) 2024-03-26T14:11:26.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8bde00640 1 -- 172.21.15.137:0/776050986 learned_addr learned my addr 172.21.15.137:0/776050986 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.189+0000 7fc8bd400640 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c0229902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8bde00640 1 -- 172.21.15.137:0/776050986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 msgr2=0x55c022993300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8bde00640 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022993300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.193 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8bde00640 1 -- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 msgr2=0x55c0229902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8bde00640 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c0229902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8bde00640 1 -- 172.21.15.137:0/776050986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0229938f0 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8bde00640 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298fda0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00059f0 tx=0x7fc8b000b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60538375946acb84 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc89f400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc89f400640 1 -- 172.21.15.137:0/776050986 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0038070 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc89f400640 1 -- 172.21.15.137:0/776050986 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc8b00052d0 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.190+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b000b3a0 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc89f400640 1 -- 172.21.15.137:0/776050986 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b0028040 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 10 client.17568 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c0229f4960 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc89f400640 1 -- 172.21.15.137:0/776050986 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8b000b3a0 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc89f400640 1 -- 172.21.15.137:0/776050986 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc8b0011440 con 0x55c022983810 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 10 client.17568 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:26.194 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 10 client.17568 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 20 client.17568 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c0229f4fc0 0x55c0229f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 4 client.17568 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.191+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0228efb50 con 0x55c0229f4fc0 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.192+0000 7fc8bd400640 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c0229f4fc0 0x55c0229f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.192+0000 7fc8bd400640 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c0229f4fc0 0x55c0229f73b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9b6d2dd7cc3bc56 server_cookie=b75e5756c7f98d6e in_seq=0 out_seq=0 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.192+0000 7fc89f400640 10 client.17568 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc89f400640 1 -- 172.21.15.137:0/776050986 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0228efb50 con 0x55c0229f4fc0 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc89f400640 10 client.17568 handle_command_reply: tid=0 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 client.17568 shutdown 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 msgr2=0x55c02298fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298fda0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc8b00059f0 tx=0x7fc8b000b2c0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 shutdown_connections 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c0229f4fc0 0x55c0229f73b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.195 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0229849e0 0x55c0229902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c0229853a0 0x55c022993300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 --2- 172.21.15.137:0/776050986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c022983810 0x55c02298fda0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 >> 172.21.15.137:0/776050986 conn(0x55c02297ec00 msgr2=0x55c02297d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc89f400640 0 client.17568 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 shutdown_connections 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 1 -- 172.21.15.137:0/776050986 wait complete. 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.193+0000 7fc8c77ed740 20 client.17568 trim_cache size 0 max 16384 2024-03-26T14:11:26.196 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc8a40a2880 msgr2=0x7fc8a40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc8a40a2880 0x7fc8a40a4d40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55c0229848e0 tx=0x7fc8b8009600 comp rx=0 tx=0).stop 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 msgr2=0x7fc8c0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0197eb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005ab0 tx=0x7fc8ac0042f0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 shutdown_connections 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc8a40a2880 0x7fc8a40a4d40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8c0107390 0x7fc8c019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc8c0104a60 0x7fc8c0197eb0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.197 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 --2- 172.21.15.137:0/1017392726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc8c0102130 0x7fc8c0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.194+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 >> 172.21.15.137:0/1017392726 conn(0x7fc8c00fbb40 msgr2=0x7fc8c0102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.195+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 shutdown_connections 2024-03-26T14:11:26.198 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.195+0000 7fc8c4a00640 1 -- 172.21.15.137:0/1017392726 wait complete. 2024-03-26T14:11:26.211 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:26 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4059699750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:11:26.214 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:11:26.215 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:11:26.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:26 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4059699750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 -- 172.21.15.137:0/3932586427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0104a50 msgr2=0x7f2df0106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3932586427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0104a50 0x7f2df0106e40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009900 tx=0x7f2ddc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 -- 172.21.15.137:0/3932586427 shutdown_connections 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3932586427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0104a50 0x7f2df0106e40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3932586427 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0107380 0x7f2df01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3932586427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df0102120 0x7f2df0104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 -- 172.21.15.137:0/3932586427 >> 172.21.15.137:0/3932586427 conn(0x7f2df00fbb30 msgr2=0x7f2df00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 -- 172.21.15.137:0/3932586427 shutdown_connections 2024-03-26T14:11:26.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.443+0000 7f2df5c00640 1 -- 172.21.15.137:0/3932586427 wait complete. 2024-03-26T14:11:26.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.444+0000 7f2df5c00640 1 Processor -- start 2024-03-26T14:11:26.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.444+0000 7f2df5c00640 1 -- start start 2024-03-26T14:11:26.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df0102120 0x7f2df0197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 0x7f2df0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0107380 0x7f2df019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2df5c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df010d1d0 con 0x7f2df0104a50 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2df5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df010d050 con 0x7f2df0107380 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2df5c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df010d350 con 0x7f2df0102120 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2dee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 0x7f2df0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2dee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 0x7f2df0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42560/0 (socket says 172.21.15.137:42560) 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2dee000640 1 -- 172.21.15.137:0/3448589705 learned_addr learned my addr 172.21.15.137:0/3448589705 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2def400640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0107380 0x7f2df019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.445+0000 7f2deea00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df0102120 0x7f2df0197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2dee000640 1 -- 172.21.15.137:0/3448589705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df0102120 msgr2=0x7f2df0197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2dee000640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df0102120 0x7f2df0197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2dee000640 1 -- 172.21.15.137:0/3448589705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0107380 msgr2=0x7f2df019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2dee000640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0107380 0x7f2df019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2dee000640 1 -- 172.21.15.137:0/3448589705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df019f540 con 0x7f2df0104a50 2024-03-26T14:11:26.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2dee000640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 0x7f2df0197eb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc000c00 tx=0x7f2ddc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2de7400640 1 -- 172.21.15.137:0/3448589705 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc043020 con 0x7f2df0104a50 2024-03-26T14:11:26.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2de7400640 1 -- 172.21.15.137:0/3448589705 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ddc004350 con 0x7f2df0104a50 2024-03-26T14:11:26.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.446+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df019f7d0 con 0x7f2df0104a50 2024-03-26T14:11:26.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.447+0000 7f2de7400640 1 -- 172.21.15.137:0/3448589705 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc003b80 con 0x7f2df0104a50 2024-03-26T14:11:26.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.447+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df019fd70 con 0x7f2df0104a50 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2de7400640 1 -- 172.21.15.137:0/3448589705 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2ddc046020 con 0x7f2df0104a50 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 Processor -- start 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 -- start start 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091aac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3091072c0 con 0x55e3091a39e0 2024-03-26T14:11:26.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3090afc90 con 0x55e3091a2810 2024-03-26T14:11:26.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2df89e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e309103360 con 0x55e3091a43a0 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.451+0000 7f2deea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2def400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2deea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42568/0 (socket says 172.21.15.137:42568) 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2deea00640 1 -- 172.21.15.137:0/4066521673 learned_addr learned my addr 172.21.15.137:0/4066521673 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2dee000640 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2def400640 1 -- 172.21.15.137:0/4066521673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 msgr2=0x55e3091aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2def400640 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091aac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2def400640 1 -- 172.21.15.137:0/4066521673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 msgr2=0x55e3091a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2def400640 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2def400640 1 -- 172.21.15.137:0/4066521673 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3090d4080 con 0x55e3091a2810 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.452+0000 7f2de7400640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2dd40a2880 0x7f2dd40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.453+0000 7f2def400640 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091a2c30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f00 tx=0x7f2de8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca5bb6f76d4988b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.453+0000 7f2de7400640 1 -- 172.21.15.137:0/3448589705 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2ddc0f0f90 con 0x7f2df0104a50 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.453+0000 7f2deea00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2dd40a2880 0x7f2dd40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.453+0000 7f2ddbe00640 1 -- 172.21.15.137:0/4066521673 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8004d20 con 0x55e3091a2810 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.453+0000 7f2ddbe00640 1 -- 172.21.15.137:0/4066521673 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2de8004030 con 0x55e3091a2810 2024-03-26T14:11:26.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.453+0000 7f2ddbe00640 1 -- 172.21.15.137:0/4066521673 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8005280 con 0x55e3091a2810 2024-03-26T14:11:26.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:26 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4059699750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:11:26.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 -- 172.21.15.137:0/4066521673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 msgr2=0x55e3091a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091a2c30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f00 tx=0x7f2de8011810 comp rx=0 tx=0).stop 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2deea00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2dd40a2880 0x7f2dd40a4d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009e10 tx=0x7f2de0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 -- 172.21.15.137:0/4066521673 shutdown_connections 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 --2- 172.21.15.137:0/4066521673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091a2c30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 -- 172.21.15.137:0/4066521673 >> 172.21.15.137:0/4066521673 conn(0x55e30919dc00 msgr2=0x55e30919e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 -- 172.21.15.137:0/4066521673 shutdown_connections 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.454+0000 7f2df89e2740 1 -- 172.21.15.137:0/4066521673 wait complete. 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 -- start start 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 Processor -- start 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091acac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3091ab3b0 con 0x55e3091a39e0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3090c30d0 con 0x55e3091a2810 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3091ab530 con 0x55e3091a43a0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.455+0000 7f2df89e2740 10 client.0 _ll_register_callbacks cb 0x55e3090c7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091ad000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42578/0 (socket says 172.21.15.137:42578) 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 -- 172.21.15.137:0/1971023346 learned_addr learned my addr 172.21.15.137:0/1971023346 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2deea00640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2def400640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 -- 172.21.15.137:0/1971023346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 msgr2=0x55e3091b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 -- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 msgr2=0x55e3091acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 -- 172.21.15.137:0/1971023346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e308e1b3f0 con 0x55e3091a39e0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dee000640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091ad000 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0bd260 tx=0x7f2ddc004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c745a7913ee4ffce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.456+0000 7f2dd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2dd8c00640 1 -- 172.21.15.137:0/1971023346 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc0bd0a0 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2dd8c00640 1 -- 172.21.15.137:0/1971023346 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2ddc003d20 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de8004030 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2dd8c00640 1 -- 172.21.15.137:0/1971023346 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ddc0bcec0 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2df89e2740 10 client.17580 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e308ee57d0 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.457+0000 7f2dd8c00640 1 -- 172.21.15.137:0/1971023346 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2de8004030 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2dd8c00640 1 -- 172.21.15.137:0/1971023346 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2ddc0c5870 con 0x55e3091a39e0 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2df89e2740 10 client.17580 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2df89e2740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2df89e2740 20 client.17580 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2df89e2740 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3091b8520 0x55e3091ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2df89e2740 4 client.17580 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:26.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e309114800 con 0x55e3091b8520 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.458+0000 7f2deea00640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3091b8520 0x55e3091ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.459+0000 7f2dd8c00640 10 client.17580 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.459+0000 7f2deea00640 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3091b8520 0x55e3091ba950 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaf54a75f005ab41 server_cookie=f8b7954956799501 in_seq=0 out_seq=0 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.460+0000 7f2dd8c00640 1 -- 172.21.15.137:0/1971023346 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e309114800 con 0x55e3091b8520 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.460+0000 7f2dd8c00640 10 client.17580 handle_command_reply: tid=0 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.460+0000 7f2df89e2740 1 client.17580 shutdown 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 msgr2=0x55e3091ad000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091ad000 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0bd260 tx=0x7f2ddc004500 comp rx=0 tx=0).stop 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 shutdown_connections 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3091b8520 0x55e3091ba950 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a43a0 0x55e3091b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a39e0 0x55e3091ad000 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 --2- 172.21.15.137:0/1971023346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a2810 0x55e3091acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2dd8c00640 0 client.17580 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:26.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 >> 172.21.15.137:0/1971023346 conn(0x55e30919dc00 msgr2=0x55e30919ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 shutdown_connections 2024-03-26T14:11:26.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 1 -- 172.21.15.137:0/1971023346 wait complete. 2024-03-26T14:11:26.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.461+0000 7f2df89e2740 20 client.17580 trim_cache size 0 max 16384 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.480+0000 7f2df89e2740 1 Processor -- start 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.480+0000 7f2df89e2740 1 -- start start 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a2810 0x55e30919cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a39e0 0x55e3091b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 0x55e3091bf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2df89e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df01ac7d0 con 0x55e3091a43a0 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2df89e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df01aa2d0 con 0x55e3091a39e0 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2df89e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df01a8350 con 0x55e3091a2810 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 0x55e3091bf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2dee000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a39e0 0x55e3091b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 0x55e3091bf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42588/0 (socket says 172.21.15.137:42588) 2024-03-26T14:11:26.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 -- 172.21.15.137:0/3929439693 learned_addr learned my addr 172.21.15.137:0/3929439693 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2deea00640 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a2810 0x55e30919cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 -- 172.21.15.137:0/3929439693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a2810 msgr2=0x55e30919cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a2810 0x55e30919cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 -- 172.21.15.137:0/3929439693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a39e0 msgr2=0x55e3091b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a39e0 0x55e3091b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2def400640 1 -- 172.21.15.137:0/3929439693 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3091bafd0 con 0x55e3091a43a0 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.481+0000 7f2dee000640 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a39e0 0x55e3091b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.482+0000 7f2def400640 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 0x55e3091bf250 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009ed0 tx=0x7f2de8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9828ff8cf90279 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.482+0000 7f2ddb400640 1 -- 172.21.15.137:0/3929439693 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8003ab0 con 0x55e3091a43a0 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.482+0000 7f2ddb400640 1 -- 172.21.15.137:0/3929439693 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2de8003c50 con 0x55e3091a43a0 2024-03-26T14:11:26.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.482+0000 7f2ddb400640 1 -- 172.21.15.137:0/3929439693 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800f770 con 0x55e3091a43a0 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 -- 172.21.15.137:0/3929439693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 msgr2=0x55e3091bf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 0x55e3091bf250 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009ed0 tx=0x7f2de8009b90 comp rx=0 tx=0).stop 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 -- 172.21.15.137:0/3929439693 shutdown_connections 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a39e0 0x55e3091b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a43a0 0x55e3091bf250 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 --2- 172.21.15.137:0/3929439693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a2810 0x55e30919cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 -- 172.21.15.137:0/3929439693 >> 172.21.15.137:0/3929439693 conn(0x55e30919dc00 msgr2=0x55e30919c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 -- 172.21.15.137:0/3929439693 shutdown_connections 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.483+0000 7f2df89e2740 1 -- 172.21.15.137:0/3929439693 wait complete. 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.484+0000 7f2df89e2740 1 -- start start 2024-03-26T14:11:26.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.484+0000 7f2df89e2740 1 Processor -- start 2024-03-26T14:11:26.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 0x55e3091b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a39e0 0x55e3091b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a43a0 0x55e3091b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df01a8950 con 0x55e3091a2810 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df01a9c70 con 0x55e3091a43a0 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df01a8fb0 con 0x55e3091a39e0 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2df89e2740 10 client.0 _ll_register_callbacks cb 0x7f2ddc0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 0x55e3091b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 0x55e3091b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42590/0 (socket says 172.21.15.137:42590) 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 -- 172.21.15.137:0/2165525703 learned_addr learned my addr 172.21.15.137:0/2165525703 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2def400640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a43a0 0x55e3091b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2dee000640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a39e0 0x55e3091b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 -- 172.21.15.137:0/2165525703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a39e0 msgr2=0x55e3091b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a39e0 0x55e3091b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 -- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a43a0 msgr2=0x55e3091b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a43a0 0x55e3091b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2deea00640 1 -- 172.21.15.137:0/2165525703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e309213e20 con 0x55e3091a2810 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.485+0000 7f2dee000640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a39e0 0x55e3091b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:26.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.486+0000 7f2deea00640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 0x55e3091b3e60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009bf0 tx=0x7f2de00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf8f93c3d85d596 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.486+0000 7f2dd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.486+0000 7f2dd8c00640 1 -- 172.21.15.137:0/2165525703 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de001fd10 con 0x55e3091a2810 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.486+0000 7f2dd8c00640 1 -- 172.21.15.137:0/2165525703 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2de002b680 con 0x55e3091a2810 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.486+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de8003c50 con 0x55e3091a2810 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2dd8c00640 1 -- 172.21.15.137:0/2165525703 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de003c050 con 0x55e3091a2810 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 10 client.17586 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:26.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e3092150b0 con 0x55e3091a2810 2024-03-26T14:11:26.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2dd8c00640 1 -- 172.21.15.137:0/2165525703 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2de8003c50 con 0x55e3091a2810 2024-03-26T14:11:26.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2dd8c00640 1 -- 172.21.15.137:0/2165525703 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2de001e870 con 0x55e3091a2810 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 10 client.17586 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 20 client.17586 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3092159f0 0x55e309217e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 4 client.17586 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e30910eb50 con 0x55e3092159f0 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.487+0000 7f2dee000640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3092159f0 0x55e309217e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2dd8c00640 10 client.17586 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2dee000640 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3092159f0 0x55e309217e10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404691b4b0f6c434 server_cookie=83563a4316bf72a1 in_seq=0 out_seq=0 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2dd8c00640 1 -- 172.21.15.137:0/2165525703 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e30910eb50 con 0x55e3092159f0 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2dd8c00640 10 client.17586 handle_command_reply: tid=0 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 client.17586 shutdown 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 msgr2=0x55e3091b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 0x55e3091b3e60 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009bf0 tx=0x7f2de00046c0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 shutdown_connections 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e3092159f0 0x55e309217e10 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3091a43a0 0x55e3091b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e3091a39e0 0x55e3091b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 --2- 172.21.15.137:0/2165525703 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e3091a2810 0x55e3091b3e60 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 >> 172.21.15.137:0/2165525703 conn(0x55e30919dc00 msgr2=0x55e30919ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.488+0000 7f2dd8c00640 0 client.17586 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 shutdown_connections 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df89e2740 1 -- 172.21.15.137:0/2165525703 wait complete. 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df89e2740 20 client.17586 trim_cache size 0 max 16384 2024-03-26T14:11:26.491 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2dd40a2880 msgr2=0x7f2dd40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2dd40a2880 0x7f2dd40a4d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009e10 tx=0x7f2de0005f90 comp rx=0 tx=0).stop 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 msgr2=0x7f2df0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.489+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 0x7f2df0197eb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc000c00 tx=0x7f2ddc005dc0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 shutdown_connections 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2dd40a2880 0x7f2dd40a4d40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2df0107380 0x7f2df019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.492 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2df0104a50 0x7f2df0197eb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 --2- 172.21.15.137:0/3448589705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2df0102120 0x7f2df0197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:26.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 >> 172.21.15.137:0/3448589705 conn(0x7f2df00fbb30 msgr2=0x7f2df0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:26.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 shutdown_connections 2024-03-26T14:11:26.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:26.490+0000 7f2df5c00640 1 -- 172.21.15.137:0/3448589705 wait complete. 2024-03-26T14:11:27.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:27 smithi179 ceph-mon[35472]: pgmap v730: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:27.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:27 smithi159 ceph-mon[35501]: pgmap v730: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:27.509 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:11:27.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:11:27.509 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:11:27.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:27 smithi137 ceph-mon[31539]: pgmap v730: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 -- 172.21.15.137:0/1992739423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffe50 msgr2=0x7fa570100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 --2- 172.21.15.137:0/1992739423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffe50 0x7fa570100270 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa564009900 tx=0x7fa56402bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 -- 172.21.15.137:0/1992739423 shutdown_connections 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 --2- 172.21.15.137:0/1992739423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5701007b0 0x7fa570100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 --2- 172.21.15.137:0/1992739423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570101150 0x7fa5701015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 --2- 172.21.15.137:0/1992739423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5700ffe50 0x7fa570100270 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 -- 172.21.15.137:0/1992739423 >> 172.21.15.137:0/1992739423 conn(0x7fa5700fba10 msgr2=0x7fa5700fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:27.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 -- 172.21.15.137:0/1992739423 shutdown_connections 2024-03-26T14:11:27.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.733+0000 7fa576800640 1 -- 172.21.15.137:0/1992739423 wait complete. 2024-03-26T14:11:27.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 Processor -- start 2024-03-26T14:11:27.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 -- start start 2024-03-26T14:11:27.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 0x7fa570199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5701007b0 0x7fa570199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 0x7fa5701a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57010f2c0 con 0x7fa570101150 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57010f140 con 0x7fa5700ffe50 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa576800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57010f440 con 0x7fa5701007b0 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.735+0000 7fa575e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 0x7fa5701a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 0x7fa570199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 0x7fa5701a0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42596/0 (socket says 172.21.15.137:42596) 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 -- 172.21.15.137:0/2696056127 learned_addr learned my addr 172.21.15.137:0/2696056127 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:27.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575400640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 0x7fa570199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:47300/0 (socket says 172.21.15.137:47300) 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa574a00640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5701007b0 0x7fa570199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 -- 172.21.15.137:0/2696056127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5701007b0 msgr2=0x7fa570199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5701007b0 0x7fa570199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 -- 172.21.15.137:0/2696056127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 msgr2=0x7fa570199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 0x7fa570199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 -- 172.21.15.137:0/2696056127 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5701a1590 con 0x7fa570101150 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575400640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 0x7fa570199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa574a00640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5701007b0 0x7fa570199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.736+0000 7fa575e00640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 0x7fa5701a0f20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa56800cce0 tx=0x7fa56800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.737+0000 7fa56e000640 1 -- 172.21.15.137:0/2696056127 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800f450 con 0x7fa570101150 2024-03-26T14:11:27.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.737+0000 7fa56e000640 1 -- 172.21.15.137:0/2696056127 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa56800faa0 con 0x7fa570101150 2024-03-26T14:11:27.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.737+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5701a1880 con 0x7fa570101150 2024-03-26T14:11:27.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.737+0000 7fa56e000640 1 -- 172.21.15.137:0/2696056127 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568018b70 con 0x7fa570101150 2024-03-26T14:11:27.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.737+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5701a1dd0 con 0x7fa570101150 2024-03-26T14:11:27.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.740+0000 7fa57ab6b740 1 Processor -- start 2024-03-26T14:11:27.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa56e000640 1 -- 172.21.15.137:0/2696056127 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa568018d10 con 0x7fa570101150 2024-03-26T14:11:27.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 -- start start 2024-03-26T14:11:27.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 0x55e7adc126d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc1a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7adb77eb0 con 0x55e7adc122b0 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7adb20a60 con 0x55e7adc17e10 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.741+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7adb73f50 con 0x55e7adc12ca0 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 0x55e7adc126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 0x55e7adc126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42598/0 (socket says 172.21.15.137:42598) 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 -- 172.21.15.137:0/3701411239 learned_addr learned my addr 172.21.15.137:0/3701411239 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa575400640 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc1a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 -- 172.21.15.137:0/3701411239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 msgr2=0x55e7adc177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 -- 172.21.15.137:0/3701411239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 msgr2=0x55e7adc1a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc1a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 -- 172.21.15.137:0/3701411239 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ad9547e0 con 0x55e7adc122b0 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa56e000640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5580a2880 0x7fa5580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.742+0000 7fa574a00640 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 0x55e7adc126d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa55c013660 tx=0x7fa55c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d42673bf03934c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.743+0000 7fa56e000640 1 -- 172.21.15.137:0/2696056127 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa5680cf1b0 con 0x7fa570101150 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.743+0000 7fa575400640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5580a2880 0x7fa5580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.743+0000 7fa562600640 1 -- 172.21.15.137:0/3701411239 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c019070 con 0x55e7adc122b0 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.743+0000 7fa562600640 1 -- 172.21.15.137:0/3701411239 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa55c0054d0 con 0x55e7adc122b0 2024-03-26T14:11:27.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.743+0000 7fa562600640 1 -- 172.21.15.137:0/3701411239 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c004bd0 con 0x55e7adc122b0 2024-03-26T14:11:27.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa575400640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5580a2880 0x7fa5580a4d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa564000c00 tx=0x7fa56402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:27.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3701411239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 msgr2=0x55e7adc126d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 0x55e7adc126d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa55c013660 tx=0x7fa55c00b730 comp rx=0 tx=0).stop 2024-03-26T14:11:27.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3701411239 shutdown_connections 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc1a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3701411239 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc122b0 0x55e7adc126d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3701411239 >> 172.21.15.137:0/3701411239 conn(0x55e7adc0d820 msgr2=0x55e7adc0dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3701411239 shutdown_connections 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.744+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3701411239 wait complete. 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 -- start start 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 Processor -- start 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc12ca0 0x55e7adc1cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc23b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7adc1adc0 con 0x55e7adc17e10 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7adb33eb0 con 0x55e7adc12ca0 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7adc1af40 con 0x55e7adc122b0 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.745+0000 7fa57ab6b740 10 client.0 _ll_register_callbacks cb 0x55e7adb28150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc23b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc23b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42600/0 (socket says 172.21.15.137:42600) 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 -- 172.21.15.137:0/3717781648 learned_addr learned my addr 172.21.15.137:0/3717781648 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa574a00640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc12ca0 0x55e7adc1cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575400640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 -- 172.21.15.137:0/3717781648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 msgr2=0x55e7adc1c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc1c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 -- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc12ca0 msgr2=0x55e7adc1cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc12ca0 0x55e7adc1cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa575e00640 1 -- 172.21.15.137:0/3717781648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7ad9b4f70 con 0x55e7adc17e10 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.746+0000 7fa574a00640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc12ca0 0x55e7adc1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa575e00640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc23b00 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa568097ba0 tx=0x7fa56809b160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c48ba5cc485df7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa556a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa556a00640 1 -- 172.21.15.137:0/3717781648 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680b4060 con 0x55e7adc17e10 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa556a00640 1 -- 172.21.15.137:0/3717781648 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa5680a3a50 con 0x55e7adc17e10 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa55c0054d0 con 0x55e7adc17e10 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa556a00640 1 -- 172.21.15.137:0/3717781648 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56809f890 con 0x55e7adc17e10 2024-03-26T14:11:27.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa57ab6b740 10 client.17598 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e7ad84c170 con 0x55e7adc17e10 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.747+0000 7fa556a00640 1 -- 172.21.15.137:0/3717781648 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa55c0054d0 con 0x55e7adc17e10 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa556a00640 1 -- 172.21.15.137:0/3717781648 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa5680afb20 con 0x55e7adc17e10 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa57ab6b740 10 client.17598 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa57ab6b740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa57ab6b740 20 client.17598 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc28340 0x55e7adc2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa57ab6b740 4 client.17598 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.748+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7adb853f0 con 0x55e7adc28340 2024-03-26T14:11:27.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.749+0000 7fa575400640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc28340 0x55e7adc2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.749+0000 7fa556a00640 10 client.17598 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:27.755 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.749+0000 7fa575400640 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc28340 0x55e7adc2a760 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b995d65c520417cb server_cookie=bee166148466d069 in_seq=0 out_seq=0 2024-03-26T14:11:27.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa556a00640 1 -- 172.21.15.137:0/3717781648 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e7adb853f0 con 0x55e7adc28340 2024-03-26T14:11:27.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa556a00640 10 client.17598 handle_command_reply: tid=0 2024-03-26T14:11:27.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa57ab6b740 1 client.17598 shutdown 2024-03-26T14:11:27.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 msgr2=0x55e7adc23b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc23b00 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa568097ba0 tx=0x7fa56809b160 comp rx=0 tx=0).stop 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 shutdown_connections 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc28340 0x55e7adc2a760 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.751+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc12ca0 0x55e7adc1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc23b00 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/3717781648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc1c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 >> 172.21.15.137:0/3717781648 conn(0x55e7adc0d820 msgr2=0x55e7adc0ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa556a00640 0 client.17598 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 shutdown_connections 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa57ab6b740 1 -- 172.21.15.137:0/3717781648 wait complete. 2024-03-26T14:11:27.757 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.752+0000 7fa57ab6b740 20 client.17598 trim_cache size 0 max 16384 2024-03-26T14:11:27.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 Processor -- start 2024-03-26T14:11:27.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 -- start start 2024-03-26T14:11:27.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc13710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 0x55e7adc23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc13c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701ae680 con 0x55e7adc12ca0 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701ac180 con 0x55e7adc17e10 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.771+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701aa200 con 0x55e7adc122b0 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 0x55e7adc23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 0x55e7adc23ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42608/0 (socket says 172.21.15.137:42608) 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 -- 172.21.15.137:0/1403899779 learned_addr learned my addr 172.21.15.137:0/1403899779 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa575e00640 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc13c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa575400640 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc13710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 -- 172.21.15.137:0/1403899779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 msgr2=0x55e7adc13710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc13710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 -- 172.21.15.137:0/1403899779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 msgr2=0x55e7adc13c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc13c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.772+0000 7fa574a00640 1 -- 172.21.15.137:0/1403899779 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7adc0e290 con 0x55e7adc12ca0 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.773+0000 7fa574a00640 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 0x55e7adc23ac0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00b600 tx=0x7fa55c013540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a4ab9d8f6dfc90 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.773+0000 7fa561c00640 1 -- 172.21.15.137:0/1403899779 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c02b070 con 0x55e7adc12ca0 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.773+0000 7fa561c00640 1 -- 172.21.15.137:0/1403899779 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa55c0035f0 con 0x55e7adc12ca0 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.773+0000 7fa561c00640 1 -- 172.21.15.137:0/1403899779 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c019040 con 0x55e7adc12ca0 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 -- 172.21.15.137:0/1403899779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 msgr2=0x55e7adc23ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 0x55e7adc23ac0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00b600 tx=0x7fa55c013540 comp rx=0 tx=0).stop 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 -- 172.21.15.137:0/1403899779 shutdown_connections 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc17e10 0x55e7adc13c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc12ca0 0x55e7adc23ac0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/1403899779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc122b0 0x55e7adc13710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 -- 172.21.15.137:0/1403899779 >> 172.21.15.137:0/1403899779 conn(0x55e7adc0d820 msgr2=0x55e7adc11310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 -- 172.21.15.137:0/1403899779 shutdown_connections 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.774+0000 7fa57ab6b740 1 -- 172.21.15.137:0/1403899779 wait complete. 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 -- start start 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 Processor -- start 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc122b0 0x55e7adc21860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc21da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc22db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701aa800 con 0x55e7adc17e10 2024-03-26T14:11:27.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701abb20 con 0x55e7adc122b0 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701aae60 con 0x55e7adc12ca0 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.775+0000 7fa57ab6b740 10 client.0 _ll_register_callbacks cb 0x7fa5680077a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc22db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc22db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:42616/0 (socket says 172.21.15.137:42616) 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc122b0 0x55e7adc21860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 -- 172.21.15.137:0/2807188498 learned_addr learned my addr 172.21.15.137:0/2807188498 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa574a00640 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc21da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 -- 172.21.15.137:0/2807188498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 msgr2=0x55e7adc21da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc21da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 -- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc122b0 msgr2=0x55e7adc21860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:27.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc122b0 0x55e7adc21860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 -- 172.21.15.137:0/2807188498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7adc233a0 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa575e00640 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc22db0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa56809b6d0 tx=0x7fa568097490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72873d15cf5ba815 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa556a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa556a00640 1 -- 172.21.15.137:0/2807188498 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56809b510 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa556a00640 1 -- 172.21.15.137:0/2807188498 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa5680a3a50 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.776+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa55c0035f0 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa556a00640 1 -- 172.21.15.137:0/2807188498 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5680a3ee0 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 10 client.17604 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e7adc842f0 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa556a00640 1 -- 172.21.15.137:0/2807188498 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa55c0035f0 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa556a00640 1 -- 172.21.15.137:0/2807188498 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa568081020 con 0x55e7adc17e10 2024-03-26T14:11:27.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 10 client.17604 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:27.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:27.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 20 client.17604 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:27.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc84b80 0x55e7adc86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:27.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 4 client.17604 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:11:27.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.777+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e7adb7f740 con 0x55e7adc84b80 2024-03-26T14:11:27.782 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.778+0000 7fa575400640 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc84b80 0x55e7adc86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:27.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.778+0000 7fa575400640 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc84b80 0x55e7adc86fa0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55222f339a0a5d4f server_cookie=6f451644045ef609 in_seq=0 out_seq=0 2024-03-26T14:11:27.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.778+0000 7fa556a00640 10 client.17604 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.780+0000 7fa556a00640 1 -- 172.21.15.137:0/2807188498 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e7adb7f740 con 0x55e7adc84b80 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.780+0000 7fa556a00640 10 client.17604 handle_command_reply: tid=0 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.780+0000 7fa57ab6b740 1 client.17604 shutdown 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.780+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 msgr2=0x55e7adc22db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.780+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc22db0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa56809b6d0 tx=0x7fa568097490 comp rx=0 tx=0).stop 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 shutdown_connections 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e7adc84b80 0x55e7adc86fa0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e7adc17e10 0x55e7adc22db0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e7adc12ca0 0x55e7adc21da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 --2- 172.21.15.137:0/2807188498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7adc122b0 0x55e7adc21860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa556a00640 0 client.17604 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:27.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 >> 172.21.15.137:0/2807188498 conn(0x55e7adc0d820 msgr2=0x55e7adc0c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 shutdown_connections 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 1 -- 172.21.15.137:0/2807188498 wait complete. 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.781+0000 7fa57ab6b740 20 client.17604 trim_cache size 0 max 16384 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "11c3ad86-736c-497d-b33f-d60c24b7924f", 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:11:27.785 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:11:27.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5580a2880 msgr2=0x7fa5580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5580a2880 0x7fa5580a4d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa564000c00 tx=0x7fa56402c7e0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 msgr2=0x7fa5701a0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:27.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 0x7fa5701a0f20 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa56800cce0 tx=0x7fa56800a620 comp rx=0 tx=0).stop 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 shutdown_connections 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa5580a2880 0x7fa5580a4d40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa570101150 0x7fa5701a0f20 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa5701007b0 0x7fa570199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 --2- 172.21.15.137:0/2696056127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5700ffe50 0x7fa570199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 >> 172.21.15.137:0/2696056127 conn(0x7fa5700fba10 msgr2=0x7fa5700fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 shutdown_connections 2024-03-26T14:11:27.787 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:27.782+0000 7fa576800640 1 -- 172.21.15.137:0/2696056127 wait complete. 2024-03-26T14:11:27.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:11:27.802 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:11:28.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 -- 172.21.15.137:0/1961379789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20101670 msgr2=0x7fbb20101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/1961379789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20101670 0x7fbb20101af0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c02bc50 comp rx=0 tx=0).stop 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 -- 172.21.15.137:0/1961379789 shutdown_connections 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/1961379789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb20102030 0x7fbb201088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/1961379789 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20101670 0x7fbb20101af0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/1961379789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20100480 0x7fbb201008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 -- 172.21.15.137:0/1961379789 >> 172.21.15.137:0/1961379789 conn(0x7fbb200fba10 msgr2=0x7fbb200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 -- 172.21.15.137:0/1961379789 shutdown_connections 2024-03-26T14:11:28.057 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.056+0000 7fbb1fe00640 1 -- 172.21.15.137:0/1961379789 wait complete. 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.057+0000 7fbb1fe00640 1 Processor -- start 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.057+0000 7fbb1fe00640 1 -- start start 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 0x7fbb20197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb20101670 0x7fbb20197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20102030 0x7fbb2019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2010bfc0 con 0x7fbb20100480 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2010be40 con 0x7fbb20102030 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1fe00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2010c140 con 0x7fbb20101670 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 0x7fbb20197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 0x7fbb20197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50880/0 (socket says 172.21.15.137:50880) 2024-03-26T14:11:28.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1ea00640 1 -- 172.21.15.137:0/4038427518 learned_addr learned my addr 172.21.15.137:0/4038427518 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1f400640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20102030 0x7fbb2019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.058+0000 7fbb1e000640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb20101670 0x7fbb20197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1ea00640 1 -- 172.21.15.137:0/4038427518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb20101670 msgr2=0x7fbb20197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb20101670 0x7fbb20197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1ea00640 1 -- 172.21.15.137:0/4038427518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20102030 msgr2=0x7fbb2019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20102030 0x7fbb2019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1ea00640 1 -- 172.21.15.137:0/4038427518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2019f380 con 0x7fbb20100480 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1f400640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20102030 0x7fbb2019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 0x7fbb20197760 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d770 tx=0x7fbb1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb17400640 1 -- 172.21.15.137:0/4038427518 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10017070 con 0x7fbb20100480 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb17400640 1 -- 172.21.15.137:0/4038427518 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb10009c30 con 0x7fbb20100480 2024-03-26T14:11:28.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2019f670 con 0x7fbb20100480 2024-03-26T14:11:28.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb17400640 1 -- 172.21.15.137:0/4038427518 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10003d00 con 0x7fbb20100480 2024-03-26T14:11:28.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.059+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2019fbc0 con 0x7fbb20100480 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 Processor -- start 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb17400640 1 -- 172.21.15.137:0/4038427518 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbb10003ea0 con 0x7fbb20100480 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 -- start start 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7460 0x559852fc7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7e50 0x559852fcc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fcf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559852f28250 con 0x559852fccfc0 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559852ed0a30 con 0x559852fc7460 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.064+0000 7fbb2861f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559852f242f0 con 0x559852fc7e50 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7460 0x559852fc7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fcf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fcf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50896/0 (socket says 172.21.15.137:50896) 2024-03-26T14:11:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7460 0x559852fc7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:53096/0 (socket says 172.21.15.137:53096) 2024-03-26T14:11:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 -- 172.21.15.137:0/3304033399 learned_addr learned my addr 172.21.15.137:0/3304033399 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:28.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7e50 0x559852fcc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 -- 172.21.15.137:0/3304033399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7e50 msgr2=0x559852fcc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7e50 0x559852fcc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 -- 172.21.15.137:0/3304033399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7460 msgr2=0x559852fc7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7460 0x559852fc7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.065+0000 7fbb1e000640 1 -- 172.21.15.137:0/3304033399 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559852ebb690 con 0x559852fccfc0 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb1e000640 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fcf450 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005ab0 tx=0x7fbb0c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30716da68be69a9e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:28.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb17400640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a2800 0x7fbb040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb0be00640 1 -- 172.21.15.137:0/3304033399 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c044020 con 0x559852fccfc0 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb17400640 1 -- 172.21.15.137:0/4038427518 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb100cec10 con 0x7fbb20100480 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb0be00640 1 -- 172.21.15.137:0/3304033399 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb0c02cc40 con 0x559852fccfc0 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb0be00640 1 -- 172.21.15.137:0/3304033399 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004920 con 0x559852fccfc0 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.066+0000 7fbb1f400640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a2800 0x7fbb040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 -- 172.21.15.137:0/3304033399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 msgr2=0x559852fcf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb1f400640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a2800 0x7fbb040a4cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x559852fc86b0 tx=0x7fbb18009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fcf450 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005ab0 tx=0x7fbb0c005d50 comp rx=0 tx=0).stop 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 -- 172.21.15.137:0/3304033399 shutdown_connections 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fcf450 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7e50 0x559852fcc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3304033399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7460 0x559852fc7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 -- 172.21.15.137:0/3304033399 >> 172.21.15.137:0/3304033399 conn(0x559852f8b200 msgr2=0x559852fc2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:28.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 -- 172.21.15.137:0/3304033399 shutdown_connections 2024-03-26T14:11:28.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.067+0000 7fbb2861f740 1 -- 172.21.15.137:0/3304033399 wait complete. 2024-03-26T14:11:28.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.068+0000 7fbb2861f740 1 -- start start 2024-03-26T14:11:28.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.068+0000 7fbb2861f740 1 Processor -- start 2024-03-26T14:11:28.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.068+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb2861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559852fcff10 con 0x559852fc7460 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb2861f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559852ee3e80 con 0x559852fc7e50 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb2861f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559852fd0090 con 0x559852fccfc0 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb2861f740 10 client.0 _ll_register_callbacks cb 0x559852ef6130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50906/0 (socket says 172.21.15.137:50906) 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 -- 172.21.15.137:0/3613972822 learned_addr learned my addr 172.21.15.137:0/3613972822 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1e000640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1f400640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 -- 172.21.15.137:0/3613972822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 msgr2=0x559852fd8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 -- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 msgr2=0x559852fd18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1ea00640 1 -- 172.21.15.137:0/3613972822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559852d067d0 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.069+0000 7fbb1f400640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd1380 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbb1009b170 tx=0x7fbb1009ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3485b8f79c83ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb08c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb08c00640 1 -- 172.21.15.137:0/3613972822 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10020050 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb08c00640 1 -- 172.21.15.137:0/3613972822 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb100bb0b0 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb0c02cc40 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb08c00640 1 -- 172.21.15.137:0/3613972822 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb1009b1e0 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb2861f740 10 client.17616 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.070+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559852c3c3f0 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb08c00640 1 -- 172.21.15.137:0/3613972822 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb0c02cc40 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb08c00640 1 -- 172.21.15.137:0/3613972822 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbb100b9ce0 con 0x559852fc7460 2024-03-26T14:11:28.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb2861f740 10 client.17616 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb2861f740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb2861f740 20 client.17616 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559852fdd1f0 0x559852fdf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb2861f740 4 client.17616 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559852f35790 con 0x559852fdd1f0 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.071+0000 7fbb1e000640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559852fdd1f0 0x559852fdf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.072+0000 7fbb08c00640 10 client.17616 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.072+0000 7fbb1e000640 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559852fdd1f0 0x559852fdf640 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1e140fd0f81432 server_cookie=dcc821143aca8062 in_seq=0 out_seq=0 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.073+0000 7fbb08c00640 1 -- 172.21.15.137:0/3613972822 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559852f35790 con 0x559852fdd1f0 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.073+0000 7fbb08c00640 10 client.17616 handle_command_reply: tid=0 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 client.17616 shutdown 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 msgr2=0x559852fd1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd1380 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbb1009b170 tx=0x7fbb1009ade0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 shutdown_connections 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559852fdd1f0 0x559852fdf640 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb08c00640 0 client.17616 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 --2- 172.21.15.137:0/3613972822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd1380 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 >> 172.21.15.137:0/3613972822 conn(0x559852f8b200 msgr2=0x559852fbdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 shutdown_connections 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 1 -- 172.21.15.137:0/3613972822 wait complete. 2024-03-26T14:11:28.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.074+0000 7fbb2861f740 20 client.17616 trim_cache size 0 max 16384 2024-03-26T14:11:28.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.092+0000 7fbb2861f740 1 Processor -- start 2024-03-26T14:11:28.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.092+0000 7fbb2861f740 1 -- start start 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7460 0x559852fc6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fc6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb2861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201aa260 con 0x559852fccfc0 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb2861f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201a7d60 con 0x559852fc7e50 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb2861f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201a5de0 con 0x559852fc7460 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fc6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb1f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fc6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50914/0 (socket says 172.21.15.137:50914) 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb1e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb1f400640 1 -- 172.21.15.137:0/2657132277 learned_addr learned my addr 172.21.15.137:0/2657132277 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:28.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.093+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7460 0x559852fc6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb1e000640 1 -- 172.21.15.137:0/2657132277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7460 msgr2=0x559852fc6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb1e000640 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7460 0x559852fc6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb1e000640 1 -- 172.21.15.137:0/2657132277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 msgr2=0x559852fc6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb1e000640 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fc6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb1e000640 1 -- 172.21.15.137:0/2657132277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559852bfc0f0 con 0x559852fc7e50 2024-03-26T14:11:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb1e000640 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd88c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000c00 tx=0x7fbb0c02cdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f47d6b4eda469b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb0b400640 1 -- 172.21.15.137:0/2657132277 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c044040 con 0x559852fc7e50 2024-03-26T14:11:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb0b400640 1 -- 172.21.15.137:0/2657132277 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb0c005040 con 0x559852fc7e50 2024-03-26T14:11:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.094+0000 7fbb0b400640 1 -- 172.21.15.137:0/2657132277 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c004a90 con 0x559852fc7e50 2024-03-26T14:11:28.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 -- 172.21.15.137:0/2657132277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 msgr2=0x559852fd88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd88c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000c00 tx=0x7fbb0c02cdf0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 -- 172.21.15.137:0/2657132277 shutdown_connections 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd88c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fccfc0 0x559852fc6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 --2- 172.21.15.137:0/2657132277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fc7460 0x559852fc6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 -- 172.21.15.137:0/2657132277 >> 172.21.15.137:0/2657132277 conn(0x559852f8b200 msgr2=0x559852fc9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 -- 172.21.15.137:0/2657132277 shutdown_connections 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.095+0000 7fbb2861f740 1 -- 172.21.15.137:0/2657132277 wait complete. 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.096+0000 7fbb2861f740 1 -- start start 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.096+0000 7fbb2861f740 1 Processor -- start 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.096+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.096+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.096+0000 7fbb2861f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.096+0000 7fbb2861f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201a63e0 con 0x559852fc7460 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb2861f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201a7700 con 0x559852fc7e50 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb2861f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb201a6a40 con 0x559852fccfc0 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb2861f740 10 client.0 _ll_register_callbacks cb 0x7fbb100046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:50922/0 (socket says 172.21.15.137:50922) 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 -- 172.21.15.137:0/499428891 learned_addr learned my addr 172.21.15.137:0/499428891 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1e000640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1f400640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 -- 172.21.15.137:0/499428891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 msgr2=0x559852fd7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 -- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 msgr2=0x559852fd6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.097+0000 7fbb1ea00640 1 -- 172.21.15.137:0/499428891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559852fd82e0 con 0x559852fc7460 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb1ea00640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd67a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbb1009b3a0 tx=0x7fbb1009b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc14a37f8daddce server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb08c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb08c00640 1 -- 172.21.15.137:0/499428891 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10015020 con 0x559852fc7460 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb08c00640 1 -- 172.21.15.137:0/499428891 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb1009a630 con 0x559852fc7460 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb0c005040 con 0x559852fc7460 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb08c00640 1 -- 172.21.15.137:0/499428891 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb10020050 con 0x559852fc7460 2024-03-26T14:11:28.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb2861f740 10 client.17622 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559853039330 con 0x559852fc7460 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.098+0000 7fbb08c00640 1 -- 172.21.15.137:0/499428891 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb0c005040 con 0x559852fc7460 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb08c00640 1 -- 172.21.15.137:0/499428891 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbb100b9ce0 con 0x559852fc7460 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb2861f740 10 client.17622 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb2861f740 10 client.17622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb2861f740 20 client.17622 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb2861f740 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559853039ba0 0x55985303bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb2861f740 4 client.17622 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559852f2fae0 con 0x559853039ba0 2024-03-26T14:11:28.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.099+0000 7fbb1e000640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559853039ba0 0x55985303bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb1e000640 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559853039ba0 0x55985303bfc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa2aedfc4869239 server_cookie=7bee99b0eb9fc788 in_seq=0 out_seq=0 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb08c00640 10 client.17622 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb08c00640 1 -- 172.21.15.137:0/499428891 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559852f2fae0 con 0x559853039ba0 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb08c00640 10 client.17622 handle_command_reply: tid=0 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb2861f740 1 client.17622 shutdown 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 msgr2=0x559852fd67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb2861f740 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd67a0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbb1009b3a0 tx=0x7fbb1009b960 comp rx=0 tx=0).stop 2024-03-26T14:11:28.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 shutdown_connections 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.100+0000 7fbb2861f740 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559853039ba0 0x55985303bfc0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559852fc7e50 0x559852fd6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559852fccfc0 0x559852fd7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 1 --2- 172.21.15.137:0/499428891 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559852fc7460 0x559852fd67a0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 >> 172.21.15.137:0/499428891 conn(0x559852f8b200 msgr2=0x559852fbd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb08c00640 0 client.17622 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 shutdown_connections 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 1 -- 172.21.15.137:0/499428891 wait complete. 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb2861f740 20 client.17622 trim_cache size 0 max 16384 2024-03-26T14:11:28.106 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: "11c3ad86-736c-497d-b33f-d60c24b7924f": { 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "11c3ad86-736c-497d-b33f-d60c24b7924f", 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:11:28.107 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a2800 msgr2=0x7fbb040a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a2800 0x7fbb040a4cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x559852fc86b0 tx=0x7fbb18009800 comp rx=0 tx=0).stop 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 msgr2=0x7fbb20197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.101+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 0x7fbb20197760 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d770 tx=0x7fbb1000dc30 comp rx=0 tx=0).stop 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 shutdown_connections 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a2800 0x7fbb040a4cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb20102030 0x7fbb2019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb20101670 0x7fbb20197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 --2- 172.21.15.137:0/4038427518 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb20100480 0x7fbb20197760 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:28.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 >> 172.21.15.137:0/4038427518 conn(0x7fbb200fba10 msgr2=0x7fbb20068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:28.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 shutdown_connections 2024-03-26T14:11:28.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:28.102+0000 7fbb1fe00640 1 -- 172.21.15.137:0/4038427518 wait complete. 2024-03-26T14:11:28.121 INFO:tasks.cephfs.filesystem:scrub status for tag:11c3ad86-736c-497d-b33f-d60c24b7924f - {'path': '/', 'tag': '11c3ad86-736c-497d-b33f-d60c24b7924f', 'options': 'recursive,force'} 2024-03-26T14:11:28.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:28 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:11:28.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:28 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:11:28.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:28 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:11:28.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:28 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:11:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:28 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:11:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:28 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:11:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:28 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:11:28.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:28 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:11:28.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:28 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:11:28.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:28 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:11:28.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:28 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:11:28.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:28 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:11:29.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:29 smithi159 ceph-mon[35501]: pgmap v731: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:29.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:29 smithi137 ceph-mon[31539]: pgmap v731: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:29.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:29 smithi179 ceph-mon[35472]: pgmap v731: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:31.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:31 smithi159 ceph-mon[35501]: pgmap v732: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:31.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:31 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:11:31.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:31 smithi137 ceph-mon[31539]: pgmap v732: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:31.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:31 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:11:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:31 smithi179 ceph-mon[35472]: pgmap v732: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:31 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:11:33.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:32 smithi179 ceph-mon[35472]: pgmap v733: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:33.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:33.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:32 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:33.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:32 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:11:33.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:32 smithi159 ceph-mon[35501]: pgmap v733: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:33.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:33.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:32 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:33.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:32 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:11:33.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:32 smithi137 ceph-mon[31539]: pgmap v733: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:33.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:33.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:32 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:33.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:32 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:11:34.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:11:34.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:11:34.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:33 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:34.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:11:34.202 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:11:34.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:33 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:34.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:11:34.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:11:34.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:33 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:11:35.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:35 smithi179 ceph-mon[35472]: pgmap v734: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:11:35.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:35 smithi159 ceph-mon[35501]: pgmap v734: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:11:35.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:35 smithi137 ceph-mon[31539]: pgmap v734: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:11:37.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:37 smithi179 ceph-mon[35472]: pgmap v735: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:37.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:37 smithi159 ceph-mon[35501]: pgmap v735: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:37.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:37 smithi137 ceph-mon[31539]: pgmap v735: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:39.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:39 smithi179 ceph-mon[35472]: pgmap v736: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:11:39.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:39 smithi159 ceph-mon[35501]: pgmap v736: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:11:39.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:39 smithi137 ceph-mon[31539]: pgmap v736: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:11:41.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:41 smithi179 ceph-mon[35472]: pgmap v737: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:41.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:41 smithi159 ceph-mon[35501]: pgmap v737: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:41.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:41 smithi137 ceph-mon[31539]: pgmap v737: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:11:43.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:43 smithi159 ceph-mon[35501]: pgmap v738: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:43.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:43 smithi137 ceph-mon[31539]: pgmap v738: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:43.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:43 smithi179 ceph-mon[35472]: pgmap v738: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:11:45.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:45 smithi159 ceph-mon[35501]: pgmap v739: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:45.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:45 smithi137 ceph-mon[31539]: pgmap v739: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:45.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:45 smithi179 ceph-mon[35472]: pgmap v739: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:47.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:47 smithi159 ceph-mon[35501]: pgmap v740: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:47.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:47 smithi137 ceph-mon[31539]: pgmap v740: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:47.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:47 smithi179 ceph-mon[35472]: pgmap v740: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:49.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:49 smithi159 ceph-mon[35501]: pgmap v741: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:49.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:49 smithi137 ceph-mon[31539]: pgmap v741: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:49.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:49 smithi179 ceph-mon[35472]: pgmap v741: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:51.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:51 smithi159 ceph-mon[35501]: pgmap v742: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:51.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:51 smithi137 ceph-mon[31539]: pgmap v742: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:51.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:51 smithi179 ceph-mon[35472]: pgmap v742: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:53 smithi137 ceph-mon[31539]: pgmap v743: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:53.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:53 smithi179 ceph-mon[35472]: pgmap v743: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:53.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:53 smithi159 ceph-mon[35501]: pgmap v743: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:11:55.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:55 smithi137 ceph-mon[31539]: pgmap v744: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:55.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:55 smithi179 ceph-mon[35472]: pgmap v744: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:55.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:55 smithi159 ceph-mon[35501]: pgmap v744: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:11:57.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:57 smithi137 ceph-mon[31539]: pgmap v745: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:57.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:57 smithi179 ceph-mon[35472]: pgmap v745: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:57.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:57 smithi159 ceph-mon[35501]: pgmap v745: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:11:58.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:11:58.122 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 -- 172.21.15.137:0/2567648072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8109570 msgr2=0x7fcaa810ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2567648072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8109570 0x7fcaa810ba00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009ca0 tx=0x7fcaa002c0f0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 -- 172.21.15.137:0/2567648072 shutdown_connections 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2567648072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8109570 0x7fcaa810ba00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2567648072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa8074100 0x7fcaa8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2567648072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcaa80756e0 0x7fcaa8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 -- 172.21.15.137:0/2567648072 >> 172.21.15.137:0/2567648072 conn(0x7fcaa80fcc20 msgr2=0x7fcaa80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 -- 172.21.15.137:0/2567648072 shutdown_connections 2024-03-26T14:11:58.345 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.343+0000 7fcaaf400640 1 -- 172.21.15.137:0/2567648072 wait complete. 2024-03-26T14:11:58.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.344+0000 7fcaaf400640 1 Processor -- start 2024-03-26T14:11:58.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.344+0000 7fcaaf400640 1 -- start start 2024-03-26T14:11:58.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 0x7fcaa8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcaa80756e0 0x7fcaa819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaf400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa8109570 0x7fcaa81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaf400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa810f3c0 con 0x7fcaa8074100 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaf400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa810f240 con 0x7fcaa80756e0 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaf400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa810f540 con 0x7fcaa8109570 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 0x7fcaa8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 0x7fcaa8199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48480/0 (socket says 172.21.15.137:48480) 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 -- 172.21.15.137:0/2514623153 learned_addr learned my addr 172.21.15.137:0/2514623153 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcaa80756e0 0x7fcaa819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 -- 172.21.15.137:0/2514623153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa8109570 msgr2=0x7fcaa81a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa8109570 0x7fcaa81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 -- 172.21.15.137:0/2514623153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcaa80756e0 msgr2=0x7fcaa819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcaa80756e0 0x7fcaa819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.345+0000 7fcaaca00640 1 -- 172.21.15.137:0/2514623153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa81a16f0 con 0x7fcaa8074100 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.346+0000 7fcaaca00640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 0x7fcaa8199b20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fca9800e9f0 tx=0x7fca9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.346+0000 7fcaa5600640 1 -- 172.21.15.137:0/2514623153 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800cd80 con 0x7fcaa8074100 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.346+0000 7fcaa5600640 1 -- 172.21.15.137:0/2514623153 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca980108e0 con 0x7fcaa8074100 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.346+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa81a19e0 con 0x7fcaa8074100 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.346+0000 7fcaa5600640 1 -- 172.21.15.137:0/2514623153 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98018920 con 0x7fcaa8074100 2024-03-26T14:11:58.349 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.346+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa81a1f30 con 0x7fcaa8074100 2024-03-26T14:11:58.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.349+0000 7fcaa5600640 1 -- 172.21.15.137:0/2514623153 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fca98010a90 con 0x7fcaa8074100 2024-03-26T14:11:58.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.349+0000 7fcab1ffb740 1 Processor -- start 2024-03-26T14:11:58.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.349+0000 7fcab1ffb740 1 -- start start 2024-03-26T14:11:58.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af367322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 0x55af36737370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.353 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af367379e0 0x55af36739e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af36697250 con 0x55af36732870 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3663fa30 con 0x55af367379e0 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af366932f0 con 0x55af36731e80 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 0x55af36737370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 0x55af36737370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48486/0 (socket says 172.21.15.137:48486) 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 -- 172.21.15.137:0/2273071064 learned_addr learned my addr 172.21.15.137:0/2273071064 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af367379e0 0x55af36739e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaad400640 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af367322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 -- 172.21.15.137:0/2273071064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 msgr2=0x55af367322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af367322a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.354 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 -- 172.21.15.137:0/2273071064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af367379e0 msgr2=0x55af36739e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.350+0000 7fcaaca00640 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af367379e0 0x55af36739e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fcaaca00640 1 -- 172.21.15.137:0/2273071064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af3662a690 con 0x55af36732870 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af367379e0 0x55af36739e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fcaaca00640 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 0x55af36737370 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fca980d74b0 tx=0x7fca980d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b7be4feb81b732 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fcaa5600640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca8c0a2800 0x7fca8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fca9dc00640 1 -- 172.21.15.137:0/2273071064 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca980e2070 con 0x55af36732870 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fcaa5600640 1 -- 172.21.15.137:0/2514623153 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fca980cf2c0 con 0x7fcaa8074100 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fca9dc00640 1 -- 172.21.15.137:0/2273071064 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca98010de0 con 0x55af36732870 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fca9dc00640 1 -- 172.21.15.137:0/2273071064 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9809b1b0 con 0x55af36732870 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.351+0000 7fcaad400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca8c0a2800 0x7fca8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 -- 172.21.15.137:0/2273071064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 msgr2=0x55af36737370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcaad400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca8c0a2800 0x7fca8c0a4cc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55af367330d0 tx=0x7fcaa0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 0x55af36737370 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fca980d74b0 tx=0x7fca980d2f90 comp rx=0 tx=0).stop 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 -- 172.21.15.137:0/2273071064 shutdown_connections 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af367379e0 0x55af36739e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36732870 0x55af36737370 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/2273071064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af367322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 -- 172.21.15.137:0/2273071064 >> 172.21.15.137:0/2273071064 conn(0x55af366fa200 msgr2=0x55af3672d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 -- 172.21.15.137:0/2273071064 shutdown_connections 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.352+0000 7fcab1ffb740 1 -- 172.21.15.137:0/2273071064 wait complete. 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.353+0000 7fcab1ffb740 1 -- start start 2024-03-26T14:11:58.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.353+0000 7fcab1ffb740 1 Processor -- start 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 0x55af3673be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af3673c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af367379e0 0x55af36743400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3673a990 con 0x55af36731e80 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af36652e80 con 0x55af36732870 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3673ab10 con 0x55af367379e0 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcab1ffb740 10 client.0 _ll_register_callbacks cb 0x55af36665130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 0x55af3673be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 0x55af3673be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48490/0 (socket says 172.21.15.137:48490) 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 -- 172.21.15.137:0/3458561896 learned_addr learned my addr 172.21.15.137:0/3458561896 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af3673c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaad400640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af367379e0 0x55af36743400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 -- 172.21.15.137:0/3458561896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af367379e0 msgr2=0x55af36743400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af367379e0 0x55af36743400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 -- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 msgr2=0x55af3673c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af3673c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.354+0000 7fcaaca00640 1 -- 172.21.15.137:0/3458561896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af364757d0 con 0x55af36731e80 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fcaaca00640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 0x55af3673be80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fca980d7480 tx=0x7fca98009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bfb25b77b1bb41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fca92000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fca92000640 1 -- 172.21.15.137:0/3458561896 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9804c020 con 0x55af36731e80 2024-03-26T14:11:58.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af366a4790 con 0x55af36731e80 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fca92000640 1 -- 172.21.15.137:0/3458561896 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fca980cf080 con 0x55af36731e80 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fca92000640 1 -- 172.21.15.137:0/3458561896 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9805c750 con 0x55af36731e80 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fcab1ffb740 10 client.17634 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.355+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af363ab3f0 con 0x55af36731e80 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fca92000640 1 -- 172.21.15.137:0/3458561896 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55af366a4790 con 0x55af36731e80 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fca92000640 1 -- 172.21.15.137:0/3458561896 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fca9805c940 con 0x55af36731e80 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcab1ffb740 10 client.17634 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcab1ffb740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcab1ffb740 20 client.17634 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367478f0 0x55af36749ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcab1ffb740 4 client.17634 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af3669eae0 con 0x55af367478f0 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.356+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367478f0 0x55af36749ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.357+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367478f0 0x55af36749ce0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853b5bb7e319bdbf server_cookie=5a6bc24fc7967e79 in_seq=0 out_seq=0 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.357+0000 7fca92000640 10 client.17634 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fca92000640 1 -- 172.21.15.137:0/3458561896 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55af3669eae0 con 0x55af367478f0 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fca92000640 10 client.17634 handle_command_reply: tid=0 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 client.17634 shutdown 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 msgr2=0x55af3673be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 0x55af3673be80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fca980d7480 tx=0x7fca98009770 comp rx=0 tx=0).stop 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 shutdown_connections 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367478f0 0x55af36749ce0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af3673c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.364 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af367379e0 0x55af36743400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fca92000640 0 client.17634 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/3458561896 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af36731e80 0x55af3673be80 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.359+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 >> 172.21.15.137:0/3458561896 conn(0x55af366fa200 msgr2=0x55af3672ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.360+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 shutdown_connections 2024-03-26T14:11:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.360+0000 7fcab1ffb740 1 -- 172.21.15.137:0/3458561896 wait complete. 2024-03-26T14:11:58.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.360+0000 7fcab1ffb740 20 client.17634 trim_cache size 0 max 16384 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 Processor -- start 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 -- start start 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36731e80 0x55af36730a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36732870 0x55af36743390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36730f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81ae990 con 0x55af367379e0 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81ac490 con 0x55af36731e80 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.377+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81aa510 con 0x55af36732870 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36730f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36730f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48492/0 (socket says 172.21.15.137:48492) 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 -- 172.21.15.137:0/1471841576 learned_addr learned my addr 172.21.15.137:0/1471841576 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaaca00640 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36731e80 0x55af36730a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36732870 0x55af36743390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 -- 172.21.15.137:0/1471841576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36732870 msgr2=0x55af36743390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36732870 0x55af36743390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 -- 172.21.15.137:0/1471841576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36731e80 msgr2=0x55af36730a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36731e80 0x55af36730a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 -- 172.21.15.137:0/1471841576 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af3636b0f0 con 0x55af367379e0 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36732870 0x55af36743390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.378+0000 7fcaad400640 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36730f70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009ca0 tx=0x7fcaa003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfbbd7ebcbe8d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fca9d200640 1 -- 172.21.15.137:0/1471841576 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0046050 con 0x55af367379e0 2024-03-26T14:11:58.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fca9d200640 1 -- 172.21.15.137:0/1471841576 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcaa0050070 con 0x55af367379e0 2024-03-26T14:11:58.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fca9d200640 1 -- 172.21.15.137:0/1471841576 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0039040 con 0x55af367379e0 2024-03-26T14:11:58.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 -- 172.21.15.137:0/1471841576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 msgr2=0x55af36730f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36730f70 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009ca0 tx=0x7fcaa003d000 comp rx=0 tx=0).stop 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 -- 172.21.15.137:0/1471841576 shutdown_connections 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36730f70 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36732870 0x55af36743390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/1471841576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36731e80 0x55af36730a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 -- 172.21.15.137:0/1471841576 >> 172.21.15.137:0/1471841576 conn(0x55af366fa200 msgr2=0x55af3672ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 -- 172.21.15.137:0/1471841576 shutdown_connections 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.379+0000 7fcab1ffb740 1 -- 172.21.15.137:0/1471841576 wait complete. 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 -- start start 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 Processor -- start 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af36740c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af36741170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36742130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81aab10 con 0x55af367379e0 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81abe30 con 0x55af36732870 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa81ab170 con 0x55af36731e80 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.380+0000 7fcab1ffb740 10 client.0 _ll_register_callbacks cb 0x7fca980e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36742130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36742130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48502/0 (socket says 172.21.15.137:48502) 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 -- 172.21.15.137:0/4170127506 learned_addr learned my addr 172.21.15.137:0/4170127506 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaaca00640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af36740c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaa7e00640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af36741170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 -- 172.21.15.137:0/4170127506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 msgr2=0x55af36740c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af36740c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 -- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 msgr2=0x55af36741170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af36741170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 -- 172.21.15.137:0/4170127506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af36742720 con 0x55af367379e0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fcaad400640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36742130 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0033000 tx=0x7fcaa0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cda57fff26243e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.381+0000 7fca92000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fca92000640 1 -- 172.21.15.137:0/4170127506 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0058020 con 0x55af367379e0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fca92000640 1 -- 172.21.15.137:0/4170127506 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcaa003f880 con 0x55af367379e0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaa0050070 con 0x55af367379e0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fca92000640 1 -- 172.21.15.137:0/4170127506 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcaa0046020 con 0x55af367379e0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 10 client.17640 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af367a2f00 con 0x55af367379e0 2024-03-26T14:11:58.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fca92000640 1 -- 172.21.15.137:0/4170127506 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcaa0050070 con 0x55af367379e0 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fca92000640 1 -- 172.21.15.137:0/4170127506 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fcaa0034780 con 0x55af367379e0 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 10 client.17640 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 20 client.17640 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367a3810 0x55af367a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 4 client.17640 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.382+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af3661c3a0 con 0x55af367a3810 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.383+0000 7fcaaca00640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367a3810 0x55af367a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fca92000640 10 client.17640 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcaaca00640 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367a3810 0x55af367a5c30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99fca3514a9c538e server_cookie=b8d76164251b5f58 in_seq=0 out_seq=0 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fca92000640 1 -- 172.21.15.137:0/4170127506 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55af3661c3a0 con 0x55af367a3810 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fca92000640 10 client.17640 handle_command_reply: tid=0 2024-03-26T14:11:58.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 client.17640 shutdown 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 msgr2=0x55af36742130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36742130 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0033000 tx=0x7fcaa0003c20 comp rx=0 tx=0).stop 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 shutdown_connections 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55af367a3810 0x55af367a5c30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55af36732870 0x55af36741170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55af367379e0 0x55af36742130 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 --2- 172.21.15.137:0/4170127506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af36731e80 0x55af36740c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 >> 172.21.15.137:0/4170127506 conn(0x55af366fa200 msgr2=0x55af36733100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fca92000640 0 client.17640 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 shutdown_connections 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.384+0000 7fcab1ffb740 1 -- 172.21.15.137:0/4170127506 wait complete. 2024-03-26T14:11:58.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcab1ffb740 20 client.17640 trim_cache size 0 max 16384 2024-03-26T14:11:58.391 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:11:58.391 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:11:58.391 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:11:58.391 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:11:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca8c0a2800 msgr2=0x7fca8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca8c0a2800 0x7fca8c0a4cc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55af367330d0 tx=0x7fcaa0005c50 comp rx=0 tx=0).stop 2024-03-26T14:11:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 msgr2=0x7fcaa8199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 0x7fcaa8199b20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fca9800e9f0 tx=0x7fca9800eeb0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 shutdown_connections 2024-03-26T14:11:58.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fca8c0a2800 0x7fca8c0a4cc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcaa80756e0 0x7fcaa819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa8109570 0x7fcaa81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 --2- 172.21.15.137:0/2514623153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcaa8074100 0x7fcaa8199b20 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 >> 172.21.15.137:0/2514623153 conn(0x7fcaa80fcc20 msgr2=0x7fcaa8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.385+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 shutdown_connections 2024-03-26T14:11:58.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.386+0000 7fcaaf400640 1 -- 172.21.15.137:0/2514623153 wait complete. 2024-03-26T14:11:58.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:11:58.405 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:11:58.616 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 -- 172.21.15.137:0/4207799609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 msgr2=0x7efebc1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 --2- 172.21.15.137:0/4207799609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1014e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009ca0 tx=0x7efeb402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 -- 172.21.15.137:0/4207799609 shutdown_connections 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 --2- 172.21.15.137:0/4207799609 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1014e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 --2- 172.21.15.137:0/4207799609 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efebc1006c0 0x7efebc100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 --2- 172.21.15.137:0/4207799609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 0x7efebc100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 -- 172.21.15.137:0/4207799609 >> 172.21.15.137:0/4207799609 conn(0x7efebc0fb980 msgr2=0x7efebc0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 -- 172.21.15.137:0/4207799609 shutdown_connections 2024-03-26T14:11:58.617 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.616+0000 7efec1a00640 1 -- 172.21.15.137:0/4207799609 wait complete. 2024-03-26T14:11:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.617+0000 7efec1a00640 1 Processor -- start 2024-03-26T14:11:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.617+0000 7efec1a00640 1 -- start start 2024-03-26T14:11:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 0x7efebc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.620 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efebc1006c0 0x7efebc199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc10f1f0 con 0x7efebc101060 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc10f070 con 0x7efebc0ffd60 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efebc10f370 con 0x7efebc1006c0 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48514/0 (socket says 172.21.15.137:48514) 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efec1000640 1 -- 172.21.15.137:0/2723349742 learned_addr learned my addr 172.21.15.137:0/2723349742 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efebbe00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 0x7efebc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.618+0000 7efeb3e00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efebc1006c0 0x7efebc199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1000640 1 -- 172.21.15.137:0/2723349742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efebc1006c0 msgr2=0x7efebc199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.621 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1000640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efebc1006c0 0x7efebc199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1000640 1 -- 172.21.15.137:0/2723349742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 msgr2=0x7efebc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1000640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 0x7efebc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1000640 1 -- 172.21.15.137:0/2723349742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc1a1490 con 0x7efebc101060 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efebbe00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 0x7efebc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1000640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1a0e20 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7efeb4032380 tx=0x7efeb4032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4041020 con 0x7efebc101060 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efeb4003e20 con 0x7efebc101060 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc1a1720 con 0x7efebc101060 2024-03-26T14:11:58.622 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.619+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4039040 con 0x7efebc101060 2024-03-26T14:11:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.620+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc1a1c60 con 0x7efebc101060 2024-03-26T14:11:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.621+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efebc10aab0 con 0x7efebc101060 2024-03-26T14:11:58.623 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.623+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7efeb4044020 con 0x7efebc101060 2024-03-26T14:11:58.624 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.624+0000 7efeb9600640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efea00a2880 0x7efea00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.624+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7efeb40f0fe0 con 0x7efebc101060 2024-03-26T14:11:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.625+0000 7efebbe00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efea00a2880 0x7efea00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:58.625 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.625+0000 7efebbe00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efea00a2880 0x7efea00a4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efeac009c20 tx=0x7efeac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:58.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.629+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7efeb404b020 con 0x7efebc101060 2024-03-26T14:11:58.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.905+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efebc0622a0 con 0x7efebc101060 2024-03-26T14:11:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.907+0000 7efeb9600640 1 -- 172.21.15.137:0/2723349742 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7efeb40bdcb0 con 0x7efebc101060 2024-03-26T14:11:58.908 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:11:58.908 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:11:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:11:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.910+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efea00a2880 msgr2=0x7efea00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efea00a2880 0x7efea00a4d40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efeac009c20 tx=0x7efeac005f90 comp rx=0 tx=0).stop 2024-03-26T14:11:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 msgr2=0x7efebc1a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1a0e20 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7efeb4032380 tx=0x7efeb4032820 comp rx=0 tx=0).stop 2024-03-26T14:11:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 shutdown_connections 2024-03-26T14:11:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7efea00a2880 0x7efea00a4d40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efebc101060 0x7efebc1a0e20 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efebc1006c0 0x7efebc199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 --2- 172.21.15.137:0/2723349742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efebc0ffd60 0x7efebc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 >> 172.21.15.137:0/2723349742 conn(0x7efebc0fb980 msgr2=0x7efebc0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 shutdown_connections 2024-03-26T14:11:58.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:58.911+0000 7efec1a00640 1 -- 172.21.15.137:0/2723349742 wait complete. 2024-03-26T14:11:58.932 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:11:58.932 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:11:59.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 -- 172.21.15.137:0/3976447730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f141073e0 msgr2=0x7f4f141097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.165 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 --2- 172.21.15.137:0/3976447730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f141073e0 0x7f4f141097d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009900 tx=0x7f4f0c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 -- 172.21.15.137:0/3976447730 shutdown_connections 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 --2- 172.21.15.137:0/3976447730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f141073e0 0x7f4f141097d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 --2- 172.21.15.137:0/3976447730 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f14104ab0 0x7f4f14106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 --2- 172.21.15.137:0/3976447730 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f14102180 0x7f4f14104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 -- 172.21.15.137:0/3976447730 >> 172.21.15.137:0/3976447730 conn(0x7f4f140fbb30 msgr2=0x7f4f140fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 -- 172.21.15.137:0/3976447730 shutdown_connections 2024-03-26T14:11:59.166 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.165+0000 7f4f18c00640 1 -- 172.21.15.137:0/3976447730 wait complete. 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 Processor -- start 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 -- start start 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f14102180 0x7f4f14101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f14104ab0 0x7f4f14100130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 0x7f4f14100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1410d1d0 con 0x7f4f141073e0 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1410d050 con 0x7f4f14104ab0 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.166+0000 7f4f18c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1410d350 con 0x7f4f14102180 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 0x7f4f14100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 0x7f4f14100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48526/0 (socket says 172.21.15.137:48526) 2024-03-26T14:11:59.169 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 -- 172.21.15.137:0/1792038203 learned_addr learned my addr 172.21.15.137:0/1792038203 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f0be00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f14104ab0 0x7f4f14100130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f14102180 0x7f4f14101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 -- 172.21.15.137:0/1792038203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f14102180 msgr2=0x7f4f14101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f14102180 0x7f4f14101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 -- 172.21.15.137:0/1792038203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f14104ab0 msgr2=0x7f4f14100130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f14104ab0 0x7f4f14100130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.167+0000 7f4f12800640 1 -- 172.21.15.137:0/1792038203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f14100f00 con 0x7f4f141073e0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.168+0000 7f4f12800640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 0x7f4f14100670 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009230 tx=0x7f4f0c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.168+0000 7f4f0aa00640 1 -- 172.21.15.137:0/1792038203 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c043020 con 0x7f4f141073e0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.168+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f141a19f0 con 0x7f4f141073e0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.168+0000 7f4f0aa00640 1 -- 172.21.15.137:0/1792038203 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f0c0044c0 con 0x7f4f141073e0 2024-03-26T14:11:59.170 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.168+0000 7f4f0aa00640 1 -- 172.21.15.137:0/1792038203 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c003bc0 con 0x7f4f141073e0 2024-03-26T14:11:59.171 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.168+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f141a1f90 con 0x7f4f141073e0 2024-03-26T14:11:59.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.172+0000 7f4f1b9dd740 1 Processor -- start 2024-03-26T14:11:59.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.172+0000 7f4f0aa00640 1 -- 172.21.15.137:0/1792038203 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4f0c046020 con 0x7f4f141073e0 2024-03-26T14:11:59.172 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.172+0000 7f4f1b9dd740 1 -- start start 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f1232c0 con 0x559f8f1c03a0 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f0cbc90 con 0x559f8f1be810 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f11f360 con 0x559f8f1bf9e0 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48528/0 (socket says 172.21.15.137:48528) 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f0be00640 1 -- 172.21.15.137:0/2045454767 learned_addr learned my addr 172.21.15.137:0/2045454767 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f0be00640 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.173+0000 7f4f11e00640 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0be00640 1 -- 172.21.15.137:0/2045454767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 msgr2=0x559f8f1bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0be00640 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0be00640 1 -- 172.21.15.137:0/2045454767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 msgr2=0x559f8f1bec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0be00640 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1bec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0be00640 1 -- 172.21.15.137:0/2045454767 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f8f0f0080 con 0x559f8f1c03a0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0aa00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ef80a2880 0x7f4ef80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0be00640 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1c6c30 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00cfa0 tx=0x7f4efc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455bd66d0b471040 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f0aa00640 1 -- 172.21.15.137:0/1792038203 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4f0c0f11f0 con 0x7f4f141073e0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.174+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ef80a2880 0x7f4ef80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.175+0000 7f4f03000640 1 -- 172.21.15.137:0/2045454767 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc011500 con 0x559f8f1c03a0 2024-03-26T14:11:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.175+0000 7f4f03000640 1 -- 172.21.15.137:0/2045454767 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4efc011b50 con 0x559f8f1c03a0 2024-03-26T14:11:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.175+0000 7f4f03000640 1 -- 172.21.15.137:0/2045454767 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0109e0 con 0x559f8f1c03a0 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2045454767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 msgr2=0x559f8f1c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1c6c30 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00cfa0 tx=0x7f4efc00c490 comp rx=0 tx=0).stop 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ef80a2880 0x7f4ef80a4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f04005fd0 tx=0x7f4f04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2045454767 shutdown_connections 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1c6c30 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2045454767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1bec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2045454767 >> 172.21.15.137:0/2045454767 conn(0x559f8f1b9c00 msgr2=0x559f8f1ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2045454767 shutdown_connections 2024-03-26T14:11:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2045454767 wait complete. 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.176+0000 7f4f1b9dd740 1 -- start start 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 Processor -- start 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f1c73b0 con 0x559f8f1c03a0 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f0df0d0 con 0x559f8f1be810 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f1c7530 con 0x559f8f1bf9e0 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f1b9dd740 10 client.0 _ll_register_callbacks cb 0x559f8f0e3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f12800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f12800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48540/0 (socket says 172.21.15.137:48540) 2024-03-26T14:11:59.182 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.177+0000 7f4f12800640 1 -- 172.21.15.137:0/2072674700 learned_addr learned my addr 172.21.15.137:0/2072674700 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f11e00640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f0be00640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f12800640 1 -- 172.21.15.137:0/2072674700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 msgr2=0x559f8f1c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f12800640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f12800640 1 -- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 msgr2=0x559f8f1c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f12800640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f12800640 1 -- 172.21.15.137:0/2072674700 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f8ee373f0 con 0x559f8f1c03a0 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f11e00640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f0be00640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4f12800640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1d0000 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0bd480 tx=0x7f4f0c0bd380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3fc7b10ba2b13d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4ef7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:59.183 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.178+0000 7f4ef7400640 1 -- 172.21.15.137:0/2072674700 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0bd780 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4ef7400640 1 -- 172.21.15.137:0/2072674700 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4f0c003d60 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4efc011b50 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4ef7400640 1 -- 172.21.15.137:0/2072674700 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f0c0c18d0 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4f1b9dd740 10 client.17658 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559f8ef017d0 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4ef7400640 1 -- 172.21.15.137:0/2072674700 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4efc011b50 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.179+0000 7f4ef7400640 1 -- 172.21.15.137:0/2072674700 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4f0c0b9040 con 0x559f8f1c03a0 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f1b9dd740 10 client.17658 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f1b9dd740 10 client.17658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f1b9dd740 20 client.17658 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f1d44a0 0x559f8f1d6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f1b9dd740 4 client.17658 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f8f130800 con 0x559f8f1d44a0 2024-03-26T14:11:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.180+0000 7f4f11e00640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f1d44a0 0x559f8f1d6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.181+0000 7f4ef7400640 10 client.17658 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.181+0000 7f4f11e00640 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f1d44a0 0x559f8f1d6900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c246154e05458be server_cookie=47f805bcb8a26e68 in_seq=0 out_seq=0 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4ef7400640 1 -- 172.21.15.137:0/2072674700 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559f8f130800 con 0x559f8f1d44a0 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4ef7400640 10 client.17658 handle_command_reply: tid=0 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 client.17658 shutdown 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 msgr2=0x559f8f1d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1d0000 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0bd480 tx=0x7f4f0c0bd380 comp rx=0 tx=0).stop 2024-03-26T14:11:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 shutdown_connections 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f1d44a0 0x559f8f1d6900 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1d0000 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1bf9e0 0x559f8f1c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/2072674700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4ef7400640 0 client.17658 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.183+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 >> 172.21.15.137:0/2072674700 conn(0x559f8f1b9c00 msgr2=0x559f8f1b8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.184+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 shutdown_connections 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.184+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/2072674700 wait complete. 2024-03-26T14:11:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.184+0000 7f4f1b9dd740 20 client.17658 trim_cache size 0 max 16384 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 Processor -- start 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 -- start start 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1be810 0x559f8f1b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1db200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141ae870 con 0x559f8f1c03a0 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141ac370 con 0x559f8f1bf9e0 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141aa3f0 con 0x559f8f1be810 2024-03-26T14:11:59.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f12800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1db200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f12800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1db200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48548/0 (socket says 172.21.15.137:48548) 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f12800640 1 -- 172.21.15.137:0/3252053240 learned_addr learned my addr 172.21.15.137:0/3252053240 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f0be00640 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.201+0000 7f4f11e00640 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1be810 0x559f8f1b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f12800640 1 -- 172.21.15.137:0/3252053240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1be810 msgr2=0x559f8f1b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f12800640 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1be810 0x559f8f1b8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f12800640 1 -- 172.21.15.137:0/3252053240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1bf9e0 msgr2=0x559f8f1cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f12800640 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f12800640 1 -- 172.21.15.137:0/3252053240 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f8f1c7ea0 con 0x559f8f1c03a0 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f12800640 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1db200 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0b9be0 tx=0x7f4f0c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cd3f05149794b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f02600640 1 -- 172.21.15.137:0/3252053240 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0bd780 con 0x559f8f1c03a0 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.202+0000 7f4f02600640 1 -- 172.21.15.137:0/3252053240 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f0c0c5f20 con 0x559f8f1c03a0 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f02600640 1 -- 172.21.15.137:0/3252053240 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0b9730 con 0x559f8f1c03a0 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/3252053240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 msgr2=0x559f8f1db200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1db200 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c0b9be0 tx=0x7f4f0c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/3252053240 shutdown_connections 2024-03-26T14:11:59.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1c03a0 0x559f8f1db200 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/3252053240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1be810 0x559f8f1b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/3252053240 >> 172.21.15.137:0/3252053240 conn(0x559f8f1b9c00 msgr2=0x559f8f1b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/3252053240 shutdown_connections 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.203+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/3252053240 wait complete. 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.204+0000 7f4f1b9dd740 1 -- start start 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.204+0000 7f4f1b9dd740 1 Processor -- start 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1c03a0 0x559f8f1cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141aa9f0 con 0x559f8f1bf9e0 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141abd10 con 0x559f8f1be810 2024-03-26T14:11:59.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141ab050 con 0x559f8f1c03a0 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f1b9dd740 10 client.0 _ll_register_callbacks cb 0x7f4f0c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48550/0 (socket says 172.21.15.137:48550) 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 -- 172.21.15.137:0/1871579007 learned_addr learned my addr 172.21.15.137:0/1871579007 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f12800640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1c03a0 0x559f8f1cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 -- 172.21.15.137:0/1871579007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1c03a0 msgr2=0x559f8f1cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1c03a0 0x559f8f1cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 -- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 msgr2=0x559f8f1cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f0be00640 1 -- 172.21.15.137:0/1871579007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f8f1cc890 con 0x559f8f1bf9e0 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.205+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4f0be00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cfcf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x559f8f1c9f60 tx=0x7f4efc011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e1fa126a4b112e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4ef7400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4ef7400640 1 -- 172.21.15.137:0/1871579007 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc00cdf0 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4ef7400640 1 -- 172.21.15.137:0/1871579007 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4efc005030 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f0c0c5f20 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4ef7400640 1 -- 172.21.15.137:0/1871579007 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4efc032050 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4f1b9dd740 10 client.17664 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.206+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559f8f230940 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4ef7400640 1 -- 172.21.15.137:0/1871579007 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4f0c0c5f20 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4ef7400640 1 -- 172.21.15.137:0/1871579007 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4efc004c20 con 0x559f8f1bf9e0 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4f1b9dd740 10 client.17664 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4f1b9dd740 10 client.17664 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:11:59.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4f1b9dd740 20 client.17664 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:59.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f230fb0 0x559f8f2333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4f1b9dd740 4 client.17664 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:11:59.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.207+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f8f1c8870 con 0x559f8f230fb0 2024-03-26T14:11:59.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.208+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f230fb0 0x559f8f2333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.208+0000 7f4ef7400640 10 client.17664 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:59.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.208+0000 7f4f11e00640 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f230fb0 0x559f8f2333a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=290b4f31a6cbe635 server_cookie=e6a7a689f6b2809f in_seq=0 out_seq=0 2024-03-26T14:11:59.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4ef7400640 1 -- 172.21.15.137:0/1871579007 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f8f1c8870 con 0x559f8f230fb0 2024-03-26T14:11:59.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4ef7400640 10 client.17664 handle_command_reply: tid=0 2024-03-26T14:11:59.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 client.17664 shutdown 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 msgr2=0x559f8f1cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cfcf0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x559f8f1c9f60 tx=0x7f4efc011be0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 shutdown_connections 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x559f8f230fb0 0x559f8f2333a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f8f1c03a0 0x559f8f1cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559f8f1bf9e0 0x559f8f1cfcf0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 --2- 172.21.15.137:0/1871579007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559f8f1be810 0x559f8f1cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4ef7400640 0 client.17664 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 >> 172.21.15.137:0/1871579007 conn(0x559f8f1b9c00 msgr2=0x559f8f1b8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 shutdown_connections 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 1 -- 172.21.15.137:0/1871579007 wait complete. 2024-03-26T14:11:59.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.209+0000 7f4f1b9dd740 20 client.17664 trim_cache size 0 max 16384 2024-03-26T14:11:59.214 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ef80a2880 msgr2=0x7f4ef80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ef80a2880 0x7f4ef80a4d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f04005fd0 tx=0x7f4f04009500 comp rx=0 tx=0).stop 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 msgr2=0x7f4f14100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 0x7f4f14100670 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009230 tx=0x7f4f0c0040c0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 shutdown_connections 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ef80a2880 0x7f4ef80a4d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f14104ab0 0x7f4f14100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f141073e0 0x7f4f14100670 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 --2- 172.21.15.137:0/1792038203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f14102180 0x7f4f14101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 >> 172.21.15.137:0/1792038203 conn(0x7f4f140fbb30 msgr2=0x7f4f14102ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 shutdown_connections 2024-03-26T14:11:59.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.210+0000 7f4f18c00640 1 -- 172.21.15.137:0/1792038203 wait complete. 2024-03-26T14:11:59.230 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:11:59.231 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:11:59.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.439+0000 7fa472000640 1 -- 172.21.15.137:0/2374655944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c107390 msgr2=0x7fa46c1097e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.439+0000 7fa472000640 1 --2- 172.21.15.137:0/2374655944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c107390 0x7fa46c1097e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fa464009ca0 tx=0x7fa46402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 -- 172.21.15.137:0/2374655944 shutdown_connections 2024-03-26T14:11:59.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 --2- 172.21.15.137:0/2374655944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa46c104a60 0x7fa46c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 --2- 172.21.15.137:0/2374655944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c107390 0x7fa46c1097e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 --2- 172.21.15.137:0/2374655944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c102130 0x7fa46c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 -- 172.21.15.137:0/2374655944 >> 172.21.15.137:0/2374655944 conn(0x7fa46c0fbb40 msgr2=0x7fa46c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 -- 172.21.15.137:0/2374655944 shutdown_connections 2024-03-26T14:11:59.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.440+0000 7fa472000640 1 -- 172.21.15.137:0/2374655944 wait complete. 2024-03-26T14:11:59.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 Processor -- start 2024-03-26T14:11:59.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 -- start start 2024-03-26T14:11:59.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 0x7fa46c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c104a60 0x7fa46c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa46c107390 0x7fa46c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c10d200 con 0x7fa46c102130 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c10d080 con 0x7fa46c107390 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa472000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c10d380 con 0x7fa46c104a60 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.441+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 0x7fa46c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 0x7fa46c1978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48566/0 (socket says 172.21.15.137:48566) 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 -- 172.21.15.137:0/2865582741 learned_addr learned my addr 172.21.15.137:0/2865582741 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46be00640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa46c107390 0x7fa46c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa463e00640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c104a60 0x7fa46c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 -- 172.21.15.137:0/2865582741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c104a60 msgr2=0x7fa46c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c104a60 0x7fa46c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 -- 172.21.15.137:0/2865582741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa46c107390 msgr2=0x7fa46c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa46c107390 0x7fa46c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 -- 172.21.15.137:0/2865582741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c19f4f0 con 0x7fa46c102130 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa463e00640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c104a60 0x7fa46c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.442+0000 7fa46b400640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 0x7fa46c1978d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00d770 tx=0x7fa45c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.443+0000 7fa468c00640 1 -- 172.21.15.137:0/2865582741 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c017070 con 0x7fa46c102130 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.443+0000 7fa468c00640 1 -- 172.21.15.137:0/2865582741 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa45c0054d0 con 0x7fa46c102130 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.443+0000 7fa468c00640 1 -- 172.21.15.137:0/2865582741 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c004cf0 con 0x7fa46c102130 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.443+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c19f7e0 con 0x7fa46c102130 2024-03-26T14:11:59.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.443+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c19fce0 con 0x7fa46c102130 2024-03-26T14:11:59.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.446+0000 7fa474d78740 1 Processor -- start 2024-03-26T14:11:59.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.446+0000 7fa468c00640 1 -- 172.21.15.137:0/2865582741 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa45c004e90 con 0x7fa46c102130 2024-03-26T14:11:59.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 -- start start 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4be9810 0x55ccc4be9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4beae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc4b4e2c0 con 0x55ccc4beb3a0 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc4af6c90 con 0x55ccc4bea9e0 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.447+0000 7fa474d78740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc4b4a360 con 0x55ccc4be9810 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa46be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4be9810 0x55ccc4be9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa46b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4beae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa46be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4be9810 0x55ccc4be9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:36102/0 (socket says 172.21.15.137:36102) 2024-03-26T14:11:59.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa46be00640 1 -- 172.21.15.137:0/2003084887 learned_addr learned my addr 172.21.15.137:0/2003084887 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48578/0 (socket says 172.21.15.137:48578) 2024-03-26T14:11:59.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 -- 172.21.15.137:0/2003084887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4be9810 msgr2=0x55ccc4be9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4be9810 0x55ccc4be9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 -- 172.21.15.137:0/2003084887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 msgr2=0x55ccc4beae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4beae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa463e00640 1 -- 172.21.15.137:0/2003084887 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccc4b1b080 con 0x55ccc4beb3a0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.448+0000 7fa468c00640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4500a2800 0x7fa4500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.449+0000 7fa468c00640 1 -- 172.21.15.137:0/2865582741 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa45c0ce980 con 0x7fa46c102130 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.449+0000 7fa463e00640 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf1c30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa454005fd0 tx=0x7fa45400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59be64a2de89e2c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.449+0000 7fa46be00640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4500a2800 0x7fa4500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.449+0000 7fa45be00640 1 -- 172.21.15.137:0/2003084887 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454016070 con 0x55ccc4beb3a0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.449+0000 7fa45be00640 1 -- 172.21.15.137:0/2003084887 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4540054d0 con 0x55ccc4beb3a0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.449+0000 7fa45be00640 1 -- 172.21.15.137:0/2003084887 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454011470 con 0x55ccc4beb3a0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 -- 172.21.15.137:0/2003084887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 msgr2=0x55ccc4bf1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa46be00640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4500a2800 0x7fa4500a4cc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55ccc4bea8e0 tx=0x7fa464005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf1c30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa454005fd0 tx=0x7fa45400e6c0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 -- 172.21.15.137:0/2003084887 shutdown_connections 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4beae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf1c30 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 --2- 172.21.15.137:0/2003084887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4be9810 0x55ccc4be9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 -- 172.21.15.137:0/2003084887 >> 172.21.15.137:0/2003084887 conn(0x55ccc4be4c00 msgr2=0x55ccc4be5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 -- 172.21.15.137:0/2003084887 shutdown_connections 2024-03-26T14:11:59.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.450+0000 7fa474d78740 1 -- 172.21.15.137:0/2003084887 wait complete. 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.451+0000 7fa474d78740 1 -- start start 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.451+0000 7fa474d78740 1 Processor -- start 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.451+0000 7fa474d78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bf3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa474d78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bf4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa474d78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bfaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa474d78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc4bf23b0 con 0x55ccc4be9810 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa474d78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc4b0a0d0 con 0x55ccc4bea9e0 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa474d78740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc4bf2530 con 0x55ccc4beb3a0 2024-03-26T14:11:59.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa474d78740 10 client.0 _ll_register_callbacks cb 0x55ccc4b0ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bf3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bf3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48588/0 (socket says 172.21.15.137:48588) 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 -- 172.21.15.137:0/3966750650 learned_addr learned my addr 172.21.15.137:0/3966750650 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa463e00640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bf4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46be00640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bfaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 -- 172.21.15.137:0/3966750650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4beb3a0 msgr2=0x55ccc4bfaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bfaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 -- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 msgr2=0x55ccc4bf4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bf4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa46b400640 1 -- 172.21.15.137:0/3966750650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccc48623f0 con 0x55ccc4be9810 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.452+0000 7fa463e00640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bf4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa46b400640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bf3ae0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0978e0 tx=0x7fa45c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734e82a1018be40f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa458c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:59.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa458c00640 1 -- 172.21.15.137:0/3966750650 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c020070 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa458c00640 1 -- 172.21.15.137:0/3966750650 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa45c09a360 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4540054d0 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa458c00640 1 -- 172.21.15.137:0/3966750650 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45c09a870 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa474d78740 10 client.17676 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.453+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ccc492c7d0 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa458c00640 1 -- 172.21.15.137:0/3966750650 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa4540054d0 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa458c00640 1 -- 172.21.15.137:0/3966750650 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa45c0b0020 con 0x55ccc4be9810 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa474d78740 10 client.17676 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa474d78740 10 client.17676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa474d78740 20 client.17676 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa474d78740 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4bff570 0x55ccc4c019f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa474d78740 4 client.17676 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccc4b5b800 con 0x55ccc4bff570 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.454+0000 7fa463e00640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4bff570 0x55ccc4c019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.455+0000 7fa458c00640 10 client.17676 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.455+0000 7fa463e00640 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4bff570 0x55ccc4c019f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93c4beb86479edf3 server_cookie=6e441223b4f7221c in_seq=0 out_seq=0 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa458c00640 1 -- 172.21.15.137:0/3966750650 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ccc4b5b800 con 0x55ccc4bff570 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa458c00640 10 client.17676 handle_command_reply: tid=0 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 client.17676 shutdown 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 msgr2=0x55ccc4bf3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bf3ae0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0978e0 tx=0x7fa45c00df70 comp rx=0 tx=0).stop 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 shutdown_connections 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4bff570 0x55ccc4c019f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bf4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bfaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 --2- 172.21.15.137:0/3966750650 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bf3ae0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa458c00640 0 client.17676 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.457+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 >> 172.21.15.137:0/3966750650 conn(0x55ccc4be4c00 msgr2=0x55ccc4be3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.458+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 shutdown_connections 2024-03-26T14:11:59.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.458+0000 7fa474d78740 1 -- 172.21.15.137:0/3966750650 wait complete. 2024-03-26T14:11:59.463 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.458+0000 7fa474d78740 20 client.17676 trim_cache size 0 max 16384 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.474+0000 7fa474d78740 1 Processor -- start 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.474+0000 7fa474d78740 1 -- start start 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa474d78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4be3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa474d78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa474d78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4c062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa474d78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1ac790 con 0x55ccc4be9810 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa474d78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1aa290 con 0x55ccc4beb3a0 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa474d78740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1a8310 con 0x55ccc4bea9e0 2024-03-26T14:11:59.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4be3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4be3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48602/0 (socket says 172.21.15.137:48602) 2024-03-26T14:11:59.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46b400640 1 -- 172.21.15.137:0/4162223163 learned_addr learned my addr 172.21.15.137:0/4162223163 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46be00640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4c062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46b400640 1 -- 172.21.15.137:0/4162223163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 msgr2=0x55ccc4bfaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa463e00640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46b400640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.475+0000 7fa46b400640 1 -- 172.21.15.137:0/4162223163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 msgr2=0x55ccc4c062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa46b400640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4c062b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa46b400640 1 -- 172.21.15.137:0/4162223163 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccc4bea460 con 0x55ccc4be9810 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa463e00640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa46be00640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4c062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa46b400640 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4be3d30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0057a0 tx=0x7fa45c009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba9a40e8dffa560 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa45b400640 1 -- 172.21.15.137:0/4162223163 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c020070 con 0x55ccc4be9810 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa45b400640 1 -- 172.21.15.137:0/4162223163 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa45c0d4220 con 0x55ccc4be9810 2024-03-26T14:11:59.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.476+0000 7fa45b400640 1 -- 172.21.15.137:0/4162223163 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c005180 con 0x55ccc4be9810 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 -- 172.21.15.137:0/4162223163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 msgr2=0x55ccc4be3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4be3d30 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0057a0 tx=0x7fa45c009740 comp rx=0 tx=0).stop 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 -- 172.21.15.137:0/4162223163 shutdown_connections 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4c062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 --2- 172.21.15.137:0/4162223163 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4be3d30 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 -- 172.21.15.137:0/4162223163 >> 172.21.15.137:0/4162223163 conn(0x55ccc4be4c00 msgr2=0x55ccc4be3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 -- 172.21.15.137:0/4162223163 shutdown_connections 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 -- 172.21.15.137:0/4162223163 wait complete. 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.477+0000 7fa474d78740 1 -- start start 2024-03-26T14:11:59.479 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 Processor -- start 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bfa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1a8910 con 0x55ccc4be9810 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1a9c30 con 0x55ccc4beb3a0 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c1a8f70 con 0x55ccc4bea9e0 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa474d78740 10 client.0 _ll_register_callbacks cb 0x7fa45c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bfa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa46b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bfa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48608/0 (socket says 172.21.15.137:48608) 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.478+0000 7fa46b400640 1 -- 172.21.15.137:0/3896795391 learned_addr learned my addr 172.21.15.137:0/3896795391 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46be00640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa463e00640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46b400640 1 -- 172.21.15.137:0/3896795391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 msgr2=0x55ccc4bfaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46b400640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46b400640 1 -- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 msgr2=0x55ccc4bf7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46b400640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46b400640 1 -- 172.21.15.137:0/3896795391 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ccc4bf7cc0 con 0x55ccc4be9810 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa463e00640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa46b400640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bfa9f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0978e0 tx=0x7fa45c096e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c6180d5192387e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa458c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.479+0000 7fa458c00640 1 -- 172.21.15.137:0/3896795391 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45c020050 con 0x55ccc4be9810 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa458c00640 1 -- 172.21.15.137:0/3896795391 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa45c09ff80 con 0x55ccc4be9810 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa45c0d4220 con 0x55ccc4be9810 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa458c00640 1 -- 172.21.15.137:0/3896795391 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa45c083050 con 0x55ccc4be9810 2024-03-26T14:11:59.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 10 client.17682 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ccc4c5bb60 con 0x55ccc4be9810 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa458c00640 1 -- 172.21.15.137:0/3896795391 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa45c0d4220 con 0x55ccc4be9810 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa458c00640 1 -- 172.21.15.137:0/3896795391 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa45c0b0050 con 0x55ccc4be9810 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 10 client.17682 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 10 client.17682 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 20 client.17682 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4c5c410 0x55ccc4c5e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 4 client.17682 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.480+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ccc4b55b50 con 0x55ccc4c5c410 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.481+0000 7fa463e00640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4c5c410 0x55ccc4c5e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa463e00640 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4c5c410 0x55ccc4c5e830 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eef729ef8deb74fc server_cookie=40603209556a5a3b in_seq=0 out_seq=0 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa458c00640 10 client.17682 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa458c00640 1 -- 172.21.15.137:0/3896795391 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ccc4b55b50 con 0x55ccc4c5c410 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa458c00640 10 client.17682 handle_command_reply: tid=0 2024-03-26T14:11:59.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 client.17682 shutdown 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 msgr2=0x55ccc4bfa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bfa9f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0978e0 tx=0x7fa45c096e90 comp rx=0 tx=0).stop 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 shutdown_connections 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55ccc4c5c410 0x55ccc4c5e830 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccc4beb3a0 0x55ccc4bf7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ccc4bea9e0 0x55ccc4bfaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 --2- 172.21.15.137:0/3896795391 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ccc4be9810 0x55ccc4bfa9f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa458c00640 0 client.17682 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.482+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 >> 172.21.15.137:0/3896795391 conn(0x55ccc4be4c00 msgr2=0x55ccc4be3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.483+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 shutdown_connections 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.483+0000 7fa474d78740 1 -- 172.21.15.137:0/3896795391 wait complete. 2024-03-26T14:11:59.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.483+0000 7fa474d78740 20 client.17682 trim_cache size 0 max 16384 2024-03-26T14:11:59.487 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:11:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4500a2800 msgr2=0x7fa4500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4500a2800 0x7fa4500a4cc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55ccc4bea8e0 tx=0x7fa464005d80 comp rx=0 tx=0).stop 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 msgr2=0x7fa46c1978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 0x7fa46c1978d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00d770 tx=0x7fa45c00dc30 comp rx=0 tx=0).stop 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 shutdown_connections 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa4500a2800 0x7fa4500a4cc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa46c107390 0x7fa46c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa46c104a60 0x7fa46c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 --2- 172.21.15.137:0/2865582741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa46c102130 0x7fa46c1978d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 >> 172.21.15.137:0/2865582741 conn(0x7fa46c0fbb40 msgr2=0x7fa46c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 shutdown_connections 2024-03-26T14:11:59.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.484+0000 7fa472000640 1 -- 172.21.15.137:0/2865582741 wait complete. 2024-03-26T14:11:59.500 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:59 smithi137 ceph-mon[31539]: pgmap v746: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:59.500 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:11:59 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2723349742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:11:59.504 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:11:59.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:11:59.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:59 smithi179 ceph-mon[35472]: pgmap v746: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:59.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:11:59 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2723349742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:11:59.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:59 smithi159 ceph-mon[35501]: pgmap v746: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:11:59.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:11:59 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2723349742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.707+0000 7f1ffda00640 1 -- 172.21.15.137:0/3511381414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff81037a0 msgr2=0x7f1ff8103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.707+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3511381414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff81037a0 0x7f1ff8103bc0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009900 tx=0x7f1fe802bc30 comp rx=0 tx=0).stop 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 -- 172.21.15.137:0/3511381414 shutdown_connections 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3511381414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff8105350 0x7f1ff810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3511381414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ff8104990 0x7f1ff8104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3511381414 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff81037a0 0x7f1ff8103bc0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 -- 172.21.15.137:0/3511381414 >> 172.21.15.137:0/3511381414 conn(0x7f1ff80fed10 msgr2=0x7f1ff8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 -- 172.21.15.137:0/3511381414 shutdown_connections 2024-03-26T14:11:59.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.708+0000 7f1ffda00640 1 -- 172.21.15.137:0/3511381414 wait complete. 2024-03-26T14:11:59.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 Processor -- start 2024-03-26T14:11:59.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 -- start start 2024-03-26T14:11:59.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff81037a0 0x7f1ff8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 0x7f1ff8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ff8105350 0x7f1ff81a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff810f2c0 con 0x7f1ff8104990 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff810f140 con 0x7f1ff81037a0 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ffda00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff810f440 con 0x7f1ff8105350 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ff7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 0x7f1ff8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ff7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 0x7f1ff8199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48616/0 (socket says 172.21.15.137:48616) 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.709+0000 7f1ff7400640 1 -- 172.21.15.137:0/3500997022 learned_addr learned my addr 172.21.15.137:0/3500997022 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff81037a0 0x7f1ff8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ffd000640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ff8105350 0x7f1ff81a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7400640 1 -- 172.21.15.137:0/3500997022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ff8105350 msgr2=0x7f1ff81a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7400640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ff8105350 0x7f1ff81a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7400640 1 -- 172.21.15.137:0/3500997022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff81037a0 msgr2=0x7f1ff8199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7400640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff81037a0 0x7f1ff8199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7400640 1 -- 172.21.15.137:0/3500997022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff81a1620 con 0x7f1ff8104990 2024-03-26T14:11:59.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff7400640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 0x7f1ff8199f40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400d770 tx=0x7f1fe400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.710+0000 7f1ff4c00640 1 -- 172.21.15.137:0/3500997022 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4017070 con 0x7f1ff8104990 2024-03-26T14:11:59.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.711+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff81a1910 con 0x7f1ff8104990 2024-03-26T14:11:59.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.711+0000 7f1ff4c00640 1 -- 172.21.15.137:0/3500997022 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1fe40054d0 con 0x7f1ff8104990 2024-03-26T14:11:59.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.711+0000 7f1ff4c00640 1 -- 172.21.15.137:0/3500997022 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4004cf0 con 0x7f1ff8104990 2024-03-26T14:11:59.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.711+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff81a1e10 con 0x7f1ff8104990 2024-03-26T14:11:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 Processor -- start 2024-03-26T14:11:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f1ff4c00640 1 -- 172.21.15.137:0/3500997022 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1fe4004e90 con 0x7f1ff8104990 2024-03-26T14:11:59.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 -- start start 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8696112c0 con 0x55a8696adbc0 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8695b9c90 con 0x55a8696ac9f0 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.715+0000 7f2001c6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a86960d360 con 0x55a8696ae580 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ffd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ffd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:57650/0 (socket says 172.21.15.137:57650) 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48618/0 (socket says 172.21.15.137:48618) 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ffd000640 1 -- 172.21.15.137:0/2042420137 learned_addr learned my addr 172.21.15.137:0/2042420137 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7400640 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 -- 172.21.15.137:0/2042420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 msgr2=0x55a8696b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 -- 172.21.15.137:0/2042420137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 msgr2=0x55a8696ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.716+0000 7f1ff7e00640 1 -- 172.21.15.137:0/2042420137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8695de080 con 0x55a8696adbc0 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1ff4c00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1fdc0a2800 0x7f1fdc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696ae040 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8036040 tx=0x7f1fe80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b68675bb9140d7d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1ff4c00640 1 -- 172.21.15.137:0/3500997022 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1fe40ce980 con 0x7f1ff8104990 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1ffd000640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1fdc0a2800 0x7f1fdc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1fed600640 1 -- 172.21.15.137:0/2042420137 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8044020 con 0x55a8696adbc0 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1fed600640 1 -- 172.21.15.137:0/2042420137 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1fe802cba0 con 0x55a8696adbc0 2024-03-26T14:11:59.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.717+0000 7f1fed600640 1 -- 172.21.15.137:0/2042420137 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80046c0 con 0x55a8696adbc0 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 -- 172.21.15.137:0/2042420137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 msgr2=0x55a8696ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f1ffd000640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1fdc0a2800 0x7f1fdc0a4cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55a8696adac0 tx=0x7f1ff0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696ae040 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8036040 tx=0x7f1fe80095e0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 -- 172.21.15.137:0/2042420137 shutdown_connections 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696ae040 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 --2- 172.21.15.137:0/2042420137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 -- 172.21.15.137:0/2042420137 >> 172.21.15.137:0/2042420137 conn(0x55a8696a7de0 msgr2=0x55a8696a81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.718+0000 7f2001c6f740 1 -- 172.21.15.137:0/2042420137 shutdown_connections 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.719+0000 7f2001c6f740 1 -- 172.21.15.137:0/2042420137 wait complete. 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.719+0000 7f2001c6f740 1 -- start start 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.719+0000 7f2001c6f740 1 Processor -- start 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8696b5530 con 0x55a8696adbc0 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8695cd0d0 con 0x55a8696ac9f0 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8696b56b0 con 0x55a8696ae580 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f2001c6f740 10 client.0 _ll_register_callbacks cb 0x55a8695d1e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ff7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ff7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48630/0 (socket says 172.21.15.137:48630) 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ff7400640 1 -- 172.21.15.137:0/352827833 learned_addr learned my addr 172.21.15.137:0/352827833 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ffd000640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ff7400640 1 -- 172.21.15.137:0/352827833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 msgr2=0x55a8696be1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.720+0000 7f1ff7400640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696be1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1ff7400640 1 -- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 msgr2=0x55a8696b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1ff7400640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1ff7400640 1 -- 172.21.15.137:0/352827833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8693253f0 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1ffd000640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696be1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1ff7400640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696b71b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4097370 tx=0x7f1fe4097270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a82a2ae85ab46 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1fe2000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1fe2000640 1 -- 172.21.15.137:0/352827833 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4020070 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1fe2000640 1 -- 172.21.15.137:0/352827833 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1fe409a360 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fe802cba0 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f1fe2000640 1 -- 172.21.15.137:0/352827833 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe4005180 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f2001c6f740 10 client.17694 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.721+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a8693ef7d0 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f1fe2000640 1 -- 172.21.15.137:0/352827833 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1fe802cba0 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f1fe2000640 1 -- 172.21.15.137:0/352827833 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1fe40b0030 con 0x55a8696adbc0 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f2001c6f740 10 client.17694 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f2001c6f740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f2001c6f740 20 client.17694 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f2001c6f740 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a8696caea0 0x55a8696cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f2001c6f740 4 client.17694 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:11:59.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.722+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a86961e800 con 0x55a8696caea0 2024-03-26T14:11:59.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.723+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a8696caea0 0x55a8696cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.723+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a8696caea0 0x55a8696cd2b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd7805da6fcfe80 server_cookie=81620e2793989e2d in_seq=0 out_seq=0 2024-03-26T14:11:59.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.723+0000 7f1fe2000640 10 client.17694 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f1fe2000640 1 -- 172.21.15.137:0/352827833 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a86961e800 con 0x55a8696caea0 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f1fe2000640 10 client.17694 handle_command_reply: tid=0 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 client.17694 shutdown 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 msgr2=0x55a8696b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696b71b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4097370 tx=0x7f1fe4097270 comp rx=0 tx=0).stop 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 shutdown_connections 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a8696caea0 0x55a8696cd2b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696be1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696adbc0 0x55a8696b71b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 --2- 172.21.15.137:0/352827833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696ac9f0 0x55a8696b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 >> 172.21.15.137:0/352827833 conn(0x55a8696a7de0 msgr2=0x55a8696a7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f1fe2000640 0 client.17694 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:59.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 shutdown_connections 2024-03-26T14:11:59.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.725+0000 7f2001c6f740 1 -- 172.21.15.137:0/352827833 wait complete. 2024-03-26T14:11:59.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.726+0000 7f2001c6f740 20 client.17694 trim_cache size 0 max 16384 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 Processor -- start 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 -- start start 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ac9f0 0x55a8696b6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 0x55a8696b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81aa380 con 0x55a8696ae580 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81a7e80 con 0x55a8696adbc0 2024-03-26T14:11:59.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f2001c6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81a5f00 con 0x55a8696ac9f0 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.742+0000 7f1ffd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 0x55a8696b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 0x55a8696b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48646/0 (socket says 172.21.15.137:48646) 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 -- 172.21.15.137:0/1932109180 learned_addr learned my addr 172.21.15.137:0/1932109180 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ff7400640 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ac9f0 0x55a8696b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 -- 172.21.15.137:0/1932109180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ac9f0 msgr2=0x55a8696b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ac9f0 0x55a8696b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 -- 172.21.15.137:0/1932109180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 msgr2=0x55a8696be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 -- 172.21.15.137:0/1932109180 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8696b57b0 con 0x55a8696ae580 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.743+0000 7f1ffd000640 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 0x55a8696b0c60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0003b20 tx=0x7f1ff0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c036bf07954e29 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.744+0000 7f1fecc00640 1 -- 172.21.15.137:0/1932109180 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0010550 con 0x55a8696ae580 2024-03-26T14:11:59.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.744+0000 7f1fecc00640 1 -- 172.21.15.137:0/1932109180 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ff0024070 con 0x55a8696ae580 2024-03-26T14:11:59.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.744+0000 7f1fecc00640 1 -- 172.21.15.137:0/1932109180 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0020930 con 0x55a8696ae580 2024-03-26T14:11:59.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.744+0000 7f2001c6f740 1 -- 172.21.15.137:0/1932109180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 msgr2=0x55a8696b0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.744+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 0x55a8696b0c60 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0003b20 tx=0x7f1ff0015000 comp rx=0 tx=0).stop 2024-03-26T14:11:59.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.744+0000 7f2001c6f740 1 -- 172.21.15.137:0/1932109180 shutdown_connections 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ae580 0x55a8696b0c60 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1932109180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ac9f0 0x55a8696b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 -- 172.21.15.137:0/1932109180 >> 172.21.15.137:0/1932109180 conn(0x55a8696a7de0 msgr2=0x55a8696ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 -- 172.21.15.137:0/1932109180 shutdown_connections 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 -- 172.21.15.137:0/1932109180 wait complete. 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 -- start start 2024-03-26T14:11:59.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.745+0000 7f2001c6f740 1 Processor -- start 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 0x55a8696bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81a6500 con 0x55a8696ac9f0 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81a7820 con 0x55a8696adbc0 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81a6b60 con 0x55a8696ae580 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f2001c6f740 10 client.0 _ll_register_callbacks cb 0x7f1fe4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f1ff7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 0x55a8696bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f1ff7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 0x55a8696bd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48662/0 (socket says 172.21.15.137:48662) 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f1ff7e00640 1 -- 172.21.15.137:0/1962431187 learned_addr learned my addr 172.21.15.137:0/1962431187 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f1ff7400640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.746+0000 7f1ffd000640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ff7e00640 1 -- 172.21.15.137:0/1962431187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 msgr2=0x55a8696ba4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696ba4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ff7e00640 1 -- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 msgr2=0x55a8696bdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696bdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ff7e00640 1 -- 172.21.15.137:0/1962431187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8696bab60 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ffd000640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1ff7e00640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 0x55a8696bd9e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8036040 tx=0x7f1fe802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6153bde9fc671b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1fe2000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1fe2000640 1 -- 172.21.15.137:0/1962431187 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8044040 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ff0024070 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.747+0000 7f1fe2000640 1 -- 172.21.15.137:0/1962431187 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1fe8004240 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f1fe2000640 1 -- 172.21.15.137:0/1962431187 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe805d050 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 10 client.17700 fetch_fsmap learned FSMap version 24 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a8697273b0 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f1fe2000640 1 -- 172.21.15.137:0/1962431187 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1ff0024070 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f1fe2000640 1 -- 172.21.15.137:0/1962431187 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1fe8040470 con 0x55a8696ac9f0 2024-03-26T14:11:59.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 10 client.17700 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:11:59.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:11:59.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 20 client.17700 populate_metadata read hostname 'smithi137' 2024-03-26T14:11:59.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a869727860 0x55a869729c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:11:59.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 4 client.17700 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:11:59.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.748+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a869618b50 con 0x55a869727860 2024-03-26T14:11:59.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.749+0000 7f1ff7400640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a869727860 0x55a869729c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.749+0000 7f1ff7400640 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a869727860 0x55a869729c80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb51e8d7901056b server_cookie=b02319f5e07124f9 in_seq=0 out_seq=0 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.749+0000 7f1fe2000640 10 client.17700 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f1fe2000640 1 -- 172.21.15.137:0/1962431187 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a869618b50 con 0x55a869727860 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f1fe2000640 10 client.17700 handle_command_reply: tid=0 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 client.17700 shutdown 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 msgr2=0x55a8696bd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 0x55a8696bd9e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8036040 tx=0x7f1fe802c660 comp rx=0 tx=0).stop 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 shutdown_connections 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a869727860 0x55a869729c80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a8696adbc0 0x55a8696bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8696ae580 0x55a8696ba4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 --2- 172.21.15.137:0/1962431187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a8696ac9f0 0x55a8696bd9e0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 >> 172.21.15.137:0/1962431187 conn(0x55a8696a7de0 msgr2=0x55a8696ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f1fe2000640 0 client.17700 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:11:59.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 shutdown_connections 2024-03-26T14:11:59.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 1 -- 172.21.15.137:0/1962431187 wait complete. 2024-03-26T14:11:59.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.750+0000 7f2001c6f740 20 client.17700 trim_cache size 0 max 16384 2024-03-26T14:11:59.753 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1fdc0a2800 msgr2=0x7f1fdc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1fdc0a2800 0x7f1fdc0a4cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55a8696adac0 tx=0x7f1ff0009500 comp rx=0 tx=0).stop 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 msgr2=0x7f1ff8199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 0x7f1ff8199f40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400d770 tx=0x7f1fe400dc30 comp rx=0 tx=0).stop 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 shutdown_connections 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1fdc0a2800 0x7f1fdc0a4cc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ff8105350 0x7f1ff81a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ff8104990 0x7f1ff8199f40 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 --2- 172.21.15.137:0/3500997022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff81037a0 0x7f1ff8199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 >> 172.21.15.137:0/3500997022 conn(0x7f1ff80fed10 msgr2=0x7f1ff8107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 shutdown_connections 2024-03-26T14:11:59.754 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:11:59.751+0000 7f1ffda00640 1 -- 172.21.15.137:0/3500997022 wait complete. 2024-03-26T14:12:00.771 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:12:00.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:12:00.772 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.004+0000 7ff30de00640 1 -- 172.21.15.137:0/3702028818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 msgr2=0x7ff308101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.004+0000 7ff30de00640 1 --2- 172.21.15.137:0/3702028818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308101af0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009900 tx=0x7ff2f402bc30 comp rx=0 tx=0).stop 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 -- 172.21.15.137:0/3702028818 shutdown_connections 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 --2- 172.21.15.137:0/3702028818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff308102030 0x7ff3081088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 --2- 172.21.15.137:0/3702028818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308101af0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 --2- 172.21.15.137:0/3702028818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff308100480 0x7ff3081008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 -- 172.21.15.137:0/3702028818 >> 172.21.15.137:0/3702028818 conn(0x7ff3080fba10 msgr2=0x7ff3080fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 -- 172.21.15.137:0/3702028818 shutdown_connections 2024-03-26T14:12:01.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.005+0000 7ff30de00640 1 -- 172.21.15.137:0/3702028818 wait complete. 2024-03-26T14:12:01.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 Processor -- start 2024-03-26T14:12:01.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 -- start start 2024-03-26T14:12:01.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff308100480 0x7ff3081999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff308102030 0x7ff3081a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810c010 con 0x7ff308101670 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810be90 con 0x7ff308102030 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.006+0000 7ff30de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30810c190 con 0x7ff308100480 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48686/0 (socket says 172.21.15.137:48686) 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 -- 172.21.15.137:0/2719908498 learned_addr learned my addr 172.21.15.137:0/2719908498 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff30d400640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff308102030 0x7ff3081a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff30ca00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff308100480 0x7ff3081999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 -- 172.21.15.137:0/2719908498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff308100480 msgr2=0x7ff3081999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff308100480 0x7ff3081999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 -- 172.21.15.137:0/2719908498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff308102030 msgr2=0x7ff3081a0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff308102030 0x7ff3081a0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.007+0000 7ff307e00640 1 -- 172.21.15.137:0/2719908498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3081a15e0 con 0x7ff308101670 2024-03-26T14:12:01.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.008+0000 7ff307e00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308199f00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.008+0000 7ff305600640 1 -- 172.21.15.137:0/2719908498 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4043020 con 0x7ff308101670 2024-03-26T14:12:01.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.008+0000 7ff305600640 1 -- 172.21.15.137:0/2719908498 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2f4004aa0 con 0x7ff308101670 2024-03-26T14:12:01.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.008+0000 7ff305600640 1 -- 172.21.15.137:0/2719908498 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4003d70 con 0x7ff308101670 2024-03-26T14:12:01.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.008+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3081a1870 con 0x7ff308101670 2024-03-26T14:12:01.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.008+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081a1d70 con 0x7ff308101670 2024-03-26T14:12:01.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 Processor -- start 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff305600640 1 -- 172.21.15.137:0/2719908498 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff2f4046020 con 0x7ff308101670 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 -- start start 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b1556896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15568e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1555eeeb0 con 0x55b15568ee10 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b155597a60 con 0x55b1556892b0 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.012+0000 7ff311d22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1555eaf50 con 0x55b155689ca0 2024-03-26T14:12:01.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b1556896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff307e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b1556896d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:57686/0 (socket says 172.21.15.137:57686) 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 -- 172.21.15.137:0/2872073086 learned_addr learned my addr 172.21.15.137:0/2872073086 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30ca00640 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15568e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 -- 172.21.15.137:0/2872073086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 msgr2=0x55b15568e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15568e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 -- 172.21.15.137:0/2872073086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 msgr2=0x55b1556912a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556912a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff30d400640 1 -- 172.21.15.137:0/2872073086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1553cb7e0 con 0x55b1556892b0 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.013+0000 7ff307e00640 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556912a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff305600640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2f00a2880 0x7ff2f00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff30d400640 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b1556896d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff300010370 tx=0x7ff300010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce3f301ff40b10d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff305600640 1 -- 172.21.15.137:0/2719908498 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff2f40f1130 con 0x7ff308101670 2024-03-26T14:12:01.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff30ca00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2f00a2880 0x7ff2f00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff2f9c00640 1 -- 172.21.15.137:0/2872073086 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300016070 con 0x55b1556892b0 2024-03-26T14:12:01.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff2f9c00640 1 -- 172.21.15.137:0/2872073086 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff300004d20 con 0x55b1556892b0 2024-03-26T14:12:01.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.014+0000 7ff2f9c00640 1 -- 172.21.15.137:0/2872073086 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300005310 con 0x55b1556892b0 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 -- 172.21.15.137:0/2872073086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 msgr2=0x55b1556896d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff30ca00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2f00a2880 0x7ff2f00a4d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55b15568a0a0 tx=0x7ff2fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b1556896d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff300010370 tx=0x7ff300010830 comp rx=0 tx=0).stop 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 -- 172.21.15.137:0/2872073086 shutdown_connections 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556912a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15568e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 --2- 172.21.15.137:0/2872073086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b1556896d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.015+0000 7ff311d22740 1 -- 172.21.15.137:0/2872073086 >> 172.21.15.137:0/2872073086 conn(0x55b155684820 msgr2=0x55b155684c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.016+0000 7ff311d22740 1 -- 172.21.15.137:0/2872073086 shutdown_connections 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.016+0000 7ff311d22740 1 -- 172.21.15.137:0/2872073086 wait complete. 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.016+0000 7ff311d22740 1 -- start start 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.016+0000 7ff311d22740 1 Processor -- start 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155693550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b155693a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b15569aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b155691dc0 con 0x55b15568ee10 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1555aaeb0 con 0x55b1556892b0 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b155691f40 con 0x55b155689ca0 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff311d22740 10 client.0 _ll_register_callbacks cb 0x55b15559f150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff30d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b15569aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff30ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155693550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff30d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b15569aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48708/0 (socket says 172.21.15.137:48708) 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.017+0000 7ff30d400640 1 -- 172.21.15.137:0/3454289800 learned_addr learned my addr 172.21.15.137:0/3454289800 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff307e00640 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b155693a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff30d400640 1 -- 172.21.15.137:0/3454289800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 msgr2=0x55b155693a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff30d400640 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b155693a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff30d400640 1 -- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 msgr2=0x55b155693550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff30d400640 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155693550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff30d400640 1 -- 172.21.15.137:0/3454289800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b15542bf70 con 0x55b15568ee10 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff30d400640 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b15569aa60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff300006010 tx=0x7ff300009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd83cf477b2ef4e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff2ee000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:01.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff2ee000640 1 -- 172.21.15.137:0/3454289800 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300016070 con 0x55b15568ee10 2024-03-26T14:12:01.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff300004d20 con 0x55b15568ee10 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.018+0000 7ff2ee000640 1 -- 172.21.15.137:0/3454289800 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff300003c80 con 0x55b15568ee10 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff2ee000640 1 -- 172.21.15.137:0/3454289800 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff300012510 con 0x55b15568ee10 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 10 client.17712 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b1552c3170 con 0x55b15568ee10 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff2ee000640 1 -- 172.21.15.137:0/3454289800 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff300004d20 con 0x55b15568ee10 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff2ee000640 1 -- 172.21.15.137:0/3454289800 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff30000f040 con 0x55b15568ee10 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 10 client.17712 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 20 client.17712 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b15569f010 0x55b1556a1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.019+0000 7ff311d22740 4 client.17712 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.020+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1555fc3f0 con 0x55b15569f010 2024-03-26T14:12:01.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.020+0000 7ff30ca00640 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b15569f010 0x55b1556a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.021+0000 7ff2ee000640 10 client.17712 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.021+0000 7ff30ca00640 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b15569f010 0x55b1556a1450 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc3f65355b207040 server_cookie=fa021067d301c8a5 in_seq=0 out_seq=0 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.022+0000 7ff2ee000640 1 -- 172.21.15.137:0/3454289800 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b1555fc3f0 con 0x55b15569f010 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.022+0000 7ff2ee000640 10 client.17712 handle_command_reply: tid=0 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.022+0000 7ff311d22740 1 client.17712 shutdown 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 msgr2=0x55b15569aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b15569aa60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff300006010 tx=0x7ff300009d70 comp rx=0 tx=0).stop 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 shutdown_connections 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b15569f010 0x55b1556a1450 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b15569aa60 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b155693a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 --2- 172.21.15.137:0/3454289800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155693550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 >> 172.21.15.137:0/3454289800 conn(0x55b155684820 msgr2=0x55b155683a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff2ee000640 0 client.17712 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 shutdown_connections 2024-03-26T14:12:01.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 1 -- 172.21.15.137:0/3454289800 wait complete. 2024-03-26T14:12:01.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.023+0000 7ff311d22740 20 client.17712 trim_cache size 0 max 16384 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.040+0000 7ff311d22740 1 Processor -- start 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 -- start start 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155683900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081ae770 con 0x55b15568ee10 2024-03-26T14:12:01.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081ac270 con 0x55b1556892b0 2024-03-26T14:12:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff311d22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081aa2f0 con 0x55b155689ca0 2024-03-26T14:12:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff30d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff30d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556a5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48720/0 (socket says 172.21.15.137:48720) 2024-03-26T14:12:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff30d400640 1 -- 172.21.15.137:0/2136272994 learned_addr learned my addr 172.21.15.137:0/2136272994 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff30ca00640 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155683900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.041+0000 7ff307e00640 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff30d400640 1 -- 172.21.15.137:0/2136272994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 msgr2=0x55b15569aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff30d400640 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff30d400640 1 -- 172.21.15.137:0/2136272994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 msgr2=0x55b155683900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff30d400640 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155683900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff30d400640 1 -- 172.21.15.137:0/2136272994 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1556920e0 con 0x55b15568ee10 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff30d400640 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556a5d10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff300000f80 tx=0x7ff300009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cccae1ed081ba5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff2f9200640 1 -- 172.21.15.137:0/2136272994 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300016040 con 0x55b15568ee10 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff2f9200640 1 -- 172.21.15.137:0/2136272994 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff300005bd0 con 0x55b15568ee10 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.042+0000 7ff2f9200640 1 -- 172.21.15.137:0/2136272994 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300005250 con 0x55b15568ee10 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 -- 172.21.15.137:0/2136272994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 msgr2=0x55b1556a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556a5d10 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff300000f80 tx=0x7ff300009e70 comp rx=0 tx=0).stop 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 -- 172.21.15.137:0/2136272994 shutdown_connections 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b1556a5d10 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 --2- 172.21.15.137:0/2136272994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b155683900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 -- 172.21.15.137:0/2136272994 >> 172.21.15.137:0/2136272994 conn(0x55b155684820 msgr2=0x55b1556831f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 -- 172.21.15.137:0/2136272994 shutdown_connections 2024-03-26T14:12:01.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.043+0000 7ff311d22740 1 -- 172.21.15.137:0/2136272994 wait complete. 2024-03-26T14:12:01.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 -- start start 2024-03-26T14:12:01.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 Processor -- start 2024-03-26T14:12:01.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b15569a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b155698c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081aa8f0 con 0x55b15568ee10 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081abc10 con 0x55b1556892b0 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.044+0000 7ff311d22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3081aaf50 con 0x55b155689ca0 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff311d22740 10 client.0 _ll_register_callbacks cb 0x7ff2f40f4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b155698c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b155698c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48732/0 (socket says 172.21.15.137:48732) 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 -- 172.21.15.137:0/657278343 learned_addr learned my addr 172.21.15.137:0/657278343 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30ca00640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b15569a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff307e00640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 -- 172.21.15.137:0/657278343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 msgr2=0x55b15569aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 -- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 msgr2=0x55b15569a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b15569a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.045+0000 7ff30d400640 1 -- 172.21.15.137:0/657278343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1556994c0 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff30d400640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b155698c90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff300009470 tx=0x7ff300009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4676b9621f300b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff2ee000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff2ee000640 1 -- 172.21.15.137:0/657278343 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300016040 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff300005bd0 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff2ee000640 1 -- 172.21.15.137:0/657278343 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff300002030 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff2ee000640 1 -- 172.21.15.137:0/657278343 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff300004f90 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff311d22740 10 client.17718 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.046+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b1556fb4e0 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff2ee000640 1 -- 172.21.15.137:0/657278343 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff300005bd0 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff2ee000640 1 -- 172.21.15.137:0/657278343 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff300012ad0 con 0x55b15568ee10 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff311d22740 10 client.17718 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff311d22740 10 client.17718 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff311d22740 20 client.17718 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff311d22740 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b1556fbc20 0x55b1556fe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff311d22740 4 client.17718 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:12:01.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.047+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b155693340 con 0x55b1556fbc20 2024-03-26T14:12:01.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.048+0000 7ff30ca00640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b1556fbc20 0x55b1556fe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.048+0000 7ff2ee000640 10 client.17718 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.048+0000 7ff30ca00640 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b1556fbc20 0x55b1556fe040 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb064a8a282022c server_cookie=19f84bb290829563 in_seq=0 out_seq=0 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.052+0000 7ff2ee000640 1 -- 172.21.15.137:0/657278343 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b155693340 con 0x55b1556fbc20 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.052+0000 7ff2ee000640 10 client.17718 handle_command_reply: tid=0 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.052+0000 7ff311d22740 1 client.17718 shutdown 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.052+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 msgr2=0x55b155698c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.052+0000 7ff311d22740 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b155698c90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff300009470 tx=0x7ff300009b60 comp rx=0 tx=0).stop 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 shutdown_connections 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b1556fbc20 0x55b1556fe040 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b15568ee10 0x55b155698c90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b155689ca0 0x55b15569aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 --2- 172.21.15.137:0/657278343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1556892b0 0x55b15569a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 >> 172.21.15.137:0/657278343 conn(0x55b155684820 msgr2=0x55b155683a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff2ee000640 0 client.17718 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 shutdown_connections 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 1 -- 172.21.15.137:0/657278343 wait complete. 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.053+0000 7ff311d22740 20 client.17718 trim_cache size 0 max 16384 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "b1d295fd-08ce-482b-9774-b427f677c395", 2024-03-26T14:12:01.054 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2f00a2880 msgr2=0x7ff2f00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2f00a2880 0x7ff2f00a4d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55b15568a0a0 tx=0x7ff2fc005f90 comp rx=0 tx=0).stop 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 msgr2=0x7ff308199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308199f00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4000c00 tx=0x7ff2f40042f0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 shutdown_connections 2024-03-26T14:12:01.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff2f00a2880 0x7ff2f00a4d40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff308102030 0x7ff3081a0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff308101670 0x7ff308199f00 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 --2- 172.21.15.137:0/2719908498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff308100480 0x7ff3081999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 >> 172.21.15.137:0/2719908498 conn(0x7ff3080fba10 msgr2=0x7ff308068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 shutdown_connections 2024-03-26T14:12:01.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.054+0000 7ff30de00640 1 -- 172.21.15.137:0/2719908498 wait complete. 2024-03-26T14:12:01.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:12:01.074 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 -- 172.21.15.137:0/4239518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210103150 msgr2=0x7fe2101099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 --2- 172.21.15.137:0/4239518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210103150 0x7fe2101099e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe208009ca0 tx=0x7fe20802c0f0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 -- 172.21.15.137:0/4239518536 shutdown_connections 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 --2- 172.21.15.137:0/4239518536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe210102790 0x7fe210102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 --2- 172.21.15.137:0/4239518536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe210103150 0x7fe2101099e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 --2- 172.21.15.137:0/4239518536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe2101015a0 0x7fe2101019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 -- 172.21.15.137:0/4239518536 >> 172.21.15.137:0/4239518536 conn(0x7fe2100fcb10 msgr2=0x7fe2100fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 -- 172.21.15.137:0/4239518536 shutdown_connections 2024-03-26T14:12:01.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.284+0000 7fe215e00640 1 -- 172.21.15.137:0/4239518536 wait complete. 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.285+0000 7fe215e00640 1 Processor -- start 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.285+0000 7fe215e00640 1 -- start start 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 0x7fe210199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe210102790 0x7fe210199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe210103150 0x7fe2101a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21010d0e0 con 0x7fe2101015a0 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21010cf60 con 0x7fe210102790 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21010d260 con 0x7fe210103150 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 0x7fe210199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 0x7fe210199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48752/0 (socket says 172.21.15.137:48752) 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 -- 172.21.15.137:0/170251663 learned_addr learned my addr 172.21.15.137:0/170251663 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe20fe00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe210102790 0x7fe210199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe215400640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe210103150 0x7fe2101a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 -- 172.21.15.137:0/170251663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe210103150 msgr2=0x7fe2101a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe210103150 0x7fe2101a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 -- 172.21.15.137:0/170251663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe210102790 msgr2=0x7fe210199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe210102790 0x7fe210199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.286+0000 7fe214a00640 1 -- 172.21.15.137:0/170251663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2101a1540 con 0x7fe2101015a0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.287+0000 7fe214a00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 0x7fe210199920 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe20400d950 tx=0x7fe20400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.287+0000 7fe20d600640 1 -- 172.21.15.137:0/170251663 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204017070 con 0x7fe2101015a0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.287+0000 7fe20d600640 1 -- 172.21.15.137:0/170251663 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe204009c30 con 0x7fe2101015a0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.287+0000 7fe20d600640 1 -- 172.21.15.137:0/170251663 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400f620 con 0x7fe2101015a0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.287+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2101a1830 con 0x7fe2101015a0 2024-03-26T14:12:01.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.287+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2101a1d80 con 0x7fe2101015a0 2024-03-26T14:12:01.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.291+0000 7fe21a186740 1 Processor -- start 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.291+0000 7fe21a186740 1 -- start start 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe21a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769050 0x55a09a769470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe21a186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a76e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe21a186740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 0x55a09a771040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe21a186740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09a6ce250 con 0x55a09a76ebb0 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe21a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09a676a30 con 0x55a09a769050 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe21a186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09a6ca2f0 con 0x55a09a769a40 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20d600640 1 -- 172.21.15.137:0/170251663 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe20401a070 con 0x7fe2101015a0 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 0x55a09a771040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe215400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769050 0x55a09a769470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 0x55a09a771040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48760/0 (socket says 172.21.15.137:48760) 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 -- 172.21.15.137:0/2601902773 learned_addr learned my addr 172.21.15.137:0/2601902773 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe214a00640 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a76e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 -- 172.21.15.137:0/2601902773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 msgr2=0x55a09a76e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a76e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 -- 172.21.15.137:0/2601902773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769050 msgr2=0x55a09a769470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769050 0x55a09a769470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.292+0000 7fe20fe00640 1 -- 172.21.15.137:0/2601902773 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a09a661690 con 0x55a09a76ebb0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.293+0000 7fe20fe00640 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 0x55a09a771040 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009c90 tx=0x7fe1fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa2ca9640bc5fa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.293+0000 7fe201c00640 1 -- 172.21.15.137:0/2601902773 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc01e070 con 0x55a09a76ebb0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.293+0000 7fe201c00640 1 -- 172.21.15.137:0/2601902773 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1fc019ae0 con 0x55a09a76ebb0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.293+0000 7fe201c00640 1 -- 172.21.15.137:0/2601902773 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00ea40 con 0x55a09a76ebb0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.293+0000 7fe20d600640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2900 0x7fe1f80a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe20d600640 1 -- 172.21.15.137:0/170251663 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe2040ce820 con 0x7fe2101015a0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe215400640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2900 0x7fe1f80a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 -- 172.21.15.137:0/2601902773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 msgr2=0x55a09a771040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 0x55a09a771040 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009c90 tx=0x7fe1fc011810 comp rx=0 tx=0).stop 2024-03-26T14:12:01.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 -- 172.21.15.137:0/2601902773 shutdown_connections 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a76ebb0 0x55a09a771040 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a76e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 --2- 172.21.15.137:0/2601902773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769050 0x55a09a769470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.294+0000 7fe21a186740 1 -- 172.21.15.137:0/2601902773 >> 172.21.15.137:0/2601902773 conn(0x55a09a731200 msgr2=0x55a09a7649c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.295+0000 7fe215400640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2900 0x7fe1f80a4dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe20802c630 tx=0x7fe208005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.295+0000 7fe21a186740 1 -- 172.21.15.137:0/2601902773 shutdown_connections 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.295+0000 7fe21a186740 1 -- 172.21.15.137:0/2601902773 wait complete. 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.295+0000 7fe21a186740 1 -- start start 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.295+0000 7fe21a186740 1 Processor -- start 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769050 0x55a09a7730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 0x55a09a773630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a77a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09a771b60 con 0x55a09a769a40 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09a689e80 con 0x55a09a76ebb0 2024-03-26T14:12:01.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a09a771ce0 con 0x55a09a769050 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe21a186740 10 client.0 _ll_register_callbacks cb 0x55a09a69c130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe20fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 0x55a09a773630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe20fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 0x55a09a773630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48776/0 (socket says 172.21.15.137:48776) 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe20fe00640 1 -- 172.21.15.137:0/3867954906 learned_addr learned my addr 172.21.15.137:0/3867954906 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.296+0000 7fe215400640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a77a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe214a00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769050 0x55a09a7730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe20fe00640 1 -- 172.21.15.137:0/3867954906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769050 msgr2=0x55a09a7730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769050 0x55a09a7730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe20fe00640 1 -- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 msgr2=0x55a09a77a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a77a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe20fe00640 1 -- 172.21.15.137:0/3867954906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a09a4ac7d0 con 0x55a09a769a40 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe214a00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769050 0x55a09a7730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 0x55a09a773630 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc005bc0 tx=0x7fe1fc019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3274a2cc47f067 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.302 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe1f6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe1f6000640 1 -- 172.21.15.137:0/3867954906 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc01e040 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe1f6000640 1 -- 172.21.15.137:0/3867954906 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe1fc004030 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.297+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1fc019ae0 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe1f6000640 1 -- 172.21.15.137:0/3867954906 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1fc033050 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe21a186740 10 client.17730 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a09a3e23f0 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe1f6000640 1 -- 172.21.15.137:0/3867954906 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1fc019ae0 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe1f6000640 1 -- 172.21.15.137:0/3867954906 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe1fc00e740 con 0x55a09a769a40 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe21a186740 10 client.17730 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.298+0000 7fe21a186740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.299+0000 7fe21a186740 20 client.17730 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.299+0000 7fe21a186740 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a77ebd0 0x55a09a780fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.299+0000 7fe21a186740 4 client.17730 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:01.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.299+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a09a6db790 con 0x55a09a77ebd0 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.299+0000 7fe214a00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a77ebd0 0x55a09a780fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.300+0000 7fe214a00640 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a77ebd0 0x55a09a780fe0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0cfe6eadf5bc0e server_cookie=e6e8eb87efde6c4e in_seq=0 out_seq=0 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.300+0000 7fe1f6000640 10 client.17730 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe1f6000640 1 -- 172.21.15.137:0/3867954906 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a09a6db790 con 0x55a09a77ebd0 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe1f6000640 10 client.17730 handle_command_reply: tid=0 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 client.17730 shutdown 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 msgr2=0x55a09a773630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 0x55a09a773630 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc005bc0 tx=0x7fe1fc019660 comp rx=0 tx=0).stop 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 shutdown_connections 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a77ebd0 0x55a09a780fe0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a77a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769a40 0x55a09a773630 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe1f6000640 0 client.17730 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 --2- 172.21.15.137:0/3867954906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769050 0x55a09a7730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 >> 172.21.15.137:0/3867954906 conn(0x55a09a731200 msgr2=0x55a09a763bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 shutdown_connections 2024-03-26T14:12:01.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 1 -- 172.21.15.137:0/3867954906 wait complete. 2024-03-26T14:12:01.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.301+0000 7fe21a186740 20 client.17730 trim_cache size 0 max 16384 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 Processor -- start 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 -- start start 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a75f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a77a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a763330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101ae6f0 con 0x55a09a769050 2024-03-26T14:12:01.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101ac1f0 con 0x55a09a76ebb0 2024-03-26T14:12:01.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe21a186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101aa270 con 0x55a09a769a40 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe214a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a75f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.319+0000 7fe214a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a75f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48792/0 (socket says 172.21.15.137:48792) 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 -- 172.21.15.137:0/3591303298 learned_addr learned my addr 172.21.15.137:0/3591303298 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe215400640 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a763330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a77a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 -- 172.21.15.137:0/3591303298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 msgr2=0x55a09a77a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a77a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 -- 172.21.15.137:0/3591303298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 msgr2=0x55a09a763330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a763330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 -- 172.21.15.137:0/3591303298 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a09a3a20f0 con 0x55a09a769050 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe214a00640 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a75f920 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe20409ad40 tx=0x7fe20400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d130ac70e05312bb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.320+0000 7fe201200640 1 -- 172.21.15.137:0/3591303298 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204021040 con 0x55a09a769050 2024-03-26T14:12:01.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe201200640 1 -- 172.21.15.137:0/3591303298 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe20409adb0 con 0x55a09a769050 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe201200640 1 -- 172.21.15.137:0/3591303298 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040cebd0 con 0x55a09a769050 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 -- 172.21.15.137:0/3591303298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 msgr2=0x55a09a75f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a75f920 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe20409ad40 tx=0x7fe20400bed0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 -- 172.21.15.137:0/3591303298 shutdown_connections 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a76ebb0 0x55a09a763330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a769a40 0x55a09a77a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 --2- 172.21.15.137:0/3591303298 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a75f920 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 -- 172.21.15.137:0/3591303298 >> 172.21.15.137:0/3591303298 conn(0x55a09a731200 msgr2=0x55a09a768490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.321+0000 7fe21a186740 1 -- 172.21.15.137:0/3591303298 shutdown_connections 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 -- 172.21.15.137:0/3591303298 wait complete. 2024-03-26T14:12:01.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 -- start start 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 Processor -- start 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a778420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769a40 0x55a09a778960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a76ebb0 0x55a09a779920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.322+0000 7fe21a186740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101aa870 con 0x55a09a769050 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe21a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101abb90 con 0x55a09a769a40 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe21a186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2101aaed0 con 0x55a09a76ebb0 2024-03-26T14:12:01.327 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe21a186740 10 client.0 _ll_register_callbacks cb 0x7fe1fc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a778420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a778420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48806/0 (socket says 172.21.15.137:48806) 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 -- 172.21.15.137:0/3356418457 learned_addr learned my addr 172.21.15.137:0/3356418457 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769a40 0x55a09a778960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe215400640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a76ebb0 0x55a09a779920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 -- 172.21.15.137:0/3356418457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a76ebb0 msgr2=0x55a09a779920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a76ebb0 0x55a09a779920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 -- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769a40 msgr2=0x55a09a778960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769a40 0x55a09a778960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.323+0000 7fe214a00640 1 -- 172.21.15.137:0/3356418457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a09a779f10 con 0x55a09a769050 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe214a00640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a778420 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fe2040cbf60 tx=0x7fe2040991b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417511bdd3ec6304 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe1f6000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe1f6000640 1 -- 172.21.15.137:0/3356418457 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040b8060 con 0x55a09a769050 2024-03-26T14:12:01.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe1f6000640 1 -- 172.21.15.137:0/3356418457 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe2040cebd0 con 0x55a09a769050 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe20409adb0 con 0x55a09a769050 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe1f6000640 1 -- 172.21.15.137:0/3356418457 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe204021070 con 0x55a09a769050 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe21a186740 10 client.17736 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a09a7dab40 con 0x55a09a769050 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.324+0000 7fe1f6000640 1 -- 172.21.15.137:0/3356418457 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe20409adb0 con 0x55a09a769050 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe1f6000640 1 -- 172.21.15.137:0/3356418457 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe204099c70 con 0x55a09a769050 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe21a186740 10 client.17736 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe21a186740 10 client.17736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe21a186740 20 client.17736 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe21a186740 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a7db480 0x55a09a7dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe21a186740 4 client.17736 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a09a6d5ae0 con 0x55a09a7db480 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.325+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a7db480 0x55a09a7dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe1f6000640 10 client.17736 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe20fe00640 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a7db480 0x55a09a7dd8a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0aad8297e03817f server_cookie=1a53466d391cc0cf in_seq=0 out_seq=0 2024-03-26T14:12:01.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe1f6000640 1 -- 172.21.15.137:0/3356418457 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a09a6d5ae0 con 0x55a09a7db480 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe1f6000640 10 client.17736 handle_command_reply: tid=0 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 client.17736 shutdown 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 msgr2=0x55a09a778420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a778420 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fe2040cbf60 tx=0x7fe2040991b0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 shutdown_connections 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a09a7db480 0x55a09a7dd8a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a09a769a40 0x55a09a778960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a09a76ebb0 0x55a09a779920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe1f6000640 0 client.17736 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 --2- 172.21.15.137:0/3356418457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a09a769050 0x55a09a778420 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.326+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 >> 172.21.15.137:0/3356418457 conn(0x55a09a731200 msgr2=0x55a09a767c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 shutdown_connections 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe21a186740 1 -- 172.21.15.137:0/3356418457 wait complete. 2024-03-26T14:12:01.330 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe21a186740 20 client.17736 trim_cache size 0 max 16384 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: "b1d295fd-08ce-482b-9774-b427f677c395": { 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "b1d295fd-08ce-482b-9774-b427f677c395", 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:12:01.331 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:12:01.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2900 msgr2=0x7fe1f80a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe215e00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2900 0x7fe1f80a4dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe20802c630 tx=0x7fe208005d80 comp rx=0 tx=0).stop 2024-03-26T14:12:01.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 msgr2=0x7fe210199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:01.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe215e00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 0x7fe210199920 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe20400d950 tx=0x7fe20400de10 comp rx=0 tx=0).stop 2024-03-26T14:12:01.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 shutdown_connections 2024-03-26T14:12:01.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.327+0000 7fe215e00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2900 0x7fe1f80a4dc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.328+0000 7fe215e00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe210102790 0x7fe210199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.328+0000 7fe215e00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe210103150 0x7fe2101a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.328+0000 7fe215e00640 1 --2- 172.21.15.137:0/170251663 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe2101015a0 0x7fe210199920 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:01.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.328+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 >> 172.21.15.137:0/170251663 conn(0x7fe2100fcb10 msgr2=0x7fe2100738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:01.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.328+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 shutdown_connections 2024-03-26T14:12:01.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:01.328+0000 7fe215e00640 1 -- 172.21.15.137:0/170251663 wait complete. 2024-03-26T14:12:01.348 INFO:tasks.cephfs.filesystem:scrub status for tag:b1d295fd-08ce-482b-9774-b427f677c395 - {'path': '/', 'tag': 'b1d295fd-08ce-482b-9774-b427f677c395', 'options': 'recursive,force'} 2024-03-26T14:12:01.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:01 smithi137 ceph-mon[31539]: pgmap v747: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:01.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:01 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:12:01.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:01 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:01.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:01 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:12:01.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:01 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:01 smithi179 ceph-mon[35472]: pgmap v747: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:01 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:12:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:01 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:01 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:12:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:01 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:01.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:01 smithi159 ceph-mon[35501]: pgmap v747: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:01.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:01 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:12:01.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:01 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:01.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:01 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:12:01.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:01 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:02.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:02 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:12:02.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:02 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:12:02.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:02 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:12:03.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:03 smithi137 ceph-mon[31539]: pgmap v748: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:12:03.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:03 smithi179 ceph-mon[35472]: pgmap v748: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:12:03.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:03 smithi159 ceph-mon[35501]: pgmap v748: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:12:05.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:05 smithi137 ceph-mon[31539]: pgmap v749: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:12:05.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:05 smithi179 ceph-mon[35472]: pgmap v749: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:12:05.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:05 smithi159 ceph-mon[35501]: pgmap v749: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:12:07.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:07 smithi137 ceph-mon[31539]: pgmap v750: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:07.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:07 smithi179 ceph-mon[35472]: pgmap v750: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:07.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:07 smithi159 ceph-mon[35501]: pgmap v750: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:09.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:09 smithi137 ceph-mon[31539]: pgmap v751: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:12:09.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:09 smithi179 ceph-mon[35472]: pgmap v751: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:12:09.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:09 smithi159 ceph-mon[35501]: pgmap v751: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:12:11.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:11 smithi137 ceph-mon[31539]: pgmap v752: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:12:11.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:11 smithi179 ceph-mon[35472]: pgmap v752: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:12:11.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:11 smithi159 ceph-mon[35501]: pgmap v752: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:12:13.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:13 smithi179 ceph-mon[35472]: pgmap v753: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:12:13.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:13 smithi159 ceph-mon[35501]: pgmap v753: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:12:13.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:13 smithi137 ceph-mon[31539]: pgmap v753: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:12:15.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:15 smithi179 ceph-mon[35472]: pgmap v754: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:15 smithi159 ceph-mon[35501]: pgmap v754: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:15.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:15 smithi137 ceph-mon[31539]: pgmap v754: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:17.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:17 smithi179 ceph-mon[35472]: pgmap v755: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:17.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:17 smithi159 ceph-mon[35501]: pgmap v755: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:17.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:17 smithi137 ceph-mon[31539]: pgmap v755: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:19 smithi179 ceph-mon[35472]: pgmap v756: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-26T14:12:19.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:19 smithi159 ceph-mon[35501]: pgmap v756: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-26T14:12:19.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:19 smithi137 ceph-mon[31539]: pgmap v756: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-03-26T14:12:21.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:21 smithi179 ceph-mon[35472]: pgmap v757: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:21.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:21 smithi159 ceph-mon[35501]: pgmap v757: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:21.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:21 smithi137 ceph-mon[31539]: pgmap v757: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-03-26T14:12:23.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:23 smithi179 ceph-mon[35472]: pgmap v758: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:12:23.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:23 smithi159 ceph-mon[35501]: pgmap v758: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:12:23.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:23 smithi137 ceph-mon[31539]: pgmap v758: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2024-03-26T14:12:25.588 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:25 smithi137 ceph-mon[31539]: pgmap v759: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:25 smithi179 ceph-mon[35472]: pgmap v759: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:25.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:25 smithi159 ceph-mon[35501]: pgmap v759: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:27.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:27 smithi179 ceph-mon[35472]: pgmap v760: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:27.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:27 smithi159 ceph-mon[35501]: pgmap v760: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:27.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:27 smithi137 ceph-mon[31539]: pgmap v760: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:29.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:29 smithi179 ceph-mon[35472]: pgmap v761: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:12:29.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:29 smithi159 ceph-mon[35501]: pgmap v761: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:12:29.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:29 smithi137 ceph-mon[31539]: pgmap v761: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:12:31.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:12:31.349 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:12:31.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 -- 172.21.15.137:0/3472268130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 msgr2=0x7fae4c10b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 --2- 172.21.15.137:0/3472268130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 0x7fae4c10b9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fae44009be0 tx=0x7fae4402bf80 comp rx=0 tx=0).stop 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 -- 172.21.15.137:0/3472268130 shutdown_connections 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 --2- 172.21.15.137:0/3472268130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 0x7fae4c10b9f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 --2- 172.21.15.137:0/3472268130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c074080 0x7fae4c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 --2- 172.21.15.137:0/3472268130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c075660 0x7fae4c073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 -- 172.21.15.137:0/3472268130 >> 172.21.15.137:0/3472268130 conn(0x7fae4c0fcc40 msgr2=0x7fae4c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 -- 172.21.15.137:0/3472268130 shutdown_connections 2024-03-26T14:12:31.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.589+0000 7fae50e00640 1 -- 172.21.15.137:0/3472268130 wait complete. 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 Processor -- start 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 -- start start 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c074080 0x7fae4c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 0x7fae4c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 0x7fae4c1a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c10f3d0 con 0x7fae4c075660 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c10f250 con 0x7fae4c109560 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.590+0000 7fae50e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c10f550 con 0x7fae4c074080 2024-03-26T14:12:31.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 0x7fae4c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 0x7fae4c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46874/0 (socket says 172.21.15.137:46874) 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 -- 172.21.15.137:0/1299195135 learned_addr learned my addr 172.21.15.137:0/1299195135 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae4a800640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 0x7fae4c1a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c074080 0x7fae4c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 -- 172.21.15.137:0/1299195135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c074080 msgr2=0x7fae4c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c074080 0x7fae4c199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 -- 172.21.15.137:0/1299195135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 msgr2=0x7fae4c1a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 0x7fae4c1a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.591+0000 7fae49400640 1 -- 172.21.15.137:0/1299195135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c1a16b0 con 0x7fae4c075660 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.592+0000 7fae49400640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 0x7fae4c199fd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fae3800e9c0 tx=0x7fae3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.592+0000 7fae42a00640 1 -- 172.21.15.137:0/1299195135 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800cde0 con 0x7fae4c075660 2024-03-26T14:12:31.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.592+0000 7fae42a00640 1 -- 172.21.15.137:0/1299195135 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae380035f0 con 0x7fae4c075660 2024-03-26T14:12:31.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.592+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c1a19a0 con 0x7fae4c075660 2024-03-26T14:12:31.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.592+0000 7fae42a00640 1 -- 172.21.15.137:0/1299195135 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38010690 con 0x7fae4c075660 2024-03-26T14:12:31.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.592+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c1a1f40 con 0x7fae4c075660 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.596+0000 7fae53a61740 1 Processor -- start 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.596+0000 7fae53a61740 1 -- start start 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae53a61740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae53a61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 0x555a605e4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae53a61740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605e49e0 0x555a605e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae53a61740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a60544250 con 0x555a605e49e0 2024-03-26T14:12:31.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae53a61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a604eca30 con 0x555a605df870 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae53a61740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a605402f0 con 0x555a605dee80 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae42a00640 1 -- 172.21.15.137:0/1299195135 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fae380037a0 con 0x7fae4c075660 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae49e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 0x555a605e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae49e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 0x555a605e4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37614/0 (socket says 172.21.15.137:37614) 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae49e00640 1 -- 172.21.15.137:0/2848333627 learned_addr learned my addr 172.21.15.137:0/2848333627 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae49400640 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605e49e0 0x555a605e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.597+0000 7fae4a800640 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.598+0000 7fae49e00640 1 -- 172.21.15.137:0/2848333627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 msgr2=0x555a605df2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.598+0000 7fae49e00640 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605df2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.598+0000 7fae49e00640 1 -- 172.21.15.137:0/2848333627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605e49e0 msgr2=0x555a605e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.598+0000 7fae49e00640 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605e49e0 0x555a605e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.598+0000 7fae49e00640 1 -- 172.21.15.137:0/2848333627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a604d7690 con 0x555a605df870 2024-03-26T14:12:31.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.598+0000 7fae49e00640 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 0x555a605e4370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009c80 tx=0x7fae3c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100e1ef9ddef3463 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.599+0000 7fae42a00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae300a2880 0x7fae300a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.599+0000 7fae42a00640 1 -- 172.21.15.137:0/1299195135 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae380cf0d0 con 0x7fae4c075660 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.599+0000 7fae4a800640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae300a2880 0x7fae300a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.599+0000 7fae37400640 1 -- 172.21.15.137:0/2848333627 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c012450 con 0x555a605df870 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.599+0000 7fae37400640 1 -- 172.21.15.137:0/2848333627 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae3c012aa0 con 0x555a605df870 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.599+0000 7fae37400640 1 -- 172.21.15.137:0/2848333627 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c00fb70 con 0x555a605df870 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 -- 172.21.15.137:0/2848333627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 msgr2=0x555a605e4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae4a800640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae300a2880 0x7fae300a4d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fae44004af0 tx=0x7fae44033400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 0x555a605e4370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fae3c009c80 tx=0x7fae3c0117b0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 -- 172.21.15.137:0/2848333627 shutdown_connections 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605df870 0x555a605e4370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605e49e0 0x555a605e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 --2- 172.21.15.137:0/2848333627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605df2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 -- 172.21.15.137:0/2848333627 >> 172.21.15.137:0/2848333627 conn(0x555a605a7200 msgr2=0x555a605da7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 -- 172.21.15.137:0/2848333627 shutdown_connections 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.600+0000 7fae53a61740 1 -- 172.21.15.137:0/2848333627 wait complete. 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 -- start start 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 Processor -- start 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605f0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a605e7990 con 0x555a605df870 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a604ffe80 con 0x555a605dee80 2024-03-26T14:12:31.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a605e7b10 con 0x555a605e49e0 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.601+0000 7fae53a61740 10 client.0 _ll_register_callbacks cb 0x555a60512130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46892/0 (socket says 172.21.15.137:46892) 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 -- 172.21.15.137:0/4087606310 learned_addr learned my addr 172.21.15.137:0/4087606310 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49e00640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae4a800640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 -- 172.21.15.137:0/4087606310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 msgr2=0x555a605f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 -- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 msgr2=0x555a605e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae49400640 1 -- 172.21.15.137:0/4087606310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a603227d0 con 0x555a605df870 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.602+0000 7fae4a800640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:12:31.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae49400640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605e9330 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fae3809b3a0 tx=0x7fae38097a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c41affe280e34d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae2be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae2be00640 1 -- 172.21.15.137:0/4087606310 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38003b10 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae2be00640 1 -- 172.21.15.137:0/4087606310 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae380cc910 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae3c012aa0 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae2be00640 1 -- 172.21.15.137:0/4087606310 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3809bda0 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae53a61740 10 client.17748 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.603+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555a602583f0 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae2be00640 1 -- 172.21.15.137:0/4087606310 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae3c012aa0 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae2be00640 1 -- 172.21.15.137:0/4087606310 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae380d1110 con 0x555a605df870 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae53a61740 10 client.17748 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae53a61740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae53a61740 20 client.17748 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:31.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae53a61740 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a605f4c80 0x555a605f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae53a61740 4 client.17748 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:31.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.604+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a60551790 con 0x555a605f4c80 2024-03-26T14:12:31.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.605+0000 7fae49e00640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a605f4c80 0x555a605f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.605+0000 7fae2be00640 10 client.17748 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.605+0000 7fae49e00640 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a605f4c80 0x555a605f70c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9514425af5d223c4 server_cookie=f06ed6a4b7e401cf in_seq=0 out_seq=0 2024-03-26T14:12:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.607+0000 7fae2be00640 1 -- 172.21.15.137:0/4087606310 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555a60551790 con 0x555a605f4c80 2024-03-26T14:12:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.607+0000 7fae2be00640 10 client.17748 handle_command_reply: tid=0 2024-03-26T14:12:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.607+0000 7fae53a61740 1 client.17748 shutdown 2024-03-26T14:12:31.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.607+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 msgr2=0x555a605e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.607+0000 7fae53a61740 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605e9330 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fae3809b3a0 tx=0x7fae38097a00 comp rx=0 tx=0).stop 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 shutdown_connections 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a605f4c80 0x555a605f70c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605e9330 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 --2- 172.21.15.137:0/4087606310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 >> 172.21.15.137:0/4087606310 conn(0x555a605a7200 msgr2=0x555a605d9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae2be00640 0 client.17748 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 shutdown_connections 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 1 -- 172.21.15.137:0/4087606310 wait complete. 2024-03-26T14:12:31.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.608+0000 7fae53a61740 20 client.17748 trim_cache size 0 max 16384 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.629+0000 7fae53a61740 1 Processor -- start 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.629+0000 7fae53a61740 1 -- start start 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae53a61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae53a61740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605f0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae53a61740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae53a61740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1ae960 con 0x555a605df870 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae53a61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1ac460 con 0x555a605dee80 2024-03-26T14:12:31.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae53a61740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1aa4e0 con 0x555a605e49e0 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605f0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605f0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46900/0 (socket says 172.21.15.137:46900) 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae49400640 1 -- 172.21.15.137:0/3451403301 learned_addr learned my addr 172.21.15.137:0/3451403301 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae49e00640 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae4a800640 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae49400640 1 -- 172.21.15.137:0/3451403301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 msgr2=0x555a605e0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.630+0000 7fae49400640 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605e0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae49400640 1 -- 172.21.15.137:0/3451403301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 msgr2=0x555a605e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae49400640 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae49400640 1 -- 172.21.15.137:0/3451403301 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a602180f0 con 0x555a605df870 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae49400640 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605f0330 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fae38098070 tx=0x7fae38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb49b644bbbc6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae36a00640 1 -- 172.21.15.137:0/3451403301 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3809b1e0 con 0x555a605df870 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae36a00640 1 -- 172.21.15.137:0/3451403301 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae380d2840 con 0x555a605df870 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.631+0000 7fae36a00640 1 -- 172.21.15.137:0/3451403301 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38097440 con 0x555a605df870 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 -- 172.21.15.137:0/3451403301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 msgr2=0x555a605f0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605f0330 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fae38098070 tx=0x7fae38000bc0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 -- 172.21.15.137:0/3451403301 shutdown_connections 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605e49e0 0x555a605e0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605f0330 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.634 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 --2- 172.21.15.137:0/3451403301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605dee80 0x555a605e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 -- 172.21.15.137:0/3451403301 >> 172.21.15.137:0/3451403301 conn(0x555a605a7200 msgr2=0x555a605e3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 -- 172.21.15.137:0/3451403301 shutdown_connections 2024-03-26T14:12:31.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.632+0000 7fae53a61740 1 -- 172.21.15.137:0/3451403301 wait complete. 2024-03-26T14:12:31.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 -- start start 2024-03-26T14:12:31.635 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 Processor -- start 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605ed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605e49e0 0x555a6064e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1aaae0 con 0x555a605df870 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1abe00 con 0x555a605e49e0 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c1ab140 con 0x555a605dee80 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae53a61740 10 client.0 _ll_register_callbacks cb 0x7fae380046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:31.639 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.633+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46910/0 (socket says 172.21.15.137:46910) 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 -- 172.21.15.137:0/22098213 learned_addr learned my addr 172.21.15.137:0/22098213 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae4a800640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605e49e0 0x555a6064e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49e00640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605ed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 -- 172.21.15.137:0/22098213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 msgr2=0x555a605ed160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605ed160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 -- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605e49e0 msgr2=0x555a6064e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605e49e0 0x555a6064e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 -- 172.21.15.137:0/22098213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a605edd40 con 0x555a605df870 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49e00640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605ed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae49400640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605ed6a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fae380d29f0 tx=0x7fae380d1340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f09ffacf6bac6ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae2be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:31.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae2be00640 1 -- 172.21.15.137:0/22098213 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3809acf0 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.634+0000 7fae2be00640 1 -- 172.21.15.137:0/22098213 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fae380b4210 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae380d2840 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae2be00640 1 -- 172.21.15.137:0/22098213 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae38082050 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 10 client.17754 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555a6064fba0 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae2be00640 1 -- 172.21.15.137:0/22098213 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fae380d2840 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae2be00640 1 -- 172.21.15.137:0/22098213 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fae380a0710 con 0x555a605df870 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 10 client.17754 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 10 client.17754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 20 client.17754 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a60650310 0x555a60652730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 4 client.17754 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.635+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a6054bae0 con 0x555a60650310 2024-03-26T14:12:31.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.636+0000 7fae49e00640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a60650310 0x555a60652730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae2be00640 10 client.17754 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae49e00640 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a60650310 0x555a60652730 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d33bd17aab9b99c server_cookie=ef605236fbedb093 in_seq=0 out_seq=0 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae2be00640 1 -- 172.21.15.137:0/22098213 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555a6054bae0 con 0x555a60650310 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae2be00640 10 client.17754 handle_command_reply: tid=0 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 client.17754 shutdown 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 msgr2=0x555a605ed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605ed6a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fae380d29f0 tx=0x7fae380d1340 comp rx=0 tx=0).stop 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 shutdown_connections 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x555a60650310 0x555a60652730 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555a605e49e0 0x555a6064e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555a605df870 0x555a605ed6a0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.642 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 --2- 172.21.15.137:0/22098213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555a605dee80 0x555a605ed160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 >> 172.21.15.137:0/22098213 conn(0x555a605a7200 msgr2=0x555a605e0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae2be00640 0 client.17754 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 shutdown_connections 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.637+0000 7fae53a61740 1 -- 172.21.15.137:0/22098213 wait complete. 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae53a61740 20 client.17754 trim_cache size 0 max 16384 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:12:31.643 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:12:31.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae300a2880 msgr2=0x7fae300a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae300a2880 0x7fae300a4d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fae44004af0 tx=0x7fae44033400 comp rx=0 tx=0).stop 2024-03-26T14:12:31.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 msgr2=0x7fae4c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 0x7fae4c199fd0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fae3800e9c0 tx=0x7fae3800ee80 comp rx=0 tx=0).stop 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 shutdown_connections 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fae300a2880 0x7fae300a4d40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae4c109560 0x7fae4c1a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae4c075660 0x7fae4c199fd0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 --2- 172.21.15.137:0/1299195135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fae4c074080 0x7fae4c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.638+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 >> 172.21.15.137:0/1299195135 conn(0x7fae4c0fcc40 msgr2=0x7fae4c1054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.639+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 shutdown_connections 2024-03-26T14:12:31.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.639+0000 7fae50e00640 1 -- 172.21.15.137:0/1299195135 wait complete. 2024-03-26T14:12:31.656 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:31 smithi137 ceph-mon[31539]: pgmap v762: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:31.658 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:12:31.658 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:12:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:31 smithi179 ceph-mon[35472]: pgmap v762: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:31 smithi159 ceph-mon[35501]: pgmap v762: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:31.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.883+0000 7f89bd400640 1 -- 172.21.15.137:0/486936786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b8101fa0 msgr2=0x7f89b8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.883+0000 7f89bd400640 1 --2- 172.21.15.137:0/486936786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b8101fa0 0x7f89b8108830 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009900 tx=0x7f89b002bc30 comp rx=0 tx=0).stop 2024-03-26T14:12:31.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 -- 172.21.15.137:0/486936786 shutdown_connections 2024-03-26T14:12:31.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 --2- 172.21.15.137:0/486936786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b81015e0 0x7f89b8101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 --2- 172.21.15.137:0/486936786 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b8101fa0 0x7f89b8108830 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 --2- 172.21.15.137:0/486936786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81003f0 0x7f89b8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 -- 172.21.15.137:0/486936786 >> 172.21.15.137:0/486936786 conn(0x7f89b80fb980 msgr2=0x7f89b80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:31.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 -- 172.21.15.137:0/486936786 shutdown_connections 2024-03-26T14:12:31.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.884+0000 7f89bd400640 1 -- 172.21.15.137:0/486936786 wait complete. 2024-03-26T14:12:31.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 Processor -- start 2024-03-26T14:12:31.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 -- start start 2024-03-26T14:12:31.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 0x7f89b81976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81015e0 0x7f89b8197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b8101fa0 0x7f89b819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b810bf80 con 0x7f89b81003f0 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b810be00 con 0x7f89b8101fa0 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.885+0000 7f89bd400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b810c100 con 0x7f89b81015e0 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 0x7f89b81976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 0x7f89b81976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46930/0 (socket says 172.21.15.137:46930) 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 -- 172.21.15.137:0/1064442013 learned_addr learned my addr 172.21.15.137:0/1064442013 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89bca00640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b8101fa0 0x7f89b819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89af400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81015e0 0x7f89b8197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 -- 172.21.15.137:0/1064442013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81015e0 msgr2=0x7f89b8197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81015e0 0x7f89b8197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 -- 172.21.15.137:0/1064442013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b8101fa0 msgr2=0x7f89b819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b8101fa0 0x7f89b819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 -- 172.21.15.137:0/1064442013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b819f310 con 0x7f89b81003f0 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89bca00640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b8101fa0 0x7f89b819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89af400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81015e0 0x7f89b8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.886+0000 7f89b7e00640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 0x7f89b81976f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f89a800ea80 tx=0x7f89a800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.887+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a800ce40 con 0x7f89b81003f0 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.887+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89a8011920 con 0x7f89b81003f0 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.887+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b819f600 con 0x7f89b81003f0 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.887+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a801aaa0 con 0x7f89b81003f0 2024-03-26T14:12:31.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.887+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b819fb50 con 0x7f89b81003f0 2024-03-26T14:12:31.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.888+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b819fe10 con 0x7f89b81003f0 2024-03-26T14:12:31.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.890+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f89a8011ad0 con 0x7f89b81003f0 2024-03-26T14:12:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.892+0000 7f89b5600640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f899c0a2800 0x7f899c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.892+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f89a80cf4f0 con 0x7f89b81003f0 2024-03-26T14:12:31.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.892+0000 7f89af400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f899c0a2800 0x7f899c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:31.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.893+0000 7f89af400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f899c0a2800 0x7f899c0a4cc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89b8198c00 tx=0x7f89a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:31.897 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:31.897+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f89a8016030 con 0x7f89b81003f0 2024-03-26T14:12:32.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.154+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89b819ffe0 con 0x7f89b81003f0 2024-03-26T14:12:32.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.156+0000 7f89b5600640 1 -- 172.21.15.137:0/1064442013 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f89a809c180 con 0x7f89b81003f0 2024-03-26T14:12:32.157 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:12:32.157 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:12:32.157 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:12:32.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f899c0a2800 msgr2=0x7f899c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f899c0a2800 0x7f899c0a4cc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f89b8198c00 tx=0x7f89a0005f90 comp rx=0 tx=0).stop 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 msgr2=0x7f89b81976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 0x7f89b81976f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f89a800ea80 tx=0x7f89a800ef40 comp rx=0 tx=0).stop 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 shutdown_connections 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f899c0a2800 0x7f899c0a4cc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89b8101fa0 0x7f89b819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f89b81015e0 0x7f89b8197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 --2- 172.21.15.137:0/1064442013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f89b81003f0 0x7f89b81976f0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 >> 172.21.15.137:0/1064442013 conn(0x7f89b80fb980 msgr2=0x7f89b80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 shutdown_connections 2024-03-26T14:12:32.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.159+0000 7f89bd400640 1 -- 172.21.15.137:0/1064442013 wait complete. 2024-03-26T14:12:32.179 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:12:32.179 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:12:32.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 -- 172.21.15.137:0/3134265781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 msgr2=0x7fe340103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 --2- 172.21.15.137:0/3134265781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340103ce0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002bc30 comp rx=0 tx=0).stop 2024-03-26T14:12:32.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 -- 172.21.15.137:0/3134265781 shutdown_connections 2024-03-26T14:12:32.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 --2- 172.21.15.137:0/3134265781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe340105470 0x7fe34010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.408 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 --2- 172.21.15.137:0/3134265781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe340104ab0 0x7fe340104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 --2- 172.21.15.137:0/3134265781 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340103ce0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.407+0000 7fe345000640 1 -- 172.21.15.137:0/3134265781 >> 172.21.15.137:0/3134265781 conn(0x7fe3400fee50 msgr2=0x7fe340101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.408+0000 7fe345000640 1 -- 172.21.15.137:0/3134265781 shutdown_connections 2024-03-26T14:12:32.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.408+0000 7fe345000640 1 -- 172.21.15.137:0/3134265781 wait complete. 2024-03-26T14:12:32.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 Processor -- start 2024-03-26T14:12:32.411 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 -- start start 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe340104ab0 0x7fe340199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe340105470 0x7fe3401a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34010f410 con 0x7fe3401038c0 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34010f290 con 0x7fe340105470 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe345000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34010f590 con 0x7fe340104ab0 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46952/0 (socket says 172.21.15.137:46952) 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.409+0000 7fe33de00640 1 -- 172.21.15.137:0/4216138805 learned_addr learned my addr 172.21.15.137:0/4216138805 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33e800640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe340105470 0x7fe3401a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33d400640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe340104ab0 0x7fe340199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33de00640 1 -- 172.21.15.137:0/4216138805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe340104ab0 msgr2=0x7fe340199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33de00640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe340104ab0 0x7fe340199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33de00640 1 -- 172.21.15.137:0/4216138805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe340105470 msgr2=0x7fe3401a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33de00640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe340105470 0x7fe3401a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33de00640 1 -- 172.21.15.137:0/4216138805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3401a16a0 con 0x7fe3401038c0 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe33de00640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340199a80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe330005ab0 tx=0x7fe3300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe336a00640 1 -- 172.21.15.137:0/4216138805 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330043020 con 0x7fe3401038c0 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe336a00640 1 -- 172.21.15.137:0/4216138805 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe3300054c0 con 0x7fe3401038c0 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3401a1930 con 0x7fe3401038c0 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.410+0000 7fe336a00640 1 -- 172.21.15.137:0/4216138805 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330003d10 con 0x7fe3401038c0 2024-03-26T14:12:32.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.411+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3401a1e80 con 0x7fe3401038c0 2024-03-26T14:12:32.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 Processor -- start 2024-03-26T14:12:32.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 -- start start 2024-03-26T14:12:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d4212810 0x5567d4212c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d4213e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 0x5567d421ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d41772c0 con 0x5567d42143a0 2024-03-26T14:12:32.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d411fc90 con 0x5567d4212810 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.414+0000 7fe347e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d4173360 con 0x5567d42139e0 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe336a00640 1 -- 172.21.15.137:0/4216138805 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe330046020 con 0x7fe3401038c0 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 0x5567d421ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 0x5567d421ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46954/0 (socket says 172.21.15.137:46954) 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 -- 172.21.15.137:0/3441943775 learned_addr learned my addr 172.21.15.137:0/3441943775 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33e800640 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d4212810 0x5567d4212c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33de00640 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d4213e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 -- 172.21.15.137:0/3441943775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 msgr2=0x5567d4213e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d4213e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 -- 172.21.15.137:0/3441943775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d4212810 msgr2=0x5567d4212c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d4212810 0x5567d4212c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33d400640 1 -- 172.21.15.137:0/3441943775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d4144080 con 0x5567d42143a0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.415+0000 7fe33de00640 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d4213e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.416+0000 7fe33d400640 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 0x5567d421ac30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00bb10 tx=0x7fe32c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a665b959a743183e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.416+0000 7fe32b400640 1 -- 172.21.15.137:0/3441943775 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c017420 con 0x5567d42143a0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.416+0000 7fe32b400640 1 -- 172.21.15.137:0/3441943775 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe32c017a70 con 0x5567d42143a0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.416+0000 7fe32b400640 1 -- 172.21.15.137:0/3441943775 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00fa20 con 0x5567d42143a0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.416+0000 7fe336a00640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe3240a2880 0x7fe3240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe336a00640 1 -- 172.21.15.137:0/4216138805 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe3300f1240 con 0x7fe3401038c0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe33e800640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe3240a2880 0x7fe3240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 -- 172.21.15.137:0/3441943775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 msgr2=0x5567d421ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 0x5567d421ac30 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00bb10 tx=0x7fe32c00d340 comp rx=0 tx=0).stop 2024-03-26T14:12:32.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 -- 172.21.15.137:0/3441943775 shutdown_connections 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d42143a0 0x5567d421ac30 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d4213e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 --2- 172.21.15.137:0/3441943775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d4212810 0x5567d4212c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 -- 172.21.15.137:0/3441943775 >> 172.21.15.137:0/3441943775 conn(0x5567d420dc00 msgr2=0x5567d420e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 -- 172.21.15.137:0/3441943775 shutdown_connections 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe347e02740 1 -- 172.21.15.137:0/3441943775 wait complete. 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.417+0000 7fe33e800640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe3240a2880 0x7fe3240a4d40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x5567d42138e0 tx=0x7fe338009740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.418+0000 7fe347e02740 1 -- start start 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.418+0000 7fe347e02740 1 Processor -- start 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.418+0000 7fe347e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.418+0000 7fe347e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d421cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.418+0000 7fe347e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.418+0000 7fe347e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d421b3b0 con 0x5567d4212810 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe347e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d41330d0 con 0x5567d42143a0 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe347e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d421b530 con 0x5567d42139e0 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe347e02740 10 client.0 _ll_register_callbacks cb 0x5567d4137e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46964/0 (socket says 172.21.15.137:46964) 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 -- 172.21.15.137:0/2139707175 learned_addr learned my addr 172.21.15.137:0/2139707175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33e800640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33d400640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d421cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 -- 172.21.15.137:0/2139707175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 msgr2=0x5567d421cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d421cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 -- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 msgr2=0x5567d4223f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 -- 172.21.15.137:0/2139707175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567d3e8b3f0 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.419+0000 7fe33de00640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421c9f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe3300b9c30 tx=0x7fe330000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fe02bdead565c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe31fe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe31fe00640 1 -- 172.21.15.137:0/2139707175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300bd310 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe31fe00640 1 -- 172.21.15.137:0/2139707175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe3300bbad0 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe32c017a70 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe31fe00640 1 -- 172.21.15.137:0/2139707175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3300bd130 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe347e02740 10 client.17772 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5567d3f557d0 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.420+0000 7fe31fe00640 1 -- 172.21.15.137:0/2139707175 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe32c017a70 con 0x5567d4212810 2024-03-26T14:12:32.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe31fe00640 1 -- 172.21.15.137:0/2139707175 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe3300c5ae0 con 0x5567d4212810 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe347e02740 10 client.17772 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe347e02740 10 client.17772 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe347e02740 20 client.17772 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe347e02740 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42284b0 0x5567d422a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe347e02740 4 client.17772 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567d4184800 con 0x5567d42284b0 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.421+0000 7fe33d400640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42284b0 0x5567d422a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.422+0000 7fe33d400640 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42284b0 0x5567d422a920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74568366e641705a server_cookie=f6e1d1b16b98412f in_seq=0 out_seq=0 2024-03-26T14:12:32.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.422+0000 7fe31fe00640 10 client.17772 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:12:32.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe31fe00640 1 -- 172.21.15.137:0/2139707175 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5567d4184800 con 0x5567d42284b0 2024-03-26T14:12:32.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe31fe00640 10 client.17772 handle_command_reply: tid=0 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 client.17772 shutdown 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 msgr2=0x5567d421c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421c9f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe3300b9c30 tx=0x7fe330000bc0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 shutdown_connections 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42284b0 0x5567d422a920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d421cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 --2- 172.21.15.137:0/2139707175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421c9f0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 >> 172.21.15.137:0/2139707175 conn(0x5567d420dc00 msgr2=0x5567d420ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe31fe00640 0 client.17772 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 shutdown_connections 2024-03-26T14:12:32.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 1 -- 172.21.15.137:0/2139707175 wait complete. 2024-03-26T14:12:32.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.424+0000 7fe347e02740 20 client.17772 trim_cache size 0 max 16384 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 Processor -- start 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 -- start start 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42139e0 0x5567d4223f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42143a0 0x5567d4211570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401ae900 con 0x5567d4212810 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401ac400 con 0x5567d42139e0 2024-03-26T14:12:32.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe347e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401aa480 con 0x5567d42143a0 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46978/0 (socket says 172.21.15.137:46978) 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe33de00640 1 -- 172.21.15.137:0/1841872466 learned_addr learned my addr 172.21.15.137:0/1841872466 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.442+0000 7fe33d400640 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42139e0 0x5567d4223f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33e800640 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42143a0 0x5567d4211570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33de00640 1 -- 172.21.15.137:0/1841872466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42143a0 msgr2=0x5567d4211570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33de00640 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42143a0 0x5567d4211570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33de00640 1 -- 172.21.15.137:0/1841872466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42139e0 msgr2=0x5567d4223f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33de00640 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42139e0 0x5567d4223f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33de00640 1 -- 172.21.15.137:0/1841872466 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d422ac90 con 0x5567d4212810 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33e800640 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42143a0 0x5567d4211570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe33de00640 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421ce90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe3300ba1a0 tx=0x7fe3300f11b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0bab74aa1e45c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe32aa00640 1 -- 172.21.15.137:0/1841872466 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300bd310 con 0x5567d4212810 2024-03-26T14:12:32.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe32aa00640 1 -- 172.21.15.137:0/1841872466 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe330004c50 con 0x5567d4212810 2024-03-26T14:12:32.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.443+0000 7fe32aa00640 1 -- 172.21.15.137:0/1841872466 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300bbad0 con 0x5567d4212810 2024-03-26T14:12:32.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 -- 172.21.15.137:0/1841872466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 msgr2=0x5567d421ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421ce90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fe3300ba1a0 tx=0x7fe3300f11b0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 -- 172.21.15.137:0/1841872466 shutdown_connections 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42139e0 0x5567d4223f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42143a0 0x5567d4211570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 --2- 172.21.15.137:0/1841872466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d421ce90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 -- 172.21.15.137:0/1841872466 >> 172.21.15.137:0/1841872466 conn(0x5567d420dc00 msgr2=0x5567d4210e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 -- 172.21.15.137:0/1841872466 shutdown_connections 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.444+0000 7fe347e02740 1 -- 172.21.15.137:0/1841872466 wait complete. 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 -- start start 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 Processor -- start 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d4221d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d42222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401aaa80 con 0x5567d4212810 2024-03-26T14:12:32.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401abda0 con 0x5567d42143a0 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3401ab0e0 con 0x5567d42139e0 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.445+0000 7fe347e02740 10 client.0 _ll_register_callbacks cb 0x7fe330005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d4221d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d4221d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46990/0 (socket says 172.21.15.137:46990) 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 -- 172.21.15.137:0/3394008692 learned_addr learned my addr 172.21.15.137:0/3394008692 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33e800640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33d400640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d42222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 -- 172.21.15.137:0/3394008692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 msgr2=0x5567d42222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d42222d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 -- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 msgr2=0x5567d4223270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33de00640 1 -- 172.21.15.137:0/3394008692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567d4223860 con 0x5567d4212810 2024-03-26T14:12:32.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.446+0000 7fe33e800640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe33de00640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d4221d90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe3300c5a40 tx=0x7fe33002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3e578970d40c3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe31fe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe31fe00640 1 -- 172.21.15.137:0/3394008692 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300bd310 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe31fe00640 1 -- 172.21.15.137:0/3394008692 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe3300d1bb0 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe330004c50 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe31fe00640 1 -- 172.21.15.137:0/3394008692 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3300bded0 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe347e02740 10 client.17778 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5567d4223860 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe31fe00640 1 -- 172.21.15.137:0/3394008692 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe330004c50 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.447+0000 7fe31fe00640 1 -- 172.21.15.137:0/3394008692 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe3300beb00 con 0x5567d4212810 2024-03-26T14:12:32.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe347e02740 10 client.17778 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe347e02740 10 client.17778 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe347e02740 20 client.17778 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe347e02740 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42848c0 0x5567d4286ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe347e02740 4 client.17778 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5567d417eb50 con 0x5567d42848c0 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.448+0000 7fe33d400640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42848c0 0x5567d4286ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe33d400640 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42848c0 0x5567d4286ce0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4770a9104d155e80 server_cookie=f252bbb661fe7e3 in_seq=0 out_seq=0 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe31fe00640 10 client.17778 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe31fe00640 1 -- 172.21.15.137:0/3394008692 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5567d417eb50 con 0x5567d42848c0 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe31fe00640 10 client.17778 handle_command_reply: tid=0 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 client.17778 shutdown 2024-03-26T14:12:32.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 msgr2=0x5567d4221d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d4221d90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe3300c5a40 tx=0x7fe33002bc30 comp rx=0 tx=0).stop 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 shutdown_connections 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5567d42848c0 0x5567d4286ce0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567d42143a0 0x5567d4223270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5567d42139e0 0x5567d42222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 --2- 172.21.15.137:0/3394008692 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5567d4212810 0x5567d4221d90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 >> 172.21.15.137:0/3394008692 conn(0x5567d420dc00 msgr2=0x5567d4210e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.449+0000 7fe31fe00640 0 client.17778 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 shutdown_connections 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe347e02740 1 -- 172.21.15.137:0/3394008692 wait complete. 2024-03-26T14:12:32.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe347e02740 20 client.17778 trim_cache size 0 max 16384 2024-03-26T14:12:32.456 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:12:32.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe3240a2880 msgr2=0x7fe3240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe3240a2880 0x7fe3240a4d40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x5567d42138e0 tx=0x7fe338009740 comp rx=0 tx=0).stop 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 msgr2=0x7fe340199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340199a80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fe330005ab0 tx=0x7fe3300042f0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 shutdown_connections 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe3240a2880 0x7fe3240a4d40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe340105470 0x7fe3401a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe340104ab0 0x7fe340199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.450+0000 7fe345000640 1 --2- 172.21.15.137:0/4216138805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe3401038c0 0x7fe340199a80 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.451+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 >> 172.21.15.137:0/4216138805 conn(0x7fe3400fee50 msgr2=0x7fe340107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.451+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 shutdown_connections 2024-03-26T14:12:32.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.451+0000 7fe345000640 1 -- 172.21.15.137:0/4216138805 wait complete. 2024-03-26T14:12:32.465 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1064442013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:12:32.468 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:12:32.468 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:12:32.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1064442013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:12:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1064442013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:12:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.702+0000 7f25ab200640 1 -- 172.21.15.137:0/3442180785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac1015a0 msgr2=0x7f25ac1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.702+0000 7f25ab200640 1 --2- 172.21.15.137:0/3442180785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac1015a0 0x7f25ac1019c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f259c009900 tx=0x7f259c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:12:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 -- 172.21.15.137:0/3442180785 shutdown_connections 2024-03-26T14:12:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 --2- 172.21.15.137:0/3442180785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25ac102790 0x7f25ac102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 --2- 172.21.15.137:0/3442180785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac103150 0x7f25ac1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 --2- 172.21.15.137:0/3442180785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac1015a0 0x7f25ac1019c0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 -- 172.21.15.137:0/3442180785 >> 172.21.15.137:0/3442180785 conn(0x7f25ac0fcb10 msgr2=0x7f25ac0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 -- 172.21.15.137:0/3442180785 shutdown_connections 2024-03-26T14:12:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.703+0000 7f25ab200640 1 -- 172.21.15.137:0/3442180785 wait complete. 2024-03-26T14:12:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 Processor -- start 2024-03-26T14:12:32.705 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 -- start start 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac1015a0 0x7f25ac1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 0x7f25ac199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25ac103150 0x7f25ac1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10d110 con 0x7f25ac102790 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10cf90 con 0x7f25ac103150 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.704+0000 7f25ab200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac10d290 con 0x7f25ac1015a0 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 0x7f25ac199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 0x7f25ac199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47000/0 (socket says 172.21.15.137:47000) 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 -- 172.21.15.137:0/1175584479 learned_addr learned my addr 172.21.15.137:0/1175584479 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.707 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac1015a0 0x7f25ac1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25aa800640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25ac103150 0x7f25ac1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 -- 172.21.15.137:0/1175584479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac1015a0 msgr2=0x7f25ac1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac1015a0 0x7f25ac1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 -- 172.21.15.137:0/1175584479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25ac103150 msgr2=0x7f25ac1a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25ac103150 0x7f25ac1a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9400640 1 -- 172.21.15.137:0/1175584479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac1a15f0 con 0x7f25ac102790 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.705+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac1015a0 0x7f25ac1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.706+0000 7f25a9400640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 0x7f25ac199f10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f259800d950 tx=0x7f259800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.706+0000 7f25a2a00640 1 -- 172.21.15.137:0/1175584479 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598017070 con 0x7f25ac102790 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.706+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac1a18e0 con 0x7f25ac102790 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.706+0000 7f25a2a00640 1 -- 172.21.15.137:0/1175584479 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2598009c30 con 0x7f25ac102790 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.706+0000 7f25a2a00640 1 -- 172.21.15.137:0/1175584479 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259800f620 con 0x7f25ac102790 2024-03-26T14:12:32.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.706+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac1a1e30 con 0x7f25ac102790 2024-03-26T14:12:32.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.709+0000 7f25b3fc0740 1 Processor -- start 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.709+0000 7f25b3fc0740 1 -- start start 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x559347916e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347918040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347918580 0x55934791ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55934787b2c0 con 0x5593479169f0 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347823c90 con 0x559347917bc0 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559347877360 con 0x559347918580 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25aa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x559347916e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25aa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x559347916e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47016/0 (socket says 172.21.15.137:47016) 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25aa800640 1 -- 172.21.15.137:0/3263036829 learned_addr learned my addr 172.21.15.137:0/3263036829 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25a2a00640 1 -- 172.21.15.137:0/1175584479 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f25980109a0 con 0x7f25ac102790 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.710+0000 7f25a9e00640 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347918040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25aa800640 1 -- 172.21.15.137:0/3263036829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347918580 msgr2=0x55934791ee10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25aa800640 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347918580 0x55934791ee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25aa800640 1 -- 172.21.15.137:0/3263036829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 msgr2=0x559347918040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25aa800640 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347918040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25aa800640 1 -- 172.21.15.137:0/3263036829 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559347848080 con 0x5593479169f0 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25a9e00640 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347918040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f25aa800640 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x559347916e10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009ee0 tx=0x7f25a4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba864e25a4d2d581 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f2597400640 1 -- 172.21.15.137:0/3263036829 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4016070 con 0x5593479169f0 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f2597400640 1 -- 172.21.15.137:0/3263036829 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25a4004d20 con 0x5593479169f0 2024-03-26T14:12:32.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.711+0000 7f2597400640 1 -- 172.21.15.137:0/3263036829 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4012740 con 0x5593479169f0 2024-03-26T14:12:32.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25a2a00640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f25900a2800 0x7f25900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 -- 172.21.15.137:0/3263036829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 msgr2=0x559347916e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x559347916e10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009ee0 tx=0x7f25a4010740 comp rx=0 tx=0).stop 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25a2a00640 1 -- 172.21.15.137:0/1175584479 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f25980ceb30 con 0x7f25ac102790 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f25900a2800 0x7f25900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 -- 172.21.15.137:0/3263036829 shutdown_connections 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347918040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347918580 0x55934791ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/3263036829 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x559347916e10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.712+0000 7f25b3fc0740 1 -- 172.21.15.137:0/3263036829 >> 172.21.15.137:0/3263036829 conn(0x559347911de0 msgr2=0x5593479121e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.713+0000 7f25b3fc0740 1 -- 172.21.15.137:0/3263036829 shutdown_connections 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.713+0000 7f25b3fc0740 1 -- 172.21.15.137:0/3263036829 wait complete. 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.713+0000 7f25b3fc0740 1 -- start start 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.713+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f25900a2800 0x7f25900a4cc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f259c02c240 tx=0x7f259c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.713+0000 7f25b3fc0740 1 Processor -- start 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347920d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 0x559347921240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x559347928210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55934791f590 con 0x559347917bc0 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593478370d0 con 0x559347918580 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55934791f710 con 0x5593479169f0 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25b3fc0740 10 client.0 _ll_register_callbacks cb 0x55934783be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25a9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 0x559347921240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25a9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 0x559347921240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47026/0 (socket says 172.21.15.137:47026) 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25a9400640 1 -- 172.21.15.137:0/904347001 learned_addr learned my addr 172.21.15.137:0/904347001 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.714+0000 7f25aa800640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x559347928210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9e00640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347920d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9400640 1 -- 172.21.15.137:0/904347001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 msgr2=0x559347920d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9400640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347920d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9400640 1 -- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 msgr2=0x559347928210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9400640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x559347928210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9400640 1 -- 172.21.15.137:0/904347001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55934758f3f0 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25a9400640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 0x559347921240 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2598097ad0 tx=0x7f259809ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54b095c4d7dd164 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f258be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f258be00640 1 -- 172.21.15.137:0/904347001 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598020070 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f258be00640 1 -- 172.21.15.137:0/904347001 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f259809a550 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.715+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a4004d20 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f258be00640 1 -- 172.21.15.137:0/904347001 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259809f250 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 10 client.17790 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5593476597d0 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f258be00640 1 -- 172.21.15.137:0/904347001 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25a4004d20 con 0x559347917bc0 2024-03-26T14:12:32.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f258be00640 1 -- 172.21.15.137:0/904347001 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2598010460 con 0x559347917bc0 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 10 client.17790 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 10 client.17790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 20 client.17790 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55934792c710 0x55934792eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 4 client.17790 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.716+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559347888800 con 0x55934792c710 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.717+0000 7f25a9e00640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55934792c710 0x55934792eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.718+0000 7f258be00640 10 client.17790 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:12:32.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.718+0000 7f25a9e00640 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55934792c710 0x55934792eb30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c08097a972caa7cc server_cookie=570662476c245f08 in_seq=0 out_seq=0 2024-03-26T14:12:32.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f258be00640 1 -- 172.21.15.137:0/904347001 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559347888800 con 0x55934792c710 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f258be00640 10 client.17790 handle_command_reply: tid=0 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 client.17790 shutdown 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 msgr2=0x559347921240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 0x559347921240 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2598097ad0 tx=0x7f259809ab60 comp rx=0 tx=0).stop 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 shutdown_connections 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55934792c710 0x55934792eb30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x559347928210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347917bc0 0x559347921240 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/904347001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347920d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 >> 172.21.15.137:0/904347001 conn(0x559347911de0 msgr2=0x559347911160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f258be00640 0 client.17790 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 shutdown_connections 2024-03-26T14:12:32.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.719+0000 7f25b3fc0740 1 -- 172.21.15.137:0/904347001 wait complete. 2024-03-26T14:12:32.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.720+0000 7f25b3fc0740 20 client.17790 trim_cache size 0 max 16384 2024-03-26T14:12:32.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 Processor -- start 2024-03-26T14:12:32.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 -- start start 2024-03-26T14:12:32.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x55934791b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347917bc0 0x5593479281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x55934791b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1ae790 con 0x5593479169f0 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1ac290 con 0x559347918580 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.735+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1aa310 con 0x559347917bc0 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x55934791b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x55934791b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47028/0 (socket says 172.21.15.137:47028) 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 -- 172.21.15.137:0/1144791543 learned_addr learned my addr 172.21.15.137:0/1144791543 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9400640 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347917bc0 0x5593479281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25aa800640 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x55934791b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 -- 172.21.15.137:0/1144791543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347917bc0 msgr2=0x5593479281d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347917bc0 0x5593479281d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 -- 172.21.15.137:0/1144791543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 msgr2=0x55934791b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x55934791b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.736+0000 7f25a9e00640 1 -- 172.21.15.137:0/1144791543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55934791be90 con 0x5593479169f0 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.737+0000 7f25a9e00640 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x55934791b410 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f25ac1025f0 tx=0x7f259c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db1c2a10c152dd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.737+0000 7f2596a00640 1 -- 172.21.15.137:0/1144791543 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c005100 con 0x5593479169f0 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.737+0000 7f2596a00640 1 -- 172.21.15.137:0/1144791543 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f259c058070 con 0x5593479169f0 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.737+0000 7f2596a00640 1 -- 172.21.15.137:0/1144791543 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0035f0 con 0x5593479169f0 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 -- 172.21.15.137:0/1144791543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 msgr2=0x55934791b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x55934791b410 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f25ac1025f0 tx=0x7f259c043000 comp rx=0 tx=0).stop 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 -- 172.21.15.137:0/1144791543 shutdown_connections 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347918580 0x55934791b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559347917bc0 0x5593479281d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/1144791543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5593479169f0 0x55934791b410 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 -- 172.21.15.137:0/1144791543 >> 172.21.15.137:0/1144791543 conn(0x559347911de0 msgr2=0x55934791ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 -- 172.21.15.137:0/1144791543 shutdown_connections 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 -- 172.21.15.137:0/1144791543 wait complete. 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.738+0000 7f25b3fc0740 1 -- start start 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 Processor -- start 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347924300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347924840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 0x559347927860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1aa910 con 0x559347918580 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1abc30 con 0x559347917bc0 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1aaf70 con 0x5593479169f0 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.739+0000 7f25b3fc0740 10 client.0 _ll_register_callbacks cb 0x7f259800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:32.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 0x559347927860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 0x559347927860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47038/0 (socket says 172.21.15.137:47038) 2024-03-26T14:12:32.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 -- 172.21.15.137:0/393963588 learned_addr learned my addr 172.21.15.137:0/393963588 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25a9400640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347924840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25a9e00640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347924300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 -- 172.21.15.137:0/393963588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 msgr2=0x559347924300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347924300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 -- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 msgr2=0x559347924840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347924840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25aa800640 1 -- 172.21.15.137:0/393963588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559347927e50 con 0x559347918580 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.740+0000 7f25a9e00640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347924300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f25aa800640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 0x559347927860 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009ee0 tx=0x7f25a4009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb7f64aa4370ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f258be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f258be00640 1 -- 172.21.15.137:0/393963588 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4016040 con 0x559347918580 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f258be00640 1 -- 172.21.15.137:0/393963588 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f25a4012450 con 0x559347918580 2024-03-26T14:12:32.745 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f259c058070 con 0x559347918580 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f258be00640 1 -- 172.21.15.137:0/393963588 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a4033050 con 0x559347918580 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f25b3fc0740 10 client.17796 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.741+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559347988a70 con 0x559347918580 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f258be00640 1 -- 172.21.15.137:0/393963588 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f259c058070 con 0x559347918580 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f258be00640 1 -- 172.21.15.137:0/393963588 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f25a400f040 con 0x559347918580 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f25b3fc0740 10 client.17796 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f25b3fc0740 10 client.17796 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f25b3fc0740 20 client.17796 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559347989280 0x55934798b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f25b3fc0740 4 client.17796 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:12:32.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.742+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559347920af0 con 0x559347989280 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.743+0000 7f25a9e00640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559347989280 0x55934798b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.743+0000 7f258be00640 10 client.17796 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.743+0000 7f25a9e00640 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559347989280 0x55934798b6a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee6ec5a09b2a089a server_cookie=e2bedca2b616a931 in_seq=0 out_seq=0 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f258be00640 1 -- 172.21.15.137:0/393963588 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559347920af0 con 0x559347989280 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f258be00640 10 client.17796 handle_command_reply: tid=0 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 client.17796 shutdown 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 msgr2=0x559347927860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 0x559347927860 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009ee0 tx=0x7f25a4009c50 comp rx=0 tx=0).stop 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 shutdown_connections 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559347989280 0x55934798b6a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559347917bc0 0x559347924840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559347918580 0x559347927860 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 --2- 172.21.15.137:0/393963588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593479169f0 0x559347924300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 >> 172.21.15.137:0/393963588 conn(0x559347911de0 msgr2=0x559347933c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f258be00640 0 client.17796 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:12:32.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 shutdown_connections 2024-03-26T14:12:32.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 1 -- 172.21.15.137:0/393963588 wait complete. 2024-03-26T14:12:32.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.744+0000 7f25b3fc0740 20 client.17796 trim_cache size 0 max 16384 2024-03-26T14:12:32.748 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f25900a2800 msgr2=0x7f25900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f25900a2800 0x7f25900a4cc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f259c02c240 tx=0x7f259c02c7e0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 msgr2=0x7f25ac199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 0x7f25ac199f10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f259800d950 tx=0x7f259800de10 comp rx=0 tx=0).stop 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 shutdown_connections 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f25900a2800 0x7f25900a4cc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25ac103150 0x7f25ac1a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25ac102790 0x7f25ac199f10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 --2- 172.21.15.137:0/1175584479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25ac1015a0 0x7f25ac1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.745+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 >> 172.21.15.137:0/1175584479 conn(0x7f25ac0fcb10 msgr2=0x7f25ac107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.746+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 shutdown_connections 2024-03-26T14:12:32.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.746+0000 7f25ab200640 1 -- 172.21.15.137:0/1175584479 wait complete. 2024-03-26T14:12:32.762 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:12:32.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:12:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.979+0000 7f710aa00640 1 -- 172.21.15.137:0/3495611982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104102790 msgr2=0x7f7104102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.979+0000 7f710aa00640 1 --2- 172.21.15.137:0/3495611982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104102790 0x7f7104102c10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009900 tx=0x7f70f002bc30 comp rx=0 tx=0).stop 2024-03-26T14:12:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 -- 172.21.15.137:0/3495611982 shutdown_connections 2024-03-26T14:12:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 --2- 172.21.15.137:0/3495611982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7104103150 0x7f71041099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 --2- 172.21.15.137:0/3495611982 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104102790 0x7f7104102c10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 --2- 172.21.15.137:0/3495611982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71041015a0 0x7f71041019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 -- 172.21.15.137:0/3495611982 >> 172.21.15.137:0/3495611982 conn(0x7f71040fcb10 msgr2=0x7f71040fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 -- 172.21.15.137:0/3495611982 shutdown_connections 2024-03-26T14:12:32.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.980+0000 7f710aa00640 1 -- 172.21.15.137:0/3495611982 wait complete. 2024-03-26T14:12:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 Processor -- start 2024-03-26T14:12:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 -- start start 2024-03-26T14:12:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71041015a0 0x7f71041999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7104102790 0x7f7104199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 0x7f71041a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710410d0b0 con 0x7f7104103150 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710410cf30 con 0x7f71041015a0 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.981+0000 7f710aa00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710410d230 con 0x7f7104102790 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 0x7f71041a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 0x7f71041a0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47066/0 (socket says 172.21.15.137:47066) 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f7109600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71041015a0 0x7f71041999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 -- 172.21.15.137:0/3995747285 learned_addr learned my addr 172.21.15.137:0/3995747285 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f7108c00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7104102790 0x7f7104199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 -- 172.21.15.137:0/3995747285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7104102790 msgr2=0x7f7104199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7104102790 0x7f7104199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 -- 172.21.15.137:0/3995747285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71041015a0 msgr2=0x7f71041999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71041015a0 0x7f71041999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 -- 172.21.15.137:0/3995747285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71041a15c0 con 0x7f7104103150 2024-03-26T14:12:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.982+0000 7f710a000640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 0x7f71041a0f50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00eaa0 tx=0x7f70fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.983+0000 7f7102000640 1 -- 172.21.15.137:0/3995747285 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00cdb0 con 0x7f7104103150 2024-03-26T14:12:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.983+0000 7f7102000640 1 -- 172.21.15.137:0/3995747285 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70fc005020 con 0x7f7104103150 2024-03-26T14:12:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.983+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71041a18b0 con 0x7f7104103150 2024-03-26T14:12:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.983+0000 7f7102000640 1 -- 172.21.15.137:0/3995747285 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0116d0 con 0x7f7104103150 2024-03-26T14:12:32.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.983+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71041a1db0 con 0x7f7104103150 2024-03-26T14:12:32.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.986+0000 7f710ed9c740 1 Processor -- start 2024-03-26T14:12:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f7102000640 1 -- 172.21.15.137:0/3995747285 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f70fc0051d0 con 0x7f7104103150 2024-03-26T14:12:32.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 -- start start 2024-03-26T14:12:32.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721ca9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cafe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557721c0c2c0 con 0x557721ca9580 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557721bb4c90 con 0x557721ca79f0 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.987+0000 7f710ed9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557721c08360 con 0x557721ca8bc0 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cafe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cafe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47072/0 (socket says 172.21.15.137:47072) 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 -- 172.21.15.137:0/2927438454 learned_addr learned my addr 172.21.15.137:0/2927438454 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7109600640 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721ca9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 -- 172.21.15.137:0/2927438454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 msgr2=0x557721ca9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721ca9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 -- 172.21.15.137:0/2927438454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 msgr2=0x557721ca7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.988+0000 7f7108c00640 1 -- 172.21.15.137:0/2927438454 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557721bd9080 con 0x557721ca9580 2024-03-26T14:12:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f7102000640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70ec0a2800 0x7f70ec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f7108c00640 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cafe10 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x557721ca8ac0 tx=0x7f70f00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ffd87e89c00474 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f7102000640 1 -- 172.21.15.137:0/3995747285 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f70fc0cf080 con 0x7f7104103150 2024-03-26T14:12:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f7109600640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70ec0a2800 0x7f70ec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f70f6600640 1 -- 172.21.15.137:0/2927438454 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0049070 con 0x557721ca9580 2024-03-26T14:12:32.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f70f6600640 1 -- 172.21.15.137:0/2927438454 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70f0005500 con 0x557721ca9580 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.989+0000 7f70f6600640 1 -- 172.21.15.137:0/2927438454 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00049b0 con 0x557721ca9580 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 -- 172.21.15.137:0/2927438454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 msgr2=0x557721cafe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f7109600640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70ec0a2800 0x7f70ec0a4cc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71041025f0 tx=0x7f70f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cafe10 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x557721ca8ac0 tx=0x7f70f00058b0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 -- 172.21.15.137:0/2927438454 shutdown_connections 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cafe10 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721ca9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2927438454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.990+0000 7f710ed9c740 1 -- 172.21.15.137:0/2927438454 >> 172.21.15.137:0/2927438454 conn(0x557721ca2de0 msgr2=0x557721ca31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.991+0000 7f710ed9c740 1 -- 172.21.15.137:0/2927438454 shutdown_connections 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.991+0000 7f710ed9c740 1 -- 172.21.15.137:0/2927438454 wait complete. 2024-03-26T14:12:32.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.991+0000 7f710ed9c740 1 -- start start 2024-03-26T14:12:32.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.991+0000 7f710ed9c740 1 Processor -- start 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x557721cb0530 con 0x557721ca9580 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x557721bc80d0 con 0x557721ca79f0 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x557721cb06b0 con 0x557721ca8bc0 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710ed9c740 10 client.0 _ll_register_callbacks cb 0x557721bcce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47078/0 (socket says 172.21.15.137:47078) 2024-03-26T14:12:32.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f710a000640 1 -- 172.21.15.137:0/2367302141 learned_addr learned my addr 172.21.15.137:0/2367302141 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f7109600640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.992+0000 7f7108c00640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710a000640 1 -- 172.21.15.137:0/2367302141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 msgr2=0x557721cb21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710a000640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710a000640 1 -- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 msgr2=0x557721cb1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710a000640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710a000640 1 -- 172.21.15.137:0/2367302141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577219203f0 con 0x557721ca9580 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710a000640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb9220 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f70fc09b5a0 tx=0x7f70fc086110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5bb50e802b1d35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f70eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f70eaa00640 1 -- 172.21.15.137:0/2367302141 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0b4060 con 0x557721ca9580 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f70eaa00640 1 -- 172.21.15.137:0/2367302141 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f70fc084ba0 con 0x557721ca9580 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f0005500 con 0x557721ca9580 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.993+0000 7f70eaa00640 1 -- 172.21.15.137:0/2367302141 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70fc01f040 con 0x557721ca9580 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 10 client.17808 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:32.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5577219ea7d0 con 0x557721ca9580 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f70eaa00640 1 -- 172.21.15.137:0/2367302141 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70f0005500 con 0x557721ca9580 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f70eaa00640 1 -- 172.21.15.137:0/2367302141 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f70fc00f040 con 0x557721ca9580 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 10 client.17808 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 20 client.17808 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721cbd780 0x557721cbfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 4 client.17808 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.994+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557721c19800 con 0x557721cbd780 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.995+0000 7f7109600640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721cbd780 0x557721cbfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.996+0000 7f7109600640 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721cbd780 0x557721cbfbc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c442a5e8d64000e server_cookie=4e0597e96a5bbca0 in_seq=0 out_seq=0 2024-03-26T14:12:32.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.996+0000 7f70eaa00640 10 client.17808 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:32.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.997+0000 7f70eaa00640 1 -- 172.21.15.137:0/2367302141 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557721c19800 con 0x557721cbd780 2024-03-26T14:12:32.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.997+0000 7f70eaa00640 10 client.17808 handle_command_reply: tid=0 2024-03-26T14:12:32.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.997+0000 7f710ed9c740 1 client.17808 shutdown 2024-03-26T14:12:32.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 msgr2=0x557721cb9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb9220 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f70fc09b5a0 tx=0x7f70fc086110 comp rx=0 tx=0).stop 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 shutdown_connections 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721cbd780 0x557721cbfbc0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb9220 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 --2- 172.21.15.137:0/2367302141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 >> 172.21.15.137:0/2367302141 conn(0x557721ca2de0 msgr2=0x557721ca2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f70eaa00640 0 client.17808 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 shutdown_connections 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 1 -- 172.21.15.137:0/2367302141 wait complete. 2024-03-26T14:12:33.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:32.998+0000 7f710ed9c740 20 client.17808 trim_cache size 0 max 16384 2024-03-26T14:12:33.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.015+0000 7f710ed9c740 1 Processor -- start 2024-03-26T14:12:33.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 -- start start 2024-03-26T14:12:33.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721ca1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71041ae720 con 0x557721ca9580 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71041ac220 con 0x557721ca79f0 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710ed9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71041aa2a0 con 0x557721ca8bc0 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721ca1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721ca1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47082/0 (socket says 172.21.15.137:47082) 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f710a000640 1 -- 172.21.15.137:0/3161281344 learned_addr learned my addr 172.21.15.137:0/3161281344 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f7109600640 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.016+0000 7f7108c00640 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f710a000640 1 -- 172.21.15.137:0/3161281344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 msgr2=0x557721cb91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:33.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f710a000640 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f710a000640 1 -- 172.21.15.137:0/3161281344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 msgr2=0x557721ca18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f710a000640 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f710a000640 1 -- 172.21.15.137:0/3161281344 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557721cbd560 con 0x557721ca9580 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f710a000640 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721ca1e10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0973f0 tx=0x7f70fc099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20be0e8ab861b86b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f70f5c00640 1 -- 172.21.15.137:0/3161281344 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0b4040 con 0x557721ca9580 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f70f5c00640 1 -- 172.21.15.137:0/3161281344 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70fc097040 con 0x557721ca9580 2024-03-26T14:12:33.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.017+0000 7f70f5c00640 1 -- 172.21.15.137:0/3161281344 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc084ba0 con 0x557721ca9580 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 -- 172.21.15.137:0/3161281344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 msgr2=0x557721ca1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721ca1e10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0973f0 tx=0x7f70fc099b20 comp rx=0 tx=0).stop 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 -- 172.21.15.137:0/3161281344 shutdown_connections 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721ca1e10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 --2- 172.21.15.137:0/3161281344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721ca18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 -- 172.21.15.137:0/3161281344 >> 172.21.15.137:0/3161281344 conn(0x557721ca2de0 msgr2=0x557721ca31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 -- 172.21.15.137:0/3161281344 shutdown_connections 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.018+0000 7f710ed9c740 1 -- 172.21.15.137:0/3161281344 wait complete. 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 -- start start 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 Processor -- start 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71041aa8a0 con 0x557721ca9580 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71041abbc0 con 0x557721ca79f0 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71041aaf00 con 0x557721ca8bc0 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.019+0000 7f710ed9c740 10 client.0 _ll_register_callbacks cb 0x7f70fc0971f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47086/0 (socket says 172.21.15.137:47086) 2024-03-26T14:12:33.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 -- 172.21.15.137:0/318230960 learned_addr learned my addr 172.21.15.137:0/318230960 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f7108c00640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f7109600640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 -- 172.21.15.137:0/318230960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 msgr2=0x557721cb5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 -- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 msgr2=0x557721cb4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 -- 172.21.15.137:0/318230960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557721cb88a0 con 0x557721ca9580 2024-03-26T14:12:33.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.020+0000 7f710a000640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb82b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0998c0 tx=0x7f70fc09be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4e7c0582290694 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:33.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f70eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:33.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f70eaa00640 1 -- 172.21.15.137:0/318230960 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0a9060 con 0x557721ca9580 2024-03-26T14:12:33.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f70eaa00640 1 -- 172.21.15.137:0/318230960 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f70fc084da0 con 0x557721ca9580 2024-03-26T14:12:33.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70fc097040 con 0x557721ca9580 2024-03-26T14:12:33.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f70eaa00640 1 -- 172.21.15.137:0/318230960 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70fc0978b0 con 0x557721ca9580 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f710ed9c740 10 client.17814 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557721cb88a0 con 0x557721ca9580 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f70eaa00640 1 -- 172.21.15.137:0/318230960 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f70fc097040 con 0x557721ca9580 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.021+0000 7f70eaa00640 1 -- 172.21.15.137:0/318230960 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f70fc082070 con 0x557721ca9580 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f710ed9c740 10 client.17814 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f710ed9c740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f710ed9c740 20 client.17814 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f710ed9c740 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721d19a10 0x557721d1be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f710ed9c740 4 client.17814 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557721c13b50 con 0x557721d19a10 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.022+0000 7f7109600640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721d19a10 0x557721d1be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f7109600640 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721d19a10 0x557721d1be30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ce228f01456dd7 server_cookie=af41f6176a01ea41 in_seq=0 out_seq=0 2024-03-26T14:12:33.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f70eaa00640 10 client.17814 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f70eaa00640 1 -- 172.21.15.137:0/318230960 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557721c13b50 con 0x557721d19a10 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f70eaa00640 10 client.17814 handle_command_reply: tid=0 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 client.17814 shutdown 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 msgr2=0x557721cb82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb82b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f70fc0998c0 tx=0x7f70fc09be40 comp rx=0 tx=0).stop 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 shutdown_connections 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x557721d19a10 0x557721d1be30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x557721ca9580 0x557721cb82b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557721ca8bc0 0x557721cb5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f70eaa00640 0 client.17814 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 --2- 172.21.15.137:0/318230960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557721ca79f0 0x557721cb4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 >> 172.21.15.137:0/318230960 conn(0x557721ca2de0 msgr2=0x557721ca6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 shutdown_connections 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.023+0000 7f710ed9c740 1 -- 172.21.15.137:0/318230960 wait complete. 2024-03-26T14:12:33.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710ed9c740 20 client.17814 trim_cache size 0 max 16384 2024-03-26T14:12:33.030 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70ec0a2800 msgr2=0x7f70ec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70ec0a2800 0x7f70ec0a4cc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71041025f0 tx=0x7f70f8005f90 comp rx=0 tx=0).stop 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 msgr2=0x7f71041a0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 0x7f71041a0f50 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00eaa0 tx=0x7f70fc00ef60 comp rx=0 tx=0).stop 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 shutdown_connections 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f70ec0a2800 0x7f70ec0a4cc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7104103150 0x7f71041a0f50 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7104102790 0x7f7104199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 --2- 172.21.15.137:0/3995747285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71041015a0 0x7f71041999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.024+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 >> 172.21.15.137:0/3995747285 conn(0x7f71040fcb10 msgr2=0x7f71040738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:33.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.025+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 shutdown_connections 2024-03-26T14:12:33.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:33.025+0000 7f710aa00640 1 -- 172.21.15.137:0/3995747285 wait complete. 2024-03-26T14:12:33.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:33 smithi179 ceph-mon[35472]: pgmap v763: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:12:33.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:33 smithi159 ceph-mon[35501]: pgmap v763: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:12:33.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:33 smithi137 ceph-mon[31539]: pgmap v763: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:12:34.047 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:12:34.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:12:34.048 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:12:34.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 -- 172.21.15.137:0/2017895099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41072e0 msgr2=0x7f0ff41096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2017895099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41072e0 0x7f0ff41096d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009be0 tx=0x7f0fec02bef0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 -- 172.21.15.137:0/2017895099 shutdown_connections 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2017895099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff41049b0 0x7f0ff4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2017895099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41072e0 0x7f0ff41096d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2017895099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ff4102080 0x7f0ff4104470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 -- 172.21.15.137:0/2017895099 >> 172.21.15.137:0/2017895099 conn(0x7f0ff40fba10 msgr2=0x7f0ff40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 -- 172.21.15.137:0/2017895099 shutdown_connections 2024-03-26T14:12:34.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.262+0000 7f0ffac00640 1 -- 172.21.15.137:0/2017895099 wait complete. 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.263+0000 7f0ffac00640 1 Processor -- start 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.263+0000 7f0ffac00640 1 -- start start 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ff4102080 0x7f0ff41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffac00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 0x7f0ff4199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff41072e0 0x7f0ff41a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffac00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff410d110 con 0x7f0ff41049b0 2024-03-26T14:12:34.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff410cf90 con 0x7f0ff41072e0 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff410d290 con 0x7f0ff4102080 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 0x7f0ff4199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 0x7f0ff4199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47102/0 (socket says 172.21.15.137:47102) 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2461953040 learned_addr learned my addr 172.21.15.137:0/2461953040 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ff4102080 0x7f0ff41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.264+0000 7f0ffa200640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff41072e0 0x7f0ff41a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2461953040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ff4102080 msgr2=0x7f0ff41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ff4102080 0x7f0ff41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2461953040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff41072e0 msgr2=0x7f0ff41a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff41072e0 0x7f0ff41a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2461953040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff41a15c0 con 0x7f0ff41049b0 2024-03-26T14:12:34.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 0x7f0ff4199ee0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000e9a0 tx=0x7f0fe000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff2000640 1 -- 172.21.15.137:0/2461953040 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe000cd80 con 0x7f0ff41049b0 2024-03-26T14:12:34.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff2000640 1 -- 172.21.15.137:0/2461953040 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0fe00035f0 con 0x7f0ff41049b0 2024-03-26T14:12:34.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff41a18b0 con 0x7f0ff41049b0 2024-03-26T14:12:34.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.265+0000 7f0ff2000640 1 -- 172.21.15.137:0/2461953040 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0010690 con 0x7f0ff41049b0 2024-03-26T14:12:34.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.266+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41a1e50 con 0x7f0ff41049b0 2024-03-26T14:12:34.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.270+0000 7f0ff2000640 1 -- 172.21.15.137:0/2461953040 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0fe00037a0 con 0x7f0ff41049b0 2024-03-26T14:12:34.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.270+0000 7f0ffefdc740 1 Processor -- start 2024-03-26T14:12:34.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 -- start start 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 0x562865f996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f99ca0 0x562865f9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f9ee10 0x562865fa12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865efeeb0 con 0x562865f99ca0 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865ea7a60 con 0x562865f992b0 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865efaf50 con 0x562865f9ee10 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffa200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 0x562865f996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffa200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 0x562865f996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37866/0 (socket says 172.21.15.137:37866) 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.271+0000 7f0ffa200640 1 -- 172.21.15.137:0/3702357498 learned_addr learned my addr 172.21.15.137:0/3702357498 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ff9800640 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f99ca0 0x562865f9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f9ee10 0x562865fa12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ffa200640 1 -- 172.21.15.137:0/3702357498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f9ee10 msgr2=0x562865fa12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ffa200640 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f9ee10 0x562865fa12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ffa200640 1 -- 172.21.15.137:0/3702357498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f99ca0 msgr2=0x562865f9e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ffa200640 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f99ca0 0x562865f9e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ffa200640 1 -- 172.21.15.137:0/3702357498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562865cdb7e0 con 0x562865f992b0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.272+0000 7f0ff2000640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.273+0000 7f0ff2000640 1 -- 172.21.15.137:0/2461953040 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0fe00cf040 con 0x7f0ff41049b0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.273+0000 7f0ffa200640 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 0x562865f996d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fec02c500 tx=0x7f0fec009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ccf73c7bc6c652 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.273+0000 7f0ff9800640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.273+0000 7f0fe6a00640 1 -- 172.21.15.137:0/3702357498 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec043030 con 0x562865f992b0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.273+0000 7f0fe6a00640 1 -- 172.21.15.137:0/3702357498 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0fec033510 con 0x562865f992b0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.273+0000 7f0fe6a00640 1 -- 172.21.15.137:0/3702357498 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003e20 con 0x562865f992b0 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 -- 172.21.15.137:0/3702357498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 msgr2=0x562865f996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ff9800640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x562865f9a0a0 tx=0x7f0fe8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 0x562865f996d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fec02c500 tx=0x7f0fec009840 comp rx=0 tx=0).stop 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 -- 172.21.15.137:0/3702357498 shutdown_connections 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f9ee10 0x562865fa12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f99ca0 0x562865f9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/3702357498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f992b0 0x562865f996d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 -- 172.21.15.137:0/3702357498 >> 172.21.15.137:0/3702357498 conn(0x562865f94820 msgr2=0x562865f94c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 -- 172.21.15.137:0/3702357498 shutdown_connections 2024-03-26T14:12:34.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.274+0000 7f0ffefdc740 1 -- 172.21.15.137:0/3702357498 wait complete. 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.275+0000 7f0ffefdc740 1 -- start start 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.275+0000 7f0ffefdc740 1 Processor -- start 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865fa3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865fa3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865faa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865fa1d60 con 0x562865f992b0 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865ebaeb0 con 0x562865f9ee10 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562865fa1ee0 con 0x562865f99ca0 2024-03-26T14:12:34.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffefdc740 10 client.0 _ll_register_callbacks cb 0x562865eaf150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865fa3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865fa3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865fa3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47114/0 (socket says 172.21.15.137:47114) 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865fa3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:50388/0 (socket says 172.21.15.137:50388) 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ff9800640 1 -- 172.21.15.137:0/4012035966 learned_addr learned my addr 172.21.15.137:0/4012035966 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.276+0000 7f0ffa200640 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865faa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0ff9800640 1 -- 172.21.15.137:0/4012035966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 msgr2=0x562865fa3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0ff9800640 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865fa3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0ff9800640 1 -- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 msgr2=0x562865faa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0ff9800640 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865faa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0ff9800640 1 -- 172.21.15.137:0/4012035966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562865d3bf70 con 0x562865f992b0 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0ff9800640 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865fa3450 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0fe801e9d0 tx=0x7f0fe8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d9435fa648fc83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.277+0000 7f0fdb400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:12:34.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.278+0000 7f0fdb400640 1 -- 172.21.15.137:0/4012035966 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe802e060 con 0x562865f992b0 2024-03-26T14:12:34.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.278+0000 7f0fdb400640 1 -- 172.21.15.137:0/4012035966 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0fe8018030 con 0x562865f992b0 2024-03-26T14:12:34.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.278+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fec033510 con 0x562865f992b0 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.278+0000 7f0fdb400640 1 -- 172.21.15.137:0/4012035966 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe8035050 con 0x562865f992b0 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.278+0000 7f0ffefdc740 10 client.17826 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.278+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562865bd3170 con 0x562865f992b0 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0fdb400640 1 -- 172.21.15.137:0/4012035966 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0fec033510 con 0x562865f992b0 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0fdb400640 1 -- 172.21.15.137:0/4012035966 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0fe800f450 con 0x562865f992b0 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0ffefdc740 10 client.17826 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:34.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0ffefdc740 10 client.17826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0ffefdc740 20 client.17826 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562865faf340 0x562865fb17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0ffefdc740 4 client.17826 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.279+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562865f0c3f0 con 0x562865faf340 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.280+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562865faf340 0x562865fb17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.280+0000 7f0fdb400640 10 client.17826 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.280+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562865faf340 0x562865fb17c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3de32c3d23709e server_cookie=ad0f3fb354040318 in_seq=0 out_seq=0 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0fdb400640 1 -- 172.21.15.137:0/4012035966 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562865f0c3f0 con 0x562865faf340 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0fdb400640 10 client.17826 handle_command_reply: tid=0 2024-03-26T14:12:34.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 client.17826 shutdown 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 msgr2=0x562865fa3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865fa3450 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0fe801e9d0 tx=0x7f0fe8004730 comp rx=0 tx=0).stop 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 shutdown_connections 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562865faf340 0x562865fb17c0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865faa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865fa3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/4012035966 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865fa3450 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0fdb400640 0 client.17826 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.281+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 >> 172.21.15.137:0/4012035966 conn(0x562865f94820 msgr2=0x562865f93a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.282+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 shutdown_connections 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.282+0000 7f0ffefdc740 1 -- 172.21.15.137:0/4012035966 wait complete. 2024-03-26T14:12:34.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.282+0000 7f0ffefdc740 20 client.17826 trim_cache size 0 max 16384 2024-03-26T14:12:34.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.308+0000 7f0ffefdc740 1 Processor -- start 2024-03-26T14:12:34.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.308+0000 7f0ffefdc740 1 -- start start 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865f93900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865faa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865f9a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41ae6b0 con 0x562865f992b0 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41ac1b0 con 0x562865f9ee10 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41aa230 con 0x562865f99ca0 2024-03-26T14:12:34.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865f93900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865faa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865f93900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47130/0 (socket says 172.21.15.137:47130) 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.309+0000 7f0ff9800640 1 -- 172.21.15.137:0/2299969257 learned_addr learned my addr 172.21.15.137:0/2299969257 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ffa200640 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865f9a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2299969257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 msgr2=0x562865f9a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865f9a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2299969257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 msgr2=0x562865f93900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865f93900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2299969257 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562865f9eb50 con 0x562865f99ca0 2024-03-26T14:12:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff9800640 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865f93900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.310+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865faa990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fe009b5a0 tx=0x7f0fe00d14d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e5ecbf329fd8ba server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.311+0000 7f0fe6000640 1 -- 172.21.15.137:0/2299969257 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe009af70 con 0x562865f99ca0 2024-03-26T14:12:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.311+0000 7f0fe6000640 1 -- 172.21.15.137:0/2299969257 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0fe0097040 con 0x562865f99ca0 2024-03-26T14:12:34.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.311+0000 7f0fe6000640 1 -- 172.21.15.137:0/2299969257 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00a3db0 con 0x562865f99ca0 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2299969257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 msgr2=0x562865faa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865faa990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0fe009b5a0 tx=0x7f0fe00d14d0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2299969257 shutdown_connections 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f9ee10 0x562865f9a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f99ca0 0x562865faa990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2299969257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f992b0 0x562865f93900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.312+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2299969257 >> 172.21.15.137:0/2299969257 conn(0x562865f94820 msgr2=0x562865f931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.313+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2299969257 shutdown_connections 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.313+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2299969257 wait complete. 2024-03-26T14:12:34.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.313+0000 7f0ffefdc740 1 -- start start 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.313+0000 7f0ffefdc740 1 Processor -- start 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 0x562865faa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 0x562865fa8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f9ee10 0x562865fa92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41aa830 con 0x562865f9ee10 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41abb50 con 0x562865f99ca0 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41aae90 con 0x562865f992b0 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffefdc740 10 client.0 _ll_register_callbacks cb 0x7f0fe8009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 0x562865fa8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 0x562865faa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ffa200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f9ee10 0x562865fa92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ff8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 0x562865fa8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37896/0 (socket says 172.21.15.137:37896) 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 0x562865faa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:50396/0 (socket says 172.21.15.137:50396) 2024-03-26T14:12:34.318 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.314+0000 7f0ff8e00640 1 -- 172.21.15.137:0/2019760448 learned_addr learned my addr 172.21.15.137:0/2019760448 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ff9800640 1 -- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 msgr2=0x562865fa8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ff9800640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 0x562865fa8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ff9800640 1 -- 172.21.15.137:0/2019760448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f9ee10 msgr2=0x562865fa92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ff9800640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f9ee10 0x562865fa92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ff9800640 1 -- 172.21.15.137:0/2019760448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562865fa9930 con 0x562865f992b0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ffa200640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f9ee10 0x562865fa92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.315+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 0x562865fa8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.316+0000 7f0ff9800640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 0x562865faa7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x562865f98a60 tx=0x7f0fe8012d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42d0fe4342871380 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.316+0000 7f0fdb400640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.316+0000 7f0fdb400640 1 -- 172.21.15.137:0/2019760448 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe803f070 con 0x562865f992b0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.316+0000 7f0fdb400640 1 -- 172.21.15.137:0/2019760448 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0fe800fa60 con 0x562865f992b0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.316+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fe0097040 con 0x562865f992b0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0fdb400640 1 -- 172.21.15.137:0/2019760448 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe804b050 con 0x562865f992b0 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0ffefdc740 10 client.26653 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:34.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562866014380 con 0x562865f992b0 2024-03-26T14:12:34.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0fdb400640 1 -- 172.21.15.137:0/2019760448 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0fe0097040 con 0x562865f992b0 2024-03-26T14:12:34.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0fdb400640 1 -- 172.21.15.137:0/2019760448 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0fe8044bb0 con 0x562865f992b0 2024-03-26T14:12:34.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0ffefdc740 10 client.26653 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0ffefdc740 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0ffefdc740 20 client.26653 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.317+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562866014c50 0x562866017070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.318+0000 7f0ffefdc740 4 client.26653 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.318+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562865f06740 con 0x562866014c50 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.318+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562866014c50 0x562866017070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.318+0000 7f0ff8e00640 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562866014c50 0x562866017070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d87900f3fe7a9111 server_cookie=cfac91ce568236b4 in_seq=0 out_seq=0 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.318+0000 7f0fdb400640 10 client.26653 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.320+0000 7f0fdb400640 1 -- 172.21.15.137:0/2019760448 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562865f06740 con 0x562866014c50 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.320+0000 7f0fdb400640 10 client.26653 handle_command_reply: tid=0 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.320+0000 7f0ffefdc740 1 client.26653 shutdown 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 msgr2=0x562865faa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 0x562865faa7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x562865f98a60 tx=0x7f0fe8012d70 comp rx=0 tx=0).stop 2024-03-26T14:12:34.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 shutdown_connections 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562866014c50 0x562866017070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562865f99ca0 0x562865fa8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562865f9ee10 0x562865fa92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 --2- 172.21.15.137:0/2019760448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562865f992b0 0x562865faa7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0fdb400640 0 client.26653 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 >> 172.21.15.137:0/2019760448 conn(0x562865f94820 msgr2=0x562865f93a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 shutdown_connections 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 1 -- 172.21.15.137:0/2019760448 wait complete. 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.321+0000 7f0ffefdc740 20 client.26653 trim_cache size 0 max 16384 2024-03-26T14:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "189ff8eb-491a-4187-9a2c-1e48fcf00783", 2024-03-26T14:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:12:34.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0fdc0a2880 msgr2=0x7f0fdc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x562865f9a0a0 tx=0x7f0fe8005f90 comp rx=0 tx=0).stop 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 msgr2=0x7f0ff4199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 0x7f0ff4199ee0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000e9a0 tx=0x7f0fe000ee60 comp rx=0 tx=0).stop 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 shutdown_connections 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f0fdc0a2880 0x7f0fdc0a4d40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff41072e0 0x7f0ff41a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ff41049b0 0x7f0ff4199ee0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 --2- 172.21.15.137:0/2461953040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0ff4102080 0x7f0ff41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 >> 172.21.15.137:0/2461953040 conn(0x7f0ff40fba10 msgr2=0x7f0ff41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 shutdown_connections 2024-03-26T14:12:34.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.322+0000 7f0ffac00640 1 -- 172.21.15.137:0/2461953040 wait complete. 2024-03-26T14:12:34.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:12:34.342 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:12:34.417 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:34 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:12:34.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 -- 172.21.15.137:0/459666867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8105490 msgr2=0x7ff9b810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 --2- 172.21.15.137:0/459666867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8105490 0x7ff9b810bd20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009ca0 tx=0x7ff9b002c100 comp rx=0 tx=0).stop 2024-03-26T14:12:34.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 -- 172.21.15.137:0/459666867 shutdown_connections 2024-03-26T14:12:34.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 --2- 172.21.15.137:0/459666867 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8105490 0x7ff9b810bd20 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 --2- 172.21.15.137:0/459666867 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 --2- 172.21.15.137:0/459666867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b81038e0 0x7ff9b8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 -- 172.21.15.137:0/459666867 >> 172.21.15.137:0/459666867 conn(0x7ff9b80fee50 msgr2=0x7ff9b8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 -- 172.21.15.137:0/459666867 shutdown_connections 2024-03-26T14:12:34.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.560+0000 7ff9bce00640 1 -- 172.21.15.137:0/459666867 wait complete. 2024-03-26T14:12:34.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.561+0000 7ff9bce00640 1 Processor -- start 2024-03-26T14:12:34.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.561+0000 7ff9bce00640 1 -- start start 2024-03-26T14:12:34.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9bce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b81038e0 0x7ff9b8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9bce00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9bce00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b8105490 0x7ff9b81a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9bce00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b810f410 con 0x7ff9b8104ad0 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9bce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b810f290 con 0x7ff9b81038e0 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9bce00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b810f590 con 0x7ff9b8105490 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47148/0 (socket says 172.21.15.137:47148) 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 -- 172.21.15.137:0/3501914144 learned_addr learned my addr 172.21.15.137:0/3501914144 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b81038e0 0x7ff9b8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9b6800640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b8105490 0x7ff9b81a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 -- 172.21.15.137:0/3501914144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b8105490 msgr2=0x7ff9b81a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b8105490 0x7ff9b81a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 -- 172.21.15.137:0/3501914144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b81038e0 msgr2=0x7ff9b8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b81038e0 0x7ff9b8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.562+0000 7ff9afe00640 1 -- 172.21.15.137:0/3501914144 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b81a16b0 con 0x7ff9b8104ad0 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.563+0000 7ff9afe00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8199fd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000e9f0 tx=0x7ff9a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.563+0000 7ff9aea00640 1 -- 172.21.15.137:0/3501914144 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000cd80 con 0x7ff9b8104ad0 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.563+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b81a19a0 con 0x7ff9b8104ad0 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.563+0000 7ff9aea00640 1 -- 172.21.15.137:0/3501914144 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9a00054d0 con 0x7ff9b8104ad0 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.563+0000 7ff9aea00640 1 -- 172.21.15.137:0/3501914144 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0010490 con 0x7ff9b8104ad0 2024-03-26T14:12:34.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.563+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b81a1ef0 con 0x7ff9b8104ad0 2024-03-26T14:12:34.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.567+0000 7ff9aea00640 1 -- 172.21.15.137:0/3501914144 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff9a0005020 con 0x7ff9b8104ad0 2024-03-26T14:12:34.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.567+0000 7ff9bfbd5740 1 Processor -- start 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 -- start start 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a2332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a238370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a2389e0 0x55870a23ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a198250 con 0x55870a233870 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a140a30 con 0x55870a2389e0 2024-03-26T14:12:34.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a1942f0 con 0x55870a232e80 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a238370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9b6800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a2332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9b6800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a2332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:50424/0 (socket says 172.21.15.137:50424) 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a238370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47150/0 (socket says 172.21.15.137:47150) 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9b6800640 1 -- 172.21.15.137:0/137524979 learned_addr learned my addr 172.21.15.137:0/137524979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.568+0000 7ff9afe00640 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a2389e0 0x55870a23ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9b6800640 1 -- 172.21.15.137:0/137524979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a2389e0 msgr2=0x55870a23ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9b6800640 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a2389e0 0x55870a23ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9b6800640 1 -- 172.21.15.137:0/137524979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 msgr2=0x55870a238370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9b6800640 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a238370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9b6800640 1 -- 172.21.15.137:0/137524979 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55870a12b690 con 0x55870a232e80 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9aea00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff99c0a2900 0x7ff99c0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9b6800640 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a2332a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005cc0 tx=0x7ff9b00093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed0efd99324b3f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.569+0000 7ff9aea00640 1 -- 172.21.15.137:0/3501914144 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff9a00cf220 con 0x7ff9b8104ad0 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.570+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff99c0a2900 0x7ff99c0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.570+0000 7ff9a7400640 1 -- 172.21.15.137:0/137524979 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0043020 con 0x55870a232e80 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.570+0000 7ff9a7400640 1 -- 172.21.15.137:0/137524979 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9b0032dc0 con 0x55870a232e80 2024-03-26T14:12:34.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.570+0000 7ff9a7400640 1 -- 172.21.15.137:0/137524979 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0039040 con 0x55870a232e80 2024-03-26T14:12:34.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/137524979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 msgr2=0x55870a2332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff99c0a2900 0x7ff99c0a4dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55870a233c70 tx=0x7ff9a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a2332a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005cc0 tx=0x7ff9b00093e0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/137524979 shutdown_connections 2024-03-26T14:12:34.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a2389e0 0x55870a23ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a238370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/137524979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a2332a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/137524979 >> 172.21.15.137:0/137524979 conn(0x55870a1fb200 msgr2=0x55870a22e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/137524979 shutdown_connections 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/137524979 wait complete. 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.571+0000 7ff9bfbd5740 1 -- start start 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 Processor -- start 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a23ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a233870 0x55870a23d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a2389e0 0x55870a2443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a23b930 con 0x55870a2389e0 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a153e80 con 0x55870a233870 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55870a23bab0 con 0x55870a232e80 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.572+0000 7ff9bfbd5740 10 client.0 _ll_register_callbacks cb 0x55870a166130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a23ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a23ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:50430/0 (socket says 172.21.15.137:50430) 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9b5e00640 1 -- 172.21.15.137:0/734635804 learned_addr learned my addr 172.21.15.137:0/734635804 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9b6800640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a2389e0 0x55870a2443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9afe00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a233870 0x55870a23d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9afe00640 1 -- 172.21.15.137:0/734635804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 msgr2=0x55870a23ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9afe00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a23ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9afe00640 1 -- 172.21.15.137:0/734635804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a2389e0 msgr2=0x55870a2443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9afe00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a2389e0 0x55870a2443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9afe00640 1 -- 172.21.15.137:0/734635804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558709f767d0 con 0x55870a233870 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a23ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.573+0000 7ff9b6800640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a2389e0 0x55870a2443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff9afe00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a233870 0x55870a23d370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0097c10 tx=0x7ff9a0004b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bcbb25ba528bbe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff99be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff99be00640 1 -- 172.21.15.137:0/734635804 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a009b110 con 0x55870a233870 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff99be00640 1 -- 172.21.15.137:0/734635804 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff9a00046c0 con 0x55870a233870 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9b0032dc0 con 0x55870a233870 2024-03-26T14:12:34.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff99be00640 1 -- 172.21.15.137:0/734635804 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a0005310 con 0x55870a233870 2024-03-26T14:12:34.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff9bfbd5740 10 client.37400 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:34.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558709eac3f0 con 0x55870a233870 2024-03-26T14:12:34.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.574+0000 7ff99be00640 1 -- 172.21.15.137:0/734635804 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9b0032dc0 con 0x55870a233870 2024-03-26T14:12:34.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff99be00640 1 -- 172.21.15.137:0/734635804 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff9a0086070 con 0x55870a233870 2024-03-26T14:12:34.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9bfbd5740 10 client.37400 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:34.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9bfbd5740 10 client.37400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:34.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9bfbd5740 20 client.37400 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a248bd0 0x55870a24aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9bfbd5740 4 client.37400 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55870a1a5790 con 0x55870a248bd0 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.575+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a248bd0 0x55870a24aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.576+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a248bd0 0x55870a24aff0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262093f2b1208601 server_cookie=665da4cc8267f9dc in_seq=0 out_seq=0 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.576+0000 7ff99be00640 10 client.37400 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.576+0000 7ff99be00640 1 -- 172.21.15.137:0/734635804 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55870a1a5790 con 0x55870a248bd0 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.576+0000 7ff99be00640 10 client.37400 handle_command_reply: tid=0 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 client.37400 shutdown 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a233870 msgr2=0x55870a23d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a233870 0x55870a23d370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0097c10 tx=0x7ff9a0004b10 comp rx=0 tx=0).stop 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 shutdown_connections 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a248bd0 0x55870a24aff0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a233870 0x55870a23d370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a2389e0 0x55870a2443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff99be00640 0 client.37400 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/734635804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a23ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 >> 172.21.15.137:0/734635804 conn(0x55870a1fb200 msgr2=0x55870a22da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 shutdown_connections 2024-03-26T14:12:34.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/734635804 wait complete. 2024-03-26T14:12:34.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.577+0000 7ff9bfbd5740 20 client.37400 trim_cache size 0 max 16384 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.595+0000 7ff9bfbd5740 1 Processor -- start 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.595+0000 7ff9bfbd5740 1 -- start start 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a1fb200 0x55870a234100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a232e80 0x55870a244370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a234640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81ae8b0 con 0x55870a233870 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81ac3b0 con 0x55870a232e80 2024-03-26T14:12:34.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81aa430 con 0x55870a1fb200 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9b6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a234640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9b6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a234640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47160/0 (socket says 172.21.15.137:47160) 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9b6800640 1 -- 172.21.15.137:0/1016780772 learned_addr learned my addr 172.21.15.137:0/1016780772 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a1fb200 0x55870a234100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.596+0000 7ff9afe00640 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a232e80 0x55870a244370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b6800640 1 -- 172.21.15.137:0/1016780772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a1fb200 msgr2=0x55870a234100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b6800640 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a1fb200 0x55870a234100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b6800640 1 -- 172.21.15.137:0/1016780772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a232e80 msgr2=0x55870a244370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b6800640 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a232e80 0x55870a244370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b6800640 1 -- 172.21.15.137:0/1016780772 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558709e6c0f0 con 0x55870a233870 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a1fb200 0x55870a234100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9b6800640 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a234640 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005cc0 tx=0x7ff9b00324e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9568f7bde74b925 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.597+0000 7ff9a6a00640 1 -- 172.21.15.137:0/1016780772 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0043070 con 0x55870a233870 2024-03-26T14:12:34.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9a6a00640 1 -- 172.21.15.137:0/1016780772 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9b004c070 con 0x55870a233870 2024-03-26T14:12:34.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9a6a00640 1 -- 172.21.15.137:0/1016780772 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0039040 con 0x55870a233870 2024-03-26T14:12:34.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/1016780772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 msgr2=0x55870a234640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a234640 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005cc0 tx=0x7ff9b00324e0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/1016780772 shutdown_connections 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a232e80 0x55870a244370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a234640 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/1016780772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a1fb200 0x55870a234100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.598+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/1016780772 >> 172.21.15.137:0/1016780772 conn(0x55870a23c9f0 msgr2=0x55870a22da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.599+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/1016780772 shutdown_connections 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.599+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/1016780772 wait complete. 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.599+0000 7ff9bfbd5740 1 -- start start 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.599+0000 7ff9bfbd5740 1 Processor -- start 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 0x55870a241c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a242180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a243190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81aaa30 con 0x55870a233870 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81abd50 con 0x55870a1fb200 2024-03-26T14:12:34.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b81ab090 con 0x55870a232e80 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9bfbd5740 10 client.0 _ll_register_callbacks cb 0x7ff9a0010630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 0x55870a241c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9b6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a243190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9b5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 0x55870a241c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37936/0 (socket says 172.21.15.137:37936) 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9b5e00640 1 -- 172.21.15.137:0/2781297746 learned_addr learned my addr 172.21.15.137:0/2781297746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.600+0000 7ff9afe00640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a242180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b5e00640 1 -- 172.21.15.137:0/2781297746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 msgr2=0x55870a242180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a242180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b5e00640 1 -- 172.21.15.137:0/2781297746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 msgr2=0x55870a243190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a243190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b5e00640 1 -- 172.21.15.137:0/2781297746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55870a243780 con 0x55870a1fb200 2024-03-26T14:12:34.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b6800640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a243190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:12:34.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9b5e00640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 0x55870a241c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8004570 tx=0x7ff9a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3ca041af897e4a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff99be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff99be00640 1 -- 172.21.15.137:0/2781297746 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8003c00 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff99be00640 1 -- 172.21.15.137:0/2781297746 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff9a8003da0 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.601+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9b004c070 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff99be00640 1 -- 172.21.15.137:0/2781297746 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a8005020 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 10 client.37406 fetch_fsmap learned FSMap version 24 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55870a2a46d0 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff99be00640 1 -- 172.21.15.137:0/2781297746 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9b004c070 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff99be00640 1 -- 172.21.15.137:0/2781297746 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff9a802b680 con 0x55870a1fb200 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 10 client.37406 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 10 client.37406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 20 client.37406 populate_metadata read hostname 'smithi137' 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a2389e0 0x55870a2a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:12:34.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 4 client.37406 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.602+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55870a19fae0 con 0x55870a2389e0 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.603+0000 7ff9afe00640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a2389e0 0x55870a2a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.603+0000 7ff99be00640 10 client.37406 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.603+0000 7ff9afe00640 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a2389e0 0x55870a2a70f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dde4bb9e302720e server_cookie=6d6de72c5f1250db in_seq=0 out_seq=0 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff99be00640 1 -- 172.21.15.137:0/2781297746 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55870a19fae0 con 0x55870a2389e0 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff99be00640 10 client.37406 handle_command_reply: tid=0 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 client.37406 shutdown 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 msgr2=0x55870a241c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 0x55870a241c40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8004570 tx=0x7ff9a80046c0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 shutdown_connections 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55870a2389e0 0x55870a2a70f0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55870a233870 0x55870a243190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55870a232e80 0x55870a242180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 --2- 172.21.15.137:0/2781297746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55870a1fb200 0x55870a241c40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 >> 172.21.15.137:0/2781297746 conn(0x55870a23c9f0 msgr2=0x55870a231a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff99be00640 0 client.37406 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 shutdown_connections 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.604+0000 7ff9bfbd5740 1 -- 172.21.15.137:0/2781297746 wait complete. 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.605+0000 7ff9bfbd5740 20 client.37406 trim_cache size 0 max 16384 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout: "189ff8eb-491a-4187-9a2c-1e48fcf00783": { 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "189ff8eb-491a-4187-9a2c-1e48fcf00783", 2024-03-26T14:12:34.607 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.605+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff99c0a2900 msgr2=0x7ff99c0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.605+0000 7ff9bce00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff99c0a2900 0x7ff99c0a4dc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55870a233c70 tx=0x7ff9a8005f90 comp rx=0 tx=0).stop 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.605+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 msgr2=0x7ff9b8199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.605+0000 7ff9bce00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8199fd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000e9f0 tx=0x7ff9a000edf0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 shutdown_connections 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff99c0a2900 0x7ff99c0a4dc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b8105490 0x7ff9b81a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff9b8104ad0 0x7ff9b8199fd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 --2- 172.21.15.137:0/3501914144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9b81038e0 0x7ff9b8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 >> 172.21.15.137:0/3501914144 conn(0x7ff9b80fee50 msgr2=0x7ff9b8107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 shutdown_connections 2024-03-26T14:12:34.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:12:34.606+0000 7ff9bce00640 1 -- 172.21.15.137:0/3501914144 wait complete. 2024-03-26T14:12:34.624 INFO:tasks.cephfs.filesystem:scrub status for tag:189ff8eb-491a-4187-9a2c-1e48fcf00783 - {'path': '/', 'tag': '189ff8eb-491a-4187-9a2c-1e48fcf00783', 'options': 'recursive,force'} 2024-03-26T14:12:34.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:34 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:12:34.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:34 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:12:35.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: pgmap v764: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:35.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:12:35.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:35.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:12:35.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:35.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:35.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:35 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:35.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: pgmap v764: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:35.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:12:35.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:35.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:12:35.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:35.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:35.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:35 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: pgmap v764: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:35.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:35 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: pgmap v765: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:37 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: pgmap v765: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:37 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: pgmap v765: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:37 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:39.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:39 smithi179 ceph-mon[35472]: pgmap v766: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:12:39.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:12:39.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:12:39.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:39 smithi159 ceph-mon[35501]: pgmap v766: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:12:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:12:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:12:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:39.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:39 smithi137 ceph-mon[31539]: pgmap v766: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:12:39.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:12:39.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:12:39.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:12:41.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:41 smithi179 ceph-mon[35472]: pgmap v767: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:41 smithi159 ceph-mon[35501]: pgmap v767: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:41.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:41 smithi137 ceph-mon[31539]: pgmap v767: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:43.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:43 smithi179 ceph-mon[35472]: pgmap v768: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:12:43.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:43 smithi159 ceph-mon[35501]: pgmap v768: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:12:43.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:43 smithi137 ceph-mon[31539]: pgmap v768: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:12:44.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:44 smithi179 ceph-mon[35472]: pgmap v769: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:12:44.702 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:44 smithi159 ceph-mon[35501]: pgmap v769: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:12:44.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:44 smithi137 ceph-mon[31539]: pgmap v769: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:12:46.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:46 smithi179 ceph-mon[35472]: pgmap v770: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:46.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:46 smithi159 ceph-mon[35501]: pgmap v770: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:47.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:46 smithi137 ceph-mon[31539]: pgmap v770: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:12:48.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:48 smithi179 ceph-mon[35472]: pgmap v771: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:12:48.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:48 smithi159 ceph-mon[35501]: pgmap v771: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:12:49.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:48 smithi137 ceph-mon[31539]: pgmap v771: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:12:51.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:51 smithi179 ceph-mon[35472]: pgmap v772: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:51.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:51 smithi159 ceph-mon[35501]: pgmap v772: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:51.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:51 smithi137 ceph-mon[31539]: pgmap v772: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:53.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:53 smithi179 ceph-mon[35472]: pgmap v773: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:12:53.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:53 smithi159 ceph-mon[35501]: pgmap v773: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:12:53.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:53 smithi137 ceph-mon[31539]: pgmap v773: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:12:55.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:55 smithi179 ceph-mon[35472]: pgmap v774: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:55.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:55 smithi159 ceph-mon[35501]: pgmap v774: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:55.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:55 smithi137 ceph-mon[31539]: pgmap v774: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:12:57.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:57 smithi179 ceph-mon[35472]: pgmap v775: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:57.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:57 smithi159 ceph-mon[35501]: pgmap v775: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:57.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:57 smithi137 ceph-mon[31539]: pgmap v775: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:12:59.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:12:59 smithi179 ceph-mon[35472]: pgmap v776: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:12:59.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:12:59 smithi159 ceph-mon[35501]: pgmap v776: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:12:59.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:12:59 smithi137 ceph-mon[31539]: pgmap v776: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:01.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:01 smithi179 ceph-mon[35472]: pgmap v777: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:01.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:01 smithi159 ceph-mon[35501]: pgmap v777: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:01.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:01 smithi137 ceph-mon[31539]: pgmap v777: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:03.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:03 smithi179 ceph-mon[35472]: pgmap v778: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:13:03.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:03 smithi159 ceph-mon[35501]: pgmap v778: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:13:03.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:03 smithi137 ceph-mon[31539]: pgmap v778: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:13:04.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:13:04.625 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 -- 172.21.15.137:0/3043758735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4101580 msgr2=0x7fb8e41019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 --2- 172.21.15.137:0/3043758735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4101580 0x7fb8e41019a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009900 tx=0x7fb8d402bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 -- 172.21.15.137:0/3043758735 shutdown_connections 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 --2- 172.21.15.137:0/3043758735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4103130 0x7fb8e41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 --2- 172.21.15.137:0/3043758735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e4102770 0x7fb8e4102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 --2- 172.21.15.137:0/3043758735 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4101580 0x7fb8e41019a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 -- 172.21.15.137:0/3043758735 >> 172.21.15.137:0/3043758735 conn(0x7fb8e40fcb10 msgr2=0x7fb8e40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:04.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 -- 172.21.15.137:0/3043758735 shutdown_connections 2024-03-26T14:13:04.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.866+0000 7fb8e9800640 1 -- 172.21.15.137:0/3043758735 wait complete. 2024-03-26T14:13:04.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.868+0000 7fb8e9800640 1 Processor -- start 2024-03-26T14:13:04.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.868+0000 7fb8e9800640 1 -- start start 2024-03-26T14:13:04.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4101580 0x7fb8e41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e9800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e4102770 0x7fb8e4197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e9800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 0x7fb8e419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e9800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e410d100 con 0x7fb8e4103130 2024-03-26T14:13:04.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e410cf80 con 0x7fb8e4101580 2024-03-26T14:13:04.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e9800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e410d280 con 0x7fb8e4102770 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 0x7fb8e419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 0x7fb8e419ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43244/0 (socket says 172.21.15.137:43244) 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e8e00640 1 -- 172.21.15.137:0/284350712 learned_addr learned my addr 172.21.15.137:0/284350712 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4101580 0x7fb8e41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.869+0000 7fb8e3400640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e4102770 0x7fb8e4197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e8e00640 1 -- 172.21.15.137:0/284350712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e4102770 msgr2=0x7fb8e4197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e4102770 0x7fb8e4197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e8e00640 1 -- 172.21.15.137:0/284350712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4101580 msgr2=0x7fb8e41977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4101580 0x7fb8e41977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e8e00640 1 -- 172.21.15.137:0/284350712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e419f3f0 con 0x7fb8e4103130 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4101580 0x7fb8e41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:04.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 0x7fb8e419ed80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00cce0 tx=0x7fb8dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:04.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e0c00640 1 -- 172.21.15.137:0/284350712 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00f450 con 0x7fb8e4103130 2024-03-26T14:13:04.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e0c00640 1 -- 172.21.15.137:0/284350712 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8dc00faa0 con 0x7fb8e4103130 2024-03-26T14:13:04.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e419f6e0 con 0x7fb8e4103130 2024-03-26T14:13:04.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e0c00640 1 -- 172.21.15.137:0/284350712 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc018b70 con 0x7fb8e4103130 2024-03-26T14:13:04.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.870+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e419fc60 con 0x7fb8e4103130 2024-03-26T14:13:04.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.874+0000 7fb8eda55740 1 Processor -- start 2024-03-26T14:13:04.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.874+0000 7fb8e0c00640 1 -- 172.21.15.137:0/284350712 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb8dc018d10 con 0x7fb8e4103130 2024-03-26T14:13:04.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.874+0000 7fb8eda55740 1 -- start start 2024-03-26T14:13:04.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 0x559efd69a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69aa40 0x559efd69f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6a2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8eda55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efd5ff250 con 0x559efd69a050 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8eda55740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efd5a7a30 con 0x559efd69fbb0 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8eda55740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efd5fb2f0 con 0x559efd69aa40 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 0x559efd69a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 0x559efd69a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43260/0 (socket says 172.21.15.137:43260) 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8e3400640 1 -- 172.21.15.137:0/2423450608 learned_addr learned my addr 172.21.15.137:0/2423450608 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:04.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.875+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69aa40 0x559efd69f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6a2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 -- 172.21.15.137:0/2423450608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69aa40 msgr2=0x559efd69f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69aa40 0x559efd69f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 -- 172.21.15.137:0/2423450608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 msgr2=0x559efd6a2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6a2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 -- 172.21.15.137:0/2423450608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efd592690 con 0x559efd69a050 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e0c00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3400640 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 0x559efd69a470 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f10 tx=0x7fb8d0009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c411af19b9ba037a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e0c00640 1 -- 172.21.15.137:0/284350712 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb8dc0cf340 con 0x7fb8e4103130 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.876+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8d9600640 1 -- 172.21.15.137:0/2423450608 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d001c070 con 0x559efd69a050 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8d9600640 1 -- 172.21.15.137:0/2423450608 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8d0010040 con 0x559efd69a050 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8d9600640 1 -- 172.21.15.137:0/2423450608 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0014730 con 0x559efd69a050 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8eda55740 1 -- 172.21.15.137:0/2423450608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 msgr2=0x559efd69a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8eda55740 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 0x559efd69a470 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009f10 tx=0x7fb8d0009420 comp rx=0 tx=0).stop 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4006010 tx=0x7fb8d4005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.877+0000 7fb8eda55740 1 -- 172.21.15.137:0/2423450608 shutdown_connections 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6a2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69aa40 0x559efd69f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 --2- 172.21.15.137:0/2423450608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69a050 0x559efd69a470 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 -- 172.21.15.137:0/2423450608 >> 172.21.15.137:0/2423450608 conn(0x559efd662200 msgr2=0x559efd6959c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:04.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 -- 172.21.15.137:0/2423450608 shutdown_connections 2024-03-26T14:13:04.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 -- 172.21.15.137:0/2423450608 wait complete. 2024-03-26T14:13:04.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 -- start start 2024-03-26T14:13:04.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.878+0000 7fb8eda55740 1 Processor -- start 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69a050 0x559efd6a8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69fbb0 0x559efd6afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efd6a2b60 con 0x559efd69aa40 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efd5bae80 con 0x559efd69a050 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efd6a2ce0 con 0x559efd69fbb0 2024-03-26T14:13:04.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8eda55740 10 client.0 _ll_register_callbacks cb 0x559efd5cd130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6a8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43262/0 (socket says 172.21.15.137:43262) 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8e3400640 1 -- 172.21.15.137:0/1939423971 learned_addr learned my addr 172.21.15.137:0/1939423971 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.879+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69a050 0x559efd6a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69fbb0 0x559efd6afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3400640 1 -- 172.21.15.137:0/1939423971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69fbb0 msgr2=0x559efd6afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3400640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69fbb0 0x559efd6afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3400640 1 -- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69a050 msgr2=0x559efd6a8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3400640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69a050 0x559efd6a8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3400640 1 -- 172.21.15.137:0/1939423971 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efd3dd7d0 con 0x559efd69aa40 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69a050 0x559efd6a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69fbb0 0x559efd6afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:04.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8e3400640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6a8a60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000f9e0 tx=0x7fb8d0017b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3170b574f7a76cf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8ce000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8ce000640 1 -- 172.21.15.137:0/1939423971 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0026020 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8d0010040 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.880+0000 7fb8ce000640 1 -- 172.21.15.137:0/1939423971 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb8d00146b0 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8ce000640 1 -- 172.21.15.137:0/1939423971 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8d0031050 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8eda55740 10 client.17862 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559efd3133f0 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8ce000640 1 -- 172.21.15.137:0/1939423971 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8d0010040 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8ce000640 1 -- 172.21.15.137:0/1939423971 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb8d00224d0 con 0x559efd69aa40 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8eda55740 10 client.17862 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.881+0000 7fb8eda55740 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.882+0000 7fb8eda55740 20 client.17862 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.882+0000 7fb8eda55740 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd6b3fd0 0x559efd6b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.882+0000 7fb8eda55740 4 client.17862 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:04.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.882+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559efd60c790 con 0x559efd6b3fd0 2024-03-26T14:13:04.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.882+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd6b3fd0 0x559efd6b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.883+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd6b3fd0 0x559efd6b63c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919f4dd59ce32121 server_cookie=cec4ff1a137f7019 in_seq=0 out_seq=0 2024-03-26T14:13:04.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.883+0000 7fb8ce000640 10 client.17862 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:04.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8ce000640 1 -- 172.21.15.137:0/1939423971 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559efd60c790 con 0x559efd6b3fd0 2024-03-26T14:13:04.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8ce000640 10 client.17862 handle_command_reply: tid=0 2024-03-26T14:13:04.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 client.17862 shutdown 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 msgr2=0x559efd6a8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6a8a60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000f9e0 tx=0x7fb8d0017b40 comp rx=0 tx=0).stop 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 shutdown_connections 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd6b3fd0 0x559efd6b63c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69fbb0 0x559efd6afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8ce000640 0 client.17862 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6a8a60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 --2- 172.21.15.137:0/1939423971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69a050 0x559efd6a8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 >> 172.21.15.137:0/1939423971 conn(0x559efd662200 msgr2=0x559efd694bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.885+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 shutdown_connections 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.886+0000 7fb8eda55740 1 -- 172.21.15.137:0/1939423971 wait complete. 2024-03-26T14:13:04.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.886+0000 7fb8eda55740 20 client.17862 trim_cache size 0 max 16384 2024-03-26T14:13:04.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 Processor -- start 2024-03-26T14:13:04.904 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 -- start start 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd694330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69aa40 0x559efd6af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 0x559efd694870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41aa2e0 con 0x559efd69fbb0 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41a7de0 con 0x559efd69aa40 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.903+0000 7fb8eda55740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41a5e60 con 0x559efd69a050 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 0x559efd694870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 0x559efd694870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43272/0 (socket says 172.21.15.137:43272) 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 -- 172.21.15.137:0/361995388 learned_addr learned my addr 172.21.15.137:0/361995388 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e3400640 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69aa40 0x559efd6af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd694330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 -- 172.21.15.137:0/361995388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 msgr2=0x559efd694330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd694330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.905 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 -- 172.21.15.137:0/361995388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69aa40 msgr2=0x559efd6af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69aa40 0x559efd6af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 -- 172.21.15.137:0/361995388 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efd2d30f0 con 0x559efd69fbb0 2024-03-26T14:13:04.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.904+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 0x559efd694870 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0982a0 tx=0x7fb8dc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6484a2c5eda3ccf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:04.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.905+0000 7fb8d8c00640 1 -- 172.21.15.137:0/361995388 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0b4070 con 0x559efd69fbb0 2024-03-26T14:13:04.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.905+0000 7fb8d8c00640 1 -- 172.21.15.137:0/361995388 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8dc09fa20 con 0x559efd69fbb0 2024-03-26T14:13:04.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.905+0000 7fb8d8c00640 1 -- 172.21.15.137:0/361995388 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc084e60 con 0x559efd69fbb0 2024-03-26T14:13:04.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.905+0000 7fb8eda55740 1 -- 172.21.15.137:0/361995388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 msgr2=0x559efd694870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.905+0000 7fb8eda55740 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 0x559efd694870 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0982a0 tx=0x7fb8dc00fd10 comp rx=0 tx=0).stop 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 -- 172.21.15.137:0/361995388 shutdown_connections 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69aa40 0x559efd6af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69fbb0 0x559efd694870 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 --2- 172.21.15.137:0/361995388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd694330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 -- 172.21.15.137:0/361995388 >> 172.21.15.137:0/361995388 conn(0x559efd662200 msgr2=0x559efd699490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 -- 172.21.15.137:0/361995388 shutdown_connections 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 -- 172.21.15.137:0/361995388 wait complete. 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 -- start start 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.906+0000 7fb8eda55740 1 Processor -- start 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd6acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41a6460 con 0x559efd69aa40 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41a7780 con 0x559efd69fbb0 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e41a6ac0 con 0x559efd69a050 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8eda55740 10 client.0 _ll_register_callbacks cb 0x7fb8d001a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43284/0 (socket says 172.21.15.137:43284) 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 -- 172.21.15.137:0/487055410 learned_addr learned my addr 172.21.15.137:0/487055410 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd6acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 -- 172.21.15.137:0/487055410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 msgr2=0x559efd6acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd6acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 -- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 msgr2=0x559efd6ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3400640 1 -- 172.21.15.137:0/487055410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efd6aeb00 con 0x559efd69aa40 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.907+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd6acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8e8e00640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8e3400640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6ad500 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0022920 tx=0x7fb8d0009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a10fd9d3ab3519b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8ce000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8ce000640 1 -- 172.21.15.137:0/487055410 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0026050 con 0x559efd69aa40 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8dc09fa20 con 0x559efd69aa40 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8ce000640 1 -- 172.21.15.137:0/487055410 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb8d0022620 con 0x559efd69aa40 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8ce000640 1 -- 172.21.15.137:0/487055410 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8d001c070 con 0x559efd69aa40 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8eda55740 10 client.17868 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.908+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559efd70f870 con 0x559efd69aa40 2024-03-26T14:13:04.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8ce000640 1 -- 172.21.15.137:0/487055410 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8dc09fa20 con 0x559efd69aa40 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8ce000640 1 -- 172.21.15.137:0/487055410 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb8d0006040 con 0x559efd69aa40 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8eda55740 10 client.17868 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8eda55740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8eda55740 20 client.17868 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8eda55740 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd710130 0x559efd712550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8eda55740 4 client.17868 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:13:04.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.909+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559efd606ae0 con 0x559efd710130 2024-03-26T14:13:04.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.910+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd710130 0x559efd712550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:04.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8e3e00640 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd710130 0x559efd712550 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7e39b574074d17 server_cookie=7bca8977ece5ebe9 in_seq=0 out_seq=0 2024-03-26T14:13:04.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8ce000640 10 client.17868 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:04.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8ce000640 1 -- 172.21.15.137:0/487055410 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559efd606ae0 con 0x559efd710130 2024-03-26T14:13:04.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8ce000640 10 client.17868 handle_command_reply: tid=0 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 client.17868 shutdown 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 msgr2=0x559efd6ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6ad500 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0022920 tx=0x7fb8d0009470 comp rx=0 tx=0).stop 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 shutdown_connections 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x559efd710130 0x559efd712550 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559efd69fbb0 0x559efd6ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559efd69aa40 0x559efd6ad500 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 --2- 172.21.15.137:0/487055410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559efd69a050 0x559efd6acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 >> 172.21.15.137:0/487055410 conn(0x559efd662200 msgr2=0x559efd69b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8ce000640 0 client.17868 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 shutdown_connections 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.911+0000 7fb8eda55740 1 -- 172.21.15.137:0/487055410 wait complete. 2024-03-26T14:13:04.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8eda55740 20 client.17868 trim_cache size 0 max 16384 2024-03-26T14:13:04.916 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:13:04.916 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:13:04.916 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:13:04.916 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:13:04.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8c80a2800 msgr2=0x7fb8c80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4006010 tx=0x7fb8d4005d50 comp rx=0 tx=0).stop 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 msgr2=0x7fb8e419ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 0x7fb8e419ed80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00cce0 tx=0x7fb8dc00a620 comp rx=0 tx=0).stop 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 shutdown_connections 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8c80a2800 0x7fb8c80a4cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8e4103130 0x7fb8e419ed80 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8e4102770 0x7fb8e4197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 --2- 172.21.15.137:0/284350712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8e4101580 0x7fb8e41977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.912+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 >> 172.21.15.137:0/284350712 conn(0x7fb8e40fcb10 msgr2=0x7fb8e4105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.913+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 shutdown_connections 2024-03-26T14:13:04.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:04.913+0000 7fb8e9800640 1 -- 172.21.15.137:0/284350712 wait complete. 2024-03-26T14:13:04.930 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:13:04.930 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 -- 172.21.15.137:0/81103754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414100030 msgr2=0x7fc414100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 --2- 172.21.15.137:0/81103754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414100030 0x7fc414100450 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc404009900 tx=0x7fc40402bcd0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 -- 172.21.15.137:0/81103754 shutdown_connections 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 --2- 172.21.15.137:0/81103754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc414101330 0x7fc4141017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 --2- 172.21.15.137:0/81103754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc414100990 0x7fc414100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 --2- 172.21.15.137:0/81103754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414100030 0x7fc414100450 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.133 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 -- 172.21.15.137:0/81103754 >> 172.21.15.137:0/81103754 conn(0x7fc4140fbbd0 msgr2=0x7fc4140fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 -- 172.21.15.137:0/81103754 shutdown_connections 2024-03-26T14:13:05.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.133+0000 7fc418a00640 1 -- 172.21.15.137:0/81103754 wait complete. 2024-03-26T14:13:05.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.134+0000 7fc418a00640 1 Processor -- start 2024-03-26T14:13:05.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.134+0000 7fc418a00640 1 -- start start 2024-03-26T14:13:05.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc418a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc414100030 0x7fc414199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc418a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc414100990 0x7fc41419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc418a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 0x7fc4141a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc418a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41410f450 con 0x7fc414101330 2024-03-26T14:13:05.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc418a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41410f2d0 con 0x7fc414100030 2024-03-26T14:13:05.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc418a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41410f5d0 con 0x7fc414100990 2024-03-26T14:13:05.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc412800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 0x7fc4141a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc412800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 0x7fc4141a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43310/0 (socket says 172.21.15.137:43310) 2024-03-26T14:13:05.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc412800640 1 -- 172.21.15.137:0/3995904808 learned_addr learned my addr 172.21.15.137:0/3995904808 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc411e00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc414100030 0x7fc414199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.135+0000 7fc411400640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc414100990 0x7fc41419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.136+0000 7fc412800640 1 -- 172.21.15.137:0/3995904808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc414100990 msgr2=0x7fc41419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.136+0000 7fc412800640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc414100990 0x7fc41419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.136+0000 7fc412800640 1 -- 172.21.15.137:0/3995904808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc414100030 msgr2=0x7fc414199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.136+0000 7fc412800640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc414100030 0x7fc414199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.136+0000 7fc412800640 1 -- 172.21.15.137:0/3995904808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4141a1710 con 0x7fc414101330 2024-03-26T14:13:05.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.136+0000 7fc412800640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 0x7fc4141a10a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00eb10 tx=0x7fc40c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.138+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c004b20 con 0x7fc414101330 2024-03-26T14:13:05.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.138+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc40c004cc0 con 0x7fc414101330 2024-03-26T14:13:05.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.138+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c010690 con 0x7fc414101330 2024-03-26T14:13:05.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.138+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4141a1a00 con 0x7fc414101330 2024-03-26T14:13:05.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.138+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4141a1f00 con 0x7fc414101330 2024-03-26T14:13:05.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.139+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41410ad80 con 0x7fc414101330 2024-03-26T14:13:05.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.142+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc40c005020 con 0x7fc414101330 2024-03-26T14:13:05.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.143+0000 7fc40aa00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3f80a2800 0x7fc3f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.143+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc40c0cf3c0 con 0x7fc414101330 2024-03-26T14:13:05.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.144+0000 7fc411e00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3f80a2800 0x7fc3f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.144+0000 7fc411e00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3f80a2800 0x7fc3f80a4cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc40402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.148+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc40c09c090 con 0x7fc414101330 2024-03-26T14:13:05.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:05 smithi137 ceph-mon[31539]: pgmap v779: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:05.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.420+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc41402d070 con 0x7fc414101330 2024-03-26T14:13:05.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.421+0000 7fc40aa00640 1 -- 172.21.15.137:0/3995904808 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fc40c09b990 con 0x7fc414101330 2024-03-26T14:13:05.421 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:13:05.422 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:13:05.422 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:13:05.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3f80a2800 msgr2=0x7fc3f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3f80a2800 0x7fc3f80a4cc0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc404000c00 tx=0x7fc40402c880 comp rx=0 tx=0).stop 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 msgr2=0x7fc4141a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 0x7fc4141a10a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00eb10 tx=0x7fc40c00c2d0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 shutdown_connections 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3f80a2800 0x7fc3f80a4cc0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc414101330 0x7fc4141a10a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc414100990 0x7fc41419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 --2- 172.21.15.137:0/3995904808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc414100030 0x7fc414199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 >> 172.21.15.137:0/3995904808 conn(0x7fc4140fbbd0 msgr2=0x7fc4140fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 shutdown_connections 2024-03-26T14:13:05.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.424+0000 7fc418a00640 1 -- 172.21.15.137:0/3995904808 wait complete. 2024-03-26T14:13:05.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:05 smithi179 ceph-mon[35472]: pgmap v779: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:05.443 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:13:05.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:13:05.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:05 smithi159 ceph-mon[35501]: pgmap v779: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.667+0000 7f56a3200640 1 -- 172.21.15.137:0/3106045389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 msgr2=0x7f56a41096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.667+0000 7f56a3200640 1 --2- 172.21.15.137:0/3106045389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a41096b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f569c009ca0 tx=0x7f569c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 -- 172.21.15.137:0/3106045389 shutdown_connections 2024-03-26T14:13:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 --2- 172.21.15.137:0/3106045389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a41096b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 --2- 172.21.15.137:0/3106045389 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56a4104930 0x7f56a4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 --2- 172.21.15.137:0/3106045389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a4102000 0x7f56a41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 -- 172.21.15.137:0/3106045389 >> 172.21.15.137:0/3106045389 conn(0x7f56a40fba10 msgr2=0x7f56a40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 -- 172.21.15.137:0/3106045389 shutdown_connections 2024-03-26T14:13:05.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.668+0000 7f56a3200640 1 -- 172.21.15.137:0/3106045389 wait complete. 2024-03-26T14:13:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 Processor -- start 2024-03-26T14:13:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 -- start start 2024-03-26T14:13:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a4102000 0x7f56a4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56a4104930 0x7f56a4197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a410d0c0 con 0x7f56a4107260 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a410cf40 con 0x7f56a4102000 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.669+0000 7f56a3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a410d240 con 0x7f56a4104930 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43320/0 (socket says 172.21.15.137:43320) 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 -- 172.21.15.137:0/1212490805 learned_addr learned my addr 172.21.15.137:0/1212490805 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a1e00640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a4102000 0x7f56a4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a1400640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56a4104930 0x7f56a4197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 -- 172.21.15.137:0/1212490805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56a4104930 msgr2=0x7f56a4197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56a4104930 0x7f56a4197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 -- 172.21.15.137:0/1212490805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a4102000 msgr2=0x7f56a4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a4102000 0x7f56a4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.673 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 -- 172.21.15.137:0/1212490805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a419f370 con 0x7f56a4107260 2024-03-26T14:13:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.670+0000 7f56a2800640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a419ed00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f569c032380 tx=0x7f569c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.671+0000 7f569aa00640 1 -- 172.21.15.137:0/1212490805 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c041020 con 0x7f56a4107260 2024-03-26T14:13:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.671+0000 7f569aa00640 1 -- 172.21.15.137:0/1212490805 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569c003e20 con 0x7f56a4107260 2024-03-26T14:13:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.671+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a419f600 con 0x7f56a4107260 2024-03-26T14:13:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.671+0000 7f569aa00640 1 -- 172.21.15.137:0/1212490805 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c039040 con 0x7f56a4107260 2024-03-26T14:13:05.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.671+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a419fb40 con 0x7f56a4107260 2024-03-26T14:13:05.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.674+0000 7f56abfef740 1 Processor -- start 2024-03-26T14:13:05.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f569aa00640 1 -- 172.21.15.137:0/1212490805 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f569c044020 con 0x7f56a4107260 2024-03-26T14:13:05.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 -- start start 2024-03-26T14:13:05.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 0x55f95b27be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27cbc0 0x55f95b27d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27d580 0x55f95b283e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95b1e02c0 con 0x55f95b27b9f0 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95b188c90 con 0x55f95b27d580 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.675+0000 7f56abfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95b1dc360 con 0x55f95b27cbc0 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 0x55f95b27be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 0x55f95b27be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43330/0 (socket says 172.21.15.137:43330) 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 -- 172.21.15.137:0/3524321069 learned_addr learned my addr 172.21.15.137:0/3524321069 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1e00640 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27cbc0 0x55f95b27d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27d580 0x55f95b283e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 -- 172.21.15.137:0/3524321069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27cbc0 msgr2=0x55f95b27d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27cbc0 0x55f95b27d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 -- 172.21.15.137:0/3524321069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27d580 msgr2=0x55f95b283e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27d580 0x55f95b283e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.676+0000 7f56a1400640 1 -- 172.21.15.137:0/3524321069 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f95b1ad080 con 0x55f95b27b9f0 2024-03-26T14:13:05.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f56a1400640 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 0x55f95b27be10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f569000bfd0 tx=0x7f569000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3709bb3be3d8768 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f569aa00640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56880a2880 0x7f56880a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f569aa00640 1 -- 172.21.15.137:0/1212490805 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f569c0f1080 con 0x7f56a4107260 2024-03-26T14:13:05.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f568f400640 1 -- 172.21.15.137:0/3524321069 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690019030 con 0x55f95b27b9f0 2024-03-26T14:13:05.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f56a1e00640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56880a2880 0x7f56880a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f568f400640 1 -- 172.21.15.137:0/3524321069 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5690011970 con 0x55f95b27b9f0 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.677+0000 7f568f400640 1 -- 172.21.15.137:0/3524321069 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569001d720 con 0x55f95b27b9f0 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 -- 172.21.15.137:0/3524321069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 msgr2=0x55f95b27be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56a1e00640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56880a2880 0x7f56880a4d40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5694004610 tx=0x7f569400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 0x55f95b27be10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f569000bfd0 tx=0x7f569000b650 comp rx=0 tx=0).stop 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 -- 172.21.15.137:0/3524321069 shutdown_connections 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27d580 0x55f95b283e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27cbc0 0x55f95b27d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 --2- 172.21.15.137:0/3524321069 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27b9f0 0x55f95b27be10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 -- 172.21.15.137:0/3524321069 >> 172.21.15.137:0/3524321069 conn(0x55f95b276de0 msgr2=0x55f95b2771e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 -- 172.21.15.137:0/3524321069 shutdown_connections 2024-03-26T14:13:05.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.678+0000 7f56abfef740 1 -- 172.21.15.137:0/3524321069 wait complete. 2024-03-26T14:13:05.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.679+0000 7f56abfef740 1 -- start start 2024-03-26T14:13:05.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.679+0000 7f56abfef740 1 Processor -- start 2024-03-26T14:13:05.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b285c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b2861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b28d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95b284590 con 0x55f95b27d580 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95b19c0d0 con 0x55f95b27cbc0 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f95b284710 con 0x55f95b27b9f0 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56abfef740 10 client.0 _ll_register_callbacks cb 0x55f95b1a0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56a2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b28d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56a2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b28d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43332/0 (socket says 172.21.15.137:43332) 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56a2800640 1 -- 172.21.15.137:0/2491848162 learned_addr learned my addr 172.21.15.137:0/2491848162 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56a1400640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b2861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.680+0000 7f56a1e00640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b285c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56a2800640 1 -- 172.21.15.137:0/2491848162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 msgr2=0x55f95b285c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56a2800640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b285c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56a2800640 1 -- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 msgr2=0x55f95b2861c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56a2800640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b2861c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56a2800640 1 -- 172.21.15.137:0/2491848162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f95aef43f0 con 0x55f95b27d580 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56a2800640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b28d200 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f569c0ba020 tx=0x7f569c0bd420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc744e723dc6b738 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f5683e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f5683e00640 1 -- 172.21.15.137:0/2491848162 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c004040 con 0x55f95b27d580 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f5683e00640 1 -- 172.21.15.137:0/2491848162 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f569c004680 con 0x55f95b27d580 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5690011970 con 0x55f95b27d580 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f5683e00640 1 -- 172.21.15.137:0/2491848162 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f569c0a8070 con 0x55f95b27d580 2024-03-26T14:13:05.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56abfef740 10 client.17886 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.681+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f95afbe7d0 con 0x55f95b27d580 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f5683e00640 1 -- 172.21.15.137:0/2491848162 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5690011970 con 0x55f95b27d580 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f5683e00640 1 -- 172.21.15.137:0/2491848162 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f569c03c030 con 0x55f95b27d580 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f56abfef740 10 client.17886 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f56abfef740 10 client.17886 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f56abfef740 20 client.17886 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f56abfef740 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2916a0 0x55f95b293ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f56abfef740 4 client.17886 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.682+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f95b1ed800 con 0x55f95b2916a0 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.683+0000 7f56a1e00640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2916a0 0x55f95b293ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.683+0000 7f5683e00640 10 client.17886 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:05.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.683+0000 7f56a1e00640 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2916a0 0x55f95b293ab0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21e7798cc01cae30 server_cookie=1a313aa4433d7d92 in_seq=0 out_seq=0 2024-03-26T14:13:05.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.685+0000 7f5683e00640 1 -- 172.21.15.137:0/2491848162 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f95b1ed800 con 0x55f95b2916a0 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.685+0000 7f5683e00640 10 client.17886 handle_command_reply: tid=0 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.685+0000 7f56abfef740 1 client.17886 shutdown 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 msgr2=0x55f95b28d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b28d200 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f569c0ba020 tx=0x7f569c0bd420 comp rx=0 tx=0).stop 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 shutdown_connections 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2916a0 0x55f95b293ab0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b2861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b28d200 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f5683e00640 0 client.17886 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 --2- 172.21.15.137:0/2491848162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b285c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 >> 172.21.15.137:0/2491848162 conn(0x55f95b276de0 msgr2=0x55f95b276160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 shutdown_connections 2024-03-26T14:13:05.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 1 -- 172.21.15.137:0/2491848162 wait complete. 2024-03-26T14:13:05.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.686+0000 7f56abfef740 20 client.17886 trim_cache size 0 max 16384 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 Processor -- start 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 -- start start 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27b9f0 0x55f95b271ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 0x55f95b28d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27d580 0x55f95b27fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41ac4c0 con 0x55f95b27cbc0 2024-03-26T14:13:05.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a9fc0 con 0x55f95b27b9f0 2024-03-26T14:13:05.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.701+0000 7f56abfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a8040 con 0x55f95b27d580 2024-03-26T14:13:05.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 0x55f95b28d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27b9f0 0x55f95b271ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 0x55f95b28d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43346/0 (socket says 172.21.15.137:43346) 2024-03-26T14:13:05.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 -- 172.21.15.137:0/1713478733 learned_addr learned my addr 172.21.15.137:0/1713478733 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a2800640 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27d580 0x55f95b27fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 -- 172.21.15.137:0/1713478733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27d580 msgr2=0x55f95b27fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27d580 0x55f95b27fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 -- 172.21.15.137:0/1713478733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27b9f0 msgr2=0x55f95b271ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27b9f0 0x55f95b271ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 -- 172.21.15.137:0/1713478733 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f95b2801a0 con 0x55f95b27cbc0 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.702+0000 7f56a1400640 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 0x55f95b28d1c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f569000bcd0 tx=0x7f5690005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c16e59fb65a07 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.703+0000 7f568ea00640 1 -- 172.21.15.137:0/1713478733 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690019040 con 0x55f95b27cbc0 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.703+0000 7f568ea00640 1 -- 172.21.15.137:0/1713478733 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f569000b3c0 con 0x55f95b27cbc0 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.703+0000 7f568ea00640 1 -- 172.21.15.137:0/1713478733 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56900048e0 con 0x55f95b27cbc0 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.703+0000 7f56abfef740 1 -- 172.21.15.137:0/1713478733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 msgr2=0x55f95b28d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.703+0000 7f56abfef740 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 0x55f95b28d1c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f569000bcd0 tx=0x7f5690005bf0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 -- 172.21.15.137:0/1713478733 shutdown_connections 2024-03-26T14:13:05.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27d580 0x55f95b27fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27cbc0 0x55f95b28d1c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 --2- 172.21.15.137:0/1713478733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27b9f0 0x55f95b271ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 -- 172.21.15.137:0/1713478733 >> 172.21.15.137:0/1713478733 conn(0x55f95b276de0 msgr2=0x55f95b27a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 -- 172.21.15.137:0/1713478733 shutdown_connections 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 -- 172.21.15.137:0/1713478733 wait complete. 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 -- start start 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.704+0000 7f56abfef740 1 Processor -- start 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b28ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b28cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b289510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a8640 con 0x55f95b27d580 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a9960 con 0x55f95b27cbc0 2024-03-26T14:13:05.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a41a8ca0 con 0x55f95b27b9f0 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56abfef740 10 client.0 _ll_register_callbacks cb 0x7f569c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56a2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b289510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56a1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b28cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56a2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b289510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43352/0 (socket says 172.21.15.137:43352) 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56a2800640 1 -- 172.21.15.137:0/3364568060 learned_addr learned my addr 172.21.15.137:0/3364568060 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.705+0000 7f56a1e00640 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b28ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56a2800640 1 -- 172.21.15.137:0/3364568060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 msgr2=0x55f95b28ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56a2800640 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b28ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56a2800640 1 -- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 msgr2=0x55f95b28cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56a2800640 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b28cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56a2800640 1 -- 172.21.15.137:0/3364568060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f95b289b50 con 0x55f95b27d580 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56a2800640 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b289510 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f569c0b9ab0 tx=0x7f569c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec14fc5cfb4c907 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f5683e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f5683e00640 1 -- 172.21.15.137:0/3364568060 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c003b70 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f5683e00640 1 -- 172.21.15.137:0/3364568060 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f569c004b40 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f95b1e7b50 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f5683e00640 1 -- 172.21.15.137:0/3364568060 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f569c0a5050 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56abfef740 10 client.17892 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.706+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f95b289b50 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f5683e00640 1 -- 172.21.15.137:0/3364568060 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55f95b1e7b50 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f5683e00640 1 -- 172.21.15.137:0/3364568060 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f569c0b93a0 con 0x55f95b27d580 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56abfef740 10 client.17892 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56abfef740 10 client.17892 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56abfef740 20 client.17892 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56abfef740 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2ee020 0x55f95b2f0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56abfef740 4 client.17892 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:13:05.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f95b165500 con 0x55f95b2ee020 2024-03-26T14:13:05.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.707+0000 7f56a1e00640 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2ee020 0x55f95b2f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.708+0000 7f56a1e00640 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2ee020 0x55f95b2f0440 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b516e8ea91d21a3a server_cookie=e6df139263831d27 in_seq=0 out_seq=0 2024-03-26T14:13:05.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.708+0000 7f5683e00640 10 client.17892 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.708+0000 7f5683e00640 1 -- 172.21.15.137:0/3364568060 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f95b165500 con 0x55f95b2ee020 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.708+0000 7f5683e00640 10 client.17892 handle_command_reply: tid=0 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.708+0000 7f56abfef740 1 client.17892 shutdown 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 msgr2=0x55f95b289510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b289510 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f569c0b9ab0 tx=0x7f569c009650 comp rx=0 tx=0).stop 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 shutdown_connections 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55f95b2ee020 0x55f95b2f0440 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f95b27cbc0 0x55f95b28cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f95b27d580 0x55f95b289510 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 --2- 172.21.15.137:0/3364568060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f95b27b9f0 0x55f95b28ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 >> 172.21.15.137:0/3364568060 conn(0x55f95b276de0 msgr2=0x55f95b27a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f5683e00640 0 client.17892 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:05.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 shutdown_connections 2024-03-26T14:13:05.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 1 -- 172.21.15.137:0/3364568060 wait complete. 2024-03-26T14:13:05.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.709+0000 7f56abfef740 20 client.17892 trim_cache size 0 max 16384 2024-03-26T14:13:05.716 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56880a2880 msgr2=0x7f56880a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56880a2880 0x7f56880a4d40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5694004610 tx=0x7f569400b390 comp rx=0 tx=0).stop 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 msgr2=0x7f56a419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a419ed00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f569c032380 tx=0x7f569c032820 comp rx=0 tx=0).stop 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 shutdown_connections 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f56880a2880 0x7f56880a4d40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f56a4107260 0x7f56a419ed00 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56a4104930 0x7f56a4197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 --2- 172.21.15.137:0/1212490805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a4102000 0x7f56a4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 >> 172.21.15.137:0/1212490805 conn(0x7f56a40fba10 msgr2=0x7f56a41028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 shutdown_connections 2024-03-26T14:13:05.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.710+0000 7f56a3200640 1 -- 172.21.15.137:0/1212490805 wait complete. 2024-03-26T14:13:05.730 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:13:05.730 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 -- 172.21.15.137:0/3266432946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98105350 msgr2=0x7f2c9810bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/3266432946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98105350 0x7f2c9810bbe0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ca0 tx=0x7f2c9002c0f0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 -- 172.21.15.137:0/3266432946 shutdown_connections 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/3266432946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c98104990 0x7f2c98104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/3266432946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98105350 0x7f2c9810bbe0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/3266432946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c981037a0 0x7f2c98103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 -- 172.21.15.137:0/3266432946 >> 172.21.15.137:0/3266432946 conn(0x7f2c980fed10 msgr2=0x7f2c98101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 -- 172.21.15.137:0/3266432946 shutdown_connections 2024-03-26T14:13:05.952 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.950+0000 7f2c9cc00640 1 -- 172.21.15.137:0/3266432946 wait complete. 2024-03-26T14:13:05.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.951+0000 7f2c9cc00640 1 Processor -- start 2024-03-26T14:13:05.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.951+0000 7f2c9cc00640 1 -- start start 2024-03-26T14:13:05.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c981037a0 0x7f2c98199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 0x7f2c98199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c9cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c98105350 0x7f2c981a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c9cc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9810f2d0 con 0x7f2c98104990 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c9cc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9810f150 con 0x7f2c98105350 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c9cc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9810f450 con 0x7f2c981037a0 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 0x7f2c98199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 0x7f2c98199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43382/0 (socket says 172.21.15.137:43382) 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c96a00640 1 -- 172.21.15.137:0/2318403874 learned_addr learned my addr 172.21.15.137:0/2318403874 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c97e00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c98105350 0x7f2c981a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.952+0000 7f2c97400640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c981037a0 0x7f2c98199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c96a00640 1 -- 172.21.15.137:0/2318403874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c981037a0 msgr2=0x7f2c98199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c96a00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c981037a0 0x7f2c98199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c96a00640 1 -- 172.21.15.137:0/2318403874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c98105350 msgr2=0x7f2c981a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c96a00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c98105350 0x7f2c981a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c96a00640 1 -- 172.21.15.137:0/2318403874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c981a1560 con 0x7f2c98104990 2024-03-26T14:13:05.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c96a00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 0x7f2c98199e80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400e9f0 tx=0x7f2c8400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c8fe00640 1 -- 172.21.15.137:0/2318403874 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8400cd80 con 0x7f2c98104990 2024-03-26T14:13:05.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c8fe00640 1 -- 172.21.15.137:0/2318403874 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c840108e0 con 0x7f2c98104990 2024-03-26T14:13:05.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c8fe00640 1 -- 172.21.15.137:0/2318403874 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84018920 con 0x7f2c98104990 2024-03-26T14:13:05.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.953+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c981a1850 con 0x7f2c98104990 2024-03-26T14:13:05.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.954+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c981a1d50 con 0x7f2c98104990 2024-03-26T14:13:05.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.958+0000 7f2ca0ffa740 1 Processor -- start 2024-03-26T14:13:05.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.958+0000 7f2ca0ffa740 1 -- start start 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2c8fe00640 1 -- 172.21.15.137:0/2318403874 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2c84010a90 con 0x7f2c98104990 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b71040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b77e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dab5ad42c0 con 0x55dab5b70bc0 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dab5a7cc90 con 0x55dab5b6f9f0 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dab5ad0360 con 0x55dab5b71580 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2c97400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b71040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2c97400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b71040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43392/0 (socket says 172.21.15.137:43392) 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2c97400640 1 -- 172.21.15.137:0/1343957569 learned_addr learned my addr 172.21.15.137:0/1343957569 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.959+0000 7f2c97e00640 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c96a00640 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b77e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c97400640 1 -- 172.21.15.137:0/1343957569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 msgr2=0x55dab5b77e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c97400640 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b77e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c97400640 1 -- 172.21.15.137:0/1343957569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 msgr2=0x55dab5b6fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c97400640 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c97400640 1 -- 172.21.15.137:0/1343957569 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dab5aa1080 con 0x55dab5b70bc0 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c97400640 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b71040 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800bc80 tx=0x7f2c8800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc93141fc5d2ffdf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c83e00640 1 -- 172.21.15.137:0/1343957569 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88015070 con 0x55dab5b70bc0 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.960+0000 7f2c83e00640 1 -- 172.21.15.137:0/1343957569 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c8801a910 con 0x55dab5b70bc0 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.961+0000 7f2c8fe00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2c7c0a2800 0x7f2c7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.961+0000 7f2c83e00640 1 -- 172.21.15.137:0/1343957569 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88011870 con 0x55dab5b70bc0 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.961+0000 7f2c8fe00640 1 -- 172.21.15.137:0/2318403874 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c840cf3b0 con 0x7f2c98104990 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.961+0000 7f2c97e00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2c7c0a2800 0x7f2c7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.961+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/1343957569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 msgr2=0x55dab5b71040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.961+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b71040 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800bc80 tx=0x7f2c8800f730 comp rx=0 tx=0).stop 2024-03-26T14:13:05.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/1343957569 shutdown_connections 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b77e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b71040 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/1343957569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/1343957569 >> 172.21.15.137:0/1343957569 conn(0x55dab5b6ade0 msgr2=0x55dab5b6b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/1343957569 shutdown_connections 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2c97e00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2c7c0a2800 0x7f2c7c0a4cc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c9002c630 tx=0x7f2c90009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/1343957569 wait complete. 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 -- start start 2024-03-26T14:13:05.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.962+0000 7f2ca0ffa740 1 Processor -- start 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b79c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b70bc0 0x55dab5b7a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 0x55dab5b811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dab5b785e0 con 0x55dab5b71580 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dab5a900d0 con 0x55dab5b6f9f0 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dab5b78760 con 0x55dab5b70bc0 2024-03-26T14:13:05.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2ca0ffa740 10 client.0 _ll_register_callbacks cb 0x55dab5a94e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2c97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 0x55dab5b811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2c97e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 0x55dab5b811b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43400/0 (socket says 172.21.15.137:43400) 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2c97e00640 1 -- 172.21.15.137:0/826019672 learned_addr learned my addr 172.21.15.137:0/826019672 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.963+0000 7f2c97400640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b79c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c96a00640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b70bc0 0x55dab5b7a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c97e00640 1 -- 172.21.15.137:0/826019672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b70bc0 msgr2=0x55dab5b7a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c97e00640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b70bc0 0x55dab5b7a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c97e00640 1 -- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 msgr2=0x55dab5b79c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c97e00640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b79c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c97e00640 1 -- 172.21.15.137:0/826019672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dab57e83f0 con 0x55dab5b71580 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c97e00640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 0x55dab5b811b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ca0 tx=0x7f2c900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a0da7c984f3634 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c80c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:05.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.964+0000 7f2c80c00640 1 -- 172.21.15.137:0/826019672 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90051020 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2c80c00640 1 -- 172.21.15.137:0/826019672 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c90004180 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c8801a910 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2c80c00640 1 -- 172.21.15.137:0/826019672 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c9005c050 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2ca0ffa740 10 client.17904 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dab58b27d0 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2c80c00640 1 -- 172.21.15.137:0/826019672 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c8801a910 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.965+0000 7f2c80c00640 1 -- 172.21.15.137:0/826019672 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2c9004d9e0 con 0x55dab5b71580 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2ca0ffa740 10 client.17904 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2ca0ffa740 10 client.17904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2ca0ffa740 20 client.17904 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5b85750 0x55dab5b87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2ca0ffa740 4 client.17904 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dab5ae1800 con 0x55dab5b85750 2024-03-26T14:13:05.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.966+0000 7f2c97400640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5b85750 0x55dab5b87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.967+0000 7f2c80c00640 10 client.17904 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:05.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.967+0000 7f2c97400640 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5b85750 0x55dab5b87bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57dffe5405e5e104 server_cookie=17766ef6e304d21e in_seq=0 out_seq=0 2024-03-26T14:13:05.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2c80c00640 1 -- 172.21.15.137:0/826019672 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55dab5ae1800 con 0x55dab5b85750 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2c80c00640 10 client.17904 handle_command_reply: tid=0 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 client.17904 shutdown 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 msgr2=0x55dab5b811b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 0x55dab5b811b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009ca0 tx=0x7f2c900095c0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 shutdown_connections 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5b85750 0x55dab5b87bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b71580 0x55dab5b811b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b70bc0 0x55dab5b7a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/826019672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b79c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2c80c00640 0 client.17904 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.968+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 >> 172.21.15.137:0/826019672 conn(0x55dab5b6ade0 msgr2=0x55dab5b6a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.969+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 shutdown_connections 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.969+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/826019672 wait complete. 2024-03-26T14:13:05.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.969+0000 7f2ca0ffa740 20 client.17904 trim_cache size 0 max 16384 2024-03-26T14:13:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 Processor -- start 2024-03-26T14:13:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 -- start start 2024-03-26T14:13:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b81170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b71580 0x55dab5b73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981ae6d0 con 0x55dab5b70bc0 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981ac1d0 con 0x55dab5b71580 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.986+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981aa250 con 0x55dab5b6f9f0 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b81170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b81170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43412/0 (socket says 172.21.15.137:43412) 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 -- 172.21.15.137:0/3389000816 learned_addr learned my addr 172.21.15.137:0/3389000816 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c97e00640 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b71580 0x55dab5b73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c97400640 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 -- 172.21.15.137:0/3389000816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b6f9f0 msgr2=0x55dab5b6e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 -- 172.21.15.137:0/3389000816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b71580 msgr2=0x55dab5b73bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b71580 0x55dab5b73bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c96a00640 1 -- 172.21.15.137:0/3389000816 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dab5b70640 con 0x55dab5b70bc0 2024-03-26T14:13:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.987+0000 7f2c97400640 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.988+0000 7f2c96a00640 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b81170 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c6a0 tx=0x7f2c84097c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd57e78036e382a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.988+0000 7f2c83400640 1 -- 172.21.15.137:0/3389000816 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840a40e0 con 0x55dab5b70bc0 2024-03-26T14:13:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.988+0000 7f2c83400640 1 -- 172.21.15.137:0/3389000816 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c840ccbb0 con 0x55dab5b70bc0 2024-03-26T14:13:05.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.988+0000 7f2c83400640 1 -- 172.21.15.137:0/3389000816 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84086310 con 0x55dab5b70bc0 2024-03-26T14:13:05.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/3389000816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 msgr2=0x55dab5b81170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b81170 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400c6a0 tx=0x7f2c84097c60 comp rx=0 tx=0).stop 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/3389000816 shutdown_connections 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b71580 0x55dab5b73bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b81170 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/3389000816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b6f9f0 0x55dab5b6e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/3389000816 >> 172.21.15.137:0/3389000816 conn(0x55dab5b6ade0 msgr2=0x55dab5b6e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/3389000816 shutdown_connections 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/3389000816 wait complete. 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.989+0000 7f2ca0ffa740 1 -- start start 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 Processor -- start 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b7d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b7d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981aa850 con 0x55dab5b70bc0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981abb70 con 0x55dab5b6f9f0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c981aaeb0 con 0x55dab5b71580 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2ca0ffa740 10 client.0 _ll_register_callbacks cb 0x7f2c90044070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b7d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2c96a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b7d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43418/0 (socket says 172.21.15.137:43418) 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2c96a00640 1 -- 172.21.15.137:0/914069237 learned_addr learned my addr 172.21.15.137:0/914069237 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.990+0000 7f2c97400640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b7d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c97e00640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c96a00640 1 -- 172.21.15.137:0/914069237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 msgr2=0x55dab5b806e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c96a00640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b806e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c96a00640 1 -- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 msgr2=0x55dab5b7d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c96a00640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b7d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c96a00640 1 -- 172.21.15.137:0/914069237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dab5b80cd0 con 0x55dab5b70bc0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c97400640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b7d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c96a00640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b7d6c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2c84098310 tx=0x7f2c840ccd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb315ad25de77e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:05.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c80c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c80c00640 1 -- 172.21.15.137:0/914069237 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c84010630 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c80c00640 1 -- 172.21.15.137:0/914069237 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c8409ad90 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c840ccbb0 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.991+0000 7f2c80c00640 1 -- 172.21.15.137:0/914069237 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c8409af90 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 10 client.17910 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55dab5be1870 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2c80c00640 1 -- 172.21.15.137:0/914069237 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c840ccbb0 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2c80c00640 1 -- 172.21.15.137:0/914069237 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2c8409a800 con 0x55dab5b70bc0 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 10 client.17910 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 10 client.17910 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 20 client.17910 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5be2170 0x55dab5be4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 4 client.17910 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.992+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dab5adbb50 con 0x55dab5be2170 2024-03-26T14:13:05.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.993+0000 7f2c97400640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5be2170 0x55dab5be4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.993+0000 7f2c97400640 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5be2170 0x55dab5be4590 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da0768e0d0d3ead server_cookie=ea9957ffc85ef121 in_seq=0 out_seq=0 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.993+0000 7f2c80c00640 10 client.17910 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2c80c00640 1 -- 172.21.15.137:0/914069237 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dab5adbb50 con 0x55dab5be2170 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2c80c00640 10 client.17910 handle_command_reply: tid=0 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 client.17910 shutdown 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 msgr2=0x55dab5b7d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b7d6c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2c84098310 tx=0x7f2c840ccd60 comp rx=0 tx=0).stop 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 shutdown_connections 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55dab5be2170 0x55dab5be4590 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55dab5b71580 0x55dab5b806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55dab5b70bc0 0x55dab5b7d6c0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 --2- 172.21.15.137:0/914069237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dab5b6f9f0 0x55dab5b7d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2c80c00640 0 client.17910 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 >> 172.21.15.137:0/914069237 conn(0x55dab5b6ade0 msgr2=0x55dab5b6e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 shutdown_connections 2024-03-26T14:13:05.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 1 -- 172.21.15.137:0/914069237 wait complete. 2024-03-26T14:13:06.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.994+0000 7f2ca0ffa740 20 client.17910 trim_cache size 0 max 16384 2024-03-26T14:13:06.000 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:13:06.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2c7c0a2800 msgr2=0x7f2c7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2c7c0a2800 0x7f2c7c0a4cc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c9002c630 tx=0x7f2c90009360 comp rx=0 tx=0).stop 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 msgr2=0x7f2c98199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 0x7f2c98199e80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400e9f0 tx=0x7f2c8400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 shutdown_connections 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2c7c0a2800 0x7f2c7c0a4cc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c98105350 0x7f2c981a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c98104990 0x7f2c98199e80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 --2- 172.21.15.137:0/2318403874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c981037a0 0x7f2c98199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 >> 172.21.15.137:0/2318403874 conn(0x7f2c980fed10 msgr2=0x7f2c98107ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 shutdown_connections 2024-03-26T14:13:06.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:05.995+0000 7f2c9cc00640 1 -- 172.21.15.137:0/2318403874 wait complete. 2024-03-26T14:13:06.015 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:13:06.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 -- 172.21.15.137:0/2841282080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c107260 msgr2=0x7f536c1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 --2- 172.21.15.137:0/2841282080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c107260 0x7f536c1096b0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f5364009900 tx=0x7f536402bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 -- 172.21.15.137:0/2841282080 shutdown_connections 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 --2- 172.21.15.137:0/2841282080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f536c104930 0x7f536c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 --2- 172.21.15.137:0/2841282080 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c107260 0x7f536c1096b0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 --2- 172.21.15.137:0/2841282080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c102000 0x7f536c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 -- 172.21.15.137:0/2841282080 >> 172.21.15.137:0/2841282080 conn(0x7f536c0fba10 msgr2=0x7f536c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.232+0000 7f5370c00640 1 -- 172.21.15.137:0/2841282080 shutdown_connections 2024-03-26T14:13:06.234 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.233+0000 7f5370c00640 1 -- 172.21.15.137:0/2841282080 wait complete. 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.233+0000 7f5370c00640 1 Processor -- start 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 -- start start 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 0x7f536c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f536c104930 0x7f536c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c107260 0x7f536c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c10d110 con 0x7f536c102000 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c10cf90 con 0x7f536c104930 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f5370c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c10d290 con 0x7f536c107260 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f536b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 0x7f536c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f536b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 0x7f536c101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43438/0 (socket says 172.21.15.137:43438) 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.234+0000 7f536b400640 1 -- 172.21.15.137:0/512299853 learned_addr learned my addr 172.21.15.137:0/512299853 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536aa00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f536c104930 0x7f536c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536be00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c107260 0x7f536c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536b400640 1 -- 172.21.15.137:0/512299853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c107260 msgr2=0x7f536c1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536b400640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c107260 0x7f536c1004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536b400640 1 -- 172.21.15.137:0/512299853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f536c104930 msgr2=0x7f536c0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536b400640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f536c104930 0x7f536c0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536b400640 1 -- 172.21.15.137:0/512299853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c100d80 con 0x7f536c102000 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f536b400640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 0x7f536c101970 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ea80 tx=0x7f535c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f5363e00640 1 -- 172.21.15.137:0/512299853 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c00ce40 con 0x7f536c102000 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f5363e00640 1 -- 172.21.15.137:0/512299853 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f535c005020 con 0x7f536c102000 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f5363e00640 1 -- 172.21.15.137:0/512299853 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c0116d0 con 0x7f536c102000 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.235+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c1a1870 con 0x7f536c102000 2024-03-26T14:13:06.238 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.236+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c1a1dc0 con 0x7f536c102000 2024-03-26T14:13:06.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.239+0000 7f5363e00640 1 -- 172.21.15.137:0/512299853 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f535c0051d0 con 0x7f536c102000 2024-03-26T14:13:06.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.239+0000 7f5374fce740 1 Processor -- start 2024-03-26T14:13:06.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 -- start start 2024-03-26T14:13:06.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 0x560f6f451e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f453040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f459e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6f3b62c0 con 0x560f6f4519f0 2024-03-26T14:13:06.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6f35ec90 con 0x560f6f452bc0 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f5374fce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6f3b2360 con 0x560f6f453580 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f536be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 0x560f6f451e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f536be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 0x560f6f451e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43444/0 (socket says 172.21.15.137:43444) 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.240+0000 7f536be00640 1 -- 172.21.15.137:0/508761922 learned_addr learned my addr 172.21.15.137:0/508761922 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536b400640 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f453040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f5363e00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f53500a2880 0x7f53500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536aa00640 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f459e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f5363e00640 1 -- 172.21.15.137:0/512299853 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f535c0cf1d0 con 0x7f536c102000 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536be00640 1 -- 172.21.15.137:0/508761922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 msgr2=0x560f6f459e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536be00640 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f459e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536be00640 1 -- 172.21.15.137:0/508761922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 msgr2=0x560f6f453040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536be00640 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f453040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536be00640 1 -- 172.21.15.137:0/508761922 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6f383080 con 0x560f6f4519f0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.241+0000 7f536be00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f53500a2880 0x7f53500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.242+0000 7f536be00640 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 0x560f6f451e10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5364005ab0 tx=0x7f5364005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cf3996a0626af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.242+0000 7f5357e00640 1 -- 172.21.15.137:0/508761922 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364045040 con 0x560f6f4519f0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.242+0000 7f5357e00640 1 -- 172.21.15.137:0/508761922 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f53640054f0 con 0x560f6f4519f0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.242+0000 7f5357e00640 1 -- 172.21.15.137:0/508761922 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364004a40 con 0x560f6f4519f0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.242+0000 7f536be00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f53500a2880 0x7f53500a4d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x560f6f4539e0 tx=0x7f536404b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 -- 172.21.15.137:0/508761922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 msgr2=0x560f6f451e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 0x560f6f451e10 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f5364005ab0 tx=0x7f5364005e20 comp rx=0 tx=0).stop 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 -- 172.21.15.137:0/508761922 shutdown_connections 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f453040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f459e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 --2- 172.21.15.137:0/508761922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f4519f0 0x560f6f451e10 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 -- 172.21.15.137:0/508761922 >> 172.21.15.137:0/508761922 conn(0x560f6f44cde0 msgr2=0x560f6f44d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 -- 172.21.15.137:0/508761922 shutdown_connections 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.243+0000 7f5374fce740 1 -- 172.21.15.137:0/508761922 wait complete. 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 -- start start 2024-03-26T14:13:06.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 Processor -- start 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f45bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f45c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f453580 0x560f6f4631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6f45a590 con 0x560f6f452bc0 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6f3720d0 con 0x560f6f453580 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.244+0000 7f5374fce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f6f45a710 con 0x560f6f4519f0 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f5374fce740 10 client.0 _ll_register_callbacks cb 0x560f6f376e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:06.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f45c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f45c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43458/0 (socket says 172.21.15.137:43458) 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 -- 172.21.15.137:0/4057757075 learned_addr learned my addr 172.21.15.137:0/4057757075 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536be00640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f453580 0x560f6f4631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536b400640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f45bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 -- 172.21.15.137:0/4057757075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 msgr2=0x560f6f45bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f45bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 -- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f453580 msgr2=0x560f6f4631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f453580 0x560f6f4631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.245+0000 7f536aa00640 1 -- 172.21.15.137:0/4057757075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f6f0ca3f0 con 0x560f6f452bc0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f536aa00640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f45c1b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f535800fde0 tx=0x7f53580117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836a27d44f13519f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5354c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5354c00640 1 -- 172.21.15.137:0/4057757075 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358004220 con 0x560f6f452bc0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5354c00640 1 -- 172.21.15.137:0/4057757075 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f53580043c0 con 0x560f6f452bc0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53640054f0 con 0x560f6f452bc0 2024-03-26T14:13:06.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5354c00640 1 -- 172.21.15.137:0/4057757075 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5358024050 con 0x560f6f452bc0 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5374fce740 10 client.17922 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.246+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560f6f1947d0 con 0x560f6f452bc0 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5354c00640 1 -- 172.21.15.137:0/4057757075 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f53640054f0 con 0x560f6f452bc0 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5354c00640 1 -- 172.21.15.137:0/4057757075 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5358004c90 con 0x560f6f452bc0 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5374fce740 10 client.17922 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5374fce740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5374fce740 20 client.17922 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5374fce740 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f467890 0x560f6f469cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5374fce740 4 client.17922 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.247+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f6f3c3800 con 0x560f6f467890 2024-03-26T14:13:06.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.248+0000 7f536b400640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f467890 0x560f6f469cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.248+0000 7f536b400640 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f467890 0x560f6f469cf0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf9cc3a29b9c66b server_cookie=5727e0ab197fd26d in_seq=0 out_seq=0 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.248+0000 7f5354c00640 10 client.17922 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5354c00640 1 -- 172.21.15.137:0/4057757075 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560f6f3c3800 con 0x560f6f467890 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5354c00640 10 client.17922 handle_command_reply: tid=0 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 client.17922 shutdown 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 msgr2=0x560f6f45c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f45c1b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f535800fde0 tx=0x7f53580117b0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 shutdown_connections 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f467890 0x560f6f469cf0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f453580 0x560f6f4631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f45c1b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 --2- 172.21.15.137:0/4057757075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f45bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5354c00640 0 client.17922 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.250+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 >> 172.21.15.137:0/4057757075 conn(0x560f6f44cde0 msgr2=0x560f6f44c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.251+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 shutdown_connections 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.251+0000 7f5374fce740 1 -- 172.21.15.137:0/4057757075 wait complete. 2024-03-26T14:13:06.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.251+0000 7f5374fce740 20 client.17922 trim_cache size 0 max 16384 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.268+0000 7f5374fce740 1 Processor -- start 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.268+0000 7f5374fce740 1 -- start start 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f5374fce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f450750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f5374fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f4631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f5374fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 0x560f6f46e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f5374fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1ae6b0 con 0x560f6f453580 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f5374fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1ac1b0 con 0x560f6f452bc0 2024-03-26T14:13:06.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f5374fce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1aa230 con 0x560f6f4519f0 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f536be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 0x560f6f46e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f536aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f4631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f536be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 0x560f6f46e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43470/0 (socket says 172.21.15.137:43470) 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f536be00640 1 -- 172.21.15.137:0/47522355 learned_addr learned my addr 172.21.15.137:0/47522355 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.269+0000 7f536b400640 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f450750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f536be00640 1 -- 172.21.15.137:0/47522355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 msgr2=0x560f6f450750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f536be00640 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f450750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f536be00640 1 -- 172.21.15.137:0/47522355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 msgr2=0x560f6f4631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f536be00640 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f4631b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f536be00640 1 -- 172.21.15.137:0/47522355 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f6f456e00 con 0x560f6f453580 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f536be00640 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 0x560f6f46e5b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f536402c6a0 tx=0x7f5364005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcdb3cfa07b336a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f5357400640 1 -- 172.21.15.137:0/47522355 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364057070 con 0x560f6f453580 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f5357400640 1 -- 172.21.15.137:0/47522355 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5364004440 con 0x560f6f453580 2024-03-26T14:13:06.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.270+0000 7f5357400640 1 -- 172.21.15.137:0/47522355 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364045040 con 0x560f6f453580 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 -- 172.21.15.137:0/47522355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 msgr2=0x560f6f46e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 0x560f6f46e5b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f536402c6a0 tx=0x7f5364005fd0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 -- 172.21.15.137:0/47522355 shutdown_connections 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f452bc0 0x560f6f4631b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f453580 0x560f6f46e5b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 --2- 172.21.15.137:0/47522355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f4519f0 0x560f6f450750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 -- 172.21.15.137:0/47522355 >> 172.21.15.137:0/47522355 conn(0x560f6f44cde0 msgr2=0x560f6f450040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 -- 172.21.15.137:0/47522355 shutdown_connections 2024-03-26T14:13:06.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.271+0000 7f5374fce740 1 -- 172.21.15.137:0/47522355 wait complete. 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 -- start start 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 Processor -- start 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f4519f0 0x560f6f461d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f462260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f4627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1aa830 con 0x560f6f452bc0 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1abb50 con 0x560f6f4519f0 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1aae90 con 0x560f6f453580 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.272+0000 7f5374fce740 10 client.0 _ll_register_callbacks cb 0x7f535800f360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f462260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f462260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43474/0 (socket says 172.21.15.137:43474) 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 -- 172.21.15.137:0/1185216698 learned_addr learned my addr 172.21.15.137:0/1185216698 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536b400640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f4519f0 0x560f6f461d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536be00640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f4627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 -- 172.21.15.137:0/1185216698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 msgr2=0x560f6f4627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f4627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 -- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f4519f0 msgr2=0x560f6f461d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f4519f0 0x560f6f461d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 -- 172.21.15.137:0/1185216698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f6f4c3470 con 0x560f6f452bc0 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f536aa00640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f462260 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5358005bc0 tx=0x7f535800f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ec7cb7f22eebb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.273+0000 7f5354c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5354c00640 1 -- 172.21.15.137:0/1185216698 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5358004230 con 0x560f6f452bc0 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5364004440 con 0x560f6f452bc0 2024-03-26T14:13:06.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5354c00640 1 -- 172.21.15.137:0/1185216698 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f53580043d0 con 0x560f6f452bc0 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5354c00640 1 -- 172.21.15.137:0/1185216698 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5358039050 con 0x560f6f452bc0 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5374fce740 10 client.17928 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560f6f4c47a0 con 0x560f6f452bc0 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.274+0000 7f5354c00640 1 -- 172.21.15.137:0/1185216698 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5364004440 con 0x560f6f452bc0 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5354c00640 1 -- 172.21.15.137:0/1185216698 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5358004910 con 0x560f6f452bc0 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5374fce740 10 client.17928 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:06.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5374fce740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5374fce740 20 client.17928 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5374fce740 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f4c50d0 0x560f6f4c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5374fce740 4 client.17928 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f6f3bdb50 con 0x560f6f4c50d0 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.275+0000 7f536b400640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f4c50d0 0x560f6f4c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.276+0000 7f5354c00640 10 client.17928 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.276+0000 7f536b400640 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f4c50d0 0x560f6f4c74f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb47f7ddf6cbfae server_cookie=2947a7da999c024d in_seq=0 out_seq=0 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.276+0000 7f5354c00640 1 -- 172.21.15.137:0/1185216698 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f6f3bdb50 con 0x560f6f4c50d0 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.276+0000 7f5354c00640 10 client.17928 handle_command_reply: tid=0 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.276+0000 7f5374fce740 1 client.17928 shutdown 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 msgr2=0x560f6f462260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f462260 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f5358005bc0 tx=0x7f535800f910 comp rx=0 tx=0).stop 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 shutdown_connections 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560f6f4c50d0 0x560f6f4c74f0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560f6f453580 0x560f6f4627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560f6f452bc0 0x560f6f462260 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 --2- 172.21.15.137:0/1185216698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560f6f4519f0 0x560f6f461d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5354c00640 0 client.17928 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 >> 172.21.15.137:0/1185216698 conn(0x560f6f44cde0 msgr2=0x560f6f4508d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 shutdown_connections 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 1 -- 172.21.15.137:0/1185216698 wait complete. 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.277+0000 7f5374fce740 20 client.17928 trim_cache size 0 max 16384 2024-03-26T14:13:06.281 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f53500a2880 msgr2=0x7f53500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f53500a2880 0x7f53500a4d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x560f6f4539e0 tx=0x7f536404b000 comp rx=0 tx=0).stop 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 msgr2=0x7f536c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 0x7f536c101970 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f535c00ea80 tx=0x7f535c00ef40 comp rx=0 tx=0).stop 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 shutdown_connections 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f53500a2880 0x7f53500a4d40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f536c104930 0x7f536c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c107260 0x7f536c1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 --2- 172.21.15.137:0/512299853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f536c102000 0x7f536c101970 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:06.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 >> 172.21.15.137:0/512299853 conn(0x7f536c0fba10 msgr2=0x7f536c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:06.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 shutdown_connections 2024-03-26T14:13:06.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:06.278+0000 7f5370c00640 1 -- 172.21.15.137:0/512299853 wait complete. 2024-03-26T14:13:06.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:06 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3995904808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:13:06.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:06 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3995904808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:13:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:06 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3995904808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:13:07.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:13:07.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:13:07.298 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:13:07.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:07 smithi179 ceph-mon[35472]: pgmap v780: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:07.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:07 smithi159 ceph-mon[35501]: pgmap v780: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:07.483 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:07 smithi137 ceph-mon[31539]: pgmap v780: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:07.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 -- 172.21.15.137:0/1365147649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 msgr2=0x7f6da8106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 --2- 172.21.15.137:0/1365147649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8106d20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009900 tx=0x7f6d9402bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 -- 172.21.15.137:0/1365147649 shutdown_connections 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 --2- 172.21.15.137:0/1365147649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8107260 0x7f6da81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 --2- 172.21.15.137:0/1365147649 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8106d20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 --2- 172.21.15.137:0/1365147649 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8102000 0x7f6da81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 -- 172.21.15.137:0/1365147649 >> 172.21.15.137:0/1365147649 conn(0x7f6da80fba10 msgr2=0x7f6da80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 -- 172.21.15.137:0/1365147649 shutdown_connections 2024-03-26T14:13:07.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.537+0000 7f6daea00640 1 -- 172.21.15.137:0/1365147649 wait complete. 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.538+0000 7f6daea00640 1 Processor -- start 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.538+0000 7f6daea00640 1 -- start start 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6daea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8102000 0x7f6da8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6daea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6daea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8107260 0x7f6da81a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6daea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da810d110 con 0x7f6da8104930 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6daea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da810cf90 con 0x7f6da8102000 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6daea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da810d290 con 0x7f6da8107260 2024-03-26T14:13:07.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8102000 0x7f6da8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43496/0 (socket says 172.21.15.137:43496) 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 -- 172.21.15.137:0/3983106688 learned_addr learned my addr 172.21.15.137:0/3983106688 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dae000640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8107260 0x7f6da81a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 -- 172.21.15.137:0/3983106688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8107260 msgr2=0x7f6da81a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8107260 0x7f6da81a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 -- 172.21.15.137:0/3983106688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8102000 msgr2=0x7f6da8199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.539+0000 7f6dacc00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8102000 0x7f6da8199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6dacc00640 1 -- 172.21.15.137:0/3983106688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da81a1660 con 0x7f6da8104930 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6dad600640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8102000 0x7f6da8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:07.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6dacc00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8199f80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000c00 tx=0x7f6d940040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6da6000640 1 -- 172.21.15.137:0/3983106688 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94043020 con 0x7f6da8104930 2024-03-26T14:13:07.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6da6000640 1 -- 172.21.15.137:0/3983106688 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d940044c0 con 0x7f6da8104930 2024-03-26T14:13:07.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6da6000640 1 -- 172.21.15.137:0/3983106688 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94003c60 con 0x7f6da8104930 2024-03-26T14:13:07.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da81a18f0 con 0x7f6da8104930 2024-03-26T14:13:07.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.540+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da81a1df0 con 0x7f6da8104930 2024-03-26T14:13:07.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.544+0000 7f6db2d9d740 1 Processor -- start 2024-03-26T14:13:07.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.544+0000 7f6db2d9d740 1 -- start start 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.544+0000 7f6da6000640 1 -- 172.21.15.137:0/3983106688 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6d94046020 con 0x7f6da8104930 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c641216eb0 con 0x55c6412b6e10 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6411bfa60 con 0x55c6412b1ca0 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c641212f50 con 0x55c6412b12b0 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6dad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6dae000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412b92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43500/0 (socket says 172.21.15.137:43500) 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6dad600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:58900/0 (socket says 172.21.15.137:58900) 2024-03-26T14:13:07.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.545+0000 7f6dacc00640 1 -- 172.21.15.137:0/4113336408 learned_addr learned my addr 172.21.15.137:0/4113336408 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6dad600640 1 -- 172.21.15.137:0/4113336408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 msgr2=0x55c6412b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6dad600640 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6dad600640 1 -- 172.21.15.137:0/4113336408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 msgr2=0x55c6412b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6dad600640 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412b92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6dad600640 1 -- 172.21.15.137:0/4113336408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c640ff37e0 con 0x55c6412b1ca0 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6dad600640 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412b67a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0103d0 tx=0x7f6d9c010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f02f5ef52b4c9a53 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6da6000640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d900a2880 0x7f6d900a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6d9a600640 1 -- 172.21.15.137:0/4113336408 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c011700 con 0x55c6412b1ca0 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6d9a600640 1 -- 172.21.15.137:0/4113336408 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d9c011d50 con 0x55c6412b1ca0 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.546+0000 7f6d9a600640 1 -- 172.21.15.137:0/4113336408 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c01ad40 con 0x55c6412b1ca0 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.547+0000 7f6da6000640 1 -- 172.21.15.137:0/3983106688 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d940f1010 con 0x7f6da8104930 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.547+0000 7f6dae000640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d900a2880 0x7f6d900a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.547+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4113336408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 msgr2=0x55c6412b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.547+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412b67a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0103d0 tx=0x7f6d9c010890 comp rx=0 tx=0).stop 2024-03-26T14:13:07.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.547+0000 7f6dae000640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d900a2880 0x7f6d900a4d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55c6412b2500 tx=0x7f6da0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4113336408 shutdown_connections 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412b67a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412b92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4113336408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4113336408 >> 172.21.15.137:0/4113336408 conn(0x55c6412ac820 msgr2=0x55c6412acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4113336408 shutdown_connections 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4113336408 wait complete. 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 -- start start 2024-03-26T14:13:07.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.548+0000 7f6db2d9d740 1 Processor -- start 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412bb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c6412c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6412b9dc0 con 0x55c6412b1ca0 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6411d2eb0 con 0x55c6412b6e10 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6412b9f40 con 0x55c6412b12b0 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6db2d9d740 10 client.0 _ll_register_callbacks cb 0x55c6411c7150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412bb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412bb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43514/0 (socket says 172.21.15.137:43514) 2024-03-26T14:13:07.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.549+0000 7f6dacc00640 1 -- 172.21.15.137:0/2759357882 learned_addr learned my addr 172.21.15.137:0/2759357882 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dae000640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c6412c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dad600640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dacc00640 1 -- 172.21.15.137:0/2759357882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 msgr2=0x55c6412bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dacc00640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412bb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dacc00640 1 -- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 msgr2=0x55c6412c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dacc00640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c6412c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dacc00640 1 -- 172.21.15.137:0/2759357882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c641053f70 con 0x55c6412b1ca0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dae000640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c6412c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6dacc00640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412bb970 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6d940f12f0 tx=0x7f6d94009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7695f14902ffb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6d8ea00640 1 -- 172.21.15.137:0/2759357882 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940bd120 con 0x55c6412b1ca0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6d8ea00640 1 -- 172.21.15.137:0/2759357882 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d940f3070 con 0x55c6412b1ca0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d9c011d50 con 0x55c6412b1ca0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6d8ea00640 1 -- 172.21.15.137:0/2759357882 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d940bcf40 con 0x55c6412b1ca0 2024-03-26T14:13:07.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.550+0000 7f6db2d9d740 10 client.17940 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c640eeb170 con 0x55c6412b1ca0 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6d8ea00640 1 -- 172.21.15.137:0/2759357882 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d9c011d50 con 0x55c6412b1ca0 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6d8ea00640 1 -- 172.21.15.137:0/2759357882 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6d940c58f0 con 0x55c6412b1ca0 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 10 client.17940 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 10 client.17940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 20 client.17940 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c6412c71b0 0x55c6412c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 4 client.17940 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.551+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6412243f0 con 0x55c6412c71b0 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.552+0000 7f6dad600640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c6412c71b0 0x55c6412c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.552+0000 7f6d8ea00640 10 client.17940 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.552+0000 7f6dad600640 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c6412c71b0 0x55c6412c9620 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36857fa312e80957 server_cookie=cf9e3d1dfd076a20 in_seq=0 out_seq=0 2024-03-26T14:13:07.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.553+0000 7f6d8ea00640 1 -- 172.21.15.137:0/2759357882 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c6412243f0 con 0x55c6412c71b0 2024-03-26T14:13:07.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.553+0000 7f6d8ea00640 10 client.17940 handle_command_reply: tid=0 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.553+0000 7f6db2d9d740 1 client.17940 shutdown 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 msgr2=0x55c6412bb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412bb970 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6d940f12f0 tx=0x7f6d94009620 comp rx=0 tx=0).stop 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 shutdown_connections 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c6412c71b0 0x55c6412c9620 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c6412c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412bb970 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6d8ea00640 0 client.17940 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/2759357882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 >> 172.21.15.137:0/2759357882 conn(0x55c6412ac820 msgr2=0x55c6412aba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 shutdown_connections 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 1 -- 172.21.15.137:0/2759357882 wait complete. 2024-03-26T14:13:07.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.554+0000 7f6db2d9d740 20 client.17940 trim_cache size 0 max 16384 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.575+0000 7f6db2d9d740 1 Processor -- start 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.575+0000 7f6db2d9d740 1 -- start start 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412c2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81ae800 con 0x55c6412b6e10 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81ac300 con 0x55c6412b1ca0 2024-03-26T14:13:07.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81aa380 con 0x55c6412b12b0 2024-03-26T14:13:07.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43528/0 (socket says 172.21.15.137:43528) 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 -- 172.21.15.137:0/4131675342 learned_addr learned my addr 172.21.15.137:0/4131675342 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dacc00640 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dad600640 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 -- 172.21.15.137:0/4131675342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 msgr2=0x55c6412b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 -- 172.21.15.137:0/4131675342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 msgr2=0x55c6412c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.576+0000 7f6dae000640 1 -- 172.21.15.137:0/4131675342 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6412b60b0 con 0x55c6412b6e10 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.577+0000 7f6dae000640 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412cdf20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6da0012580 tx=0x7f6da0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2321c67273139d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.577+0000 7f6d99c00640 1 -- 172.21.15.137:0/4131675342 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da001b430 con 0x55c6412b6e10 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.577+0000 7f6d99c00640 1 -- 172.21.15.137:0/4131675342 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6da001ba80 con 0x55c6412b6e10 2024-03-26T14:13:07.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.577+0000 7f6d99c00640 1 -- 172.21.15.137:0/4131675342 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0025ac0 con 0x55c6412b6e10 2024-03-26T14:13:07.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4131675342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 msgr2=0x55c6412cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412cdf20 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6da0012580 tx=0x7f6da0015000 comp rx=0 tx=0).stop 2024-03-26T14:13:07.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4131675342 shutdown_connections 2024-03-26T14:13:07.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b1ca0 0x55c6412c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b6e10 0x55c6412cdf20 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/4131675342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4131675342 >> 172.21.15.137:0/4131675342 conn(0x55c6412ac820 msgr2=0x55c6412aba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4131675342 shutdown_connections 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 -- 172.21.15.137:0/4131675342 wait complete. 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.578+0000 7f6db2d9d740 1 -- start start 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 Processor -- start 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c641322af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81aa980 con 0x55c6412b1ca0 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81abca0 con 0x55c6412b6e10 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da81aafe0 con 0x55c6412b12b0 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.579+0000 7f6db2d9d740 10 client.0 _ll_register_callbacks cb 0x7f6d940f4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:07.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412c1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43540/0 (socket says 172.21.15.137:43540) 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 -- 172.21.15.137:0/1103048722 learned_addr learned my addr 172.21.15.137:0/1103048722 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dad600640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dae000640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c641322af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 -- 172.21.15.137:0/1103048722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 msgr2=0x55c6412c19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412c19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 -- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 msgr2=0x55c641322af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c641322af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 -- 172.21.15.137:0/1103048722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6413230b0 con 0x55c6412b1ca0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dad600640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6dacc00640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412c1f20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6d94006010 tx=0x7f6d940c5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342acdb37ba30932 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.580+0000 7f6d8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6d8ea00640 1 -- 172.21.15.137:0/1103048722 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940bd5e0 con 0x55c6412b1ca0 2024-03-26T14:13:07.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6d8ea00640 1 -- 172.21.15.137:0/1103048722 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d940a6d80 con 0x55c6412b1ca0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6da001ba80 con 0x55c6412b1ca0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6d8ea00640 1 -- 172.21.15.137:0/1103048722 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d940d6490 con 0x55c6412b1ca0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 10 client.17946 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c641324490 con 0x55c6412b1ca0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6d8ea00640 1 -- 172.21.15.137:0/1103048722 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6da001ba80 con 0x55c6412b1ca0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6d8ea00640 1 -- 172.21.15.137:0/1103048722 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6d940b9330 con 0x55c6412b1ca0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 10 client.17946 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 20 client.17946 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c641324bc0 0x55c641326fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 4 client.17946 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.581+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c64121e740 con 0x55c641324bc0 2024-03-26T14:13:07.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.582+0000 7f6dad600640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c641324bc0 0x55c641326fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.583+0000 7f6dad600640 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c641324bc0 0x55c641326fe0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5059be392649807e server_cookie=3a5ec57a5e738415 in_seq=0 out_seq=0 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.583+0000 7f6d8ea00640 10 client.17946 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.584+0000 7f6d8ea00640 1 -- 172.21.15.137:0/1103048722 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c64121e740 con 0x55c641324bc0 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.584+0000 7f6d8ea00640 10 client.17946 handle_command_reply: tid=0 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.584+0000 7f6db2d9d740 1 client.17946 shutdown 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 msgr2=0x55c6412c1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412c1f20 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6d94006010 tx=0x7f6d940c5c00 comp rx=0 tx=0).stop 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 shutdown_connections 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55c641324bc0 0x55c641326fe0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6412b6e10 0x55c641322af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c6412b1ca0 0x55c6412c1f20 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 --2- 172.21.15.137:0/1103048722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6412b12b0 0x55c6412c19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6d8ea00640 0 client.17946 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 >> 172.21.15.137:0/1103048722 conn(0x55c6412ac820 msgr2=0x55c6412b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 shutdown_connections 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 1 -- 172.21.15.137:0/1103048722 wait complete. 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.585+0000 7f6db2d9d740 20 client.17946 trim_cache size 0 max 16384 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "3278046f-fc4a-4525-a0be-ebd4d23892e9", 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:13:07.589 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:13:07.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d900a2880 msgr2=0x7f6d900a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d900a2880 0x7f6d900a4d40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55c6412b2500 tx=0x7f6da0009500 comp rx=0 tx=0).stop 2024-03-26T14:13:07.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 msgr2=0x7f6da8199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8199f80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000c00 tx=0x7f6d940040c0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 shutdown_connections 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d900a2880 0x7f6d900a4d40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6da8107260 0x7f6da81a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6da8104930 0x7f6da8199f80 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 --2- 172.21.15.137:0/3983106688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6da8102000 0x7f6da8199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 >> 172.21.15.137:0/3983106688 conn(0x7f6da80fba10 msgr2=0x7f6da8101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 shutdown_connections 2024-03-26T14:13:07.591 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.586+0000 7f6daea00640 1 -- 172.21.15.137:0/3983106688 wait complete. 2024-03-26T14:13:07.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:13:07.606 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 -- 172.21.15.137:0/1466136212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670100480 msgr2=0x7fa6701008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 --2- 172.21.15.137:0/1466136212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670100480 0x7fa6701008a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa660009900 tx=0x7fa66002bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 -- 172.21.15.137:0/1466136212 shutdown_connections 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 --2- 172.21.15.137:0/1466136212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670102030 0x7fa6701088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 --2- 172.21.15.137:0/1466136212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa670101670 0x7fa670101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 --2- 172.21.15.137:0/1466136212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670100480 0x7fa6701008a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 -- 172.21.15.137:0/1466136212 >> 172.21.15.137:0/1466136212 conn(0x7fa6700fba10 msgr2=0x7fa6700fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 -- 172.21.15.137:0/1466136212 shutdown_connections 2024-03-26T14:13:07.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.809+0000 7fa674e00640 1 -- 172.21.15.137:0/1466136212 wait complete. 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.810+0000 7fa674e00640 1 Processor -- start 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.810+0000 7fa674e00640 1 -- start start 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa674e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 0x7fa670104410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa674e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670101670 0x7fa670102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa674e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa670102030 0x7fa670102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa674e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67010c010 con 0x7fa670101670 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa674e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67010be90 con 0x7fa670100480 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa674e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67010c190 con 0x7fa670102030 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 0x7fa670104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 0x7fa670104410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:58948/0 (socket says 172.21.15.137:58948) 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670101670 0x7fa670102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 -- 172.21.15.137:0/2730720958 learned_addr learned my addr 172.21.15.137:0/2730720958 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66fe00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa670102030 0x7fa670102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 -- 172.21.15.137:0/2730720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa670102030 msgr2=0x7fa670102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa670102030 0x7fa670102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 -- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670101670 msgr2=0x7fa670102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670101670 0x7fa670102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.811+0000 7fa66f400640 1 -- 172.21.15.137:0/2730720958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670103820 con 0x7fa670100480 2024-03-26T14:13:07.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.812+0000 7fa66f400640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 0x7fa670104410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa660000c00 tx=0x7fa6600040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.812+0000 7fa667e00640 1 -- 172.21.15.137:0/2730720958 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660043020 con 0x7fa670100480 2024-03-26T14:13:07.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.812+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6701a1900 con 0x7fa670100480 2024-03-26T14:13:07.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.812+0000 7fa667e00640 1 -- 172.21.15.137:0/2730720958 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa6600044c0 con 0x7fa670100480 2024-03-26T14:13:07.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.812+0000 7fa667e00640 1 -- 172.21.15.137:0/2730720958 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660003c60 con 0x7fa670100480 2024-03-26T14:13:07.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.812+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6701a1ea0 con 0x7fa670100480 2024-03-26T14:13:07.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 Processor -- start 2024-03-26T14:13:07.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 -- start start 2024-03-26T14:13:07.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa667e00640 1 -- 172.21.15.137:0/2730720958 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa660046020 con 0x7fa670100480 2024-03-26T14:13:07.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 0x56301e974470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e979540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e979bb0 0x56301e97c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e8d9250 con 0x56301e974050 2024-03-26T14:13:07.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e881a30 con 0x56301e979bb0 2024-03-26T14:13:07.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.816+0000 7fa679067740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e8d52f0 con 0x56301e974a40 2024-03-26T14:13:07.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 0x56301e974470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 0x56301e974470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43554/0 (socket says 172.21.15.137:43554) 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 -- 172.21.15.137:0/1615303471 learned_addr learned my addr 172.21.15.137:0/1615303471 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66f400640 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e979540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66ea00640 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e979bb0 0x56301e97c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 -- 172.21.15.137:0/1615303471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 msgr2=0x56301e979540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e979540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 -- 172.21.15.137:0/1615303471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e979bb0 msgr2=0x56301e97c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e979bb0 0x56301e97c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.817+0000 7fa66fe00640 1 -- 172.21.15.137:0/1615303471 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301e86c690 con 0x56301e974050 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 0x56301e974470 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa668009eb0 tx=0x7fa66800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e78c1c65a483b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa667e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6540a2800 0x7fa6540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa664c00640 1 -- 172.21.15.137:0/1615303471 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668011640 con 0x56301e974050 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa667e00640 1 -- 172.21.15.137:0/2730720958 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa6600f1070 con 0x7fa670100480 2024-03-26T14:13:07.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa664c00640 1 -- 172.21.15.137:0/1615303471 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa668011c90 con 0x56301e974050 2024-03-26T14:13:07.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa66f400640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6540a2800 0x7fa6540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.818+0000 7fa664c00640 1 -- 172.21.15.137:0/1615303471 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66801bc50 con 0x56301e974050 2024-03-26T14:13:07.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa66f400640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6540a2800 0x7fa6540a4cc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa66002ce10 tx=0x7fa660009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 -- 172.21.15.137:0/1615303471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 msgr2=0x56301e974470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 0x56301e974470 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa668009eb0 tx=0x7fa66800f730 comp rx=0 tx=0).stop 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 -- 172.21.15.137:0/1615303471 shutdown_connections 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e979bb0 0x56301e97c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e979540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 --2- 172.21.15.137:0/1615303471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e974050 0x56301e974470 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 -- 172.21.15.137:0/1615303471 >> 172.21.15.137:0/1615303471 conn(0x56301e93c200 msgr2=0x56301e96f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 -- 172.21.15.137:0/1615303471 shutdown_connections 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.819+0000 7fa679067740 1 -- 172.21.15.137:0/1615303471 wait complete. 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 -- start start 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 Processor -- start 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974050 0x56301e97e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974a40 0x56301e97e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e985650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e97cbb0 con 0x56301e979bb0 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e894e80 con 0x56301e974a40 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.820+0000 7fa679067740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56301e97cd30 con 0x56301e974050 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa679067740 10 client.0 _ll_register_callbacks cb 0x56301e8a7130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e985650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974a40 0x56301e97e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974a40 0x56301e97e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:58962/0 (socket says 172.21.15.137:58962) 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66ea00640 1 -- 172.21.15.137:0/2296166835 learned_addr learned my addr 172.21.15.137:0/2296166835 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974050 0x56301e97e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 -- 172.21.15.137:0/2296166835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974050 msgr2=0x56301e97e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974050 0x56301e97e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 -- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974a40 msgr2=0x56301e97e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974a40 0x56301e97e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.821+0000 7fa66fe00640 1 -- 172.21.15.137:0/2296166835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56301e6b77d0 con 0x56301e979bb0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa66fe00640 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e985650 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa668009eb0 tx=0x7fa6680098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4937b264ac38d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa659600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa659600640 1 -- 172.21.15.137:0/2296166835 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668004a40 con 0x56301e979bb0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa668011c90 con 0x56301e979bb0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa659600640 1 -- 172.21.15.137:0/2296166835 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa668004be0 con 0x56301e979bb0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa659600640 1 -- 172.21.15.137:0/2296166835 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa668004de0 con 0x56301e979bb0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa679067740 10 client.17958 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56301e5ed3f0 con 0x56301e979bb0 2024-03-26T14:13:07.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.822+0000 7fa659600640 1 -- 172.21.15.137:0/2296166835 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa668011c90 con 0x56301e979bb0 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa659600640 1 -- 172.21.15.137:0/2296166835 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa668011970 con 0x56301e979bb0 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa679067740 10 client.17958 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa679067740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa679067740 20 client.17958 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa679067740 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e989b40 0x56301e98bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa679067740 4 client.17958 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56301e8e6790 con 0x56301e989b40 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.823+0000 7fa66ea00640 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e989b40 0x56301e98bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.824+0000 7fa659600640 10 client.17958 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.824+0000 7fa66ea00640 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e989b40 0x56301e98bf30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f277ee43760a81 server_cookie=e67b61cbc2b8a2a in_seq=0 out_seq=0 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa659600640 1 -- 172.21.15.137:0/2296166835 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56301e8e6790 con 0x56301e989b40 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa659600640 10 client.17958 handle_command_reply: tid=0 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 client.17958 shutdown 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 msgr2=0x56301e985650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e985650 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa668009eb0 tx=0x7fa6680098e0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 shutdown_connections 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e989b40 0x56301e98bf30 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974a40 0x56301e97e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e985650 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 --2- 172.21.15.137:0/2296166835 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974050 0x56301e97e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa659600640 0 client.17958 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 >> 172.21.15.137:0/2296166835 conn(0x56301e93c200 msgr2=0x56301e96ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 shutdown_connections 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.825+0000 7fa679067740 1 -- 172.21.15.137:0/2296166835 wait complete. 2024-03-26T14:13:07.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.826+0000 7fa679067740 20 client.17958 trim_cache size 0 max 16384 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.845+0000 7fa679067740 1 Processor -- start 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.845+0000 7fa679067740 1 -- start start 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa679067740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e990730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa679067740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e985610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa679067740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e990c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa679067740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701ae7e0 con 0x56301e979bb0 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa679067740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701ac2e0 con 0x56301e974050 2024-03-26T14:13:07.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa679067740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701aa360 con 0x56301e974a40 2024-03-26T14:13:07.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e990c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e990c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43568/0 (socket says 172.21.15.137:43568) 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66fe00640 1 -- 172.21.15.137:0/1326439396 learned_addr learned my addr 172.21.15.137:0/1326439396 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e990730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66ea00640 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e985610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66fe00640 1 -- 172.21.15.137:0/1326439396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 msgr2=0x56301e985610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.846+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e985610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa66fe00640 1 -- 172.21.15.137:0/1326439396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 msgr2=0x56301e990730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e990730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa66fe00640 1 -- 172.21.15.137:0/1326439396 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56301e98c2a0 con 0x56301e979bb0 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e990c70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa668009690 tx=0x7fa668009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c483889c1e8c83 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa65be00640 1 -- 172.21.15.137:0/1326439396 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800fc30 con 0x56301e979bb0 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa65be00640 1 -- 172.21.15.137:0/1326439396 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa668003e20 con 0x56301e979bb0 2024-03-26T14:13:07.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.847+0000 7fa65be00640 1 -- 172.21.15.137:0/1326439396 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680045f0 con 0x56301e979bb0 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 -- 172.21.15.137:0/1326439396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 msgr2=0x56301e990c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e990c70 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa668009690 tx=0x7fa668009500 comp rx=0 tx=0).stop 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 -- 172.21.15.137:0/1326439396 shutdown_connections 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e990c70 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e985610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 --2- 172.21.15.137:0/1326439396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e990730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 -- 172.21.15.137:0/1326439396 >> 172.21.15.137:0/1326439396 conn(0x56301e93c200 msgr2=0x56301e975d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 -- 172.21.15.137:0/1326439396 shutdown_connections 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.848+0000 7fa679067740 1 -- 172.21.15.137:0/1326439396 wait complete. 2024-03-26T14:13:07.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 -- start start 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 Processor -- start 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e9831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e983720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e9846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701aa960 con 0x56301e979bb0 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701abc80 con 0x56301e974050 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701aafc0 con 0x56301e974a40 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.849+0000 7fa679067740 10 client.0 _ll_register_callbacks cb 0x7fa668015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e9846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e9846c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:43582/0 (socket says 172.21.15.137:43582) 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 -- 172.21.15.137:0/1151162637 learned_addr learned my addr 172.21.15.137:0/1151162637 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e9831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66ea00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e983720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 -- 172.21.15.137:0/1151162637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 msgr2=0x56301e983720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e983720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 -- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 msgr2=0x56301e9831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:07.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e9831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.850+0000 7fa66fe00640 1 -- 172.21.15.137:0/1151162637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56301e5ad0f0 con 0x56301e979bb0 2024-03-26T14:13:07.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa66fe00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e9846c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa668005bd0 tx=0x7fa668000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219b4c67aefdb230 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:07.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa659600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa659600640 1 -- 172.21.15.137:0/1151162637 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800fc30 con 0x56301e979bb0 2024-03-26T14:13:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa659600640 1 -- 172.21.15.137:0/1151162637 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa668026070 con 0x56301e979bb0 2024-03-26T14:13:07.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa668003e20 con 0x56301e979bb0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa659600640 1 -- 172.21.15.137:0/1151162637 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa668004800 con 0x56301e979bb0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa679067740 10 client.17964 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56301e9e54b0 con 0x56301e979bb0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.851+0000 7fa659600640 1 -- 172.21.15.137:0/1151162637 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa668003e20 con 0x56301e979bb0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa659600640 1 -- 172.21.15.137:0/1151162637 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fa668011a90 con 0x56301e979bb0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa679067740 10 client.17964 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa679067740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa679067740 20 client.17964 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa679067740 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e9e5d60 0x56301e9e8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa679067740 4 client.17964 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56301e8e0ae0 con 0x56301e9e5d60 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.852+0000 7fa66ea00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e9e5d60 0x56301e9e8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.853+0000 7fa659600640 10 client.17964 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.853+0000 7fa66ea00640 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e9e5d60 0x56301e9e8180 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884d86d7126e28b server_cookie=bde24edf31c9fb12 in_seq=0 out_seq=0 2024-03-26T14:13:07.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa659600640 1 -- 172.21.15.137:0/1151162637 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56301e8e0ae0 con 0x56301e9e5d60 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa659600640 10 client.17964 handle_command_reply: tid=0 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 client.17964 shutdown 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 msgr2=0x56301e9846c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e9846c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa668005bd0 tx=0x7fa668000f40 comp rx=0 tx=0).stop 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 shutdown_connections 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56301e9e5d60 0x56301e9e8180 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56301e979bb0 0x56301e9846c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56301e974a40 0x56301e983720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 --2- 172.21.15.137:0/1151162637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56301e974050 0x56301e9831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 >> 172.21.15.137:0/1151162637 conn(0x56301e93c200 msgr2=0x56301e972c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa659600640 0 client.17964 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 shutdown_connections 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 1 -- 172.21.15.137:0/1151162637 wait complete. 2024-03-26T14:13:07.858 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.854+0000 7fa679067740 20 client.17964 trim_cache size 0 max 16384 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: "3278046f-fc4a-4525-a0be-ebd4d23892e9": { 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "3278046f-fc4a-4525-a0be-ebd4d23892e9", 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:13:07.859 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:13:07.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6540a2800 msgr2=0x7fa6540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6540a2800 0x7fa6540a4cc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa66002ce10 tx=0x7fa660009260 comp rx=0 tx=0).stop 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 msgr2=0x7fa670104410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 0x7fa670104410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa660000c00 tx=0x7fa6600040c0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 shutdown_connections 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa6540a2800 0x7fa6540a4cc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa670102030 0x7fa670102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa670101670 0x7fa670102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 --2- 172.21.15.137:0/2730720958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa670100480 0x7fa670104410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 >> 172.21.15.137:0/2730720958 conn(0x7fa6700fba10 msgr2=0x7fa670068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 shutdown_connections 2024-03-26T14:13:07.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:07.855+0000 7fa674e00640 1 -- 172.21.15.137:0/2730720958 wait complete. 2024-03-26T14:13:07.875 INFO:tasks.cephfs.filesystem:scrub status for tag:3278046f-fc4a-4525-a0be-ebd4d23892e9 - {'path': '/', 'tag': '3278046f-fc4a-4525-a0be-ebd4d23892e9', 'options': 'recursive,force'} 2024-03-26T14:13:08.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:08 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:13:08.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:08 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:08.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:08 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:13:08.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:08 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:08.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:08 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:13:08.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:08 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:08.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:08 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:13:08.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:08 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:08.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:08 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:13:08.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:08 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:08.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:08 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:13:08.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:08 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:09.424 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:09 smithi179 ceph-mon[35472]: pgmap v781: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:09.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:09 smithi159 ceph-mon[35501]: pgmap v781: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:09.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:09 smithi137 ceph-mon[31539]: pgmap v781: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:11.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:11 smithi159 ceph-mon[35501]: pgmap v782: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:11.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:11 smithi137 ceph-mon[31539]: pgmap v782: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:11.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:11 smithi179 ceph-mon[35472]: pgmap v782: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:12.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:12 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:13:12.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:12 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:13:12.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:12 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:13:13.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:13 smithi159 ceph-mon[35501]: pgmap v783: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:13.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:13 smithi137 ceph-mon[31539]: pgmap v783: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:13.576 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:13 smithi179 ceph-mon[35472]: pgmap v783: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:15.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:15 smithi159 ceph-mon[35501]: pgmap v784: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:13:15.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:15 smithi137 ceph-mon[31539]: pgmap v784: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:13:15.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:15 smithi179 ceph-mon[35472]: pgmap v784: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:13:17.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:17 smithi159 ceph-mon[35501]: pgmap v785: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:17.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:17 smithi137 ceph-mon[31539]: pgmap v785: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:17.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:17 smithi179 ceph-mon[35472]: pgmap v785: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:19.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:19 smithi137 ceph-mon[31539]: pgmap v786: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:13:19.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:19 smithi179 ceph-mon[35472]: pgmap v786: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:13:19.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:19 smithi159 ceph-mon[35501]: pgmap v786: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:13:21.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:21 smithi137 ceph-mon[31539]: pgmap v787: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:21.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:21 smithi179 ceph-mon[35472]: pgmap v787: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:21.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:21 smithi159 ceph-mon[35501]: pgmap v787: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:23.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:23 smithi137 ceph-mon[31539]: pgmap v788: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:23.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:23 smithi179 ceph-mon[35472]: pgmap v788: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:23.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:23 smithi159 ceph-mon[35501]: pgmap v788: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:25.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:25 smithi137 ceph-mon[31539]: pgmap v789: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:25 smithi179 ceph-mon[35472]: pgmap v789: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:25.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:25 smithi159 ceph-mon[35501]: pgmap v789: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:27.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:27 smithi137 ceph-mon[31539]: pgmap v790: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:27.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:27 smithi179 ceph-mon[35472]: pgmap v790: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:27.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:27 smithi159 ceph-mon[35501]: pgmap v790: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:29.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:29 smithi137 ceph-mon[31539]: pgmap v791: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:29.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:29 smithi179 ceph-mon[35472]: pgmap v791: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:29.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:29 smithi159 ceph-mon[35501]: pgmap v791: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:31.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:31 smithi137 ceph-mon[31539]: pgmap v792: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:31 smithi179 ceph-mon[35472]: pgmap v792: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:31 smithi159 ceph-mon[35501]: pgmap v792: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:33.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:33 smithi137 ceph-mon[31539]: pgmap v793: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:13:33.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:33 smithi179 ceph-mon[35472]: pgmap v793: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:13:33.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:33 smithi159 ceph-mon[35501]: pgmap v793: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:13:35.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:35 smithi137 ceph-mon[31539]: pgmap v794: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:35.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:35 smithi179 ceph-mon[35472]: pgmap v794: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:35.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:35 smithi159 ceph-mon[35501]: pgmap v794: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:37.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:37 smithi137 ceph-mon[31539]: pgmap v795: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:37.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:37 smithi179 ceph-mon[35472]: pgmap v795: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:37 smithi159 ceph-mon[35501]: pgmap v795: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:37.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:13:37.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:13:38.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 -- 172.21.15.137:0/1430471499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8102000 msgr2=0x7fabb81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 --2- 172.21.15.137:0/1430471499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8102000 0x7fabb81043f0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba802bc40 comp rx=0 tx=0).stop 2024-03-26T14:13:38.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 -- 172.21.15.137:0/1430471499 shutdown_connections 2024-03-26T14:13:38.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 --2- 172.21.15.137:0/1430471499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 0x7fabb8106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 --2- 172.21.15.137:0/1430471499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb8107260 0x7fabb81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 --2- 172.21.15.137:0/1430471499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8102000 0x7fabb81043f0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 -- 172.21.15.137:0/1430471499 >> 172.21.15.137:0/1430471499 conn(0x7fabb80fba10 msgr2=0x7fabb80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 -- 172.21.15.137:0/1430471499 shutdown_connections 2024-03-26T14:13:38.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.105+0000 7fabbd200640 1 -- 172.21.15.137:0/1430471499 wait complete. 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.106+0000 7fabbd200640 1 Processor -- start 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 -- start start 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 0x7fabb8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 0x7fabb8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb819cef0 0x7fabb819f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb810d110 con 0x7fabb8107260 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb810cf90 con 0x7fabb8104930 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabbd200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb810d290 con 0x7fabb819cef0 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabb6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 0x7fabb8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabb6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 0x7fabb8197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59800/0 (socket says 172.21.15.137:59800) 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabb6a00640 1 -- 172.21.15.137:0/3347126680 learned_addr learned my addr 172.21.15.137:0/3347126680 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.107+0000 7fabb7400640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 0x7fabb8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb7e00640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb819cef0 0x7fabb819f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb6a00640 1 -- 172.21.15.137:0/3347126680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb819cef0 msgr2=0x7fabb819f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb6a00640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb819cef0 0x7fabb819f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb6a00640 1 -- 172.21.15.137:0/3347126680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 msgr2=0x7fabb8197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb6a00640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 0x7fabb8197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb6a00640 1 -- 172.21.15.137:0/3347126680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb80ffdf0 con 0x7fabb8107260 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb7400640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 0x7fabb8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabb6a00640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 0x7fabb8197e90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7faba4005fd0 tx=0x7faba400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabafe00640 1 -- 172.21.15.137:0/3347126680 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4017070 con 0x7fabb8107260 2024-03-26T14:13:38.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabafe00640 1 -- 172.21.15.137:0/3347126680 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faba4009c30 con 0x7fabb8107260 2024-03-26T14:13:38.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.108+0000 7fabafe00640 1 -- 172.21.15.137:0/3347126680 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4003d40 con 0x7fabb8107260 2024-03-26T14:13:38.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.109+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb81000e0 con 0x7fabb8107260 2024-03-26T14:13:38.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.109+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb8100680 con 0x7fabb8107260 2024-03-26T14:13:38.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.112+0000 7fabc161d740 1 Processor -- start 2024-03-26T14:13:38.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.112+0000 7fabc161d740 1 -- start start 2024-03-26T14:13:38.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabc161d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1912470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabc161d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c1917540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabc161d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c191a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabc161d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2c1877250 con 0x55b2c1912050 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabc161d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2c181fa30 con 0x55b2c1912a40 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabc161d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2c18732f0 con 0x55b2c1917bb0 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabafe00640 1 -- 172.21.15.137:0/3347126680 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7faba4003ee0 con 0x7fabb8107260 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1912470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1912470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59806/0 (socket says 172.21.15.137:59806) 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabb7e00640 1 -- 172.21.15.137:0/420153737 learned_addr learned my addr 172.21.15.137:0/420153737 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.113+0000 7fabb7400640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c1917540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb6a00640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c191a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7e00640 1 -- 172.21.15.137:0/420153737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 msgr2=0x55b2c191a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7e00640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c191a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7e00640 1 -- 172.21.15.137:0/420153737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 msgr2=0x55b2c1917540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7e00640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c1917540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7e00640 1 -- 172.21.15.137:0/420153737 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2c180a690 con 0x55b2c1912050 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7400640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c1917540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb6a00640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c191a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabb7e00640 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1912470 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fabb0007dd0 tx=0x7fabb000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8fea679a907f42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.114+0000 7fabacc00640 1 -- 172.21.15.137:0/420153737 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0014070 con 0x55b2c1912050 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabacc00640 1 -- 172.21.15.137:0/420153737 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fabb0004d20 con 0x55b2c1912050 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabacc00640 1 -- 172.21.15.137:0/420153737 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0010790 con 0x55b2c1912050 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabafe00640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fab9c0a2880 0x7fab9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabafe00640 1 -- 172.21.15.137:0/3347126680 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faba40cebe0 con 0x7fabb8107260 2024-03-26T14:13:38.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabb7400640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fab9c0a2880 0x7fab9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabc161d740 1 -- 172.21.15.137:0/420153737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 msgr2=0x55b2c1912470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.115+0000 7fabc161d740 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1912470 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fabb0007dd0 tx=0x7fabb000e720 comp rx=0 tx=0).stop 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 -- 172.21.15.137:0/420153737 shutdown_connections 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c1917540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c191a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 --2- 172.21.15.137:0/420153737 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1912470 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 -- 172.21.15.137:0/420153737 >> 172.21.15.137:0/420153737 conn(0x55b2c18da200 msgr2=0x55b2c190d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 -- 172.21.15.137:0/420153737 shutdown_connections 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 -- 172.21.15.137:0/420153737 wait complete. 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabb7400640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fab9c0a2880 0x7fab9c0a4d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faba8036040 tx=0x7faba802c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.116+0000 7fabc161d740 1 -- start start 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 Processor -- start 2024-03-26T14:13:38.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912050 0x55b2c191bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c191c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 0x55b2c1923540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2c191ab60 con 0x55b2c1917bb0 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2c1832e80 con 0x55b2c1912a40 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2c191ace0 con 0x55b2c1912050 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabc161d740 10 client.0 _ll_register_callbacks cb 0x55b2c1845130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 0x55b2c1923540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabb7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 0x55b2c1923540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59810/0 (socket says 172.21.15.137:59810) 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.117+0000 7fabb7e00640 1 -- 172.21.15.137:0/2183742522 learned_addr learned my addr 172.21.15.137:0/2183742522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7400640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912050 0x55b2c191bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb6a00640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c191c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7e00640 1 -- 172.21.15.137:0/2183742522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912050 msgr2=0x55b2c191bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7e00640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912050 0x55b2c191bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7e00640 1 -- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 msgr2=0x55b2c191c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7e00640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c191c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7e00640 1 -- 172.21.15.137:0/2183742522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2c16557d0 con 0x55b2c1917bb0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7fabb7e00640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 0x55b2c1923540 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fabb0004f30 tx=0x7fabb0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cda38ee6237d5df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.118+0000 7faba1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7faba1600640 1 -- 172.21.15.137:0/2183742522 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0014070 con 0x55b2c1917bb0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7faba1600640 1 -- 172.21.15.137:0/2183742522 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fabb0010510 con 0x55b2c1917bb0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fabb0004d20 con 0x55b2c1917bb0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7faba1600640 1 -- 172.21.15.137:0/2183742522 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabb002f050 con 0x55b2c1917bb0 2024-03-26T14:13:38.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7fabc161d740 10 client.17976 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b2c158b3f0 con 0x55b2c1917bb0 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.119+0000 7faba1600640 1 -- 172.21.15.137:0/2183742522 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fabb0004d20 con 0x55b2c1917bb0 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7faba1600640 1 -- 172.21.15.137:0/2183742522 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fabb00187b0 con 0x55b2c1917bb0 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabc161d740 10 client.17976 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabc161d740 10 client.17976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabc161d740 20 client.17976 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabc161d740 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1927aa0 0x55b2c1929e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabc161d740 4 client.17976 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2c1884790 con 0x55b2c1927aa0 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.120+0000 7fabb7400640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1927aa0 0x55b2c1929e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.121+0000 7fabb7400640 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1927aa0 0x55b2c1929e90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a53fa09d53977b13 server_cookie=5f06df9bfc6be5d in_seq=0 out_seq=0 2024-03-26T14:13:38.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.121+0000 7faba1600640 10 client.17976 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:38.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.122+0000 7faba1600640 1 -- 172.21.15.137:0/2183742522 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b2c1884790 con 0x55b2c1927aa0 2024-03-26T14:13:38.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.122+0000 7faba1600640 10 client.17976 handle_command_reply: tid=0 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 client.17976 shutdown 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 msgr2=0x55b2c1923540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 0x55b2c1923540 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fabb0004f30 tx=0x7fabb0005850 comp rx=0 tx=0).stop 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 shutdown_connections 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1927aa0 0x55b2c1929e90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912a40 0x55b2c191c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1917bb0 0x55b2c1923540 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 --2- 172.21.15.137:0/2183742522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912050 0x55b2c191bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7faba1600640 0 client.17976 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 >> 172.21.15.137:0/2183742522 conn(0x55b2c18da200 msgr2=0x55b2c190cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 shutdown_connections 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 1 -- 172.21.15.137:0/2183742522 wait complete. 2024-03-26T14:13:38.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.123+0000 7fabc161d740 20 client.17976 trim_cache size 0 max 16384 2024-03-26T14:13:38.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.140+0000 7fabc161d740 1 Processor -- start 2024-03-26T14:13:38.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 -- start start 2024-03-26T14:13:38.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912050 0x55b2c1911310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 0x55b2c1923500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c19132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81aecb0 con 0x55b2c1912a40 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81ac7b0 con 0x55b2c1912050 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabc161d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81aa830 con 0x55b2c1917bb0 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabb6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 0x55b2c1923500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabb6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 0x55b2c1923500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59820/0 (socket says 172.21.15.137:59820) 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabb6a00640 1 -- 172.21.15.137:0/3039964524 learned_addr learned my addr 172.21.15.137:0/3039964524 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.141+0000 7fabb7400640 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912050 0x55b2c1911310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb6a00640 1 -- 172.21.15.137:0/3039964524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 msgr2=0x55b2c19132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb7e00640 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c19132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb6a00640 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c19132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb6a00640 1 -- 172.21.15.137:0/3039964524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912050 msgr2=0x55b2c1911310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb6a00640 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912050 0x55b2c1911310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb6a00640 1 -- 172.21.15.137:0/3039964524 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2c1917900 con 0x55b2c1912a40 2024-03-26T14:13:38.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb7400640 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912050 0x55b2c1911310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:38.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7fabb6a00640 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 0x55b2c1923500 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7faba409ae70 tx=0x7faba409b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3961134f44705c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7faba3e00640 1 -- 172.21.15.137:0/3039964524 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba409f2c0 con 0x55b2c1912a40 2024-03-26T14:13:38.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7faba3e00640 1 -- 172.21.15.137:0/3039964524 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faba40a3910 con 0x55b2c1912a40 2024-03-26T14:13:38.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.142+0000 7faba3e00640 1 -- 172.21.15.137:0/3039964524 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba40041d0 con 0x55b2c1912a40 2024-03-26T14:13:38.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 -- 172.21.15.137:0/3039964524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 msgr2=0x55b2c1923500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 0x55b2c1923500 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7faba409ae70 tx=0x7faba409b870 comp rx=0 tx=0).stop 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 -- 172.21.15.137:0/3039964524 shutdown_connections 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1917bb0 0x55b2c19132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912a40 0x55b2c1923500 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 --2- 172.21.15.137:0/3039964524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1912050 0x55b2c1911310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 -- 172.21.15.137:0/3039964524 >> 172.21.15.137:0/3039964524 conn(0x55b2c18da200 msgr2=0x55b2c1910c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 -- 172.21.15.137:0/3039964524 shutdown_connections 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.143+0000 7fabc161d740 1 -- 172.21.15.137:0/3039964524 wait complete. 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 -- start start 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 Processor -- start 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1920c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912a40 0x55b2c19211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1917bb0 0x55b2c1922160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81aae30 con 0x55b2c1912050 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81ac150 con 0x55b2c1917bb0 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb81ab490 con 0x55b2c1912a40 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.144+0000 7fabc161d740 10 client.0 _ll_register_callbacks cb 0x7fabb0007c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1920c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1920c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59834/0 (socket says 172.21.15.137:59834) 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 -- 172.21.15.137:0/707066655 learned_addr learned my addr 172.21.15.137:0/707066655 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb6a00640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912a40 0x55b2c19211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7e00640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1917bb0 0x55b2c1922160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 -- 172.21.15.137:0/707066655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912a40 msgr2=0x55b2c19211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912a40 0x55b2c19211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 -- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1917bb0 msgr2=0x55b2c1922160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1917bb0 0x55b2c1922160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 -- 172.21.15.137:0/707066655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2c154b0f0 con 0x55b2c1912050 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7fabb7400640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1920c60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7faba8006010 tx=0x7faba8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c735c59219a8b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.145+0000 7faba1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7faba1600640 1 -- 172.21.15.137:0/707066655 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80054d0 con 0x55b2c1912050 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7faba1600640 1 -- 172.21.15.137:0/707066655 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faba8005670 con 0x55b2c1912050 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faba40a3910 con 0x55b2c1912050 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7faba1600640 1 -- 172.21.15.137:0/707066655 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faba8063050 con 0x55b2c1912050 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7fabc161d740 10 client.17982 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b2c1983260 con 0x55b2c1912050 2024-03-26T14:13:38.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.146+0000 7faba1600640 1 -- 172.21.15.137:0/707066655 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faba40a3910 con 0x55b2c1912050 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7faba1600640 1 -- 172.21.15.137:0/707066655 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faba80038d0 con 0x55b2c1912050 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabc161d740 10 client.17982 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabc161d740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabc161d740 20 client.17982 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabc161d740 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1983ad0 0x55b2c1985ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabc161d740 4 client.17982 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:13:38.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b2c187eae0 con 0x55b2c1983ad0 2024-03-26T14:13:38.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.147+0000 7fabb6a00640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1983ad0 0x55b2c1985ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.148+0000 7fabb6a00640 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1983ad0 0x55b2c1985ef0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fad0bf395246736 server_cookie=b4e409227a9a1236 in_seq=0 out_seq=0 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.148+0000 7faba1600640 10 client.17982 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7faba1600640 1 -- 172.21.15.137:0/707066655 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b2c187eae0 con 0x55b2c1983ad0 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7faba1600640 10 client.17982 handle_command_reply: tid=0 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 client.17982 shutdown 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 msgr2=0x55b2c1920c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1920c60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7faba8006010 tx=0x7faba8009260 comp rx=0 tx=0).stop 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 shutdown_connections 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b2c1983ad0 0x55b2c1985ef0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2c1917bb0 0x55b2c1922160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c1912a40 0x55b2c19211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 --2- 172.21.15.137:0/707066655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b2c1912050 0x55b2c1920c60 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 >> 172.21.15.137:0/707066655 conn(0x55b2c18da200 msgr2=0x55b2c1911490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7faba1600640 0 client.17982 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 shutdown_connections 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 1 -- 172.21.15.137:0/707066655 wait complete. 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.149+0000 7fabc161d740 20 client.17982 trim_cache size 0 max 16384 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:13:38.154 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:13:38.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fab9c0a2880 msgr2=0x7fab9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fab9c0a2880 0x7fab9c0a4d40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faba8036040 tx=0x7faba802c7f0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 msgr2=0x7fabb8197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 0x7fabb8197e90 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7faba4005fd0 tx=0x7faba400dde0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 shutdown_connections 2024-03-26T14:13:38.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fab9c0a2880 0x7fab9c0a4d40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabb819cef0 0x7fabb819f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fabb8107260 0x7fabb8197e90 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 --2- 172.21.15.137:0/3347126680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabb8104930 0x7fabb8197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 >> 172.21.15.137:0/3347126680 conn(0x7fabb80fba10 msgr2=0x7fabb81028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 shutdown_connections 2024-03-26T14:13:38.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.150+0000 7fabbd200640 1 -- 172.21.15.137:0/3347126680 wait complete. 2024-03-26T14:13:38.168 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:13:38.169 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:13:38.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 -- 172.21.15.137:0/3552201884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701036f0 msgr2=0x7f7270103b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 --2- 172.21.15.137:0/3552201884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701036f0 0x7f7270103b10 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7264009900 tx=0x7f726402bc40 comp rx=0 tx=0).stop 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 -- 172.21.15.137:0/3552201884 shutdown_connections 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 --2- 172.21.15.137:0/3552201884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701048e0 0x7f7270104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 --2- 172.21.15.137:0/3552201884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72701052a0 0x7f727010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 --2- 172.21.15.137:0/3552201884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701036f0 0x7f7270103b10 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 -- 172.21.15.137:0/3552201884 >> 172.21.15.137:0/3552201884 conn(0x7f72700fec80 msgr2=0x7f72701010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 -- 172.21.15.137:0/3552201884 shutdown_connections 2024-03-26T14:13:38.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.400+0000 7f7276c00640 1 -- 172.21.15.137:0/3552201884 wait complete. 2024-03-26T14:13:38.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 Processor -- start 2024-03-26T14:13:38.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 -- start start 2024-03-26T14:13:38.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701036f0 0x7f72701998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72701048e0 0x7f7270199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 0x7f72701a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727010f240 con 0x7f72701052a0 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727010f0c0 con 0x7f72701036f0 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.401+0000 7f7276c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727010f3c0 con 0x7f72701048e0 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 0x7f72701a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 0x7f72701a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59850/0 (socket says 172.21.15.137:59850) 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 -- 172.21.15.137:0/2591679674 learned_addr learned my addr 172.21.15.137:0/2591679674 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7275800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701036f0 0x7f72701998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7274e00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72701048e0 0x7f7270199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 -- 172.21.15.137:0/2591679674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72701048e0 msgr2=0x7f7270199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72701048e0 0x7f7270199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 -- 172.21.15.137:0/2591679674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701036f0 msgr2=0x7f72701998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701036f0 0x7f72701998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7276200640 1 -- 172.21.15.137:0/2591679674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72701a14d0 con 0x7f72701052a0 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.402+0000 7f7275800640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701036f0 0x7f72701998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.403+0000 7f7276200640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 0x7f72701a0e60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f726800cce0 tx=0x7f726800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.403+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726800f450 con 0x7f72701052a0 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.403+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f726800faa0 con 0x7f72701052a0 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.403+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72701a17c0 con 0x7f72701052a0 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.403+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268018b70 con 0x7f72701052a0 2024-03-26T14:13:38.406 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.403+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72701a1cc0 con 0x7f72701052a0 2024-03-26T14:13:38.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.405+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727010aa10 con 0x7f72701052a0 2024-03-26T14:13:38.407 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.407+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7268018d10 con 0x7f72701052a0 2024-03-26T14:13:38.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.409+0000 7f726e000640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f72580a2800 0x7f72580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.409 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.409+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f72680cf250 con 0x7f72701052a0 2024-03-26T14:13:38.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.409+0000 7f7275800640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f72580a2800 0x7f72580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.410 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.410+0000 7f7275800640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f72580a2800 0x7f72580a4cc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f726402c250 tx=0x7f726402c7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.414+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7268014030 con 0x7f72701052a0 2024-03-26T14:13:38.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.688+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72700622a0 con 0x7f72701052a0 2024-03-26T14:13:38.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.690+0000 7f726e000640 1 -- 172.21.15.137:0/2591679674 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f726809bee0 con 0x7f72701052a0 2024-03-26T14:13:38.691 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:13:38.691 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:13:38.692 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:13:38.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f72580a2800 msgr2=0x7f72580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f72580a2800 0x7f72580a4cc0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f726402c250 tx=0x7f726402c7f0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 msgr2=0x7f72701a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 0x7f72701a0e60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f726800cce0 tx=0x7f726800a620 comp rx=0 tx=0).stop 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 shutdown_connections 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f72580a2800 0x7f72580a4cc0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72701052a0 0x7f72701a0e60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f72701048e0 0x7f7270199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 --2- 172.21.15.137:0/2591679674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72701036f0 0x7f72701998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.694+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 >> 172.21.15.137:0/2591679674 conn(0x7f72700fec80 msgr2=0x7f7270107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.695+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 shutdown_connections 2024-03-26T14:13:38.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.695+0000 7f7276c00640 1 -- 172.21.15.137:0/2591679674 wait complete. 2024-03-26T14:13:38.716 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:13:38.716 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:13:38.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 -- 172.21.15.137:0/3479662954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 msgr2=0x7f45fc104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 --2- 172.21.15.137:0/3479662954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc104e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009900 tx=0x7f45e802bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:38.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 -- 172.21.15.137:0/3479662954 shutdown_connections 2024-03-26T14:13:38.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 --2- 172.21.15.137:0/3479662954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc104e10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 --2- 172.21.15.137:0/3479662954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc105350 0x7f45fc10bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 --2- 172.21.15.137:0/3479662954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45fc1037a0 0x7f45fc103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 -- 172.21.15.137:0/3479662954 >> 172.21.15.137:0/3479662954 conn(0x7f45fc0fed10 msgr2=0x7f45fc101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 -- 172.21.15.137:0/3479662954 shutdown_connections 2024-03-26T14:13:38.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.967+0000 7f45fb200640 1 -- 172.21.15.137:0/3479662954 wait complete. 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.968+0000 7f45fb200640 1 Processor -- start 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 -- start start 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc1037a0 0x7f45fc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45fc105350 0x7f45fc1a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc10f2d0 con 0x7f45fc1037a0 2024-03-26T14:13:38.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc10f150 con 0x7f45fc104990 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fb200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc10f450 con 0x7f45fc105350 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc1037a0 0x7f45fc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55246/0 (socket says 172.21.15.137:55246) 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc1037a0 0x7f45fc1999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59872/0 (socket says 172.21.15.137:59872) 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45f9400640 1 -- 172.21.15.137:0/3145783931 learned_addr learned my addr 172.21.15.137:0/3145783931 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.969+0000 7f45fa800640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45fc105350 0x7f45fc1a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.970+0000 7f45f9400640 1 -- 172.21.15.137:0/3145783931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45fc105350 msgr2=0x7f45fc1a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.970+0000 7f45f9400640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45fc105350 0x7f45fc1a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.970+0000 7f45f9400640 1 -- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc1037a0 msgr2=0x7f45fc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.970+0000 7f45f9400640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc1037a0 0x7f45fc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.970+0000 7f45f9400640 1 -- 172.21.15.137:0/3145783931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc1a15c0 con 0x7f45fc104990 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.970+0000 7f45f9400640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc199ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45e8006010 tx=0x7f45e8005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.971+0000 7f45f2a00640 1 -- 172.21.15.137:0/3145783931 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8043020 con 0x7f45fc104990 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.971+0000 7f45f2a00640 1 -- 172.21.15.137:0/3145783931 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e80043d0 con 0x7f45fc104990 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.971+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc1a1850 con 0x7f45fc104990 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.971+0000 7f45f2a00640 1 -- 172.21.15.137:0/3145783931 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8003b80 con 0x7f45fc104990 2024-03-26T14:13:38.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.971+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45fc1a1df0 con 0x7f45fc104990 2024-03-26T14:13:38.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f45f2a00640 1 -- 172.21.15.137:0/3145783931 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f45e8046020 con 0x7f45fc104990 2024-03-26T14:13:38.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 Processor -- start 2024-03-26T14:13:38.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 -- start start 2024-03-26T14:13:38.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac5fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ac5592c0 con 0x5565ac5f6580 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ac501c90 con 0x5565ac5f49f0 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.975+0000 7f4604010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ac555360 con 0x5565ac5f5bc0 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45f9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac5fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55258/0 (socket says 172.21.15.137:55258) 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45fa800640 1 -- 172.21.15.137:0/3695594491 learned_addr learned my addr 172.21.15.137:0/3695594491 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45f9e00640 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45f9400640 1 -- 172.21.15.137:0/3695594491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 msgr2=0x5565ac5f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.976+0000 7f45f9400640 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45f9400640 1 -- 172.21.15.137:0/3695594491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 msgr2=0x5565ac5f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45f9400640 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45f9400640 1 -- 172.21.15.137:0/3695594491 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565ac526080 con 0x5565ac5f6580 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45fa800640 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45f2a00640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f45e00a2880 0x7f45e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45f2a00640 1 -- 172.21.15.137:0/3145783931 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f45e80f0fa0 con 0x7f45fc104990 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45f9400640 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac5fce10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ee670 tx=0x7f45e80f4f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6c50104e49f2e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.977+0000 7f45fa800640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f45e00a2880 0x7f45e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.978+0000 7f45e7400640 1 -- 172.21.15.137:0/3695594491 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80b9810 con 0x5565ac5f6580 2024-03-26T14:13:38.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.978+0000 7f45e7400640 1 -- 172.21.15.137:0/3695594491 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e80bdc70 con 0x5565ac5f6580 2024-03-26T14:13:38.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.978+0000 7f45e7400640 1 -- 172.21.15.137:0/3695594491 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80bced0 con 0x5565ac5f6580 2024-03-26T14:13:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 -- 172.21.15.137:0/3695594491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 msgr2=0x5565ac5fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f45fa800640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f45e00a2880 0x7f45e00a4d40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x5565ac5f5ac0 tx=0x7f45f4009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac5fce10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ee670 tx=0x7f45e80f4f80 comp rx=0 tx=0).stop 2024-03-26T14:13:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 -- 172.21.15.137:0/3695594491 shutdown_connections 2024-03-26T14:13:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac5fce10 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 --2- 172.21.15.137:0/3695594491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 -- 172.21.15.137:0/3695594491 >> 172.21.15.137:0/3695594491 conn(0x5565ac5efde0 msgr2=0x5565ac5f01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 -- 172.21.15.137:0/3695594491 shutdown_connections 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 -- 172.21.15.137:0/3695594491 wait complete. 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.979+0000 7f4604010740 1 -- start start 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 Processor -- start 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5feec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac605f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ac5fd350 con 0x5565ac5f6580 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ac5150d0 con 0x5565ac5f49f0 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565ac5fd4d0 con 0x5565ac5f5bc0 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.980+0000 7f4604010740 10 client.0 _ll_register_callbacks cb 0x5565ac519e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5fe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55270/0 (socket says 172.21.15.137:55270) 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 -- 172.21.15.137:0/1589492220 learned_addr learned my addr 172.21.15.137:0/1589492220 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9400640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45fa800640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac605f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 -- 172.21.15.137:0/1589492220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 msgr2=0x5565ac5feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 -- 172.21.15.137:0/1589492220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 msgr2=0x5565ac605f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac605f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45f9e00640 1 -- 172.21.15.137:0/1589492220 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565ac26d3f0 con 0x5565ac5f49f0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.981+0000 7f45fa800640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac605f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f45f9e00640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5fe980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45ec012700 tx=0x7f45ec012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0fdafacda731d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f45dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f45dbe00640 1 -- 172.21.15.137:0/1589492220 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec004050 con 0x5565ac5f49f0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f45dbe00640 1 -- 172.21.15.137:0/1589492220 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f45ec0041f0 con 0x5565ac5f49f0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45e80bdc70 con 0x5565ac5f49f0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f45dbe00640 1 -- 172.21.15.137:0/1589492220 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45ec0043f0 con 0x5565ac5f49f0 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.982+0000 7f4604010740 10 client.37544 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:38.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.983+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5565ac3377d0 con 0x5565ac5f49f0 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.983+0000 7f45dbe00640 1 -- 172.21.15.137:0/1589492220 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e80bdc70 con 0x5565ac5f49f0 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.983+0000 7f45dbe00640 1 -- 172.21.15.137:0/1589492220 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f45ec003920 con 0x5565ac5f49f0 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f4604010740 10 client.37544 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f4604010740 10 client.37544 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f4604010740 20 client.37544 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f4604010740 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac60a480 0x5565ac60c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f4604010740 4 client.37544 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565ac566800 con 0x5565ac60a480 2024-03-26T14:13:38.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.984+0000 7f45f9400640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac60a480 0x5565ac60c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:38.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.985+0000 7f45f9400640 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac60a480 0x5565ac60c900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e83edbeaccb851a5 server_cookie=54007ecd0908eb05 in_seq=0 out_seq=0 2024-03-26T14:13:38.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.985+0000 7f45dbe00640 10 client.37544 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:38.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.986+0000 7f45dbe00640 1 -- 172.21.15.137:0/1589492220 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5565ac566800 con 0x5565ac60a480 2024-03-26T14:13:38.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.986+0000 7f45dbe00640 10 client.37544 handle_command_reply: tid=0 2024-03-26T14:13:38.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.986+0000 7f4604010740 1 client.37544 shutdown 2024-03-26T14:13:38.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 msgr2=0x5565ac5fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5fe980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45ec012700 tx=0x7f45ec012b00 comp rx=0 tx=0).stop 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 shutdown_connections 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac60a480 0x5565ac60c900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac605f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f5bc0 0x5565ac5feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f45dbe00640 0 client.37544 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 --2- 172.21.15.137:0/1589492220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac5fe980 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 >> 172.21.15.137:0/1589492220 conn(0x5565ac5efde0 msgr2=0x5565ac5ef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 shutdown_connections 2024-03-26T14:13:38.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 1 -- 172.21.15.137:0/1589492220 wait complete. 2024-03-26T14:13:38.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:38.987+0000 7f4604010740 20 client.37544 trim_cache size 0 max 16384 2024-03-26T14:13:39.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.003+0000 7f4604010740 1 Processor -- start 2024-03-26T14:13:39.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 -- start start 2024-03-26T14:13:39.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 0x5565ac6111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f5bc0 0x5565ac605ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac611700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1ae6a0 con 0x5565ac5f6580 2024-03-26T14:13:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1ac1a0 con 0x5565ac5f5bc0 2024-03-26T14:13:39.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f4604010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1aa220 con 0x5565ac5f49f0 2024-03-26T14:13:39.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 0x5565ac6111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f45f9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f5bc0 0x5565ac605ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 0x5565ac6111c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:52244/0 (socket says 172.21.15.137:52244) 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f45f9e00640 1 -- 172.21.15.137:0/3142939852 learned_addr learned my addr 172.21.15.137:0/3142939852 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.004+0000 7f45fa800640 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac611700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9400640 1 -- 172.21.15.137:0/3142939852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 msgr2=0x5565ac6111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9400640 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 0x5565ac6111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9400640 1 -- 172.21.15.137:0/3142939852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 msgr2=0x5565ac611700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9400640 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac611700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9400640 1 -- 172.21.15.137:0/3142939852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565ac5f5640 con 0x5565ac5f5bc0 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9e00640 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 0x5565ac6111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.005+0000 7f45f9400640 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f5bc0 0x5565ac605ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45e80a8000 tx=0x7f45e8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f73767c2d025767 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.006+0000 7f45e6a00640 1 -- 172.21.15.137:0/3142939852 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80bd0b0 con 0x5565ac5f5bc0 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.006+0000 7f45e6a00640 1 -- 172.21.15.137:0/3142939852 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e80bdc70 con 0x5565ac5f5bc0 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.006+0000 7f45e6a00640 1 -- 172.21.15.137:0/3142939852 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80bd340 con 0x5565ac5f5bc0 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.006+0000 7f4604010740 1 -- 172.21.15.137:0/3142939852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f5bc0 msgr2=0x5565ac605ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.006+0000 7f4604010740 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f5bc0 0x5565ac605ec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45e80a8000 tx=0x7f45e8000bc0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 -- 172.21.15.137:0/3142939852 shutdown_connections 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f5bc0 0x5565ac605ec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f6580 0x5565ac611700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 --2- 172.21.15.137:0/3142939852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f49f0 0x5565ac6111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 -- 172.21.15.137:0/3142939852 >> 172.21.15.137:0/3142939852 conn(0x5565ac5efde0 msgr2=0x5565ac5ef160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 -- 172.21.15.137:0/3142939852 shutdown_connections 2024-03-26T14:13:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 -- 172.21.15.137:0/3142939852 wait complete. 2024-03-26T14:13:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 -- start start 2024-03-26T14:13:39.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.007+0000 7f4604010740 1 Processor -- start 2024-03-26T14:13:39.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac603db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 0x5565ac6042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f6580 0x5565ac605300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1aa820 con 0x5565ac5f5bc0 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1abb40 con 0x5565ac5f49f0 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45fc1aae80 con 0x5565ac5f6580 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f4604010740 10 client.0 _ll_register_callbacks cb 0x7f45ec009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f45f9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 0x5565ac6042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f45f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac603db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f45f9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 0x5565ac6042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59914/0 (socket says 172.21.15.137:59914) 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f45f9400640 1 -- 172.21.15.137:0/2375221228 learned_addr learned my addr 172.21.15.137:0/2375221228 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.008+0000 7f45fa800640 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f6580 0x5565ac605300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45f9400640 1 -- 172.21.15.137:0/2375221228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f6580 msgr2=0x5565ac605300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45f9400640 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f6580 0x5565ac605300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45f9400640 1 -- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 msgr2=0x5565ac603db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45f9400640 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac603db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45f9400640 1 -- 172.21.15.137:0/2375221228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565ac6058f0 con 0x5565ac5f5bc0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45f9400640 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 0x5565ac6042f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f45e80f4c40 tx=0x7f45e8005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e4cfa6206755a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45dbe00640 1 -- 172.21.15.137:0/2375221228 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e80d6060 con 0x5565ac5f5bc0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f45dbe00640 1 -- 172.21.15.137:0/2375221228 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f45e8003d20 con 0x5565ac5f5bc0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.009+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45e80bdc70 con 0x5565ac5f5bc0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f45dbe00640 1 -- 172.21.15.137:0/2375221228 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45e80bd340 con 0x5565ac5f5bc0 2024-03-26T14:13:39.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 10 client.18006 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5565ac666590 con 0x5565ac5f5bc0 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f45dbe00640 1 -- 172.21.15.137:0/2375221228 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f45e80bdc70 con 0x5565ac5f5bc0 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f45dbe00640 1 -- 172.21.15.137:0/2375221228 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f45e80b9be0 con 0x5565ac5f5bc0 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 10 client.18006 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 10 client.18006 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 20 client.18006 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac666c00 0x5565ac669020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 4 client.18006 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.010+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5565ac560b50 con 0x5565ac666c00 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.011+0000 7f45f9e00640 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac666c00 0x5565ac669020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.011+0000 7f45f9e00640 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac666c00 0x5565ac669020 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1f514ee5a8e0e243 server_cookie=cf107ab0498f2d68 in_seq=0 out_seq=0 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.011+0000 7f45dbe00640 10 client.18006 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:39.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f45dbe00640 1 -- 172.21.15.137:0/2375221228 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5565ac560b50 con 0x5565ac666c00 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f45dbe00640 10 client.18006 handle_command_reply: tid=0 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 client.18006 shutdown 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 msgr2=0x5565ac6042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 0x5565ac6042f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f45e80f4c40 tx=0x7f45e8005b50 comp rx=0 tx=0).stop 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 shutdown_connections 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5565ac666c00 0x5565ac669020 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565ac5f6580 0x5565ac605300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f45dbe00640 0 client.18006 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5565ac5f5bc0 0x5565ac6042f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 --2- 172.21.15.137:0/2375221228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565ac5f49f0 0x5565ac603db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 >> 172.21.15.137:0/2375221228 conn(0x5565ac5efde0 msgr2=0x5565ac5f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 shutdown_connections 2024-03-26T14:13:39.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 1 -- 172.21.15.137:0/2375221228 wait complete. 2024-03-26T14:13:39.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.012+0000 7f4604010740 20 client.18006 trim_cache size 0 max 16384 2024-03-26T14:13:39.019 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:13:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f45e00a2880 msgr2=0x7f45e00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f45e00a2880 0x7f45e00a4d40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x5565ac5f5ac0 tx=0x7f45f4009500 comp rx=0 tx=0).stop 2024-03-26T14:13:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 msgr2=0x7f45fc199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc199ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45e8006010 tx=0x7f45e8005e40 comp rx=0 tx=0).stop 2024-03-26T14:13:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 shutdown_connections 2024-03-26T14:13:39.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f45e00a2880 0x7f45e00a4d40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45fc104990 0x7f45fc199ee0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45fc105350 0x7f45fc1a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 --2- 172.21.15.137:0/3145783931 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f45fc1037a0 0x7f45fc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 >> 172.21.15.137:0/3145783931 conn(0x7f45fc0fed10 msgr2=0x7f45fc107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 shutdown_connections 2024-03-26T14:13:39.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.013+0000 7f45fb200640 1 -- 172.21.15.137:0/3145783931 wait complete. 2024-03-26T14:13:39.034 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:13:39.034 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:13:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 -- 172.21.15.137:0/954989499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 msgr2=0x7fdef81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 --2- 172.21.15.137:0/954989499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81089e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ca0 tx=0x7fdef002c100 comp rx=0 tx=0).stop 2024-03-26T14:13:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 -- 172.21.15.137:0/954989499 shutdown_connections 2024-03-26T14:13:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 --2- 172.21.15.137:0/954989499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdef8101790 0x7fdef8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 --2- 172.21.15.137:0/954989499 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81089e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 --2- 172.21.15.137:0/954989499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef81005a0 0x7fdef81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 -- 172.21.15.137:0/954989499 >> 172.21.15.137:0/954989499 conn(0x7fdef80fbb30 msgr2=0x7fdef80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 -- 172.21.15.137:0/954989499 shutdown_connections 2024-03-26T14:13:39.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.242+0000 7fdefd600640 1 -- 172.21.15.137:0/954989499 wait complete. 2024-03-26T14:13:39.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.243+0000 7fdefd600640 1 Processor -- start 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.243+0000 7fdefd600640 1 -- start start 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdefd600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef81005a0 0x7fdef8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdefd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdef8101790 0x7fdef819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdefd600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdefd600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef810d1c0 con 0x7fdef8102150 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdefd600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef810d040 con 0x7fdef8101790 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdefd600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef810d340 con 0x7fdef81005a0 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdef7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdef7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59924/0 (socket says 172.21.15.137:59924) 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdef7400640 1 -- 172.21.15.137:0/3234741419 learned_addr learned my addr 172.21.15.137:0/3234741419 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdef6000640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdef8101790 0x7fdef819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.244+0000 7fdef6a00640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef81005a0 0x7fdef8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdef7400640 1 -- 172.21.15.137:0/3234741419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef81005a0 msgr2=0x7fdef8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdef7400640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef81005a0 0x7fdef8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdef7400640 1 -- 172.21.15.137:0/3234741419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdef8101790 msgr2=0x7fdef819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdef7400640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdef8101790 0x7fdef819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdef7400640 1 -- 172.21.15.137:0/3234741419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef81a1700 con 0x7fdef8102150 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdef7400640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81a1090 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fdef0032520 tx=0x7fdef0035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdeef400640 1 -- 172.21.15.137:0/3234741419 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef0039040 con 0x7fdef8102150 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.245+0000 7fdeef400640 1 -- 172.21.15.137:0/3234741419 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdef0044070 con 0x7fdef8102150 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.246+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef81a1990 con 0x7fdef8102150 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.246+0000 7fdeef400640 1 -- 172.21.15.137:0/3234741419 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00043c0 con 0x7fdef8102150 2024-03-26T14:13:39.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.246+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef81a1e80 con 0x7fdef8102150 2024-03-26T14:13:39.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.249+0000 7fdeef400640 1 -- 172.21.15.137:0/3234741419 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdef0004560 con 0x7fdef8102150 2024-03-26T14:13:39.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 Processor -- start 2024-03-26T14:13:39.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 -- start start 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 0x55c9d05a9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05aa9e0 0x55c9d05aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d050e2c0 con 0x55c9d05a9810 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d04b6c90 con 0x55c9d05aa9e0 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdf003d0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d050a360 con 0x55c9d05ab3a0 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.250+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 0x55c9d05a9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05aa9e0 0x55c9d05aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 0x55c9d05a9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59938/0 (socket says 172.21.15.137:59938) 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6000640 1 -- 172.21.15.137:0/1501642960 learned_addr learned my addr 172.21.15.137:0/1501642960 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6000640 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6a00640 1 -- 172.21.15.137:0/1501642960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 msgr2=0x55c9d05b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6a00640 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6a00640 1 -- 172.21.15.137:0/1501642960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 msgr2=0x55c9d05a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6a00640 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 0x55c9d05a9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6a00640 1 -- 172.21.15.137:0/1501642960 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9d04db080 con 0x55c9d05aa9e0 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdef6000640 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 0x55c9d05a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.251+0000 7fdeef400640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdedc0a2800 0x7fdedc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.252+0000 7fdef6a00640 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05aa9e0 0x55c9d05aae60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdee800bbb0 tx=0x7fdee800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc80337897905a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.252+0000 7fdeef400640 1 -- 172.21.15.137:0/3234741419 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdef00f1300 con 0x7fdef8102150 2024-03-26T14:13:39.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.252+0000 7fdef6000640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdedc0a2800 0x7fdedc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.252+0000 7fdee3400640 1 -- 172.21.15.137:0/1501642960 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8015070 con 0x55c9d05aa9e0 2024-03-26T14:13:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.252+0000 7fdee3400640 1 -- 172.21.15.137:0/1501642960 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdee80035f0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.252+0000 7fdee3400640 1 -- 172.21.15.137:0/1501642960 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee80116a0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 -- 172.21.15.137:0/1501642960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05aa9e0 msgr2=0x55c9d05aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdef6000640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdedc0a2800 0x7fdedc0a4cc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55c9d05aa8e0 tx=0x7fdee4009850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05aa9e0 0x55c9d05aae60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdee800bbb0 tx=0x7fdee800f730 comp rx=0 tx=0).stop 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 -- 172.21.15.137:0/1501642960 shutdown_connections 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05aa9e0 0x55c9d05aae60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 --2- 172.21.15.137:0/1501642960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05a9810 0x55c9d05a9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 -- 172.21.15.137:0/1501642960 >> 172.21.15.137:0/1501642960 conn(0x55c9d05a4c00 msgr2=0x55c9d05a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.253+0000 7fdf003d0740 1 -- 172.21.15.137:0/1501642960 shutdown_connections 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.254+0000 7fdf003d0740 1 -- 172.21.15.137:0/1501642960 wait complete. 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.254+0000 7fdf003d0740 1 -- start start 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.254+0000 7fdf003d0740 1 Processor -- start 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05bafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d05b23b0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d04ca0d0 con 0x55c9d05a9810 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9d05b2530 con 0x55c9d05ab3a0 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdf003d0740 10 client.0 _ll_register_callbacks cb 0x55c9d04cee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdef6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdef6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55326/0 (socket says 172.21.15.137:55326) 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdef6a00640 1 -- 172.21.15.137:0/3139872874 learned_addr learned my addr 172.21.15.137:0/3139872874 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.255+0000 7fdef7400640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05bafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6a00640 1 -- 172.21.15.137:0/3139872874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 msgr2=0x55c9d05bafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6a00640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05bafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6a00640 1 -- 172.21.15.137:0/3139872874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 msgr2=0x55c9d05b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6a00640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6a00640 1 -- 172.21.15.137:0/3139872874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9d02223f0 con 0x55c9d05a9810 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6000640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fdef6a00640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05b3a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdee8003e80 tx=0x7fdee800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2c9788946facb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.256+0000 7fded7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.257+0000 7fded7e00640 1 -- 172.21.15.137:0/3139872874 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8028060 con 0x55c9d05a9810 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.257+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee80035f0 con 0x55c9d05a9810 2024-03-26T14:13:39.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.257+0000 7fded7e00640 1 -- 172.21.15.137:0/3139872874 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdee8004000 con 0x55c9d05a9810 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.257+0000 7fded7e00640 1 -- 172.21.15.137:0/3139872874 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee802f050 con 0x55c9d05a9810 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.257+0000 7fdf003d0740 10 client.37562 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.257+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c9d02ec7d0 con 0x55c9d05a9810 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fded7e00640 1 -- 172.21.15.137:0/3139872874 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdee80035f0 con 0x55c9d05a9810 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fded7e00640 1 -- 172.21.15.137:0/3139872874 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdee8023c40 con 0x55c9d05a9810 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fdf003d0740 10 client.37562 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fdf003d0740 10 client.37562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fdf003d0740 20 client.37562 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fdf003d0740 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d05bf110 0x55c9d05c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fdf003d0740 4 client.37562 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:39.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.258+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9d051b800 con 0x55c9d05bf110 2024-03-26T14:13:39.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.259+0000 7fdef6000640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d05bf110 0x55c9d05c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.260+0000 7fded7e00640 10 client.37562 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.260+0000 7fdef6000640 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d05bf110 0x55c9d05c1500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67eac3fcac288f53 server_cookie=9ff59e44ecff7e91 in_seq=0 out_seq=0 2024-03-26T14:13:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.261+0000 7fded7e00640 1 -- 172.21.15.137:0/3139872874 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55c9d051b800 con 0x55c9d05bf110 2024-03-26T14:13:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.261+0000 7fded7e00640 10 client.37562 handle_command_reply: tid=0 2024-03-26T14:13:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.261+0000 7fdf003d0740 1 client.37562 shutdown 2024-03-26T14:13:39.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 msgr2=0x55c9d05b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05b3a30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdee8003e80 tx=0x7fdee800bed0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 shutdown_connections 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d05bf110 0x55c9d05c1500 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05bafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 --2- 172.21.15.137:0/3139872874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05b3a30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 >> 172.21.15.137:0/3139872874 conn(0x55c9d05a4c00 msgr2=0x55c9d05a3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fded7e00640 0 client.37562 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 shutdown_connections 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 1 -- 172.21.15.137:0/3139872874 wait complete. 2024-03-26T14:13:39.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.262+0000 7fdf003d0740 20 client.37562 trim_cache size 0 max 16384 2024-03-26T14:13:39.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:39 smithi137 ceph-mon[31539]: pgmap v796: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:39.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:39 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2591679674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:13:39.285 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:39 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.290+0000 7fdf003d0740 1 Processor -- start 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.290+0000 7fdf003d0740 1 -- start start 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.290+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.290+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05baf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdf003d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81ae8f0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdf003d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81ac3f0 con 0x55c9d05a9810 2024-03-26T14:13:39.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdf003d0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81aa470 con 0x55c9d05ab3a0 2024-03-26T14:13:39.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05baf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05baf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59962/0 (socket says 172.21.15.137:59962) 2024-03-26T14:13:39.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef6000640 1 -- 172.21.15.137:0/737443753 learned_addr learned my addr 172.21.15.137:0/737443753 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef7400640 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef6000640 1 -- 172.21.15.137:0/737443753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 msgr2=0x55c9d05c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.291+0000 7fdef6000640 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdef6000640 1 -- 172.21.15.137:0/737443753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 msgr2=0x55c9d05c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdef6000640 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdef6000640 1 -- 172.21.15.137:0/737443753 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9d05c1ac0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdef6a00640 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdef6000640 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05baf70 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005b90 tx=0x7fdee4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0b30b43f29e820 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdee2a00640 1 -- 172.21.15.137:0/737443753 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee402f230 con 0x55c9d05aa9e0 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdee2a00640 1 -- 172.21.15.137:0/737443753 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdee402f7b0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.292+0000 7fdee2a00640 1 -- 172.21.15.137:0/737443753 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4003850 con 0x55c9d05aa9e0 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.293+0000 7fdf003d0740 1 -- 172.21.15.137:0/737443753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 msgr2=0x55c9d05baf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.293+0000 7fdf003d0740 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05baf70 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005b90 tx=0x7fdee4015000 comp rx=0 tx=0).stop 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 -- 172.21.15.137:0/737443753 shutdown_connections 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05ab3a0 0x55c9d05c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05baf70 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 --2- 172.21.15.137:0/737443753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05a9810 0x55c9d05c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 -- 172.21.15.137:0/737443753 >> 172.21.15.137:0/737443753 conn(0x55c9d05a4c00 msgr2=0x55c9d05a3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.298 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 -- 172.21.15.137:0/737443753 shutdown_connections 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 -- 172.21.15.137:0/737443753 wait complete. 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 -- start start 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.294+0000 7fdf003d0740 1 Processor -- start 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05a9810 0x55c9d05ba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81aaa70 con 0x55c9d05aa9e0 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81abd90 con 0x55c9d05ab3a0 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81ab0d0 con 0x55c9d05a9810 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdf003d0740 10 client.0 _ll_register_callbacks cb 0x7fdee40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdef6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05bac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59976/0 (socket says 172.21.15.137:59976) 2024-03-26T14:13:39.299 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdef6000640 1 -- 172.21.15.137:0/2164244739 learned_addr learned my addr 172.21.15.137:0/2164244739 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdef6a00640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05a9810 0x55c9d05ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.295+0000 7fdef7400640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdef6000640 1 -- 172.21.15.137:0/2164244739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05a9810 msgr2=0x55c9d05ba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdef6000640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05a9810 0x55c9d05ba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdef6000640 1 -- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05ab3a0 msgr2=0x55c9d05b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdef6000640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdef6000640 1 -- 172.21.15.137:0/2164244739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9d05b7790 con 0x55c9d05aa9e0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdef6000640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05bac40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fdee400fa80 tx=0x7fdee402fe50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea60c11cc2cd2a49 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fded7e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fded7e00640 1 -- 172.21.15.137:0/2164244739 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4036070 con 0x55c9d05aa9e0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fded7e00640 1 -- 172.21.15.137:0/2164244739 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdee402fb60 con 0x55c9d05aa9e0 2024-03-26T14:13:39.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.296+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee402f7b0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fded7e00640 1 -- 172.21.15.137:0/2164244739 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee4005040 con 0x55c9d05aa9e0 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fdf003d0740 10 client.18024 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c9d061b480 con 0x55c9d05aa9e0 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fded7e00640 1 -- 172.21.15.137:0/2164244739 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdee402f7b0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fded7e00640 1 -- 172.21.15.137:0/2164244739 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdee40035f0 con 0x55c9d05aa9e0 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fdf003d0740 10 client.18024 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fdf003d0740 10 client.18024 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.297+0000 7fdf003d0740 20 client.18024 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.298+0000 7fdf003d0740 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d061bdc0 0x55c9d061e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.298+0000 7fdf003d0740 4 client.18024 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:13:39.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.298+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9d0515b50 con 0x55c9d061bdc0 2024-03-26T14:13:39.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.298+0000 7fdef6a00640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d061bdc0 0x55c9d061e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdef6a00640 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d061bdc0 0x55c9d061e1e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5765fae772e05daa server_cookie=ec3d214e11ba4f01 in_seq=0 out_seq=0 2024-03-26T14:13:39.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fded7e00640 10 client.18024 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:39.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fded7e00640 1 -- 172.21.15.137:0/2164244739 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9d0515b50 con 0x55c9d061bdc0 2024-03-26T14:13:39.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fded7e00640 10 client.18024 handle_command_reply: tid=0 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 client.18024 shutdown 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 msgr2=0x55c9d05bac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05bac40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fdee400fa80 tx=0x7fdee402fe50 comp rx=0 tx=0).stop 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 shutdown_connections 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55c9d061bdc0 0x55c9d061e1e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c9d05ab3a0 0x55c9d05b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fded7e00640 0 client.18024 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55c9d05aa9e0 0x55c9d05bac40 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 --2- 172.21.15.137:0/2164244739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c9d05a9810 0x55c9d05ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.299+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 >> 172.21.15.137:0/2164244739 conn(0x55c9d05a4c00 msgr2=0x55c9d05adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.300+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 shutdown_connections 2024-03-26T14:13:39.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.300+0000 7fdf003d0740 1 -- 172.21.15.137:0/2164244739 wait complete. 2024-03-26T14:13:39.305 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.300+0000 7fdf003d0740 20 client.18024 trim_cache size 0 max 16384 2024-03-26T14:13:39.305 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdedc0a2800 msgr2=0x7fdedc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdedc0a2800 0x7fdedc0a4cc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55c9d05aa8e0 tx=0x7fdee4009850 comp rx=0 tx=0).stop 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 msgr2=0x7fdef81a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81a1090 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fdef0032520 tx=0x7fdef0035ce0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 shutdown_connections 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdedc0a2800 0x7fdedc0a4cc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdef8101790 0x7fdef819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdef8102150 0x7fdef81a1090 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 --2- 172.21.15.137:0/3234741419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdef81005a0 0x7fdef8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.306 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 >> 172.21.15.137:0/3234741419 conn(0x7fdef80fbb30 msgr2=0x7fdef8109630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 shutdown_connections 2024-03-26T14:13:39.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.301+0000 7fdefd600640 1 -- 172.21.15.137:0/3234741419 wait complete. 2024-03-26T14:13:39.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:13:39.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:13:39.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 -- 172.21.15.137:0/318567695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c1015a0 msgr2=0x7f7b3c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 --2- 172.21.15.137:0/318567695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c1015a0 0x7f7b3c1019c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009900 tx=0x7f7b3002bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 -- 172.21.15.137:0/318567695 shutdown_connections 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 --2- 172.21.15.137:0/318567695 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b3c102790 0x7f7b3c102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 --2- 172.21.15.137:0/318567695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b3c103150 0x7f7b3c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 --2- 172.21.15.137:0/318567695 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c1015a0 0x7f7b3c1019c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 -- 172.21.15.137:0/318567695 >> 172.21.15.137:0/318567695 conn(0x7f7b3c0fcb10 msgr2=0x7f7b3c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 -- 172.21.15.137:0/318567695 shutdown_connections 2024-03-26T14:13:39.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.531+0000 7f7b41e00640 1 -- 172.21.15.137:0/318567695 wait complete. 2024-03-26T14:13:39.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 Processor -- start 2024-03-26T14:13:39.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 -- start start 2024-03-26T14:13:39.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b3c1015a0 0x7f7b3c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 0x7f7b3c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b3c103150 0x7f7b3c1a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c10d110 con 0x7f7b3c102790 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c10cf90 con 0x7f7b3c103150 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.532+0000 7f7b41e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c10d290 con 0x7f7b3c1015a0 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 0x7f7b3c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 0x7f7b3c199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59988/0 (socket says 172.21.15.137:59988) 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 -- 172.21.15.137:0/2765469625 learned_addr learned my addr 172.21.15.137:0/2765469625 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b41400640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b3c103150 0x7f7b3c1a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b40a00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b3c1015a0 0x7f7b3c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 -- 172.21.15.137:0/2765469625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b3c1015a0 msgr2=0x7f7b3c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b3c1015a0 0x7f7b3c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 -- 172.21.15.137:0/2765469625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b3c103150 msgr2=0x7f7b3c1a0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b3c103150 0x7f7b3c1a0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 -- 172.21.15.137:0/2765469625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c1a1640 con 0x7f7b3c102790 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.533+0000 7f7b3be00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 0x7f7b3c199f60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e9f0 tx=0x7f7b2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.534+0000 7f7b39600640 1 -- 172.21.15.137:0/2765469625 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800cd80 con 0x7f7b3c102790 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.534+0000 7f7b39600640 1 -- 172.21.15.137:0/2765469625 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b280108e0 con 0x7f7b3c102790 2024-03-26T14:13:39.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.534+0000 7f7b39600640 1 -- 172.21.15.137:0/2765469625 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28018920 con 0x7f7b3c102790 2024-03-26T14:13:39.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.534+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c1a1930 con 0x7f7b3c102790 2024-03-26T14:13:39.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.534+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c1a1e30 con 0x7f7b3c102790 2024-03-26T14:13:39.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.538+0000 7f7b39600640 1 -- 172.21.15.137:0/2765469625 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7b28010a90 con 0x7f7b3c102790 2024-03-26T14:13:39.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 Processor -- start 2024-03-26T14:13:39.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 -- start start 2024-03-26T14:13:39.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd4200 0x55a123bd4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bd5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd5d90 0x55a123bdc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a123b302c0 con 0x55a123bd4200 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a123ad8c90 con 0x55a123bd53d0 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.539+0000 7f7b46165740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a123b2c360 con 0x55a123bd5d90 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bd5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bd5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55382/0 (socket says 172.21.15.137:55382) 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 -- 172.21.15.137:0/648428214 learned_addr learned my addr 172.21.15.137:0/648428214 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b41400640 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd4200 0x55a123bd4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b39600640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b240a2800 0x7f7b240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b3be00640 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd5d90 0x55a123bdc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 -- 172.21.15.137:0/648428214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd5d90 msgr2=0x55a123bdc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd5d90 0x55a123bdc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 -- 172.21.15.137:0/648428214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd4200 msgr2=0x55a123bd4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd4200 0x55a123bd4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b39600640 1 -- 172.21.15.137:0/2765469625 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b280cf210 con 0x7f7b3c102790 2024-03-26T14:13:39.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b40a00640 1 -- 172.21.15.137:0/648428214 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a123afd080 con 0x55a123bd53d0 2024-03-26T14:13:39.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.540+0000 7f7b41400640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b240a2800 0x7f7b240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.541+0000 7f7b40a00640 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bd5850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b30005ab0 tx=0x7f7b30009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63cf1f3056d6427f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.541+0000 7f7b2dc00640 1 -- 172.21.15.137:0/648428214 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30044020 con 0x55a123bd53d0 2024-03-26T14:13:39.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.541+0000 7f7b2dc00640 1 -- 172.21.15.137:0/648428214 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3002cae0 con 0x55a123bd53d0 2024-03-26T14:13:39.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.541+0000 7f7b2dc00640 1 -- 172.21.15.137:0/648428214 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30004920 con 0x55a123bd53d0 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 -- 172.21.15.137:0/648428214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 msgr2=0x55a123bd5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bd5850 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b30005ab0 tx=0x7f7b30009670 comp rx=0 tx=0).stop 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b41400640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b240a2800 0x7f7b240a4cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55a123bd52d0 tx=0x7f7b34004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 -- 172.21.15.137:0/648428214 shutdown_connections 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bd5850 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd5d90 0x55a123bdc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 --2- 172.21.15.137:0/648428214 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd4200 0x55a123bd4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 -- 172.21.15.137:0/648428214 >> 172.21.15.137:0/648428214 conn(0x55a123bcf5f0 msgr2=0x55a123bcf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 -- 172.21.15.137:0/648428214 shutdown_connections 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.542+0000 7f7b46165740 1 -- 172.21.15.137:0/648428214 wait complete. 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 -- start start 2024-03-26T14:13:39.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 Processor -- start 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bde440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bde980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123be59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a123bdcd40 con 0x55a123bd5d90 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a123aec0d0 con 0x55a123bd53d0 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a123bdcec0 con 0x55a123bd4200 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.543+0000 7f7b46165740 10 client.0 _ll_register_callbacks cb 0x55a123af0e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bde980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b41400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123be59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bde980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55384/0 (socket says 172.21.15.137:55384) 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 -- 172.21.15.137:0/2728555577 learned_addr learned my addr 172.21.15.137:0/2728555577 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b40a00640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bde440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 -- 172.21.15.137:0/2728555577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 msgr2=0x55a123bde440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bde440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 -- 172.21.15.137:0/2728555577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 msgr2=0x55a123be59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123be59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b3be00640 1 -- 172.21.15.137:0/2728555577 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1238443f0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.544+0000 7f7b41400640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123be59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b3be00640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bde980 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cc900 tx=0x7f7b2806a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50004b01c6790d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b22000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b22000640 1 -- 172.21.15.137:0/2728555577 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2809bea0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b22000640 1 -- 172.21.15.137:0/2728555577 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b2806d0d0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b3002cae0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b22000640 1 -- 172.21.15.137:0/2728555577 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b2809b7a0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b46165740 10 client.37580 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a12390e7d0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.545+0000 7f7b22000640 1 -- 172.21.15.137:0/2728555577 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3002cae0 con 0x55a123bd53d0 2024-03-26T14:13:39.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b22000640 1 -- 172.21.15.137:0/2728555577 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7b2809abf0 con 0x55a123bd53d0 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b46165740 10 client.37580 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b46165740 10 client.37580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b46165740 20 client.37580 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b46165740 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123be9e30 0x55a123bec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b46165740 4 client.37580 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a123b3d800 con 0x55a123be9e30 2024-03-26T14:13:39.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.546+0000 7f7b40a00640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123be9e30 0x55a123bec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.547+0000 7f7b40a00640 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123be9e30 0x55a123bec290 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0b701455ac4537 server_cookie=8ea5b8c609771919 in_seq=0 out_seq=0 2024-03-26T14:13:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.547+0000 7f7b22000640 10 client.37580 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b22000640 1 -- 172.21.15.137:0/2728555577 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a123b3d800 con 0x55a123be9e30 2024-03-26T14:13:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b22000640 10 client.37580 handle_command_reply: tid=0 2024-03-26T14:13:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 client.37580 shutdown 2024-03-26T14:13:39.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 msgr2=0x55a123bde980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bde980 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b280cc900 tx=0x7f7b2806a6c0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 shutdown_connections 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123be9e30 0x55a123bec290 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123bde980 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b22000640 0 client.37580 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123be59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 --2- 172.21.15.137:0/2728555577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bde440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.548+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 >> 172.21.15.137:0/2728555577 conn(0x55a123bcf5f0 msgr2=0x55a123bd2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.549+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 shutdown_connections 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.549+0000 7f7b46165740 1 -- 172.21.15.137:0/2728555577 wait complete. 2024-03-26T14:13:39.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.549+0000 7f7b46165740 20 client.37580 trim_cache size 0 max 16384 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.570+0000 7f7b46165740 1 Processor -- start 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.570+0000 7f7b46165740 1 -- start start 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b46165740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bc6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b46165740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123be5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b46165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123bd2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b46165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1aa3f0 con 0x55a123bd5d90 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b46165740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1a7ef0 con 0x55a123bd53d0 2024-03-26T14:13:39.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b46165740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1a5f90 con 0x55a123bd4200 2024-03-26T14:13:39.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123be5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b41400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123bd2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123be5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55386/0 (socket says 172.21.15.137:55386) 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b3be00640 1 -- 172.21.15.137:0/1281838445 learned_addr learned my addr 172.21.15.137:0/1281838445 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b40a00640 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bc6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b41400640 1 -- 172.21.15.137:0/1281838445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 msgr2=0x55a123bc6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b41400640 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bc6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b41400640 1 -- 172.21.15.137:0/1281838445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 msgr2=0x55a123be5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b41400640 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123be5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.571+0000 7f7b41400640 1 -- 172.21.15.137:0/1281838445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a123bd4e50 con 0x55a123bd5d90 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.572+0000 7f7b41400640 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123bd2f60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7b340098b0 tx=0x7f7b34015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b39f4b28711924 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.572+0000 7f7b2d200640 1 -- 172.21.15.137:0/1281838445 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34012dd0 con 0x55a123bd5d90 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.572+0000 7f7b2d200640 1 -- 172.21.15.137:0/1281838445 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3401fe20 con 0x55a123bd5d90 2024-03-26T14:13:39.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.572+0000 7f7b2d200640 1 -- 172.21.15.137:0/1281838445 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34011ea0 con 0x55a123bd5d90 2024-03-26T14:13:39.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 -- 172.21.15.137:0/1281838445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 msgr2=0x55a123bd2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123bd2f60 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7b340098b0 tx=0x7f7b34015040 comp rx=0 tx=0).stop 2024-03-26T14:13:39.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 -- 172.21.15.137:0/1281838445 shutdown_connections 2024-03-26T14:13:39.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd53d0 0x55a123be5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd5d90 0x55a123bd2f60 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 --2- 172.21.15.137:0/1281838445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123bc6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 -- 172.21.15.137:0/1281838445 >> 172.21.15.137:0/1281838445 conn(0x55a123bcf5f0 msgr2=0x55a123bd2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 -- 172.21.15.137:0/1281838445 shutdown_connections 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.573+0000 7f7b46165740 1 -- 172.21.15.137:0/1281838445 wait complete. 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.574+0000 7f7b46165740 1 -- start start 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.574+0000 7f7b46165740 1 Processor -- start 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.574+0000 7f7b46165740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123be3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.574+0000 7f7b46165740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 0x55a123be3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.574+0000 7f7b46165740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd5d90 0x55a123be4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b46165740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1a6590 con 0x55a123bd53d0 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b46165740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1a7890 con 0x55a123bd5d90 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b46165740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1a6bd0 con 0x55a123bd4200 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b46165740 10 client.0 _ll_register_callbacks cb 0x7f7b28009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 0x55a123be3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 0x55a123be3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60024/0 (socket says 172.21.15.137:60024) 2024-03-26T14:13:39.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 -- 172.21.15.137:0/3194434821 learned_addr learned my addr 172.21.15.137:0/3194434821 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:39.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b41400640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd5d90 0x55a123be4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b40a00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123be3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 -- 172.21.15.137:0/3194434821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 msgr2=0x55a123be3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123be3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 -- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd5d90 msgr2=0x55a123be4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd5d90 0x55a123be4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b3be00640 1 -- 172.21.15.137:0/3194434821 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a123be51e0 con 0x55a123bd53d0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b41400640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd5d90 0x55a123be4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.575+0000 7f7b40a00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123be3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b3be00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 0x55a123be3c80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005b60 tx=0x7f7b280862e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8e8b093c97d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b22000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b22000640 1 -- 172.21.15.137:0/3194434821 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2809f8f0 con 0x55a123bd53d0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b22000640 1 -- 172.21.15.137:0/3194434821 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7b28010430 con 0x55a123bd53d0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b3401fe20 con 0x55a123bd53d0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b22000640 1 -- 172.21.15.137:0/3194434821 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b280a3f40 con 0x55a123bd53d0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b46165740 10 client.18042 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.576+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a123c45db0 con 0x55a123bd53d0 2024-03-26T14:13:39.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b22000640 1 -- 172.21.15.137:0/3194434821 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7b3401fe20 con 0x55a123bd53d0 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b22000640 1 -- 172.21.15.137:0/3194434821 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7b2806a190 con 0x55a123bd53d0 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b46165740 10 client.18042 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b46165740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b46165740 20 client.18042 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b46165740 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123c46650 0x55a123c48a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b46165740 4 client.18042 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a123b37b50 con 0x55a123c46650 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.577+0000 7f7b40a00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123c46650 0x55a123c48a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b40a00640 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123c46650 0x55a123c48a70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=716b07920dd1a2c9 server_cookie=63f9255464207512 in_seq=0 out_seq=0 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b22000640 10 client.18042 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b22000640 1 -- 172.21.15.137:0/3194434821 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a123b37b50 con 0x55a123c46650 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b22000640 10 client.18042 handle_command_reply: tid=0 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 client.18042 shutdown 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 msgr2=0x55a123be3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 0x55a123be3c80 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005b60 tx=0x7f7b280862e0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 shutdown_connections 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a123c46650 0x55a123c48a70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a123bd5d90 0x55a123be4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a123bd53d0 0x55a123be3c80 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 --2- 172.21.15.137:0/3194434821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a123bd4200 0x55a123be3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.578+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 >> 172.21.15.137:0/3194434821 conn(0x55a123bcf5f0 msgr2=0x55a123bd30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.579+0000 7f7b22000640 0 client.18042 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.579+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 shutdown_connections 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.579+0000 7f7b46165740 1 -- 172.21.15.137:0/3194434821 wait complete. 2024-03-26T14:13:39.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.579+0000 7f7b46165740 20 client.18042 trim_cache size 0 max 16384 2024-03-26T14:13:39.583 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:13:39.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b240a2800 msgr2=0x7f7b240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b240a2800 0x7f7b240a4cc0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55a123bd52d0 tx=0x7f7b34004a40 comp rx=0 tx=0).stop 2024-03-26T14:13:39.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 msgr2=0x7f7b3c199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:39.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 0x7f7b3c199f60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e9f0 tx=0x7f7b2800eeb0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 shutdown_connections 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7b240a2800 0x7f7b240a4cc0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b3c103150 0x7f7b3c1a0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b3c102790 0x7f7b3c199f60 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 --2- 172.21.15.137:0/2765469625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7b3c1015a0 0x7f7b3c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 >> 172.21.15.137:0/2765469625 conn(0x7f7b3c0fcb10 msgr2=0x7f7b3c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 shutdown_connections 2024-03-26T14:13:39.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:39.580+0000 7f7b41e00640 1 -- 172.21.15.137:0/2765469625 wait complete. 2024-03-26T14:13:39.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:39 smithi179 ceph-mon[35472]: pgmap v796: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:39.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:39 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2591679674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:13:39.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:39 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:13:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:39 smithi159 ceph-mon[35501]: pgmap v796: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:39 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2591679674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:13:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:39 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:13:40.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:13:40.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:13:40.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:13:40.599 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:13:40.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:13:40.599 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:13:40.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:13:40.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:13:40.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:13:40.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:13:40.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:13:40.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:13:40.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 -- 172.21.15.137:0/1417245309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4107260 msgr2=0x7f41b41096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 --2- 172.21.15.137:0/1417245309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4107260 0x7f41b41096b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009900 tx=0x7f41ac02bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 -- 172.21.15.137:0/1417245309 shutdown_connections 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 --2- 172.21.15.137:0/1417245309 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4107260 0x7f41b41096b0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 --2- 172.21.15.137:0/1417245309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b4104930 0x7f41b4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 --2- 172.21.15.137:0/1417245309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4102000 0x7f41b41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 -- 172.21.15.137:0/1417245309 >> 172.21.15.137:0/1417245309 conn(0x7f41b40fba10 msgr2=0x7f41b40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 -- 172.21.15.137:0/1417245309 shutdown_connections 2024-03-26T14:13:40.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.830+0000 7f41ba800640 1 -- 172.21.15.137:0/1417245309 wait complete. 2024-03-26T14:13:40.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.831+0000 7f41ba800640 1 Processor -- start 2024-03-26T14:13:40.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.831+0000 7f41ba800640 1 -- start start 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.831+0000 7f41ba800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 0x7f41b4197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.831+0000 7f41ba800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b4104930 0x7f41b4197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.831+0000 7f41ba800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4107260 0x7f41b419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.831+0000 7f41ba800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b410d0c0 con 0x7f41b4102000 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41ba800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b410cf40 con 0x7f41b4107260 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41ba800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b410d240 con 0x7f41b4104930 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 0x7f41b4197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 0x7f41b4197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60036/0 (socket says 172.21.15.137:60036) 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 -- 172.21.15.137:0/1198978379 learned_addr learned my addr 172.21.15.137:0/1198978379 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b8a00640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b4104930 0x7f41b4197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9e00640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4107260 0x7f41b419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 -- 172.21.15.137:0/1198978379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b4104930 msgr2=0x7f41b4197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b4104930 0x7f41b4197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 -- 172.21.15.137:0/1198978379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4107260 msgr2=0x7f41b419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4107260 0x7f41b419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.832+0000 7f41b9400640 1 -- 172.21.15.137:0/1198978379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b419f390 con 0x7f41b4102000 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.833+0000 7f41b9400640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 0x7f41b4197770 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f41a800ebc0 tx=0x7f41a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.833+0000 7f41b2000640 1 -- 172.21.15.137:0/1198978379 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8010630 con 0x7f41b4102000 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.833+0000 7f41b2000640 1 -- 172.21.15.137:0/1198978379 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41a8010c80 con 0x7f41b4102000 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.833+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b419f680 con 0x7f41b4102000 2024-03-26T14:13:40.835 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.833+0000 7f41b2000640 1 -- 172.21.15.137:0/1198978379 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8018b30 con 0x7f41b4102000 2024-03-26T14:13:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.833+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b419fbd0 con 0x7f41b4102000 2024-03-26T14:13:40.836 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.836+0000 7f41bea1b740 1 Processor -- start 2024-03-26T14:13:40.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.836+0000 7f41bea1b740 1 -- start start 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831cca0 0x55ab083217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab083242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41bea1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab08281eb0 con 0x55ab0831c2b0 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41bea1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0822aa60 con 0x55ab08321e10 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41bea1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0827df50 con 0x55ab0831cca0 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41b2000640 1 -- 172.21.15.137:0/1198978379 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f41a8018cd0 con 0x7f41b4102000 2024-03-26T14:13:40.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60042/0 (socket says 172.21.15.137:60042) 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41b9e00640 1 -- 172.21.15.137:0/874427929 learned_addr learned my addr 172.21.15.137:0/874427929 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.837+0000 7f41b8a00640 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab083242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9400640 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831cca0 0x55ab083217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9e00640 1 -- 172.21.15.137:0/874427929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831cca0 msgr2=0x55ab083217a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9e00640 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831cca0 0x55ab083217a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9e00640 1 -- 172.21.15.137:0/874427929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 msgr2=0x55ab083242a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9e00640 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab083242a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9e00640 1 -- 172.21.15.137:0/874427929 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0805e7e0 con 0x55ab0831c2b0 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.838+0000 7f41b9e00640 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831c6d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f41ac02c840 tx=0x7f41ac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d53ff4af5ed999a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.839+0000 7f41a6600640 1 -- 172.21.15.137:0/874427929 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac044020 con 0x55ab0831c2b0 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.839+0000 7f41a6600640 1 -- 172.21.15.137:0/874427929 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac02cb60 con 0x55ab0831c2b0 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.839+0000 7f41a6600640 1 -- 172.21.15.137:0/874427929 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac004900 con 0x55ab0831c2b0 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.839+0000 7f41b2000640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f419c0a2800 0x7f419c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.839+0000 7f41b2000640 1 -- 172.21.15.137:0/1198978379 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f41a80cf520 con 0x7f41b4102000 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.839+0000 7f41b9400640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f419c0a2800 0x7f419c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 -- 172.21.15.137:0/874427929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 msgr2=0x55ab0831c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831c6d0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f41ac02c840 tx=0x7f41ac005d50 comp rx=0 tx=0).stop 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 -- 172.21.15.137:0/874427929 shutdown_connections 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab083242a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831cca0 0x55ab083217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 --2- 172.21.15.137:0/874427929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831c6d0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 -- 172.21.15.137:0/874427929 >> 172.21.15.137:0/874427929 conn(0x55ab08317820 msgr2=0x55ab08317c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41b9400640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f419c0a2800 0x7f419c0a4cc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f41a800cba0 tx=0x7f41a800cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 -- 172.21.15.137:0/874427929 shutdown_connections 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 -- 172.21.15.137:0/874427929 wait complete. 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.840+0000 7f41bea1b740 1 -- start start 2024-03-26T14:13:40.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 Processor -- start 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831c2b0 0x55ab083264c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08326a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 0x55ab0832da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab08324d60 con 0x55ab0831cca0 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0823deb0 con 0x55ab0831c2b0 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab08324ee0 con 0x55ab08321e10 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41bea1b740 10 client.0 _ll_register_callbacks cb 0x55ab08232150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41b8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08326a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41b8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08326a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60050/0 (socket says 172.21.15.137:60050) 2024-03-26T14:13:40.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.841+0000 7f41b8a00640 1 -- 172.21.15.137:0/4260263918 learned_addr learned my addr 172.21.15.137:0/4260263918 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b8a00640 1 -- 172.21.15.137:0/4260263918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 msgr2=0x55ab0832da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b8a00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 0x55ab0832da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b8a00640 1 -- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831c2b0 msgr2=0x55ab083264c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b9e00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831c2b0 0x55ab083264c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b8a00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831c2b0 0x55ab083264c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b8a00640 1 -- 172.21.15.137:0/4260263918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab080bef70 con 0x55ab0831cca0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b9e00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831c2b0 0x55ab083264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f41b8a00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08326a00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f41a0011780 tx=0x7f41a0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2ba0250b91e921 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.842+0000 7f419aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f419aa00640 1 -- 172.21.15.137:0/4260263918 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000fd50 con 0x55ab0831cca0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f419aa00640 1 -- 172.21.15.137:0/4260263918 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f41a0005020 con 0x55ab0831cca0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41ac02cb60 con 0x55ab0831cca0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f419aa00640 1 -- 172.21.15.137:0/4260263918 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a0024050 con 0x55ab0831cca0 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f41bea1b740 10 client.18054 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:40.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab07f56170 con 0x55ab0831cca0 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.843+0000 7f419aa00640 1 -- 172.21.15.137:0/4260263918 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac02cb60 con 0x55ab0831cca0 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f419aa00640 1 -- 172.21.15.137:0/4260263918 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f41a0018030 con 0x55ab0831cca0 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41bea1b740 10 client.18054 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41bea1b740 10 client.18054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41bea1b740 20 client.18054 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41bea1b740 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab083323f0 0x55ab08334850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41bea1b740 4 client.18054 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab0828f3f0 con 0x55ab083323f0 2024-03-26T14:13:40.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.844+0000 7f41b9e00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab083323f0 0x55ab08334850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.845+0000 7f41b9e00640 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab083323f0 0x55ab08334850 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ba698f57e7aa57 server_cookie=33077bf614996ec0 in_seq=0 out_seq=0 2024-03-26T14:13:40.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.845+0000 7f419aa00640 10 client.18054 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:40.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.846+0000 7f419aa00640 1 -- 172.21.15.137:0/4260263918 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ab0828f3f0 con 0x55ab083323f0 2024-03-26T14:13:40.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.846+0000 7f419aa00640 10 client.18054 handle_command_reply: tid=0 2024-03-26T14:13:40.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.846+0000 7f41bea1b740 1 client.18054 shutdown 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 msgr2=0x55ab08326a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08326a00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f41a0011780 tx=0x7f41a0011c40 comp rx=0 tx=0).stop 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 shutdown_connections 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab083323f0 0x55ab08334850 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 0x55ab0832da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08326a00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 --2- 172.21.15.137:0/4260263918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831c2b0 0x55ab083264c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 >> 172.21.15.137:0/4260263918 conn(0x55ab08317820 msgr2=0x55ab08316a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f419aa00640 0 client.18054 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 shutdown_connections 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 1 -- 172.21.15.137:0/4260263918 wait complete. 2024-03-26T14:13:40.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.847+0000 7f41bea1b740 20 client.18054 trim_cache size 0 max 16384 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 Processor -- start 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 -- start start 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831cca0 0x55ab0831d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 0x55ab0831dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41ac640 con 0x55ab0831c2b0 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41aa140 con 0x55ab0831cca0 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41bea1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a81c0 con 0x55ab08321e10 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60060/0 (socket says 172.21.15.137:60060) 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41b9e00640 1 -- 172.21.15.137:0/181690162 learned_addr learned my addr 172.21.15.137:0/181690162 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:40.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.869+0000 7f41b8a00640 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831cca0 0x55ab0831d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41b9e00640 1 -- 172.21.15.137:0/181690162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 msgr2=0x55ab0831dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41b9e00640 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 0x55ab0831dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41b9e00640 1 -- 172.21.15.137:0/181690162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831cca0 msgr2=0x55ab0831d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41b9e00640 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831cca0 0x55ab0831d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41b9e00640 1 -- 172.21.15.137:0/181690162 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0831d530 con 0x55ab0831c2b0 2024-03-26T14:13:40.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41b9e00640 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831b190 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f41ac005940 tx=0x7f41ac02c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0b205556c3db21 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:40.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.870+0000 7f41a5c00640 1 -- 172.21.15.137:0/181690162 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac053070 con 0x55ab0831c2b0 2024-03-26T14:13:40.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.871+0000 7f41a5c00640 1 -- 172.21.15.137:0/181690162 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac0035f0 con 0x55ab0831c2b0 2024-03-26T14:13:40.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.871+0000 7f41a5c00640 1 -- 172.21.15.137:0/181690162 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac044040 con 0x55ab0831c2b0 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.871+0000 7f41bea1b740 1 -- 172.21.15.137:0/181690162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 msgr2=0x55ab0831b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.871+0000 7f41bea1b740 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831b190 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f41ac005940 tx=0x7f41ac02c1f0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 -- 172.21.15.137:0/181690162 shutdown_connections 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab0831cca0 0x55ab0831d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab08321e10 0x55ab0831dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 --2- 172.21.15.137:0/181690162 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831c2b0 0x55ab0831b190 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 -- 172.21.15.137:0/181690162 >> 172.21.15.137:0/181690162 conn(0x55ab08317820 msgr2=0x55ab0831aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 -- 172.21.15.137:0/181690162 shutdown_connections 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 -- 172.21.15.137:0/181690162 wait complete. 2024-03-26T14:13:40.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 -- start start 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.872+0000 7f41bea1b740 1 Processor -- start 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831c2b0 0x55ab08389850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08389d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab08390dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a87c0 con 0x55ab0831cca0 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a9ae0 con 0x55ab08321e10 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b41a8e20 con 0x55ab0831c2b0 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41bea1b740 10 client.0 _ll_register_callbacks cb 0x7f41a000c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08389d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08389d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60074/0 (socket says 172.21.15.137:60074) 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 -- 172.21.15.137:0/3478174413 learned_addr learned my addr 172.21.15.137:0/3478174413 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b9e00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab08390dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b9e00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831c2b0 0x55ab08389850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 -- 172.21.15.137:0/3478174413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831c2b0 msgr2=0x55ab08389850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831c2b0 0x55ab08389850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 -- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 msgr2=0x55ab08390dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:40.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab08390dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.873+0000 7f41b8a00640 1 -- 172.21.15.137:0/3478174413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab083913c0 con 0x55ab0831cca0 2024-03-26T14:13:40.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f41b9e00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab08390dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f41b8a00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08389d90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f41a000ccf0 tx=0x7f41a000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02f42897cb90eb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f419aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f419aa00640 1 -- 172.21.15.137:0/3478174413 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000fd60 con 0x55ab0831cca0 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f419aa00640 1 -- 172.21.15.137:0/3478174413 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f41a0004660 con 0x55ab0831cca0 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41ac0035f0 con 0x55ab0831cca0 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f419aa00640 1 -- 172.21.15.137:0/3478174413 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a0038050 con 0x55ab0831cca0 2024-03-26T14:13:40.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f41bea1b740 10 client.18060 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.874+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ab083927a0 con 0x55ab0831cca0 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f419aa00640 1 -- 172.21.15.137:0/3478174413 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac0035f0 con 0x55ab0831cca0 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f419aa00640 1 -- 172.21.15.137:0/3478174413 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f41a0018040 con 0x55ab0831cca0 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f41bea1b740 10 client.18060 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f41bea1b740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f41bea1b740 20 client.18060 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f41bea1b740 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab08393110 0x55ab08395530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f41bea1b740 4 client.18060 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.875+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ab08289740 con 0x55ab08393110 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.876+0000 7f41b9e00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab08393110 0x55ab08395530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.876+0000 7f41b9e00640 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab08393110 0x55ab08395530 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed8605be3ddbf8b server_cookie=6cc1def125217c77 in_seq=0 out_seq=0 2024-03-26T14:13:40.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.876+0000 7f419aa00640 10 client.18060 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:40.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f419aa00640 1 -- 172.21.15.137:0/3478174413 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ab08289740 con 0x55ab08393110 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f419aa00640 10 client.18060 handle_command_reply: tid=0 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 client.18060 shutdown 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 msgr2=0x55ab08389d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08389d90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f41a000ccf0 tx=0x7f41a000f8d0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 shutdown_connections 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ab08393110 0x55ab08395530 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab08321e10 0x55ab08390dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ab0831cca0 0x55ab08389d90 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 --2- 172.21.15.137:0/3478174413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ab0831c2b0 0x55ab08389850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f419aa00640 0 client.18060 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 >> 172.21.15.137:0/3478174413 conn(0x55ab08317820 msgr2=0x55ab0831b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 shutdown_connections 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 1 -- 172.21.15.137:0/3478174413 wait complete. 2024-03-26T14:13:40.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.878+0000 7f41bea1b740 20 client.18060 trim_cache size 0 max 16384 2024-03-26T14:13:40.881 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:13:40.881 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:13:40.881 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "6cf0fbe7-6f5d-47c3-9d7e-bc5c042cf69e", 2024-03-26T14:13:40.881 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:13:40.881 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f419c0a2800 msgr2=0x7f419c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f419c0a2800 0x7f419c0a4cc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f41a800cba0 tx=0x7f41a800cf90 comp rx=0 tx=0).stop 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 msgr2=0x7f41b4197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 0x7f41b4197770 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f41a800ebc0 tx=0x7f41a800c6a0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 shutdown_connections 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f419c0a2800 0x7f419c0a4cc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4107260 0x7f41b419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41b4104930 0x7f41b4197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 --2- 172.21.15.137:0/1198978379 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41b4102000 0x7f41b4197770 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 >> 172.21.15.137:0/1198978379 conn(0x7f41b40fba10 msgr2=0x7f41b4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 shutdown_connections 2024-03-26T14:13:40.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:40.879+0000 7f41ba800640 1 -- 172.21.15.137:0/1198978379 wait complete. 2024-03-26T14:13:40.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:13:40.898 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 -- 172.21.15.137:0/1300438444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750104990 msgr2=0x7f0750104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 --2- 172.21.15.137:0/1300438444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750104990 0x7f0750104e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0738009900 tx=0x7f073802bc30 comp rx=0 tx=0).stop 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 -- 172.21.15.137:0/1300438444 shutdown_connections 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 --2- 172.21.15.137:0/1300438444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750104990 0x7f0750104e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 --2- 172.21.15.137:0/1300438444 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0750105350 0x7f075010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 --2- 172.21.15.137:0/1300438444 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07501037a0 0x7f0750103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 -- 172.21.15.137:0/1300438444 >> 172.21.15.137:0/1300438444 conn(0x7f07500fed10 msgr2=0x7f0750101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 -- 172.21.15.137:0/1300438444 shutdown_connections 2024-03-26T14:13:41.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.106+0000 7f074f200640 1 -- 172.21.15.137:0/1300438444 wait complete. 2024-03-26T14:13:41.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.108+0000 7f074f200640 1 Processor -- start 2024-03-26T14:13:41.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.108+0000 7f074f200640 1 -- start start 2024-03-26T14:13:41.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 0x7f07501999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074f200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0750104990 0x7f0750199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750105350 0x7f07501a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075010f320 con 0x7f07501037a0 2024-03-26T14:13:41.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075010f1a0 con 0x7f0750105350 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074f200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075010f4a0 con 0x7f0750104990 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 0x7f07501999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 0x7f07501999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60090/0 (socket says 172.21.15.137:60090) 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074de00640 1 -- 172.21.15.137:0/4165679645 learned_addr learned my addr 172.21.15.137:0/4165679645 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.109+0000 7f074e800640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750105350 0x7f07501a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f0747e00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0750104990 0x7f0750199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f074de00640 1 -- 172.21.15.137:0/4165679645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0750104990 msgr2=0x7f0750199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f074de00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0750104990 0x7f0750199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f074de00640 1 -- 172.21.15.137:0/4165679645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750105350 msgr2=0x7f07501a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f074de00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750105350 0x7f07501a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f074de00640 1 -- 172.21.15.137:0/4165679645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07501a1600 con 0x7f07501037a0 2024-03-26T14:13:41.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.110+0000 7f074de00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 0x7f07501999e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f074000e9c0 tx=0x7f074000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:41.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.111+0000 7f0746a00640 1 -- 172.21.15.137:0/4165679645 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074000cd80 con 0x7f07501037a0 2024-03-26T14:13:41.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.111+0000 7f0746a00640 1 -- 172.21.15.137:0/4165679645 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07400035f0 con 0x7f07501037a0 2024-03-26T14:13:41.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.111+0000 7f0746a00640 1 -- 172.21.15.137:0/4165679645 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740010690 con 0x7f07501037a0 2024-03-26T14:13:41.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.111+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07501a18f0 con 0x7f07501037a0 2024-03-26T14:13:41.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.111+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07501a1dc0 con 0x7f07501037a0 2024-03-26T14:13:41.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.114+0000 7f0757daf740 1 Processor -- start 2024-03-26T14:13:41.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.114+0000 7f0757daf740 1 -- start start 2024-03-26T14:13:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0757daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c64470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0757daf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c69540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0757daf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c6c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0757daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560666bc9250 con 0x560666c64050 2024-03-26T14:13:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0757daf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560666b71a30 con 0x560666c64a40 2024-03-26T14:13:41.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0757daf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560666bc52f0 con 0x560666c69bb0 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0746a00640 1 -- 172.21.15.137:0/4165679645 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f07400037a0 con 0x7f07501037a0 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c64470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c64470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60094/0 (socket says 172.21.15.137:60094) 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f074e800640 1 -- 172.21.15.137:0/360381219 learned_addr learned my addr 172.21.15.137:0/360381219 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.115+0000 7f0747e00640 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c6c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074de00640 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c69540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074e800640 1 -- 172.21.15.137:0/360381219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 msgr2=0x560666c6c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074e800640 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c6c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074e800640 1 -- 172.21.15.137:0/360381219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 msgr2=0x560666c69540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074e800640 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c69540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074e800640 1 -- 172.21.15.137:0/360381219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560666b5c690 con 0x560666c64050 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f0747e00640 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c6c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:13:41.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f074e800640 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c64470 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0748009fd0 tx=0x7f07480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c2a33d8fcd6743 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:41.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f073f400640 1 -- 172.21.15.137:0/360381219 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748004d20 con 0x560666c64050 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.116+0000 7f073f400640 1 -- 172.21.15.137:0/360381219 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0748004050 con 0x560666c64050 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.117+0000 7f073f400640 1 -- 172.21.15.137:0/360381219 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748005240 con 0x560666c64050 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.117+0000 7f0746a00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07340a2880 0x7f07340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.117+0000 7f0746a00640 1 -- 172.21.15.137:0/4165679645 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f07400cf380 con 0x7f07501037a0 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.117+0000 7f074de00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07340a2880 0x7f07340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.117+0000 7f0757daf740 1 -- 172.21.15.137:0/360381219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 msgr2=0x560666c64470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.117+0000 7f0757daf740 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c64470 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0748009fd0 tx=0x7f07480118d0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 -- 172.21.15.137:0/360381219 shutdown_connections 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c69540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c6c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 --2- 172.21.15.137:0/360381219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c64470 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 -- 172.21.15.137:0/360381219 >> 172.21.15.137:0/360381219 conn(0x560666c2c200 msgr2=0x560666c5f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 -- 172.21.15.137:0/360381219 shutdown_connections 2024-03-26T14:13:41.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 -- 172.21.15.137:0/360381219 wait complete. 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f074de00640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07340a2880 0x7f07340a4d40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07400a3bc0 tx=0x7f07400d2930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.118+0000 7f0757daf740 1 -- start start 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 Processor -- start 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c6e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c75570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560666c6cb60 con 0x560666c64050 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560666b84e80 con 0x560666c64a40 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560666c6cce0 con 0x560666c69bb0 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0757daf740 10 client.0 _ll_register_callbacks cb 0x560666b97130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f0747e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c6e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c6dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60098/0 (socket says 172.21.15.137:60098) 2024-03-26T14:13:41.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.119+0000 7f074e800640 1 -- 172.21.15.137:0/729990917 learned_addr learned my addr 172.21.15.137:0/729990917 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:41.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c75570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 -- 172.21.15.137:0/729990917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 msgr2=0x560666c75570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c75570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 -- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 msgr2=0x560666c6e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c6e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 -- 172.21.15.137:0/729990917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606669a77d0 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f074e800640 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c6dff0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0748009fa0 tx=0x7f07480047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbecd0ab413f8627 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.120+0000 7f0733e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0733e00640 1 -- 172.21.15.137:0/729990917 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748029020 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0733e00640 1 -- 172.21.15.137:0/729990917 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0748005040 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0748004050 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0733e00640 1 -- 172.21.15.137:0/729990917 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0748034050 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0757daf740 10 client.18072 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5606668dd3f0 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0733e00640 1 -- 172.21.15.137:0/729990917 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0748004050 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.121+0000 7f0733e00640 1 -- 172.21.15.137:0/729990917 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f074800f850 con 0x560666c64050 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0757daf740 10 client.18072 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:41.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0757daf740 10 client.18072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0757daf740 20 client.18072 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0757daf740 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666c79c10 0x560666c7c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0757daf740 4 client.18072 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560666bd6790 con 0x560666c79c10 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.122+0000 7f0747e00640 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666c79c10 0x560666c7c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.123+0000 7f0747e00640 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666c79c10 0x560666c7c000 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7fc17872f25a08 server_cookie=e56004b4702d7718 in_seq=0 out_seq=0 2024-03-26T14:13:41.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.123+0000 7f0733e00640 10 client.18072 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.124+0000 7f0733e00640 1 -- 172.21.15.137:0/729990917 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560666bd6790 con 0x560666c79c10 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.124+0000 7f0733e00640 10 client.18072 handle_command_reply: tid=0 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 client.18072 shutdown 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 msgr2=0x560666c6dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c6dff0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0748009fa0 tx=0x7f07480047a0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 shutdown_connections 2024-03-26T14:13:41.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666c79c10 0x560666c7c000 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64a40 0x560666c6e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c69bb0 0x560666c75570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 --2- 172.21.15.137:0/729990917 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c6dff0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 >> 172.21.15.137:0/729990917 conn(0x560666c2c200 msgr2=0x560666c5ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0733e00640 0 client.18072 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 shutdown_connections 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 1 -- 172.21.15.137:0/729990917 wait complete. 2024-03-26T14:13:41.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.125+0000 7f0757daf740 20 client.18072 trim_cache size 0 max 16384 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.144+0000 7f0757daf740 1 Processor -- start 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 -- start start 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64050 0x560666c63310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c75530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 0x560666c5e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501ae830 con 0x560666c69bb0 2024-03-26T14:13:41.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501ac330 con 0x560666c64050 2024-03-26T14:13:41.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0757daf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501aa3b0 con 0x560666c64a40 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 0x560666c5e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f074e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64050 0x560666c63310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 0x560666c5e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60102/0 (socket says 172.21.15.137:60102) 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f074e800640 1 -- 172.21.15.137:0/2708419583 learned_addr learned my addr 172.21.15.137:0/2708419583 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.145+0000 7f0747e00640 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c75530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f074e800640 1 -- 172.21.15.137:0/2708419583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 msgr2=0x560666c75530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f074e800640 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c75530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f074e800640 1 -- 172.21.15.137:0/2708419583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64050 msgr2=0x560666c63310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f074e800640 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64050 0x560666c63310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f074e800640 1 -- 172.21.15.137:0/2708419583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56066689d0f0 con 0x560666c69bb0 2024-03-26T14:13:41.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f074e800640 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 0x560666c5e330 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0748009f70 tx=0x7f0748003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413976af42592516 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:41.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f073ea00640 1 -- 172.21.15.137:0/2708419583 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748029040 con 0x560666c69bb0 2024-03-26T14:13:41.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f073ea00640 1 -- 172.21.15.137:0/2708419583 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f074800f990 con 0x560666c69bb0 2024-03-26T14:13:41.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.146+0000 7f073ea00640 1 -- 172.21.15.137:0/2708419583 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748019850 con 0x560666c69bb0 2024-03-26T14:13:41.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 -- 172.21.15.137:0/2708419583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 msgr2=0x560666c5e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 0x560666c5e330 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0748009f70 tx=0x7f0748003dc0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 -- 172.21.15.137:0/2708419583 shutdown_connections 2024-03-26T14:13:41.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c69bb0 0x560666c5e330 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c75530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 --2- 172.21.15.137:0/2708419583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c64050 0x560666c63310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 -- 172.21.15.137:0/2708419583 >> 172.21.15.137:0/2708419583 conn(0x560666c2c200 msgr2=0x560666c62c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 -- 172.21.15.137:0/2708419583 shutdown_connections 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.147+0000 7f0757daf740 1 -- 172.21.15.137:0/2708419583 wait complete. 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 -- start start 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 Processor -- start 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c72c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c73170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c69bb0 0x560666c74110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501aa9b0 con 0x560666c64050 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501abcd0 con 0x560666c69bb0 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.148+0000 7f0757daf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07501ab010 con 0x560666c64a40 2024-03-26T14:13:41.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f0757daf740 10 client.0 _ll_register_callbacks cb 0x7f0748009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c72c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c72c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:60110/0 (socket says 172.21.15.137:60110) 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 -- 172.21.15.137:0/2401789962 learned_addr learned my addr 172.21.15.137:0/2401789962 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c69bb0 0x560666c74110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f0747e00640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c73170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 -- 172.21.15.137:0/2401789962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 msgr2=0x560666c73170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c73170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 -- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c69bb0 msgr2=0x560666c74110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c69bb0 0x560666c74110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.149+0000 7f074e800640 1 -- 172.21.15.137:0/2401789962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560666c74700 con 0x560666c64050 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f074e800640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c72c30 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f074800fbc0 tx=0x7f07480118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acc4e3eab1af37b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0733e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0733e00640 1 -- 172.21.15.137:0/2401789962 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748029070 con 0x560666c64050 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0733e00640 1 -- 172.21.15.137:0/2401789962 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f0748003e30 con 0x560666c64050 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f074800f990 con 0x560666c64050 2024-03-26T14:13:41.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0733e00640 1 -- 172.21.15.137:0/2401789962 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f074804b050 con 0x560666c64050 2024-03-26T14:13:41.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0757daf740 10 client.18078 fetch_fsmap learned FSMap version 24 2024-03-26T14:13:41.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560666cd5440 con 0x560666c64050 2024-03-26T14:13:41.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.150+0000 7f0733e00640 1 -- 172.21.15.137:0/2401789962 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f074800f990 con 0x560666c64050 2024-03-26T14:13:41.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0733e00640 1 -- 172.21.15.137:0/2401789962 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f0748043020 con 0x560666c64050 2024-03-26T14:13:41.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0757daf740 10 client.18078 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:13:41.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0757daf740 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0757daf740 20 client.18078 populate_metadata read hostname 'smithi137' 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0757daf740 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666cd5c80 0x560666cd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0757daf740 4 client.18078 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560666bd0ae0 con 0x560666cd5c80 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.151+0000 7f0747e00640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666cd5c80 0x560666cd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.152+0000 7f0747e00640 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666cd5c80 0x560666cd80a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84f6b91eb039873 server_cookie=bfd822358dfe5796 in_seq=0 out_seq=0 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.152+0000 7f0733e00640 10 client.18078 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.152+0000 7f0733e00640 1 -- 172.21.15.137:0/2401789962 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560666bd0ae0 con 0x560666cd5c80 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.152+0000 7f0733e00640 10 client.18078 handle_command_reply: tid=0 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.152+0000 7f0757daf740 1 client.18078 shutdown 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 msgr2=0x560666c72c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c72c30 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f074800fbc0 tx=0x7f07480118d0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 shutdown_connections 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x560666cd5c80 0x560666cd80a0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.155 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560666c69bb0 0x560666c74110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560666c64a40 0x560666c73170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 --2- 172.21.15.137:0/2401789962 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560666c64050 0x560666c72c30 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0733e00640 0 client.18078 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 >> 172.21.15.137:0/2401789962 conn(0x560666c2c200 msgr2=0x560666c63490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 shutdown_connections 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 1 -- 172.21.15.137:0/2401789962 wait complete. 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.153+0000 7f0757daf740 20 client.18078 trim_cache size 0 max 16384 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stdout: "6cf0fbe7-6f5d-47c3-9d7e-bc5c042cf69e": { 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:13:41.156 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "6cf0fbe7-6f5d-47c3-9d7e-bc5c042cf69e", 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07340a2880 msgr2=0x7f07340a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07340a2880 0x7f07340a4d40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f07400a3bc0 tx=0x7f07400d2930 comp rx=0 tx=0).stop 2024-03-26T14:13:41.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 msgr2=0x7f07501999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 0x7f07501999e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f074000e9c0 tx=0x7f074000ee80 comp rx=0 tx=0).stop 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 shutdown_connections 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f07340a2880 0x7f07340a4d40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0750105350 0x7f07501a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0750104990 0x7f0750199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 --2- 172.21.15.137:0/4165679645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501037a0 0x7f07501999e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 >> 172.21.15.137:0/4165679645 conn(0x7f07500fed10 msgr2=0x7f0750107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 shutdown_connections 2024-03-26T14:13:41.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:13:41.154+0000 7f074f200640 1 -- 172.21.15.137:0/4165679645 wait complete. 2024-03-26T14:13:41.175 INFO:tasks.cephfs.filesystem:scrub status for tag:6cf0fbe7-6f5d-47c3-9d7e-bc5c042cf69e - {'path': '/', 'tag': '6cf0fbe7-6f5d-47c3-9d7e-bc5c042cf69e', 'options': 'recursive,force'} 2024-03-26T14:13:41.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:41 smithi179 ceph-mon[35472]: pgmap v797: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:41.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:41 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:13:41.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:41 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:41.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:41 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:13:41.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:41 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:41 smithi159 ceph-mon[35501]: pgmap v797: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:41 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:13:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:41 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:41 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:13:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:41 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:41.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:41 smithi137 ceph-mon[31539]: pgmap v797: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:41.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:41 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:13:41.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:41 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:41.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:41 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:13:41.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:41 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:13:42.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:42 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:13:42.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:42 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:13:42.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:42 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:13:43.576 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:43 smithi179 ceph-mon[35472]: pgmap v798: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:43.605 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:43 smithi159 ceph-mon[35501]: pgmap v798: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:43.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:43 smithi137 ceph-mon[31539]: pgmap v798: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:45.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:45 smithi179 ceph-mon[35472]: pgmap v799: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:13:45.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:45 smithi159 ceph-mon[35501]: pgmap v799: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:13:45.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:45 smithi137 ceph-mon[31539]: pgmap v799: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:13:47.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:47 smithi179 ceph-mon[35472]: pgmap v800: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:47.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:47 smithi159 ceph-mon[35501]: pgmap v800: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:47.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:47 smithi137 ceph-mon[31539]: pgmap v800: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:49.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:49 smithi179 ceph-mon[35472]: pgmap v801: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:13:49.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:49 smithi159 ceph-mon[35501]: pgmap v801: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:13:49.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:49 smithi137 ceph-mon[31539]: pgmap v801: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:13:51.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:51 smithi179 ceph-mon[35472]: pgmap v802: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:51.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:51 smithi159 ceph-mon[35501]: pgmap v802: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:51.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:51 smithi137 ceph-mon[31539]: pgmap v802: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:13:53.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:53 smithi179 ceph-mon[35472]: pgmap v803: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:53.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:53 smithi159 ceph-mon[35501]: pgmap v803: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:53.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:53 smithi137 ceph-mon[31539]: pgmap v803: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:13:55.589 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:55 smithi137 ceph-mon[31539]: pgmap v804: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:55.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:55 smithi179 ceph-mon[35472]: pgmap v804: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:55.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:55 smithi159 ceph-mon[35501]: pgmap v804: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:13:57.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:57 smithi179 ceph-mon[35472]: pgmap v805: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:57.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:57 smithi159 ceph-mon[35501]: pgmap v805: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:57.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:57 smithi137 ceph-mon[31539]: pgmap v805: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:13:59.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:13:59 smithi179 ceph-mon[35472]: pgmap v806: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:59.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:13:59 smithi159 ceph-mon[35501]: pgmap v806: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:13:59.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:13:59 smithi137 ceph-mon[31539]: pgmap v806: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:14:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:01 smithi179 ceph-mon[35472]: pgmap v807: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:01 smithi159 ceph-mon[35501]: pgmap v807: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:01.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:01 smithi137 ceph-mon[31539]: pgmap v807: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:03.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:03 smithi179 ceph-mon[35472]: pgmap v808: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:14:03.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:03 smithi159 ceph-mon[35501]: pgmap v808: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:14:03.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:03 smithi137 ceph-mon[31539]: pgmap v808: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:14:05.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:05 smithi179 ceph-mon[35472]: pgmap v809: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:05.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:05 smithi159 ceph-mon[35501]: pgmap v809: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:05.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:05 smithi137 ceph-mon[31539]: pgmap v809: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:07.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:07 smithi179 ceph-mon[35472]: pgmap v810: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:07.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:07 smithi159 ceph-mon[35501]: pgmap v810: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:07.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:07 smithi137 ceph-mon[31539]: pgmap v810: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:09.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:09 smithi179 ceph-mon[35472]: pgmap v811: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:14:09.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:09 smithi159 ceph-mon[35501]: pgmap v811: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:14:09.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:09 smithi137 ceph-mon[31539]: pgmap v811: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:14:11.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:14:11.177 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 -- 172.21.15.137:0/3438767464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88104930 msgr2=0x7fee88106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 --2- 172.21.15.137:0/3438767464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88104930 0x7fee88106d20 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fee74009900 tx=0x7fee7402bc30 comp rx=0 tx=0).stop 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 -- 172.21.15.137:0/3438767464 shutdown_connections 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 --2- 172.21.15.137:0/3438767464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee88107260 0x7fee881096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 --2- 172.21.15.137:0/3438767464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88104930 0x7fee88106d20 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 --2- 172.21.15.137:0/3438767464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88102000 0x7fee881043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 -- 172.21.15.137:0/3438767464 >> 172.21.15.137:0/3438767464 conn(0x7fee880fba10 msgr2=0x7fee880fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 -- 172.21.15.137:0/3438767464 shutdown_connections 2024-03-26T14:14:11.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.412+0000 7fee8e800640 1 -- 172.21.15.137:0/3438767464 wait complete. 2024-03-26T14:14:11.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.413+0000 7fee8e800640 1 Processor -- start 2024-03-26T14:14:11.414 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 -- start start 2024-03-26T14:14:11.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 0x7fee88199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee88104930 0x7fee88199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88107260 0x7fee881a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8810d110 con 0x7fee88102000 2024-03-26T14:14:11.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8810cf90 con 0x7fee88107260 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8e800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8810d290 con 0x7fee88104930 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 0x7fee88199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 0x7fee88199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45178/0 (socket says 172.21.15.137:45178) 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.414+0000 7fee8d400640 1 -- 172.21.15.137:0/1121530775 learned_addr learned my addr 172.21.15.137:0/1121530775 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8de00640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88107260 0x7fee881a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8ca00640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee88104930 0x7fee88199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8d400640 1 -- 172.21.15.137:0/1121530775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee88104930 msgr2=0x7fee88199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8d400640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee88104930 0x7fee88199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8d400640 1 -- 172.21.15.137:0/1121530775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88107260 msgr2=0x7fee881a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8d400640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88107260 0x7fee881a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8d400640 1 -- 172.21.15.137:0/1121530775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee881a15a0 con 0x7fee88102000 2024-03-26T14:14:11.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8de00640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88107260 0x7fee881a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:11.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee8d400640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 0x7fee88199980 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e9f0 tx=0x7fee7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.415+0000 7fee86000640 1 -- 172.21.15.137:0/1121530775 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00cd80 con 0x7fee88102000 2024-03-26T14:14:11.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.416+0000 7fee86000640 1 -- 172.21.15.137:0/1121530775 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee7c0108e0 con 0x7fee88102000 2024-03-26T14:14:11.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.416+0000 7fee86000640 1 -- 172.21.15.137:0/1121530775 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c018920 con 0x7fee88102000 2024-03-26T14:14:11.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.416+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee881a1890 con 0x7fee88102000 2024-03-26T14:14:11.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.416+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee881a1d60 con 0x7fee88102000 2024-03-26T14:14:11.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.419+0000 7fee86000640 1 -- 172.21.15.137:0/1121530775 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fee7c010a90 con 0x7fee88102000 2024-03-26T14:14:11.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 Processor -- start 2024-03-26T14:14:11.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 -- start start 2024-03-26T14:14:11.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65b050 0x55601d65b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d660540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d660bb0 0x55601d663040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601d5c0250 con 0x55601d65ba40 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601d568a30 con 0x55601d65b050 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee92c18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601d5bc2f0 con 0x55601d660bb0 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.420+0000 7fee8d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d660540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d660540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45194/0 (socket says 172.21.15.137:45194) 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 -- 172.21.15.137:0/3106713110 learned_addr learned my addr 172.21.15.137:0/3106713110 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8de00640 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65b050 0x55601d65b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8ca00640 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d660bb0 0x55601d663040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 -- 172.21.15.137:0/3106713110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d660bb0 msgr2=0x55601d663040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d660bb0 0x55601d663040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 -- 172.21.15.137:0/3106713110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65b050 msgr2=0x55601d65b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65b050 0x55601d65b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.421+0000 7fee8d400640 1 -- 172.21.15.137:0/3106713110 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55601d553690 con 0x55601d65ba40 2024-03-26T14:14:11.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee8d400640 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d660540 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0d74b0 tx=0x7fee7c0d7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e643ff8c5b7040cc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee86000640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fee700a2880 0x7fee700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee86000640 1 -- 172.21.15.137:0/1121530775 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fee7c0cf260 con 0x7fee88102000 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee8de00640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fee700a2880 0x7fee700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee7a600640 1 -- 172.21.15.137:0/3106713110 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0e2070 con 0x55601d65ba40 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee7a600640 1 -- 172.21.15.137:0/3106713110 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee7c010de0 con 0x55601d65ba40 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.422+0000 7fee7a600640 1 -- 172.21.15.137:0/3106713110 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0d2d90 con 0x55601d65ba40 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 -- 172.21.15.137:0/3106713110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 msgr2=0x55601d660540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d660540 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0d74b0 tx=0x7fee7c0d7970 comp rx=0 tx=0).stop 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee8de00640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fee700a2880 0x7fee700a4d40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55601d65c2a0 tx=0x7fee80007720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 -- 172.21.15.137:0/3106713110 shutdown_connections 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d660bb0 0x55601d663040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d660540 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 --2- 172.21.15.137:0/3106713110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65b050 0x55601d65b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 -- 172.21.15.137:0/3106713110 >> 172.21.15.137:0/3106713110 conn(0x55601d623200 msgr2=0x55601d6569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 -- 172.21.15.137:0/3106713110 shutdown_connections 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.423+0000 7fee92c18740 1 -- 172.21.15.137:0/3106713110 wait complete. 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 -- start start 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 Processor -- start 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d665000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d665540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d66c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601d663b60 con 0x55601d65ba40 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601d57be80 con 0x55601d660bb0 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55601d663ce0 con 0x55601d65b050 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee92c18740 10 client.0 _ll_register_callbacks cb 0x55601d58e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:11.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.424+0000 7fee8ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d665540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d665540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45202/0 (socket says 172.21.15.137:45202) 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 -- 172.21.15.137:0/963290042 learned_addr learned my addr 172.21.15.137:0/963290042 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8de00640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d66c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8d400640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d665000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 -- 172.21.15.137:0/963290042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 msgr2=0x55601d665000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d665000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 -- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 msgr2=0x55601d66c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d66c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8ca00640 1 -- 172.21.15.137:0/963290042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55601d39e7d0 con 0x55601d65ba40 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.425+0000 7fee8d400640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d665000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee8ca00640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d665540 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fee7402c840 tx=0x7fee74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bc0da90d5d3d13 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee6ea00640 1 -- 172.21.15.137:0/963290042 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74005d50 con 0x55601d65ba40 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee6ea00640 1 -- 172.21.15.137:0/963290042 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fee74005230 con 0x55601d65ba40 2024-03-26T14:14:11.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55601d5cd790 con 0x55601d65ba40 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee6ea00640 1 -- 172.21.15.137:0/963290042 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee74005430 con 0x55601d65ba40 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee92c18740 10 client.18090 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.426+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55601d2d43f0 con 0x55601d65ba40 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee6ea00640 1 -- 172.21.15.137:0/963290042 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55601d5cd790 con 0x55601d65ba40 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee6ea00640 1 -- 172.21.15.137:0/963290042 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fee74003c00 con 0x55601d65ba40 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee92c18740 10 client.18090 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee92c18740 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee92c18740 20 client.18090 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee92c18740 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d670e40 0x55601d6732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee92c18740 4 client.18090 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55601d5c7ae0 con 0x55601d670e40 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.427+0000 7fee8d400640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d670e40 0x55601d6732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.428+0000 7fee8d400640 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d670e40 0x55601d6732e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59502c59d842bc server_cookie=dd9f5ee01da1c237 in_seq=0 out_seq=0 2024-03-26T14:14:11.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.428+0000 7fee6ea00640 10 client.18090 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee6ea00640 1 -- 172.21.15.137:0/963290042 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55601d5c7ae0 con 0x55601d670e40 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee6ea00640 10 client.18090 handle_command_reply: tid=0 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 client.18090 shutdown 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 msgr2=0x55601d665540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d665540 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fee7402c840 tx=0x7fee74004270 comp rx=0 tx=0).stop 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 shutdown_connections 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d670e40 0x55601d6732e0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d66c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d665540 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 --2- 172.21.15.137:0/963290042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d665000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 >> 172.21.15.137:0/963290042 conn(0x55601d623200 msgr2=0x55601d655bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee6ea00640 0 client.18090 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:11.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 shutdown_connections 2024-03-26T14:14:11.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 1 -- 172.21.15.137:0/963290042 wait complete. 2024-03-26T14:14:11.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.430+0000 7fee92c18740 20 client.18090 trim_cache size 0 max 16384 2024-03-26T14:14:11.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 Processor -- start 2024-03-26T14:14:11.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 -- start start 2024-03-26T14:14:11.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d655330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d66c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d655870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881ae760 con 0x55601d65ba40 2024-03-26T14:14:11.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881ac260 con 0x55601d660bb0 2024-03-26T14:14:11.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee92c18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881aa2e0 con 0x55601d65b050 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.448+0000 7fee8ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d66c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d66c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45214/0 (socket says 172.21.15.137:45214) 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 -- 172.21.15.137:0/1663549248 learned_addr learned my addr 172.21.15.137:0/1663549248 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8de00640 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d655870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8d400640 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d655330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 -- 172.21.15.137:0/1663549248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 msgr2=0x55601d655330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d655330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 -- 172.21.15.137:0/1663549248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 msgr2=0x55601d655870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d655870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 -- 172.21.15.137:0/1663549248 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55601d2940f0 con 0x55601d65ba40 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.449+0000 7fee8ca00640 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d66c540 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fee74034600 tx=0x7fee7402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea90e837f4e3cdc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee79c00640 1 -- 172.21.15.137:0/1663549248 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74058020 con 0x55601d65ba40 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee79c00640 1 -- 172.21.15.137:0/1663549248 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee74004d80 con 0x55601d65ba40 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee79c00640 1 -- 172.21.15.137:0/1663549248 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74000e60 con 0x55601d65ba40 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 -- 172.21.15.137:0/1663549248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 msgr2=0x55601d66c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d66c540 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fee74034600 tx=0x7fee7402bc30 comp rx=0 tx=0).stop 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 -- 172.21.15.137:0/1663549248 shutdown_connections 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d660bb0 0x55601d655870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d65ba40 0x55601d66c540 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 --2- 172.21.15.137:0/1663549248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d655330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.450+0000 7fee92c18740 1 -- 172.21.15.137:0/1663549248 >> 172.21.15.137:0/1663549248 conn(0x55601d623200 msgr2=0x55601d665000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.451+0000 7fee92c18740 1 -- 172.21.15.137:0/1663549248 shutdown_connections 2024-03-26T14:14:11.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.451+0000 7fee92c18740 1 -- 172.21.15.137:0/1663549248 wait complete. 2024-03-26T14:14:11.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.451+0000 7fee92c18740 1 -- start start 2024-03-26T14:14:11.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.451+0000 7fee92c18740 1 Processor -- start 2024-03-26T14:14:11.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d669b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65ba40 0x55601d66a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 0x55601d66b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881aa8e0 con 0x55601d660bb0 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881abc00 con 0x55601d65ba40 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee881aaf40 con 0x55601d65b050 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee92c18740 10 client.0 _ll_register_callbacks cb 0x7fee7c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 0x55601d66b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 0x55601d66b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45230/0 (socket says 172.21.15.137:45230) 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 -- 172.21.15.137:0/3649597029 learned_addr learned my addr 172.21.15.137:0/3649597029 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8ca00640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65ba40 0x55601d66a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8d400640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d669b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 -- 172.21.15.137:0/3649597029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 msgr2=0x55601d669b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d669b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 -- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65ba40 msgr2=0x55601d66a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65ba40 0x55601d66a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.456 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.452+0000 7fee8de00640 1 -- 172.21.15.137:0/3649597029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55601d66b660 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.453+0000 7fee8de00640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 0x55601d66b070 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fee800045e0 tx=0x7fee80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9322802b2f916fe5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.453+0000 7fee6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.453+0000 7fee6ea00640 1 -- 172.21.15.137:0/3649597029 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8002c060 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.453+0000 7fee6ea00640 1 -- 172.21.15.137:0/3649597029 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fee8001d420 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.453+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee74004d80 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee6ea00640 1 -- 172.21.15.137:0/3649597029 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee80033050 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 10 client.18096 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55601d6cc770 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee6ea00640 1 -- 172.21.15.137:0/3649597029 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee74004d80 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee6ea00640 1 -- 172.21.15.137:0/3649597029 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fee80026030 con 0x55601d660bb0 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 10 client.18096 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 20 client.18096 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:11.457 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d6cd150 0x55601d6cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 4 client.18096 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:14:11.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.454+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55601d5453a0 con 0x55601d6cd150 2024-03-26T14:14:11.458 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.455+0000 7fee8d400640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d6cd150 0x55601d6cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.455+0000 7fee8d400640 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d6cd150 0x55601d6cf570 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6695c554cb46f965 server_cookie=3d5daed4ad8c4ad1 in_seq=0 out_seq=0 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.455+0000 7fee6ea00640 10 client.18096 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee6ea00640 1 -- 172.21.15.137:0/3649597029 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55601d5453a0 con 0x55601d6cd150 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee6ea00640 10 client.18096 handle_command_reply: tid=0 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 client.18096 shutdown 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 msgr2=0x55601d66b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 0x55601d66b070 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fee800045e0 tx=0x7fee80004a40 comp rx=0 tx=0).stop 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 shutdown_connections 2024-03-26T14:14:11.459 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55601d6cd150 0x55601d6cf570 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601d65ba40 0x55601d66a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55601d660bb0 0x55601d66b070 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 --2- 172.21.15.137:0/3649597029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55601d65b050 0x55601d669b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 >> 172.21.15.137:0/3649597029 conn(0x55601d623200 msgr2=0x55601d65c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee6ea00640 0 client.18096 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.456+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 shutdown_connections 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee92c18740 1 -- 172.21.15.137:0/3649597029 wait complete. 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee92c18740 20 client.18096 trim_cache size 0 max 16384 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:14:11.460 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fee700a2880 msgr2=0x7fee700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fee700a2880 0x7fee700a4d40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55601d65c2a0 tx=0x7fee80007720 comp rx=0 tx=0).stop 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 msgr2=0x7fee88199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 0x7fee88199980 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e9f0 tx=0x7fee7c00eeb0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 shutdown_connections 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fee700a2880 0x7fee700a4d40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee88107260 0x7fee881a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.461 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee88104930 0x7fee88199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.457+0000 7fee8e800640 1 --2- 172.21.15.137:0/1121530775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fee88102000 0x7fee88199980 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.458+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 >> 172.21.15.137:0/1121530775 conn(0x7fee880fba10 msgr2=0x7fee881016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.458+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 shutdown_connections 2024-03-26T14:14:11.462 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.458+0000 7fee8e800640 1 -- 172.21.15.137:0/1121530775 wait complete. 2024-03-26T14:14:11.475 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:14:11.475 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:14:11.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:11 smithi137 ceph-mon[31539]: pgmap v812: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:11.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:11 smithi179 ceph-mon[35472]: pgmap v812: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:11.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:11 smithi159 ceph-mon[35501]: pgmap v812: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:11.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 -- 172.21.15.137:0/3612268006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881052b0 msgr2=0x7f9e8810bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3612268006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881052b0 0x7f9e8810bb40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009ca0 tx=0x7f9e8002c100 comp rx=0 tx=0).stop 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 -- 172.21.15.137:0/3612268006 shutdown_connections 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3612268006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881052b0 0x7f9e8810bb40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3612268006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e881048f0 0x7f9e88104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3612268006 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e88103700 0x7f9e88103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 -- 172.21.15.137:0/3612268006 >> 172.21.15.137:0/3612268006 conn(0x7f9e880fec70 msgr2=0x7f9e881010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 -- 172.21.15.137:0/3612268006 shutdown_connections 2024-03-26T14:14:11.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.709+0000 7f9e8d600640 1 -- 172.21.15.137:0/3612268006 wait complete. 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.710+0000 7f9e8d600640 1 Processor -- start 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.710+0000 7f9e8d600640 1 -- start start 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8d600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e88103700 0x7f9e881998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881048f0 0x7f9e88199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8d600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 0x7f9e881a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8d600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8810f230 con 0x7f9e881052b0 2024-03-26T14:14:11.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8810f0b0 con 0x7f9e881048f0 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8d600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8810f3b0 con 0x7f9e88103700 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 0x7f9e881a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 0x7f9e881a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45258/0 (socket says 172.21.15.137:45258) 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 -- 172.21.15.137:0/3137852324 learned_addr learned my addr 172.21.15.137:0/3137852324 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e87400640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881048f0 0x7f9e88199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e87e00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e88103700 0x7f9e881998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 -- 172.21.15.137:0/3137852324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e88103700 msgr2=0x7f9e881998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e88103700 0x7f9e881998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 -- 172.21.15.137:0/3137852324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881048f0 msgr2=0x7f9e88199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881048f0 0x7f9e88199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.711+0000 7f9e8cc00640 1 -- 172.21.15.137:0/3137852324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e881a14f0 con 0x7f9e881052b0 2024-03-26T14:14:11.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.712+0000 7f9e8cc00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 0x7f9e881a0e80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9e80032520 tx=0x7f9e80035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.712+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e80039040 con 0x7f9e881052b0 2024-03-26T14:14:11.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.712+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e80044070 con 0x7f9e881052b0 2024-03-26T14:14:11.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.712+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800043c0 con 0x7f9e881052b0 2024-03-26T14:14:11.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.712+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e881a1780 con 0x7f9e881052b0 2024-03-26T14:14:11.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.712+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e881a1c70 con 0x7f9e881052b0 2024-03-26T14:14:11.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.714+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8810aaf0 con 0x7f9e881052b0 2024-03-26T14:14:11.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.716+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9e80004560 con 0x7f9e881052b0 2024-03-26T14:14:11.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.717+0000 7f9e84c00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e6c0a2800 0x7f9e6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:11.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.717+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9e800f15e0 con 0x7f9e881052b0 2024-03-26T14:14:11.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.718+0000 7f9e87e00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e6c0a2800 0x7f9e6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:11.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.718+0000 7f9e87e00640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e6c0a2800 0x7f9e6c0a4cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9e88104750 tx=0x7f9e78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:11.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.722+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9e800be330 con 0x7f9e881052b0 2024-03-26T14:14:11.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.966+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e880622a0 con 0x7f9e881052b0 2024-03-26T14:14:11.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.967+0000 7f9e84c00640 1 -- 172.21.15.137:0/3137852324 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f9e800bdc30 con 0x7f9e881052b0 2024-03-26T14:14:11.968 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:14:11.968 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:14:11.968 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:14:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e6c0a2800 msgr2=0x7f9e6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e6c0a2800 0x7f9e6c0a4cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9e88104750 tx=0x7f9e78005f90 comp rx=0 tx=0).stop 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 msgr2=0x7f9e881a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 0x7f9e881a0e80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f9e80032520 tx=0x7f9e80035ce0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 shutdown_connections 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e6c0a2800 0x7f9e6c0a4cc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e881048f0 0x7f9e88199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e881052b0 0x7f9e881a0e80 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 --2- 172.21.15.137:0/3137852324 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e88103700 0x7f9e881998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 >> 172.21.15.137:0/3137852324 conn(0x7f9e880fec70 msgr2=0x7f9e88107cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 shutdown_connections 2024-03-26T14:14:11.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:11.971+0000 7f9e8d600640 1 -- 172.21.15.137:0/3137852324 wait complete. 2024-03-26T14:14:11.994 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:14:11.994 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 -- 172.21.15.137:0/3281969352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 msgr2=0x7f67000744e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 --2- 172.21.15.137:0/3281969352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67000744e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009900 tx=0x7f66ec02bcd0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 -- 172.21.15.137:0/3281969352 shutdown_connections 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 --2- 172.21.15.137:0/3281969352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700109450 0x7f670010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 --2- 172.21.15.137:0/3281969352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67000744e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 --2- 172.21.15.137:0/3281969352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700075660 0x7f6700073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 -- 172.21.15.137:0/3281969352 >> 172.21.15.137:0/3281969352 conn(0x7f67000fcb10 msgr2=0x7f67000fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.207+0000 7f6704e00640 1 -- 172.21.15.137:0/3281969352 shutdown_connections 2024-03-26T14:14:12.208 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.208+0000 7f6704e00640 1 -- 172.21.15.137:0/3281969352 wait complete. 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 Processor -- start 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 -- start start 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67001999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700075660 0x7f6700199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700109450 0x7f67001a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670010f300 con 0x7f6700074080 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670010f180 con 0x7f6700109450 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f6704e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670010f480 con 0x7f6700075660 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f66ff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67001999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f66ff400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67001999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45278/0 (socket says 172.21.15.137:45278) 2024-03-26T14:14:12.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.209+0000 7f66ff400640 1 -- 172.21.15.137:0/1256107333 learned_addr learned my addr 172.21.15.137:0/1256107333 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ffe00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700109450 0x7f67001a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ff400640 1 -- 172.21.15.137:0/1256107333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700075660 msgr2=0x7f6700199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66fea00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700075660 0x7f6700199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ff400640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700075660 0x7f6700199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ff400640 1 -- 172.21.15.137:0/1256107333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700109450 msgr2=0x7f67001a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ff400640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700109450 0x7f67001a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ff400640 1 -- 172.21.15.137:0/1256107333 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67001a15f0 con 0x7f6700074080 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ffe00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700109450 0x7f67001a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66fea00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700075660 0x7f6700199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.210+0000 7f66ff400640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67001999d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f66f000eab0 tx=0x7f66f000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.211+0000 7f66f7e00640 1 -- 172.21.15.137:0/1256107333 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f000ce40 con 0x7f6700074080 2024-03-26T14:14:12.212 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.211+0000 7f66f7e00640 1 -- 172.21.15.137:0/1256107333 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66f0011920 con 0x7f6700074080 2024-03-26T14:14:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.211+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67001a18e0 con 0x7f6700074080 2024-03-26T14:14:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.211+0000 7f66f7e00640 1 -- 172.21.15.137:0/1256107333 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0010a00 con 0x7f6700074080 2024-03-26T14:14:12.213 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.211+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67001a1e30 con 0x7f6700074080 2024-03-26T14:14:12.215 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.215+0000 7f67091d3740 1 Processor -- start 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.215+0000 7f67091d3740 1 -- start start 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f67091d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f67091d3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f66f7e00640 1 -- 172.21.15.137:0/1256107333 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f66f0011ad0 con 0x7f6700074080 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f67091d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f67091d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33f4152c0 con 0x55b33f4b09f0 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f67091d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33f3bdc90 con 0x55b33f4b2580 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f67091d3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33f411360 con 0x55b33f4b1bc0 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45290/0 (socket says 172.21.15.137:45290) 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f66ffe00640 1 -- 172.21.15.137:0/2682769541 learned_addr learned my addr 172.21.15.137:0/2682769541 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f66fea00640 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.216+0000 7f66ff400640 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66ffe00640 1 -- 172.21.15.137:0/2682769541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 msgr2=0x55b33f4b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66ffe00640 1 -- 172.21.15.137:0/2682769541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 msgr2=0x55b33f4b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66ffe00640 1 -- 172.21.15.137:0/2682769541 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b33f3e2080 con 0x55b33f4b09f0 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4b0e10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009bd0 tx=0x7f66f800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d74af9dbee41 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66f4c00640 1 -- 172.21.15.137:0/2682769541 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8016070 con 0x55b33f4b09f0 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66f4c00640 1 -- 172.21.15.137:0/2682769541 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66f8011920 con 0x55b33f4b09f0 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.217+0000 7f66f4c00640 1 -- 172.21.15.137:0/2682769541 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8010790 con 0x55b33f4b09f0 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f66f7e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66e40a2800 0x7f66e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f66f7e00640 1 -- 172.21.15.137:0/1256107333 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f66f00cf560 con 0x7f6700074080 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f66ff400640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66e40a2800 0x7f66e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 -- 172.21.15.137:0/2682769541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 msgr2=0x55b33f4b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4b0e10 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009bd0 tx=0x7f66f800e6c0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 -- 172.21.15.137:0/2682769541 shutdown_connections 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 --2- 172.21.15.137:0/2682769541 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4b0e10 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.218+0000 7f67091d3740 1 -- 172.21.15.137:0/2682769541 >> 172.21.15.137:0/2682769541 conn(0x55b33f4abde0 msgr2=0x55b33f4ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.219+0000 7f67091d3740 1 -- 172.21.15.137:0/2682769541 shutdown_connections 2024-03-26T14:14:12.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.219+0000 7f67091d3740 1 -- 172.21.15.137:0/2682769541 wait complete. 2024-03-26T14:14:12.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.219+0000 7f66ff400640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66e40a2800 0x7f66e40a4cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f66f00a3d60 tx=0x7f66f00094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.219+0000 7f67091d3740 1 -- start start 2024-03-26T14:14:12.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.219+0000 7f67091d3740 1 Processor -- start 2024-03-26T14:14:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b09f0 0x55b33f4babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 0x55b33f4bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 0x55b33f4c2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33f4b9590 con 0x55b33f4b1bc0 2024-03-26T14:14:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33f3d10d0 con 0x55b33f4b09f0 2024-03-26T14:14:12.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b33f4b9710 con 0x55b33f4b2580 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f67091d3740 10 client.0 _ll_register_callbacks cb 0x55b33f3d5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f66fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 0x55b33f4bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f66fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 0x55b33f4bb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45294/0 (socket says 172.21.15.137:45294) 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f66fea00640 1 -- 172.21.15.137:0/2390340754 learned_addr learned my addr 172.21.15.137:0/2390340754 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.220+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b09f0 0x55b33f4babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66fea00640 1 -- 172.21.15.137:0/2390340754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 msgr2=0x55b33f4c2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66fea00640 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 0x55b33f4c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66fea00640 1 -- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b09f0 msgr2=0x55b33f4babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66fea00640 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b09f0 0x55b33f4babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66fea00640 1 -- 172.21.15.137:0/2390340754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b33f1293f0 con 0x55b33f4b1bc0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66fea00640 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 0x55b33f4bb110 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f670019aee0 tx=0x7f66ec02cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9280132fe2d97db1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66e9600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66e9600640 1 -- 172.21.15.137:0/2390340754 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec005d50 con 0x55b33f4b1bc0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f66e9600640 1 -- 172.21.15.137:0/2390340754 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f66ec005030 con 0x55b33f4b1bc0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.221+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66f8011920 con 0x55b33f4b1bc0 2024-03-26T14:14:12.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.222+0000 7f66e9600640 1 -- 172.21.15.137:0/2390340754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec04e050 con 0x55b33f4b1bc0 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.222+0000 7f67091d3740 10 client.18114 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.222+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b33f1f37d0 con 0x55b33f4b1bc0 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.222+0000 7f66e9600640 1 -- 172.21.15.137:0/2390340754 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66f8011920 con 0x55b33f4b1bc0 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.222+0000 7f66e9600640 1 -- 172.21.15.137:0/2390340754 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f66ec0035f0 con 0x55b33f4b1bc0 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f67091d3740 10 client.18114 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f67091d3740 10 client.18114 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f67091d3740 20 client.18114 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f67091d3740 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f4c6680 0x55b33f4c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f67091d3740 4 client.18114 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:12.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b33f422800 con 0x55b33f4c6680 2024-03-26T14:14:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.223+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f4c6680 0x55b33f4c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.224+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f4c6680 0x55b33f4c8b00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97cd60c379e46a44 server_cookie=fb9df3080f092c8b in_seq=0 out_seq=0 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.224+0000 7f66e9600640 10 client.18114 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.225+0000 7f66e9600640 1 -- 172.21.15.137:0/2390340754 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b33f422800 con 0x55b33f4c6680 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.225+0000 7f66e9600640 10 client.18114 handle_command_reply: tid=0 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.225+0000 7f67091d3740 1 client.18114 shutdown 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 msgr2=0x55b33f4bb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 0x55b33f4bb110 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f670019aee0 tx=0x7f66ec02cbb0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 shutdown_connections 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f4c6680 0x55b33f4c8b00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 0x55b33f4c2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b1bc0 0x55b33f4bb110 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f66e9600640 0 client.18114 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 --2- 172.21.15.137:0/2390340754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b09f0 0x55b33f4babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 >> 172.21.15.137:0/2390340754 conn(0x55b33f4abde0 msgr2=0x55b33f4ab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 shutdown_connections 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 1 -- 172.21.15.137:0/2390340754 wait complete. 2024-03-26T14:14:12.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.226+0000 7f67091d3740 20 client.18114 trim_cache size 0 max 16384 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 Processor -- start 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 -- start start 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001ae7d0 con 0x55b33f4b09f0 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001ac2d0 con 0x55b33f4b2580 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.240+0000 7f67091d3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001aa350 con 0x55b33f4b1bc0 2024-03-26T14:14:12.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45304/0 (socket says 172.21.15.137:45304) 2024-03-26T14:14:12.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 -- 172.21.15.137:0/580744264 learned_addr learned my addr 172.21.15.137:0/580744264 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66fea00640 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 -- 172.21.15.137:0/580744264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 msgr2=0x55b33f4c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 -- 172.21.15.137:0/580744264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 msgr2=0x55b33f4cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4cd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.241+0000 7f66ffe00640 1 -- 172.21.15.137:0/580744264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b33f4b05f0 con 0x55b33f4b09f0 2024-03-26T14:14:12.243 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.242+0000 7f66ffe00640 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4aafe0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009bd0 tx=0x7f66f8009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c41fb810ee04d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.242+0000 7f66ebe00640 1 -- 172.21.15.137:0/580744264 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f802c070 con 0x55b33f4b09f0 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.242+0000 7f66ebe00640 1 -- 172.21.15.137:0/580744264 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66f8005e70 con 0x55b33f4b09f0 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.242+0000 7f66ebe00640 1 -- 172.21.15.137:0/580744264 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8016070 con 0x55b33f4b09f0 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 -- 172.21.15.137:0/580744264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 msgr2=0x55b33f4aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4aafe0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009bd0 tx=0x7f66f8009af0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 -- 172.21.15.137:0/580744264 shutdown_connections 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b2580 0x55b33f4cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 --2- 172.21.15.137:0/580744264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4aafe0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 -- 172.21.15.137:0/580744264 >> 172.21.15.137:0/580744264 conn(0x55b33f4abde0 msgr2=0x55b33f4aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 -- 172.21.15.137:0/580744264 shutdown_connections 2024-03-26T14:14:12.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 -- 172.21.15.137:0/580744264 wait complete. 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 -- start start 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.243+0000 7f67091d3740 1 Processor -- start 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 0x55b33f4c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001aa950 con 0x55b33f4b09f0 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001abc70 con 0x55b33f4b1bc0 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001aafb0 con 0x55b33f4b2580 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f67091d3740 10 client.0 _ll_register_callbacks cb 0x7f66ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f66ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4c1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45318/0 (socket says 172.21.15.137:45318) 2024-03-26T14:14:12.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f66ffe00640 1 -- 172.21.15.137:0/2435596350 learned_addr learned my addr 172.21.15.137:0/2435596350 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.244+0000 7f66fea00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66ffe00640 1 -- 172.21.15.137:0/2435596350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 msgr2=0x55b33f4c08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 0x55b33f4c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66ffe00640 1 -- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b1bc0 msgr2=0x55b33f4c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66ffe00640 1 -- 172.21.15.137:0/2435596350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b33f4c0de0 con 0x55b33f4b09f0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66fea00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66ffe00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4c1c90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c00 tx=0x7f66f8009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4d31c23ac756c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66e9600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66e9600640 1 -- 172.21.15.137:0/2435596350 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8016070 con 0x55b33f4b09f0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f66e9600640 1 -- 172.21.15.137:0/2435596350 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f66f8004bc0 con 0x55b33f4b09f0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.245+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66f8005e70 con 0x55b33f4b09f0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f66e9600640 1 -- 172.21.15.137:0/2435596350 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f802c040 con 0x55b33f4b09f0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 10 client.18120 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b33f522c60 con 0x55b33f4b09f0 2024-03-26T14:14:12.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f66e9600640 1 -- 172.21.15.137:0/2435596350 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66f8005e70 con 0x55b33f4b09f0 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f66e9600640 1 -- 172.21.15.137:0/2435596350 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f66f800f040 con 0x55b33f4b09f0 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 10 client.18120 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 10 client.18120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 20 client.18120 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f523300 0x55b33f525720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 4 client.18120 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:14:12.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.246+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b33f41cb50 con 0x55b33f523300 2024-03-26T14:14:12.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.247+0000 7f66fea00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f523300 0x55b33f525720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.247+0000 7f66fea00640 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f523300 0x55b33f525720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33f9a7fd47220064 server_cookie=74f61e51c0aa79ec in_seq=0 out_seq=0 2024-03-26T14:14:12.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.247+0000 7f66e9600640 10 client.18120 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:12.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f66e9600640 1 -- 172.21.15.137:0/2435596350 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b33f41cb50 con 0x55b33f523300 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f66e9600640 10 client.18120 handle_command_reply: tid=0 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 client.18120 shutdown 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 msgr2=0x55b33f4c1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4c1c90 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000c00 tx=0x7f66f8009cc0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 shutdown_connections 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55b33f523300 0x55b33f525720 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b33f4b1bc0 0x55b33f4c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b33f4b2580 0x55b33f4c08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 --2- 172.21.15.137:0/2435596350 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b33f4b09f0 0x55b33f4c1c90 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 >> 172.21.15.137:0/2435596350 conn(0x55b33f4abde0 msgr2=0x55b33f4b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f66e9600640 0 client.18120 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 shutdown_connections 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 1 -- 172.21.15.137:0/2435596350 wait complete. 2024-03-26T14:14:12.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.248+0000 7f67091d3740 20 client.18120 trim_cache size 0 max 16384 2024-03-26T14:14:12.252 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:14:12.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66e40a2800 msgr2=0x7f66e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66e40a2800 0x7f66e40a4cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f66f00a3d60 tx=0x7f66f00094a0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 msgr2=0x7f67001999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67001999d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f66f000eab0 tx=0x7f66f000ef70 comp rx=0 tx=0).stop 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 shutdown_connections 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f66e40a2800 0x7f66e40a4cc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6700109450 0x7f67001a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6700075660 0x7f6700199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 --2- 172.21.15.137:0/1256107333 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6700074080 0x7f67001999d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 >> 172.21.15.137:0/1256107333 conn(0x7f67000fcb10 msgr2=0x7f67001053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 shutdown_connections 2024-03-26T14:14:12.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.249+0000 7f6704e00640 1 -- 172.21.15.137:0/1256107333 wait complete. 2024-03-26T14:14:12.268 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:14:12.268 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:14:12.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 -- 172.21.15.137:0/3920230696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103150 msgr2=0x7fdb041099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3920230696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103150 0x7fdb041099e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009ca0 tx=0x7fdafc02c100 comp rx=0 tx=0).stop 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 -- 172.21.15.137:0/3920230696 shutdown_connections 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3920230696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 0x7fdb04102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3920230696 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb04103150 0x7fdb041099e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3920230696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb041015a0 0x7fdb041019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 -- 172.21.15.137:0/3920230696 >> 172.21.15.137:0/3920230696 conn(0x7fdb040fcb10 msgr2=0x7fdb040fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 -- 172.21.15.137:0/3920230696 shutdown_connections 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.486+0000 7fdb0a800640 1 -- 172.21.15.137:0/3920230696 wait complete. 2024-03-26T14:14:12.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.487+0000 7fdb0a800640 1 Processor -- start 2024-03-26T14:14:12.488 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 -- start start 2024-03-26T14:14:12.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 0x7fdb041998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 0x7fdb04199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb04103150 0x7fdb041a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0410d0b0 con 0x7fdb041015a0 2024-03-26T14:14:12.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0410cf30 con 0x7fdb04102790 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb0a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0410d230 con 0x7fdb04103150 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb09400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 0x7fdb041998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb09400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 0x7fdb041998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45332/0 (socket says 172.21.15.137:45332) 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.488+0000 7fdb09400640 1 -- 172.21.15.137:0/3952877664 learned_addr learned my addr 172.21.15.137:0/3952877664 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb08a00640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 0x7fdb04199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09e00640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb04103150 0x7fdb041a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09400640 1 -- 172.21.15.137:0/3952877664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb04103150 msgr2=0x7fdb041a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09400640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb04103150 0x7fdb041a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09400640 1 -- 172.21.15.137:0/3952877664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 msgr2=0x7fdb04199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09400640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 0x7fdb04199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09400640 1 -- 172.21.15.137:0/3952877664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb041a14c0 con 0x7fdb041015a0 2024-03-26T14:14:12.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb08a00640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 0x7fdb04199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.489+0000 7fdb09400640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 0x7fdb041998a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c970 tx=0x7fdaf800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.490+0000 7fdb02000640 1 -- 172.21.15.137:0/3952877664 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8007c10 con 0x7fdb041015a0 2024-03-26T14:14:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.490+0000 7fdb02000640 1 -- 172.21.15.137:0/3952877664 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdaf8007db0 con 0x7fdb041015a0 2024-03-26T14:14:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.490+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb041a17b0 con 0x7fdb041015a0 2024-03-26T14:14:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.490+0000 7fdb02000640 1 -- 172.21.15.137:0/3952877664 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf800f870 con 0x7fdb041015a0 2024-03-26T14:14:12.491 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.490+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb041a1d00 con 0x7fdb041015a0 2024-03-26T14:14:12.493 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.493+0000 7fdb02000640 1 -- 172.21.15.137:0/3952877664 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdaf800fa10 con 0x7fdb041015a0 2024-03-26T14:14:12.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.494+0000 7fdb0ea6a740 1 Processor -- start 2024-03-26T14:14:12.494 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.494+0000 7fdb0ea6a740 1 -- start start 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.494+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e0139f0 0x56045e013e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.494+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e015040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e01be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045df782c0 con 0x56045e015580 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045df20c90 con 0x56045e014bc0 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045df74360 con 0x56045e0139f0 2024-03-26T14:14:12.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb08a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e01be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb08a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e01be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45344/0 (socket says 172.21.15.137:45344) 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb08a00640 1 -- 172.21.15.137:0/1984866121 learned_addr learned my addr 172.21.15.137:0/1984866121 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb09400640 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e015040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb02000640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaec0a2800 0x7fdaec0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.495+0000 7fdb09e00640 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e0139f0 0x56045e013e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb09e00640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaec0a2800 0x7fdaec0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb02000640 1 -- 172.21.15.137:0/3952877664 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdaf80cf2b0 con 0x7fdb041015a0 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb08a00640 1 -- 172.21.15.137:0/1984866121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e0139f0 msgr2=0x56045e013e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb08a00640 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e0139f0 0x56045e013e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb08a00640 1 -- 172.21.15.137:0/1984866121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 msgr2=0x56045e015040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.497 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb08a00640 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e015040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb08a00640 1 -- 172.21.15.137:0/1984866121 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56045df45080 con 0x56045e015580 2024-03-26T14:14:12.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb08a00640 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e01be10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009cf0 tx=0x7fdaf00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554c03678db85082 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdaf6600640 1 -- 172.21.15.137:0/1984866121 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0012450 con 0x56045e015580 2024-03-26T14:14:12.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdaf6600640 1 -- 172.21.15.137:0/1984866121 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdaf0012aa0 con 0x56045e015580 2024-03-26T14:14:12.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdb09e00640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaec0a2800 0x7fdaec0a4cc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006010 tx=0x7fdafc032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.498 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.496+0000 7fdaf6600640 1 -- 172.21.15.137:0/1984866121 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000fb80 con 0x56045e015580 2024-03-26T14:14:12.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.497+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1984866121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 msgr2=0x56045e01be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.497+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e01be10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009cf0 tx=0x7fdaf00117b0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.497+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1984866121 shutdown_connections 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.497+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e015040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e01be10 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1984866121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e0139f0 0x56045e013e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1984866121 >> 172.21.15.137:0/1984866121 conn(0x56045e00ede0 msgr2=0x56045e00f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1984866121 shutdown_connections 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1984866121 wait complete. 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 -- start start 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.498+0000 7fdb0ea6a740 1 Processor -- start 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e0139f0 0x56045e01dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e014bc0 0x56045e01e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e0251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045e01c590 con 0x56045e015580 2024-03-26T14:14:12.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045df340d0 con 0x56045e0139f0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56045e01c710 con 0x56045e014bc0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb0ea6a740 10 client.0 _ll_register_callbacks cb 0x56045df38e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb09e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e0251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb09e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e0251b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45354/0 (socket says 172.21.15.137:45354) 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb09e00640 1 -- 172.21.15.137:0/630793895 learned_addr learned my addr 172.21.15.137:0/630793895 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb09400640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e0139f0 0x56045e01dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.499+0000 7fdb08a00640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e014bc0 0x56045e01e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb09400640 1 -- 172.21.15.137:0/630793895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e014bc0 msgr2=0x56045e01e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb09400640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e014bc0 0x56045e01e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb09400640 1 -- 172.21.15.137:0/630793895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 msgr2=0x56045e0251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb09400640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e0251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb09400640 1 -- 172.21.15.137:0/630793895 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56045dc8c3f0 con 0x56045e0139f0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb09400640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e0139f0 0x56045e01dc30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c940 tx=0x7fdaf809b2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a19c70dbc8dddc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.502 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdaeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:14:12.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdaeaa00640 1 -- 172.21.15.137:0/630793895 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80cce60 con 0x56045e0139f0 2024-03-26T14:14:12.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaf0012aa0 con 0x56045e0139f0 2024-03-26T14:14:12.503 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.500+0000 7fdaeaa00640 1 -- 172.21.15.137:0/630793895 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdaf80860b0 con 0x56045e0139f0 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdaeaa00640 1 -- 172.21.15.137:0/630793895 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaf8084e60 con 0x56045e0139f0 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 10 client.37670 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56045dd567d0 con 0x56045e0139f0 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdaeaa00640 1 -- 172.21.15.137:0/630793895 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdaf0012aa0 con 0x56045e0139f0 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdaeaa00640 1 -- 172.21.15.137:0/630793895 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdaf80a3b50 con 0x56045e0139f0 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 10 client.37670 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 10 client.37670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 20 client.37670 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e029690 0x56045e02bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 4 client.37670 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:12.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.501+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56045df85800 con 0x56045e029690 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.502+0000 7fdb08a00640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e029690 0x56045e02bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.503+0000 7fdb08a00640 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e029690 0x56045e02bb30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2097aed78a517586 server_cookie=e766b128705aaeb4 in_seq=0 out_seq=0 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.503+0000 7fdaeaa00640 10 client.37670 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdaeaa00640 1 -- 172.21.15.137:0/630793895 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56045df85800 con 0x56045e029690 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdaeaa00640 10 client.37670 handle_command_reply: tid=0 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 client.37670 shutdown 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e0139f0 msgr2=0x56045e01dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e0139f0 0x56045e01dc30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c940 tx=0x7fdaf809b2f0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 shutdown_connections 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e029690 0x56045e02bb30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e015580 0x56045e0251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e014bc0 0x56045e01e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdaeaa00640 0 client.37670 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:12.506 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/630793895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e0139f0 0x56045e01dc30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 >> 172.21.15.137:0/630793895 conn(0x56045e00ede0 msgr2=0x56045e00e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 shutdown_connections 2024-03-26T14:14:12.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.504+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/630793895 wait complete. 2024-03-26T14:14:12.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.505+0000 7fdb0ea6a740 20 client.37670 trim_cache size 0 max 16384 2024-03-26T14:14:12.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.518+0000 7fdb0ea6a740 1 Processor -- start 2024-03-26T14:14:12.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.518+0000 7fdb0ea6a740 1 -- start start 2024-03-26T14:14:12.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.518+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e012750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e025170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e017df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041ae620 con 0x56045e0139f0 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041ac120 con 0x56045e014bc0 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041aa1a0 con 0x56045e015580 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e012750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e012750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45360/0 (socket says 172.21.15.137:45360) 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 -- 172.21.15.137:0/3758262759 learned_addr learned my addr 172.21.15.137:0/3758262759 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb08a00640 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e025170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09e00640 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e017df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 -- 172.21.15.137:0/3758262759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 msgr2=0x56045e017df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e017df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 -- 172.21.15.137:0/3758262759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 msgr2=0x56045e025170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e025170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb09400640 1 -- 172.21.15.137:0/3758262759 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56045e014640 con 0x56045e0139f0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.519+0000 7fdb08a00640 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e025170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.520+0000 7fdb09400640 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e012750 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8004790 tx=0x7fdaf809b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b512bce4ee36dfe server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.520+0000 7fdaf5c00640 1 -- 172.21.15.137:0/3758262759 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80cce60 con 0x56045e0139f0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.520+0000 7fdaf5c00640 1 -- 172.21.15.137:0/3758262759 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdaf809b8d0 con 0x56045e0139f0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.520+0000 7fdaf5c00640 1 -- 172.21.15.137:0/3758262759 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8086310 con 0x56045e0139f0 2024-03-26T14:14:12.522 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/3758262759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 msgr2=0x56045e012750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e012750 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8004790 tx=0x7fdaf809b360 comp rx=0 tx=0).stop 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/3758262759 shutdown_connections 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e025170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e017df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/3758262759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e012750 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/3758262759 >> 172.21.15.137:0/3758262759 conn(0x56045e00ede0 msgr2=0x56045e012040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/3758262759 shutdown_connections 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/3758262759 wait complete. 2024-03-26T14:14:12.523 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.521+0000 7fdb0ea6a740 1 -- start start 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 Processor -- start 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e0230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e023600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e0245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041aa7a0 con 0x56045e0139f0 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041abac0 con 0x56045e014bc0 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb041aae00 con 0x56045e015580 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb0ea6a740 10 client.0 _ll_register_callbacks cb 0x7fdaf80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb09400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e0230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb09400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e0230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45364/0 (socket says 172.21.15.137:45364) 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.522+0000 7fdb09400640 1 -- 172.21.15.137:0/1266926354 learned_addr learned my addr 172.21.15.137:0/1266926354 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb08a00640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e023600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09e00640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e0245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09400640 1 -- 172.21.15.137:0/1266926354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 msgr2=0x56045e0245c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09400640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e0245c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09400640 1 -- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 msgr2=0x56045e023600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09400640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e023600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09400640 1 -- 172.21.15.137:0/1266926354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56045e024bb0 con 0x56045e0139f0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09e00640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e0245c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdb09400640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e0230c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8097640 tx=0x7fdaf80a3b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4624da2da24da833 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdaeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.523+0000 7fdaeaa00640 1 -- 172.21.15.137:0/1266926354 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80cce60 con 0x56045e0139f0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdaeaa00640 1 -- 172.21.15.137:0/1266926354 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdaf809a990 con 0x56045e0139f0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaf809b8d0 con 0x56045e0139f0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdaeaa00640 1 -- 172.21.15.137:0/1266926354 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaf809b410 con 0x56045e0139f0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 10 client.18138 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56045e0857a0 con 0x56045e0139f0 2024-03-26T14:14:12.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdaeaa00640 1 -- 172.21.15.137:0/1266926354 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdaf809b8d0 con 0x56045e0139f0 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdaeaa00640 1 -- 172.21.15.137:0/1266926354 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fdaf80bb570 con 0x56045e0139f0 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 10 client.18138 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 10 client.18138 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 20 client.18138 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e085eb0 0x56045e0882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 4 client.18138 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.524+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56045df7fb50 con 0x56045e085eb0 2024-03-26T14:14:12.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.525+0000 7fdb08a00640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e085eb0 0x56045e0882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdb08a00640 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e085eb0 0x56045e0882d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f9ae3338509bb84 server_cookie=87e44a889a23bbe4 in_seq=0 out_seq=0 2024-03-26T14:14:12.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdaeaa00640 10 client.18138 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:12.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdaeaa00640 1 -- 172.21.15.137:0/1266926354 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56045df7fb50 con 0x56045e085eb0 2024-03-26T14:14:12.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdaeaa00640 10 client.18138 handle_command_reply: tid=0 2024-03-26T14:14:12.529 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdb0ea6a740 1 client.18138 shutdown 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 msgr2=0x56045e0230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e0230c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8097640 tx=0x7fdaf80a3b50 comp rx=0 tx=0).stop 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 shutdown_connections 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.526+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56045e085eb0 0x56045e0882d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56045e014bc0 0x56045e023600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56045e015580 0x56045e0245c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 1 --2- 172.21.15.137:0/1266926354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56045e0139f0 0x56045e0230c0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 >> 172.21.15.137:0/1266926354 conn(0x56045e00ede0 msgr2=0x56045e0128d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdaeaa00640 0 client.18138 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 shutdown_connections 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 1 -- 172.21.15.137:0/1266926354 wait complete. 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0ea6a740 20 client.18138 trim_cache size 0 max 16384 2024-03-26T14:14:12.530 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:14:12.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaec0a2800 msgr2=0x7fdaec0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaec0a2800 0x7fdaec0a4cc0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdafc006010 tx=0x7fdafc032340 comp rx=0 tx=0).stop 2024-03-26T14:14:12.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 msgr2=0x7fdb041998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.527+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 0x7fdb041998a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fdaf800c970 tx=0x7fdaf800ce30 comp rx=0 tx=0).stop 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 shutdown_connections 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaec0a2800 0x7fdaec0a4cc0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb04102790 0x7fdb04199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb04103150 0x7fdb041a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 --2- 172.21.15.137:0/3952877664 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb041015a0 0x7fdb041998a0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 >> 172.21.15.137:0/3952877664 conn(0x7fdb040fcb10 msgr2=0x7fdb040738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 shutdown_connections 2024-03-26T14:14:12.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.528+0000 7fdb0a800640 1 -- 172.21.15.137:0/3952877664 wait complete. 2024-03-26T14:14:12.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3137852324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:14:12.547 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:14:12.547 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:14:12.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3137852324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:14:12.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3137852324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 -- 172.21.15.137:0/3226117464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101a70 msgr2=0x7fde5c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 --2- 172.21.15.137:0/3226117464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101a70 0x7fde5c101e90 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009900 tx=0x7fde4c02bc40 comp rx=0 tx=0).stop 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 -- 172.21.15.137:0/3226117464 shutdown_connections 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 --2- 172.21.15.137:0/3226117464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 --2- 172.21.15.137:0/3226117464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde5c101080 0x7fde5c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 --2- 172.21.15.137:0/3226117464 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101a70 0x7fde5c101e90 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 -- 172.21.15.137:0/3226117464 >> 172.21.15.137:0/3226117464 conn(0x7fde5c0fb190 msgr2=0x7fde5c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 -- 172.21.15.137:0/3226117464 shutdown_connections 2024-03-26T14:14:12.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.788+0000 7fde62600640 1 -- 172.21.15.137:0/3226117464 wait complete. 2024-03-26T14:14:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.789+0000 7fde62600640 1 Processor -- start 2024-03-26T14:14:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.789+0000 7fde62600640 1 -- start start 2024-03-26T14:14:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.789+0000 7fde62600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde62600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101080 0x7fde5c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde62600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde5c101a70 0x7fde5c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde62600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c10c030 con 0x7fde5c101080 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde62600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c10beb0 con 0x7fde5c1006c0 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde62600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c10c1b0 con 0x7fde5c101a70 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101080 0x7fde5c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:44788/0 (socket says 172.21.15.137:44788) 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 -- 172.21.15.137:0/2820790352 learned_addr learned my addr 172.21.15.137:0/2820790352 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61c00640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde5c101a70 0x7fde5c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 -- 172.21.15.137:0/2820790352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde5c101a70 msgr2=0x7fde5c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde5c101a70 0x7fde5c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 -- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101080 msgr2=0x7fde5c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101080 0x7fde5c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde61200640 1 -- 172.21.15.137:0/2820790352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c19f360 con 0x7fde5c1006c0 2024-03-26T14:14:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.790+0000 7fde53e00640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101080 0x7fde5c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:14:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.791+0000 7fde61200640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c197740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde4c000c00 tx=0x7fde4c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.791+0000 7fde5a000640 1 -- 172.21.15.137:0/2820790352 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c043020 con 0x7fde5c1006c0 2024-03-26T14:14:12.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.791+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c19f5f0 con 0x7fde5c1006c0 2024-03-26T14:14:12.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.791+0000 7fde5a000640 1 -- 172.21.15.137:0/2820790352 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde4c0044c0 con 0x7fde5c1006c0 2024-03-26T14:14:12.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.791+0000 7fde5a000640 1 -- 172.21.15.137:0/2820790352 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c003c60 con 0x7fde5c1006c0 2024-03-26T14:14:12.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.791+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c19fb90 con 0x7fde5c1006c0 2024-03-26T14:14:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde5a000640 1 -- 172.21.15.137:0/2820790352 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fde4c046020 con 0x7fde5c1006c0 2024-03-26T14:14:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 Processor -- start 2024-03-26T14:14:12.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 -- start start 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a05a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a05b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 0x564f1a062220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19fba2c0 con 0x564f1a05b990 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f62c90 con 0x564f1a059e00 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.795+0000 7fde66854740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19fb6360 con 0x564f1a05afd0 2024-03-26T14:14:12.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 0x564f1a062220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde61c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a05a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 0x564f1a062220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45402/0 (socket says 172.21.15.137:45402) 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 -- 172.21.15.137:0/2003786479 learned_addr learned my addr 172.21.15.137:0/2003786479 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde61200640 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a05b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 -- 172.21.15.137:0/2003786479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 msgr2=0x564f1a05b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a05b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 -- 172.21.15.137:0/2003786479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 msgr2=0x564f1a05a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a05a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde53e00640 1 -- 172.21.15.137:0/2003786479 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f19f87080 con 0x564f1a05b990 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.796+0000 7fde61c00640 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a05a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde5a000640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fde400a2880 0x7fde400a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde53e00640 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 0x564f1a062220 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fde440114f0 tx=0x7fde440119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c9c5b08c7edbb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde5a000640 1 -- 172.21.15.137:0/2820790352 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fde4c0f1050 con 0x7fde5c1006c0 2024-03-26T14:14:12.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde61c00640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fde400a2880 0x7fde400a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde52000640 1 -- 172.21.15.137:0/2003786479 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4401e070 con 0x564f1a05b990 2024-03-26T14:14:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde52000640 1 -- 172.21.15.137:0/2003786479 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde4400fa70 con 0x564f1a05b990 2024-03-26T14:14:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.797+0000 7fde52000640 1 -- 172.21.15.137:0/2003786479 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44019b60 con 0x564f1a05b990 2024-03-26T14:14:12.803 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 -- 172.21.15.137:0/2003786479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 msgr2=0x564f1a062220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde61c00640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fde400a2880 0x7fde400a4d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x564f1a05aed0 tx=0x7fde54007680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 0x564f1a062220 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fde440114f0 tx=0x7fde440119b0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 -- 172.21.15.137:0/2003786479 shutdown_connections 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05b990 0x564f1a062220 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a05b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 --2- 172.21.15.137:0/2003786479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a05a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 -- 172.21.15.137:0/2003786479 >> 172.21.15.137:0/2003786479 conn(0x564f1a050de0 msgr2=0x564f1a0511e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 -- 172.21.15.137:0/2003786479 shutdown_connections 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.798+0000 7fde66854740 1 -- 172.21.15.137:0/2003786479 wait complete. 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 -- start start 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 Processor -- start 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a064080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 0x564f1a0645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a0629a0 con 0x564f1a05afd0 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f19f760d0 con 0x564f1a059e00 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1a062b20 con 0x564f1a05b990 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.799+0000 7fde66854740 10 client.0 _ll_register_callbacks cb 0x564f19f7ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 0x564f1a0645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a064080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde61200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a064080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:44812/0 (socket says 172.21.15.137:44812) 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde61200640 1 -- 172.21.15.137:0/2838858088 learned_addr learned my addr 172.21.15.137:0/2838858088 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 0x564f1a0645c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45404/0 (socket says 172.21.15.137:45404) 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde61c00640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 -- 172.21.15.137:0/2838858088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 msgr2=0x564f1a06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 -- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 msgr2=0x564f1a064080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a064080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde53e00640 1 -- 172.21.15.137:0/2838858088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f19cce3f0 con 0x564f1a05afd0 2024-03-26T14:14:12.805 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde61c00640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.800+0000 7fde61200640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a064080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde53e00640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 0x564f1a0645c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fde440097b0 tx=0x7fde440056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01e55a26e9a6f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde4aa00640 1 -- 172.21.15.137:0/2838858088 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4401e040 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde4400fa70 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde4aa00640 1 -- 172.21.15.137:0/2838858088 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fde44016050 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde4aa00640 1 -- 172.21.15.137:0/2838858088 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde44031050 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde66854740 10 client.18150 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.801+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564f19d987d0 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde4aa00640 1 -- 172.21.15.137:0/2838858088 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde4400fa70 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde4aa00640 1 -- 172.21.15.137:0/2838858088 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fde44021030 con 0x564f1a05afd0 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde66854740 10 client.18150 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde66854740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde66854740 20 client.18150 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:12.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde66854740 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a06f7b0 0x564f1a071be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde66854740 4 client.18150 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:12.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f19fc7800 con 0x564f1a06f7b0 2024-03-26T14:14:12.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.802+0000 7fde61200640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a06f7b0 0x564f1a071be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.803+0000 7fde61200640 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a06f7b0 0x564f1a071be0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cfe3a62195db7c7 server_cookie=5f394d51d922b7fa in_seq=0 out_seq=0 2024-03-26T14:14:12.807 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.803+0000 7fde4aa00640 10 client.18150 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.804+0000 7fde4aa00640 1 -- 172.21.15.137:0/2838858088 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564f19fc7800 con 0x564f1a06f7b0 2024-03-26T14:14:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.804+0000 7fde4aa00640 10 client.18150 handle_command_reply: tid=0 2024-03-26T14:14:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 client.18150 shutdown 2024-03-26T14:14:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 msgr2=0x564f1a0645c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 0x564f1a0645c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fde440097b0 tx=0x7fde440056d0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 shutdown_connections 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a06f7b0 0x564f1a071be0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a05afd0 0x564f1a0645c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 --2- 172.21.15.137:0/2838858088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a059e00 0x564f1a064080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 >> 172.21.15.137:0/2838858088 conn(0x564f1a050de0 msgr2=0x564f1a050160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde4aa00640 0 client.18150 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 shutdown_connections 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 1 -- 172.21.15.137:0/2838858088 wait complete. 2024-03-26T14:14:12.809 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.805+0000 7fde66854740 20 client.18150 trim_cache size 0 max 16384 2024-03-26T14:14:12.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 Processor -- start 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 -- start start 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a05e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05b990 0x564f1a05e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1aa260 con 0x564f1a059e00 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1a7d60 con 0x564f1a05b990 2024-03-26T14:14:12.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde66854740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1a5de0 con 0x564f1a05afd0 2024-03-26T14:14:12.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde61200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a05e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.819+0000 7fde61c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05b990 0x564f1a05e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a05e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45408/0 (socket says 172.21.15.137:45408) 2024-03-26T14:14:12.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 -- 172.21.15.137:0/487037914 learned_addr learned my addr 172.21.15.137:0/487037914 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde53e00640 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 -- 172.21.15.137:0/487037914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 msgr2=0x564f1a06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a06b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 -- 172.21.15.137:0/487037914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05b990 msgr2=0x564f1a05e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05b990 0x564f1a05e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 -- 172.21.15.137:0/487037914 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f1a051d50 con 0x564f1a059e00 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61c00640 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05b990 0x564f1a05e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde61200640 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a05e070 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0bd570 tx=0x7fde4c0a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef65bac541fe93a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde51600640 1 -- 172.21.15.137:0/487037914 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0bd120 con 0x564f1a059e00 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde51600640 1 -- 172.21.15.137:0/487037914 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fde4c0035f0 con 0x564f1a059e00 2024-03-26T14:14:12.822 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.820+0000 7fde51600640 1 -- 172.21.15.137:0/487037914 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0bcf40 con 0x564f1a059e00 2024-03-26T14:14:12.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 -- 172.21.15.137:0/487037914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 msgr2=0x564f1a05e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a05e070 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0bd570 tx=0x7fde4c0a8040 comp rx=0 tx=0).stop 2024-03-26T14:14:12.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 -- 172.21.15.137:0/487037914 shutdown_connections 2024-03-26T14:14:12.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05b990 0x564f1a05e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05afd0 0x564f1a06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 --2- 172.21.15.137:0/487037914 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a05e070 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 -- 172.21.15.137:0/487037914 >> 172.21.15.137:0/487037914 conn(0x564f1a050de0 msgr2=0x564f1a0548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 -- 172.21.15.137:0/487037914 shutdown_connections 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.821+0000 7fde66854740 1 -- 172.21.15.137:0/487037914 wait complete. 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 -- start start 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 Processor -- start 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a0673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05afd0 0x564f1a0678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1a63e0 con 0x564f1a059e00 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1a7700 con 0x564f1a05afd0 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1a6a40 con 0x564f1a05b990 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.822+0000 7fde66854740 10 client.0 _ll_register_callbacks cb 0x7fde4401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:12.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a0673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a0673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45410/0 (socket says 172.21.15.137:45410) 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 -- 172.21.15.137:0/372720617 learned_addr learned my addr 172.21.15.137:0/372720617 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde53e00640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05afd0 0x564f1a0678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 -- 172.21.15.137:0/372720617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 msgr2=0x564f1a06a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61c00640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 -- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05afd0 msgr2=0x564f1a0678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05afd0 0x564f1a0678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 -- 172.21.15.137:0/372720617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f1a06ae00 con 0x564f1a059e00 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde61200640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a0673a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0c1810 tx=0x7fde4c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b47201a8816773 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.823+0000 7fde4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde4aa00640 1 -- 172.21.15.137:0/372720617 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c005600 con 0x564f1a059e00 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde4aa00640 1 -- 172.21.15.137:0/372720617 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fde4c0ee850 con 0x564f1a059e00 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f19fc1b50 con 0x564f1a059e00 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde4aa00640 1 -- 172.21.15.137:0/372720617 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde4c0bd5e0 con 0x564f1a059e00 2024-03-26T14:14:12.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde66854740 10 client.18156 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564f1a0cb690 con 0x564f1a059e00 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde4aa00640 1 -- 172.21.15.137:0/372720617 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564f19fc1b50 con 0x564f1a059e00 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.824+0000 7fde4aa00640 1 -- 172.21.15.137:0/372720617 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fde4c0d6210 con 0x564f1a059e00 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde66854740 10 client.18156 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde66854740 10 client.18156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde66854740 20 client.18156 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde66854740 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a0cc020 0x564f1a0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde66854740 4 client.18156 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564f19f3f500 con 0x564f1a0cc020 2024-03-26T14:14:12.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.825+0000 7fde53e00640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a0cc020 0x564f1a0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde53e00640 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a0cc020 0x564f1a0ce440 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b6ef470d55dbd8 server_cookie=46869ab4441422f8 in_seq=0 out_seq=0 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde4aa00640 10 client.18156 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde4aa00640 1 -- 172.21.15.137:0/372720617 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564f19f3f500 con 0x564f1a0cc020 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde4aa00640 10 client.18156 handle_command_reply: tid=0 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 client.18156 shutdown 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 msgr2=0x564f1a0673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a0673a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0c1810 tx=0x7fde4c009260 comp rx=0 tx=0).stop 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 shutdown_connections 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564f1a0cc020 0x564f1a0ce440 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1a05afd0 0x564f1a0678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f1a05b990 0x564f1a06a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 --2- 172.21.15.137:0/372720617 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564f1a059e00 0x564f1a0673a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde4aa00640 0 client.18156 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 >> 172.21.15.137:0/372720617 conn(0x564f1a050de0 msgr2=0x564f1a04f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 shutdown_connections 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.826+0000 7fde66854740 1 -- 172.21.15.137:0/372720617 wait complete. 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde66854740 20 client.18156 trim_cache size 0 max 16384 2024-03-26T14:14:12.830 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fde400a2880 msgr2=0x7fde400a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fde400a2880 0x7fde400a4d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x564f1a05aed0 tx=0x7fde54007680 comp rx=0 tx=0).stop 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 msgr2=0x7fde5c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c197740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fde4c000c00 tx=0x7fde4c0040c0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 shutdown_connections 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fde400a2880 0x7fde400a4d40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde5c101a70 0x7fde5c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fde5c101080 0x7fde5c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 --2- 172.21.15.137:0/2820790352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5c1006c0 0x7fde5c197740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.827+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 >> 172.21.15.137:0/2820790352 conn(0x7fde5c0fb190 msgr2=0x7fde5c0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.828+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 shutdown_connections 2024-03-26T14:14:12.832 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:12.828+0000 7fde62600640 1 -- 172.21.15.137:0/2820790352 wait complete. 2024-03-26T14:14:13.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:13 smithi179 ceph-mon[35472]: pgmap v813: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:14:13.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:13 smithi159 ceph-mon[35501]: pgmap v813: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:14:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:13 smithi137 ceph-mon[31539]: pgmap v813: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:14:13.850 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:14:13.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:14:13.851 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:14:14.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 -- 172.21.15.137:0/417125117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f38104990 msgr2=0x7f2f38104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 --2- 172.21.15.137:0/417125117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f38104990 0x7f2f38104e10 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009860 tx=0x7f2f2402bb70 comp rx=0 tx=0).stop 2024-03-26T14:14:14.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 -- 172.21.15.137:0/417125117 shutdown_connections 2024-03-26T14:14:14.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 --2- 172.21.15.137:0/417125117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f38105350 0x7f2f3810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 --2- 172.21.15.137:0/417125117 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f38104990 0x7f2f38104e10 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 --2- 172.21.15.137:0/417125117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f381037a0 0x7f2f38103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 -- 172.21.15.137:0/417125117 >> 172.21.15.137:0/417125117 conn(0x7f2f380fed10 msgr2=0x7f2f38101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 -- 172.21.15.137:0/417125117 shutdown_connections 2024-03-26T14:14:14.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.100+0000 7f2f3e400640 1 -- 172.21.15.137:0/417125117 wait complete. 2024-03-26T14:14:14.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.101+0000 7f2f3e400640 1 Processor -- start 2024-03-26T14:14:14.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 -- start start 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 0x7f2f381999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f38104990 0x7f2f38199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38105350 0x7f2f381a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3810f2d0 con 0x7f2f381037a0 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3810f150 con 0x7f2f38105350 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3e400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3810f450 con 0x7f2f38104990 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 0x7f2f381999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 0x7f2f381999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45416/0 (socket says 172.21.15.137:45416) 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3d000640 1 -- 172.21.15.137:0/1440576296 learned_addr learned my addr 172.21.15.137:0/1440576296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.102+0000 7f2f3da00640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38105350 0x7f2f381a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f38104990 0x7f2f38199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3d000640 1 -- 172.21.15.137:0/1440576296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f38104990 msgr2=0x7f2f38199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3d000640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f38104990 0x7f2f38199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3d000640 1 -- 172.21.15.137:0/1440576296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38105350 msgr2=0x7f2f381a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3d000640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38105350 0x7f2f381a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3d000640 1 -- 172.21.15.137:0/1440576296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f381a15f0 con 0x7f2f381037a0 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3d000640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 0x7f2f381999d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e9f0 tx=0x7f2f2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f36000640 1 -- 172.21.15.137:0/1440576296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800ccc0 con 0x7f2f381037a0 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f36000640 1 -- 172.21.15.137:0/1440576296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f280035f0 con 0x7f2f381037a0 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f381a18e0 con 0x7f2f381037a0 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f36000640 1 -- 172.21.15.137:0/1440576296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800ce60 con 0x7f2f381037a0 2024-03-26T14:14:14.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.103+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f381a1e30 con 0x7f2f381037a0 2024-03-26T14:14:14.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.107+0000 7f2f36000640 1 -- 172.21.15.137:0/1440576296 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2f280037a0 con 0x7f2f381037a0 2024-03-26T14:14:14.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.107+0000 7f2f42322740 1 Processor -- start 2024-03-26T14:14:14.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.107+0000 7f2f42322740 1 -- start start 2024-03-26T14:14:14.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f42322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x5602486576d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f42322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x56024865c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f42322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x56024865f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f42322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602485bceb0 con 0x56024865ce10 2024-03-26T14:14:14.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f42322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560248565a60 con 0x5602486572b0 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f42322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602485b8f50 con 0x560248657ca0 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f3da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x5602486576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f3da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x5602486576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:44864/0 (socket says 172.21.15.137:44864) 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f3da00640 1 -- 172.21.15.137:0/2013860347 learned_addr learned my addr 172.21.15.137:0/2013860347 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x56024865f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.108+0000 7f2f3d000640 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x56024865c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3da00640 1 -- 172.21.15.137:0/2013860347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 msgr2=0x56024865c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x56024865c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3da00640 1 -- 172.21.15.137:0/2013860347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 msgr2=0x56024865f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x56024865f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3da00640 1 -- 172.21.15.137:0/2013860347 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602483997e0 con 0x5602486572b0 2024-03-26T14:14:14.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f36000640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f1c0a2800 0x7f2f1c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x56024865f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f36000640 1 -- 172.21.15.137:0/1440576296 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2f280cf1d0 con 0x7f2f381037a0 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3d000640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f1c0a2800 0x7f2f1c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x5602486576d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009ee0 tx=0x7f2f30010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18f12c471f5fbe68 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.109+0000 7f2f2e000640 1 -- 172.21.15.137:0/2013860347 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30016070 con 0x5602486572b0 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f2e000640 1 -- 172.21.15.137:0/2013860347 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f30004d20 con 0x5602486572b0 2024-03-26T14:14:14.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f2e000640 1 -- 172.21.15.137:0/2013860347 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30012740 con 0x5602486572b0 2024-03-26T14:14:14.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 -- 172.21.15.137:0/2013860347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 msgr2=0x5602486576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f3d000640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f1c0a2800 0x7f2f1c0a4cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f280a39d0 tx=0x7f2f280d2740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x5602486576d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009ee0 tx=0x7f2f30010740 comp rx=0 tx=0).stop 2024-03-26T14:14:14.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 -- 172.21.15.137:0/2013860347 shutdown_connections 2024-03-26T14:14:14.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x56024865f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x56024865c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 --2- 172.21.15.137:0/2013860347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x5602486576d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.110+0000 7f2f42322740 1 -- 172.21.15.137:0/2013860347 >> 172.21.15.137:0/2013860347 conn(0x560248652820 msgr2=0x560248652c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.111+0000 7f2f42322740 1 -- 172.21.15.137:0/2013860347 shutdown_connections 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.111+0000 7f2f42322740 1 -- 172.21.15.137:0/2013860347 wait complete. 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.111+0000 7f2f42322740 1 -- start start 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.111+0000 7f2f42322740 1 Processor -- start 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602486572b0 0x5602486614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 0x560248661a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x560248668a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024865fdc0 con 0x56024865ce10 2024-03-26T14:14:14.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560248578eb0 con 0x560248657ca0 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56024865ff40 con 0x5602486572b0 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f42322740 10 client.0 _ll_register_callbacks cb 0x56024856d150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f3da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x560248668a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 0x560248661a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 0x560248661a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:44880/0 (socket says 172.21.15.137:44880) 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f3da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x560248668a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45444/0 (socket says 172.21.15.137:45444) 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f2fe00640 1 -- 172.21.15.137:0/2327701451 learned_addr learned my addr 172.21.15.137:0/2327701451 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.112+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602486572b0 0x5602486614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f2fe00640 1 -- 172.21.15.137:0/2327701451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602486572b0 msgr2=0x5602486614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602486572b0 0x5602486614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f2fe00640 1 -- 172.21.15.137:0/2327701451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 msgr2=0x560248668a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x560248668a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f2fe00640 1 -- 172.21.15.137:0/2327701451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602483f9f70 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x560248668a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 0x560248661a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f2402c0b0 tx=0x7f2f24035cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcf5c89464b3ce09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f22a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.113+0000 7f2f22a00640 1 -- 172.21.15.137:0/2327701451 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24039040 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f22a00640 1 -- 172.21.15.137:0/2327701451 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2f24004450 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f30004d20 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f22a00640 1 -- 172.21.15.137:0/2327701451 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2404d050 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f42322740 10 client.37706 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560248291170 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f22a00640 1 -- 172.21.15.137:0/2327701451 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f30004d20 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.114+0000 7f2f22a00640 1 -- 172.21.15.137:0/2327701451 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2f24004b50 con 0x560248657ca0 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f42322740 10 client.37706 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:14.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f42322740 10 client.37706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f42322740 20 client.37706 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f42322740 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56024866d390 0x56024866f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f42322740 4 client.37706 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602485ca3f0 con 0x56024866d390 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.115+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56024866d390 0x56024866f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.116+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56024866d390 0x56024866f780 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972f22975255ecea server_cookie=856f93f82afb021a in_seq=0 out_seq=0 2024-03-26T14:14:14.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.116+0000 7f2f22a00640 10 client.37706 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f22a00640 1 -- 172.21.15.137:0/2327701451 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5602485ca3f0 con 0x56024866d390 2024-03-26T14:14:14.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f22a00640 10 client.37706 handle_command_reply: tid=0 2024-03-26T14:14:14.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 client.37706 shutdown 2024-03-26T14:14:14.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 msgr2=0x560248661a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 0x560248661a30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f2402c0b0 tx=0x7f2f24035cc0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 shutdown_connections 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56024866d390 0x56024866f780 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560248657ca0 0x560248661a30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56024865ce10 0x560248668a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f22a00640 0 client.37706 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 --2- 172.21.15.137:0/2327701451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5602486572b0 0x5602486614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 >> 172.21.15.137:0/2327701451 conn(0x560248652820 msgr2=0x560248651a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.117+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 shutdown_connections 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.118+0000 7f2f42322740 1 -- 172.21.15.137:0/2327701451 wait complete. 2024-03-26T14:14:14.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.118+0000 7f2f42322740 20 client.37706 trim_cache size 0 max 16384 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.137+0000 7f2f42322740 1 Processor -- start 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.137+0000 7f2f42322740 1 -- start start 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.137+0000 7f2f42322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 0x560248658740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.137+0000 7f2f42322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x560248668a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.137+0000 7f2f42322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56024865ce10 0x560248658c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f42322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381ae830 con 0x5602486572b0 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f42322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381ac330 con 0x56024865ce10 2024-03-26T14:14:14.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f42322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381aa3b0 con 0x560248657ca0 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 0x560248658740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 0x560248658740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45458/0 (socket says 172.21.15.137:45458) 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 -- 172.21.15.137:0/2793120583 learned_addr learned my addr 172.21.15.137:0/2793120583 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56024865ce10 0x560248658c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 -- 172.21.15.137:0/2793120583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 msgr2=0x560248668a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x560248668a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 -- 172.21.15.137:0/2793120583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56024865ce10 msgr2=0x560248658c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56024865ce10 0x560248658c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.138+0000 7f2f3da00640 1 -- 172.21.15.137:0/2793120583 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56024864cc40 con 0x5602486572b0 2024-03-26T14:14:14.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.139+0000 7f2f3da00640 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 0x560248658740 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009fd0 tx=0x7f2f300097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65076f45021828d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.139+0000 7f2f2d600640 1 -- 172.21.15.137:0/2793120583 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3002e020 con 0x5602486572b0 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.139+0000 7f2f2d600640 1 -- 172.21.15.137:0/2793120583 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f30009580 con 0x5602486572b0 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.139+0000 7f2f2d600640 1 -- 172.21.15.137:0/2793120583 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30016040 con 0x5602486572b0 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 -- 172.21.15.137:0/2793120583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 msgr2=0x560248658740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 0x560248658740 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009fd0 tx=0x7f2f300097e0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 -- 172.21.15.137:0/2793120583 shutdown_connections 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56024865ce10 0x560248658c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560248657ca0 0x560248668a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 --2- 172.21.15.137:0/2793120583 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5602486572b0 0x560248658740 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 -- 172.21.15.137:0/2793120583 >> 172.21.15.137:0/2793120583 conn(0x560248652820 msgr2=0x560248656310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 -- 172.21.15.137:0/2793120583 shutdown_connections 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 -- 172.21.15.137:0/2793120583 wait complete. 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.140+0000 7f2f42322740 1 -- start start 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 Processor -- start 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x560248668780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 0x560248666de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56024865ce10 0x560248667320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381aa9b0 con 0x560248657ca0 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381abcd0 con 0x5602486572b0 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f381ab010 con 0x56024865ce10 2024-03-26T14:14:14.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f42322740 10 client.0 _ll_register_callbacks cb 0x7f2f28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 0x560248666de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f2fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 0x560248666de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45460/0 (socket says 172.21.15.137:45460) 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.141+0000 7f2f2fe00640 1 -- 172.21.15.137:0/3298714003 learned_addr learned my addr 172.21.15.137:0/3298714003 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f3da00640 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x560248668780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f2fe00640 1 -- 172.21.15.137:0/3298714003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56024865ce10 msgr2=0x560248667320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56024865ce10 0x560248667320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f2fe00640 1 -- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 msgr2=0x560248668780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x560248668780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f2fe00640 1 -- 172.21.15.137:0/3298714003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560248667990 con 0x560248657ca0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f2fe00640 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 0x560248666de0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2f24004f60 tx=0x7f2f24003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7966390966be1e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f22a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.142+0000 7f2f22a00640 1 -- 172.21.15.137:0/3298714003 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24039040 con 0x560248657ca0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f30009580 con 0x560248657ca0 2024-03-26T14:14:14.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f22a00640 1 -- 172.21.15.137:0/3298714003 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2f24003c70 con 0x560248657ca0 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f22a00640 1 -- 172.21.15.137:0/3298714003 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f2405e050 con 0x560248657ca0 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 10 client.18174 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5602486c9b10 con 0x560248657ca0 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f22a00640 1 -- 172.21.15.137:0/3298714003 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f30009580 con 0x560248657ca0 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f22a00640 1 -- 172.21.15.137:0/3298714003 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2f24059030 con 0x560248657ca0 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 10 client.18174 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 10 client.18174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 20 client.18174 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.143+0000 7f2f42322740 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5602486ca4f0 0x5602486cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.144+0000 7f2f42322740 4 client.18174 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:14:14.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.144+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5602485c4740 con 0x5602486ca4f0 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.144+0000 7f2f3da00640 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5602486ca4f0 0x5602486cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.145+0000 7f2f3da00640 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5602486ca4f0 0x5602486cc910 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865ddc375ed42244 server_cookie=909d12de2e116e0d in_seq=0 out_seq=0 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.145+0000 7f2f22a00640 10 client.18174 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.146+0000 7f2f22a00640 1 -- 172.21.15.137:0/3298714003 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5602485c4740 con 0x5602486ca4f0 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.146+0000 7f2f22a00640 10 client.18174 handle_command_reply: tid=0 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.146+0000 7f2f42322740 1 client.18174 shutdown 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 msgr2=0x560248666de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 0x560248666de0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f2f24004f60 tx=0x7f2f24003710 comp rx=0 tx=0).stop 2024-03-26T14:14:14.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 shutdown_connections 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5602486ca4f0 0x5602486cc910 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56024865ce10 0x560248667320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560248657ca0 0x560248666de0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 --2- 172.21.15.137:0/3298714003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602486572b0 0x560248668780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 >> 172.21.15.137:0/3298714003 conn(0x560248652820 msgr2=0x5602486511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f22a00640 0 client.18174 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 shutdown_connections 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 1 -- 172.21.15.137:0/3298714003 wait complete. 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.147+0000 7f2f42322740 20 client.18174 trim_cache size 0 max 16384 2024-03-26T14:14:14.150 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:14:14.151 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:14:14.151 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "88adeacd-19af-4012-859d-bf1cfc25dcb2", 2024-03-26T14:14:14.151 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:14:14.151 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:14:14.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f1c0a2800 msgr2=0x7f2f1c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f1c0a2800 0x7f2f1c0a4cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f280a39d0 tx=0x7f2f280d2740 comp rx=0 tx=0).stop 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 msgr2=0x7f2f381999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 0x7f2f381999d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800e9f0 tx=0x7f2f2800eeb0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 shutdown_connections 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2f1c0a2800 0x7f2f1c0a4cc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f38105350 0x7f2f381a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2f38104990 0x7f2f38199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 --2- 172.21.15.137:0/1440576296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2f381037a0 0x7f2f381999d0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.148+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 >> 172.21.15.137:0/1440576296 conn(0x7f2f380fed10 msgr2=0x7f2f38107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.149+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 shutdown_connections 2024-03-26T14:14:14.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.149+0000 7f2f3e400640 1 -- 172.21.15.137:0/1440576296 wait complete. 2024-03-26T14:14:14.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:14:14.167 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.370+0000 7f2b56600640 1 -- 172.21.15.137:0/452587307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50104990 msgr2=0x7f2b50104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.370+0000 7f2b56600640 1 --2- 172.21.15.137:0/452587307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50104990 0x7f2b50104e10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009900 tx=0x7f2b3c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 -- 172.21.15.137:0/452587307 shutdown_connections 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 --2- 172.21.15.137:0/452587307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b50105350 0x7f2b5010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 --2- 172.21.15.137:0/452587307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50104990 0x7f2b50104e10 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 --2- 172.21.15.137:0/452587307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b501037a0 0x7f2b50103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 -- 172.21.15.137:0/452587307 >> 172.21.15.137:0/452587307 conn(0x7f2b500fed10 msgr2=0x7f2b50101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 -- 172.21.15.137:0/452587307 shutdown_connections 2024-03-26T14:14:14.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.371+0000 7f2b56600640 1 -- 172.21.15.137:0/452587307 wait complete. 2024-03-26T14:14:14.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 Processor -- start 2024-03-26T14:14:14.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 -- start start 2024-03-26T14:14:14.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b501037a0 0x7f2b50199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b50104990 0x7f2b50199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 0x7f2b501a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010f320 con 0x7f2b50105350 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010f1a0 con 0x7f2b501037a0 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.372+0000 7f2b56600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5010f4a0 con 0x7f2b50104990 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 0x7f2b501a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b501037a0 0x7f2b50199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 0x7f2b501a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45472/0 (socket says 172.21.15.137:45472) 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 -- 172.21.15.137:0/893152999 learned_addr learned my addr 172.21.15.137:0/893152999 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b50104990 0x7f2b50199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 -- 172.21.15.137:0/893152999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b50104990 msgr2=0x7f2b50199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b50104990 0x7f2b50199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 -- 172.21.15.137:0/893152999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b501037a0 msgr2=0x7f2b50199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b501037a0 0x7f2b50199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.373+0000 7f2b55c00640 1 -- 172.21.15.137:0/893152999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b501a1580 con 0x7f2b50105350 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.374+0000 7f2b55c00640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 0x7f2b501a0f10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800eaa0 tx=0x7f2b4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.374+0000 7f2b4d600640 1 -- 172.21.15.137:0/893152999 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4800cdb0 con 0x7f2b50105350 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.374+0000 7f2b4d600640 1 -- 172.21.15.137:0/893152999 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b48005020 con 0x7f2b50105350 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.374+0000 7f2b4d600640 1 -- 172.21.15.137:0/893152999 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b480116d0 con 0x7f2b50105350 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.374+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b501a1870 con 0x7f2b50105350 2024-03-26T14:14:14.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.374+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b501a1de0 con 0x7f2b50105350 2024-03-26T14:14:14.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.377+0000 7f2b5a99e740 1 Processor -- start 2024-03-26T14:14:14.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b4d600640 1 -- 172.21.15.137:0/893152999 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2b480051d0 con 0x7f2b50105350 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 -- start start 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92da050 0x5595b92da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92daa40 0x5595b92df540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 0x5595b92e2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b923f250 con 0x5595b92dfbb0 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b91e7a30 con 0x5595b92da050 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.378+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b923b2f0 con 0x5595b92daa40 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92da050 0x5595b92da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 0x5595b92e2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 0x5595b92e2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45480/0 (socket says 172.21.15.137:45480) 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 -- 172.21.15.137:0/2278166039 learned_addr learned my addr 172.21.15.137:0/2278166039 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b55200640 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92daa40 0x5595b92df540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 -- 172.21.15.137:0/2278166039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92daa40 msgr2=0x5595b92df540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92daa40 0x5595b92df540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 -- 172.21.15.137:0/2278166039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92da050 msgr2=0x5595b92da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92da050 0x5595b92da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.379+0000 7f2b4fe00640 1 -- 172.21.15.137:0/2278166039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b91d2690 con 0x5595b92dfbb0 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b4d600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2b380a2880 0x7f2b380a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 0x5595b92e2040 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0096c0 tx=0x7f2b3c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fee68c63a8a4c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b4d600640 1 -- 172.21.15.137:0/893152999 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2b480cefc0 con 0x7f2b50105350 2024-03-26T14:14:14.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b55200640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2b380a2880 0x7f2b380a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b42000640 1 -- 172.21.15.137:0/2278166039 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c004040 con 0x5595b92dfbb0 2024-03-26T14:14:14.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b42000640 1 -- 172.21.15.137:0/2278166039 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b3c049070 con 0x5595b92dfbb0 2024-03-26T14:14:14.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.380+0000 7f2b42000640 1 -- 172.21.15.137:0/2278166039 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c004920 con 0x5595b92dfbb0 2024-03-26T14:14:14.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 -- 172.21.15.137:0/2278166039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 msgr2=0x5595b92e2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 0x5595b92e2040 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0096c0 tx=0x7f2b3c000bc0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b55200640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2b380a2880 0x7f2b380a4d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5595b92dae40 tx=0x7f2b44005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 -- 172.21.15.137:0/2278166039 shutdown_connections 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92dfbb0 0x5595b92e2040 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92daa40 0x5595b92df540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/2278166039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92da050 0x5595b92da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.381+0000 7f2b5a99e740 1 -- 172.21.15.137:0/2278166039 >> 172.21.15.137:0/2278166039 conn(0x5595b92a2200 msgr2=0x5595b92d59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.382+0000 7f2b5a99e740 1 -- 172.21.15.137:0/2278166039 shutdown_connections 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.382+0000 7f2b5a99e740 1 -- 172.21.15.137:0/2278166039 wait complete. 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.382+0000 7f2b5a99e740 1 -- start start 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.382+0000 7f2b5a99e740 1 Processor -- start 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92e4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b92e2b00 con 0x5595b92da050 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b91fae80 con 0x5595b92daa40 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595b92e2c80 con 0x5595b92dfbb0 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b5a99e740 10 client.0 _ll_register_callbacks cb 0x5595b920d130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b55200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b55200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92e4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45490/0 (socket says 172.21.15.137:45490) 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b55200640 1 -- 172.21.15.137:0/3008983198 learned_addr learned my addr 172.21.15.137:0/3008983198 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.383+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55c00640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55200640 1 -- 172.21.15.137:0/3008983198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 msgr2=0x5595b92eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55200640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55200640 1 -- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 msgr2=0x5595b92e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55200640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55200640 1 -- 172.21.15.137:0/3008983198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b901d7d0 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b55200640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92e4080 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2b44004600 tx=0x7f2b44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4d15646a3e100a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b36a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.384+0000 7f2b36a00640 1 -- 172.21.15.137:0/3008983198 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4401f630 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b36a00640 1 -- 172.21.15.137:0/3008983198 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2b4401f7d0 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b3c049070 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b36a00640 1 -- 172.21.15.137:0/3008983198 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b4401f9d0 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b5a99e740 10 client.18186 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5595b8f533f0 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b36a00640 1 -- 172.21.15.137:0/3008983198 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b3c049070 con 0x5595b92da050 2024-03-26T14:14:14.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.385+0000 7f2b36a00640 1 -- 172.21.15.137:0/3008983198 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2b44027070 con 0x5595b92da050 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b5a99e740 10 client.18186 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b5a99e740 10 client.18186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b5a99e740 20 client.18186 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b92eff50 0x5595b92f2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b5a99e740 4 client.18186 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595b924c790 con 0x5595b92eff50 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.386+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b92eff50 0x5595b92f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.387+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b92eff50 0x5595b92f2380 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73e993e6086c541 server_cookie=da4c5521a9a4723d in_seq=0 out_seq=0 2024-03-26T14:14:14.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.387+0000 7f2b36a00640 10 client.18186 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.388+0000 7f2b36a00640 1 -- 172.21.15.137:0/3008983198 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5595b924c790 con 0x5595b92eff50 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.388+0000 7f2b36a00640 10 client.18186 handle_command_reply: tid=0 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.388+0000 7f2b5a99e740 1 client.18186 shutdown 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 msgr2=0x5595b92e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92e4080 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2b44004600 tx=0x7f2b44004730 comp rx=0 tx=0).stop 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 shutdown_connections 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b92eff50 0x5595b92f2380 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b36a00640 0 client.18186 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3008983198 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92e4080 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 >> 172.21.15.137:0/3008983198 conn(0x5595b92a2200 msgr2=0x5595b92d4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 shutdown_connections 2024-03-26T14:14:14.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3008983198 wait complete. 2024-03-26T14:14:14.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.389+0000 7f2b5a99e740 20 client.18186 trim_cache size 0 max 16384 2024-03-26T14:14:14.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.411+0000 7f2b5a99e740 1 Processor -- start 2024-03-26T14:14:14.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.411+0000 7f2b5a99e740 1 -- start start 2024-03-26T14:14:14.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.412 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92db980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501ae750 con 0x5595b92da050 2024-03-26T14:14:14.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501ac250 con 0x5595b92daa40 2024-03-26T14:14:14.413 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501aa2d0 con 0x5595b92dfbb0 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b55200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b55200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92db440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45496/0 (socket says 172.21.15.137:45496) 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b55200640 1 -- 172.21.15.137:0/3826408219 learned_addr learned my addr 172.21.15.137:0/3826408219 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.412+0000 7f2b55c00640 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92db980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b55200640 1 -- 172.21.15.137:0/3826408219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 msgr2=0x5595b92db980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b55200640 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92db980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b55200640 1 -- 172.21.15.137:0/3826408219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 msgr2=0x5595b92eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b55200640 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.415 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b55200640 1 -- 172.21.15.137:0/3826408219 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595b8f130f0 con 0x5595b92da050 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b55200640 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92db440 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2b44029d50 tx=0x7f2b44009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436247f7453d7e1d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b41600640 1 -- 172.21.15.137:0/3826408219 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4401f830 con 0x5595b92da050 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b41600640 1 -- 172.21.15.137:0/3826408219 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b440054d0 con 0x5595b92da050 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.413+0000 7f2b41600640 1 -- 172.21.15.137:0/3826408219 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b440297b0 con 0x5595b92da050 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3826408219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 msgr2=0x5595b92db440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92db440 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2b44029d50 tx=0x7f2b44009f20 comp rx=0 tx=0).stop 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3826408219 shutdown_connections 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92daa40 0x5595b92eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92dfbb0 0x5595b92db980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/3826408219 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92da050 0x5595b92db440 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.416 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.414+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3826408219 >> 172.21.15.137:0/3826408219 conn(0x5595b92a2200 msgr2=0x5595b92d9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.415+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3826408219 shutdown_connections 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.415+0000 7f2b5a99e740 1 -- 172.21.15.137:0/3826408219 wait complete. 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.415+0000 7f2b5a99e740 1 -- start start 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.415+0000 7f2b5a99e740 1 Processor -- start 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92da050 0x5595b92e91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 0x5595b92e9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92dfbb0 0x5595b92ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501aa8d0 con 0x5595b92daa40 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501abbf0 con 0x5595b92dfbb0 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b501aaf30 con 0x5595b92da050 2024-03-26T14:14:14.420 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b5a99e740 10 client.0 _ll_register_callbacks cb 0x7f2b44009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 0x5595b92e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 0x5595b92e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:45512/0 (socket says 172.21.15.137:45512) 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 -- 172.21.15.137:0/1789151157 learned_addr learned my addr 172.21.15.137:0/1789151157 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b55c00640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92dfbb0 0x5595b92ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b55200640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92da050 0x5595b92e91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 -- 172.21.15.137:0/1789151157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92da050 msgr2=0x5595b92e91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92da050 0x5595b92e91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 -- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92dfbb0 msgr2=0x5595b92ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92dfbb0 0x5595b92ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.416+0000 7f2b4fe00640 1 -- 172.21.15.137:0/1789151157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595b92ead10 con 0x5595b92daa40 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b4fe00640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 0x5595b92e9710 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009230 tx=0x7f2b3c02cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad665dbd81ba14 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:14.421 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b36a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b36a00640 1 -- 172.21.15.137:0/1789151157 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0053e0 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b36a00640 1 -- 172.21.15.137:0/1789151157 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2b3c005580 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b440054d0 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b36a00640 1 -- 172.21.15.137:0/1789151157 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b3c064050 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b5a99e740 10 client.18192 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.417+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5595b934bcc0 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b36a00640 1 -- 172.21.15.137:0/1789151157 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b440054d0 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b36a00640 1 -- 172.21.15.137:0/1789151157 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2b3c040040 con 0x5595b92daa40 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b5a99e740 10 client.18192 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b5a99e740 10 client.18192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b5a99e740 20 client.18192 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:14.422 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b934c160 0x5595b934e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:14.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b5a99e740 4 client.18192 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:14:14.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595b9246ae0 con 0x5595b934c160 2024-03-26T14:14:14.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.418+0000 7f2b55200640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b934c160 0x5595b934e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:14.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.419+0000 7f2b55200640 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b934c160 0x5595b934e550 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47eaede681a7087f server_cookie=37bde1d64774c7fb in_seq=0 out_seq=0 2024-03-26T14:14:14.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.419+0000 7f2b36a00640 10 client.18192 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b36a00640 1 -- 172.21.15.137:0/1789151157 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5595b9246ae0 con 0x5595b934c160 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b36a00640 10 client.18192 handle_command_reply: tid=0 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 client.18192 shutdown 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 msgr2=0x5595b92e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 0x5595b92e9710 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009230 tx=0x7f2b3c02cc20 comp rx=0 tx=0).stop 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 shutdown_connections 2024-03-26T14:14:14.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5595b934c160 0x5595b934e550 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5595b92dfbb0 0x5595b92ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5595b92daa40 0x5595b92e9710 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 --2- 172.21.15.137:0/1789151157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595b92da050 0x5595b92e91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 >> 172.21.15.137:0/1789151157 conn(0x5595b92a2200 msgr2=0x5595b92d8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b36a00640 0 client.18192 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 shutdown_connections 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 1 -- 172.21.15.137:0/1789151157 wait complete. 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.420+0000 7f2b5a99e740 20 client.18192 trim_cache size 0 max 16384 2024-03-26T14:14:14.425 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: "88adeacd-19af-4012-859d-bf1cfc25dcb2": { 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "88adeacd-19af-4012-859d-bf1cfc25dcb2", 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:14:14.426 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:14:14.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2b380a2880 msgr2=0x7f2b380a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2b380a2880 0x7f2b380a4d40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x5595b92dae40 tx=0x7f2b44005f90 comp rx=0 tx=0).stop 2024-03-26T14:14:14.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 msgr2=0x7f2b501a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:14.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 0x7f2b501a0f10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800eaa0 tx=0x7f2b4800ef60 comp rx=0 tx=0).stop 2024-03-26T14:14:14.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 shutdown_connections 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2b380a2880 0x7f2b380a4d40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2b50105350 0x7f2b501a0f10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2b50104990 0x7f2b50199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 --2- 172.21.15.137:0/893152999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b501037a0 0x7f2b50199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 >> 172.21.15.137:0/893152999 conn(0x7f2b500fed10 msgr2=0x7f2b50107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 shutdown_connections 2024-03-26T14:14:14.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:14.421+0000 7f2b56600640 1 -- 172.21.15.137:0/893152999 wait complete. 2024-03-26T14:14:14.442 INFO:tasks.cephfs.filesystem:scrub status for tag:88adeacd-19af-4012-859d-bf1cfc25dcb2 - {'path': '/', 'tag': '88adeacd-19af-4012-859d-bf1cfc25dcb2', 'options': 'recursive,force'} 2024-03-26T14:14:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:15 smithi179 ceph-mon[35472]: pgmap v814: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:15 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:14:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:15 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:15 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:14:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:15 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:15 smithi159 ceph-mon[35501]: pgmap v814: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:15 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:14:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:15 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:15 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:14:15.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:15 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:15 smithi137 ceph-mon[31539]: pgmap v814: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:15 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:14:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:15 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:15 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:14:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:15 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:17.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:17 smithi179 ceph-mon[35472]: pgmap v815: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:17.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:17 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:14:17.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:17 smithi159 ceph-mon[35501]: pgmap v815: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:17.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:17 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:14:17.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:17 smithi137 ceph-mon[31539]: pgmap v815: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:17.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:17 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:14:18.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:18 smithi159 ceph-mon[35501]: pgmap v816: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:18.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:18 smithi137 ceph-mon[31539]: pgmap v816: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:18.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:18 smithi179 ceph-mon[35472]: pgmap v816: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:20.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:20 smithi179 ceph-mon[35472]: pgmap v817: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:20.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:20 smithi159 ceph-mon[35501]: pgmap v817: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:21.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:20 smithi137 ceph-mon[31539]: pgmap v817: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:22.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:22 smithi179 ceph-mon[35472]: pgmap v818: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:14:22.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:22 smithi159 ceph-mon[35501]: pgmap v818: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:14:23.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:22 smithi137 ceph-mon[31539]: pgmap v818: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:14:24.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:24 smithi179 ceph-mon[35472]: pgmap v819: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:14:24.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:24 smithi159 ceph-mon[35501]: pgmap v819: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:14:25.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:24 smithi137 ceph-mon[31539]: pgmap v819: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:14:26.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:26 smithi179 ceph-mon[35472]: pgmap v820: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:26.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:26 smithi159 ceph-mon[35501]: pgmap v820: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:27.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:26 smithi137 ceph-mon[31539]: pgmap v820: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:28.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:28 smithi179 ceph-mon[35472]: pgmap v821: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:28.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:28 smithi159 ceph-mon[35501]: pgmap v821: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:29.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:28 smithi137 ceph-mon[31539]: pgmap v821: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:30.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:30 smithi179 ceph-mon[35472]: pgmap v822: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:30.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:30 smithi159 ceph-mon[35501]: pgmap v822: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:31.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:30 smithi137 ceph-mon[31539]: pgmap v822: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:32.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:32 smithi179 ceph-mon[35472]: pgmap v823: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:14:32.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:32 smithi159 ceph-mon[35501]: pgmap v823: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:14:33.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:32 smithi137 ceph-mon[31539]: pgmap v823: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:14:34.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:34 smithi179 ceph-mon[35472]: pgmap v824: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:14:34.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:34 smithi159 ceph-mon[35501]: pgmap v824: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:14:35.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:34 smithi137 ceph-mon[31539]: pgmap v824: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:14:36.924 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:36 smithi179 ceph-mon[35472]: pgmap v825: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:14:36.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:36 smithi159 ceph-mon[35501]: pgmap v825: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:14:37.035 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:36 smithi137 ceph-mon[31539]: pgmap v825: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:14:38.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:38 smithi159 ceph-mon[35501]: pgmap v826: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:14:39.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:38 smithi137 ceph-mon[31539]: pgmap v826: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:14:39.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:38 smithi179 ceph-mon[35472]: pgmap v826: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:14:40.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:40 smithi159 ceph-mon[35501]: pgmap v827: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:14:40.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:40 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:14:41.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:40 smithi137 ceph-mon[31539]: pgmap v827: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:14:41.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:40 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:14:41.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:40 smithi179 ceph-mon[35472]: pgmap v827: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-03-26T14:14:41.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:40 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:14:42.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:42 smithi159 ceph-mon[35501]: pgmap v828: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:14:42.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:14:42.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:14:42.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:42 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:14:43.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:42 smithi137 ceph-mon[31539]: pgmap v828: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:14:43.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:14:43.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:14:43.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:42 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:14:43.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:42 smithi179 ceph-mon[35472]: pgmap v828: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-03-26T14:14:43.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:14:43.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:14:43.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:42 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:14:44.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:14:44.444 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 -- 172.21.15.137:0/1441836378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82201038c0 msgr2=0x7f8220103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 --2- 172.21.15.137:0/1441836378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82201038c0 0x7f8220103ce0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f8210009900 tx=0x7f821002bc50 comp rx=0 tx=0).stop 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 -- 172.21.15.137:0/1441836378 shutdown_connections 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 --2- 172.21.15.137:0/1441836378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8220105470 0x7f822010bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 --2- 172.21.15.137:0/1441836378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8220104ab0 0x7f8220104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 --2- 172.21.15.137:0/1441836378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f82201038c0 0x7f8220103ce0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 -- 172.21.15.137:0/1441836378 >> 172.21.15.137:0/1441836378 conn(0x7f82200fee50 msgr2=0x7f8220101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 -- 172.21.15.137:0/1441836378 shutdown_connections 2024-03-26T14:14:44.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.687+0000 7f8224a00640 1 -- 172.21.15.137:0/1441836378 wait complete. 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 Processor -- start 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 -- start start 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82201038c0 0x7f8220199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 0x7f822019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8220105470 0x7f82201a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010f490 con 0x7f8220104ab0 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010f310 con 0x7f8220105470 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f8224a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822010f610 con 0x7f82201038c0 2024-03-26T14:14:44.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.689+0000 7f821d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 0x7f822019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 0x7f822019a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46744/0 (socket says 172.21.15.137:46744) 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 -- 172.21.15.137:0/4114406023 learned_addr learned my addr 172.21.15.137:0/4114406023 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821e800640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8220105470 0x7f82201a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821de00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82201038c0 0x7f8220199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 -- 172.21.15.137:0/4114406023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82201038c0 msgr2=0x7f8220199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82201038c0 0x7f8220199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 -- 172.21.15.137:0/4114406023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8220105470 msgr2=0x7f82201a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8220105470 0x7f82201a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 -- 172.21.15.137:0/4114406023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82201a1750 con 0x7f8220104ab0 2024-03-26T14:14:44.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.690+0000 7f821d400640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 0x7f822019a070 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b4a0 tx=0x7f820c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.694+0000 7f8216a00640 1 -- 172.21.15.137:0/4114406023 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c014070 con 0x7f8220104ab0 2024-03-26T14:14:44.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.694+0000 7f8216a00640 1 -- 172.21.15.137:0/4114406023 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f820c010ac0 con 0x7f8220104ab0 2024-03-26T14:14:44.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.694+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82201a1a40 con 0x7f8220104ab0 2024-03-26T14:14:44.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.694+0000 7f8216a00640 1 -- 172.21.15.137:0/4114406023 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c017c30 con 0x7f8220104ab0 2024-03-26T14:14:44.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.694+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82201a1fe0 con 0x7f8220104ab0 2024-03-26T14:14:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.698+0000 7f8216a00640 1 -- 172.21.15.137:0/4114406023 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f820c017070 con 0x7f8220104ab0 2024-03-26T14:14:44.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.698+0000 7f8227322740 1 Processor -- start 2024-03-26T14:14:44.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 -- start start 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fd42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fd9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fdbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565123f39250 con 0x565123fd99e0 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x565123ee1a30 con 0x565123fd3e80 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f8227322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x565123f352f0 con 0x565123fd4870 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f821d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fdbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f821d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fdbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46756/0 (socket says 172.21.15.137:46756) 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.699+0000 7f821d400640 1 -- 172.21.15.137:0/1153279086 learned_addr learned my addr 172.21.15.137:0/1153279086 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821de00640 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fd9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f8216a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f82040a2880 0x7f82040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821e800640 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fd42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f8216a00640 1 -- 172.21.15.137:0/4114406023 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f820c0ce560 con 0x7f8220104ab0 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821e800640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f82040a2880 0x7f82040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821d400640 1 -- 172.21.15.137:0/1153279086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 msgr2=0x565123fd9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821d400640 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fd9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821d400640 1 -- 172.21.15.137:0/1153279086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 msgr2=0x565123fd42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821d400640 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fd42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.703 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.700+0000 7f821d400640 1 -- 172.21.15.137:0/1153279086 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565123ecc690 con 0x565123fd99e0 2024-03-26T14:14:44.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.701+0000 7f821d400640 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fdbe70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f820c096f90 tx=0x7f820c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b51391915b4028d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.701+0000 7f820b000640 1 -- 172.21.15.137:0/1153279086 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c09be60 con 0x565123fd99e0 2024-03-26T14:14:44.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.701+0000 7f821e800640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f82040a2880 0x7f82040a4d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x565123fd50d0 tx=0x7f8218004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.701+0000 7f820b000640 1 -- 172.21.15.137:0/1153279086 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f820c09ab30 con 0x565123fd99e0 2024-03-26T14:14:44.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.701+0000 7f820b000640 1 -- 172.21.15.137:0/1153279086 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c01e070 con 0x565123fd99e0 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 -- 172.21.15.137:0/1153279086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 msgr2=0x565123fdbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fdbe70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f820c096f90 tx=0x7f820c0046c0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 -- 172.21.15.137:0/1153279086 shutdown_connections 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fdbe70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fd9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 --2- 172.21.15.137:0/1153279086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fd42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 -- 172.21.15.137:0/1153279086 >> 172.21.15.137:0/1153279086 conn(0x565123f9c200 msgr2=0x565123fcf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 -- 172.21.15.137:0/1153279086 shutdown_connections 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 -- 172.21.15.137:0/1153279086 wait complete. 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.702+0000 7f8227322740 1 -- start start 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 Processor -- start 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd3e80 0x565123fddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd4870 0x565123fde2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x565123fdc8d0 con 0x565123fd99e0 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x565123ef4e80 con 0x565123fd4870 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x565123fdca50 con 0x565123fd3e80 2024-03-26T14:14:44.710 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f8227322740 10 client.0 _ll_register_callbacks cb 0x565123f07130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.703+0000 7f821e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46764/0 (socket says 172.21.15.137:46764) 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 -- 172.21.15.137:0/3324126552 learned_addr learned my addr 172.21.15.137:0/3324126552 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821d400640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd4870 0x565123fde2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821de00640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd3e80 0x565123fddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 -- 172.21.15.137:0/3324126552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd3e80 msgr2=0x565123fddd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd3e80 0x565123fddd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 -- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd4870 msgr2=0x565123fde2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd4870 0x565123fde2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.704+0000 7f821e800640 1 -- 172.21.15.137:0/3324126552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565123d177d0 con 0x565123fd99e0 2024-03-26T14:14:44.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f821e800640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe52f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8218010f20 tx=0x7f8218004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18625b50e1293314 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f81ff400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f81ff400640 1 -- 172.21.15.137:0/3324126552 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218024020 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f81ff400640 1 -- 172.21.15.137:0/3324126552 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8218020410 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f820c09ab30 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f81ff400640 1 -- 172.21.15.137:0/3324126552 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f821802f050 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f8227322740 10 client.18204 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.705+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565123c4d3f0 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f81ff400640 1 -- 172.21.15.137:0/3324126552 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f820c09ab30 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f81ff400640 1 -- 172.21.15.137:0/3324126552 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8218011590 con 0x565123fd99e0 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f8227322740 10 client.18204 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f8227322740 10 client.18204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:44.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f8227322740 20 client.18204 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f8227322740 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565123fe9c80 0x565123fec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f8227322740 4 client.18204 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.706+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565123f46790 con 0x565123fe9c80 2024-03-26T14:14:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.707+0000 7f821de00640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565123fe9c80 0x565123fec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.708+0000 7f821de00640 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565123fe9c80 0x565123fec0a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d14e76f7a7e64d server_cookie=3edc1fca4fe693d0 in_seq=0 out_seq=0 2024-03-26T14:14:44.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.708+0000 7f81ff400640 10 client.18204 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:44.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f81ff400640 1 -- 172.21.15.137:0/3324126552 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x565123f46790 con 0x565123fe9c80 2024-03-26T14:14:44.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f81ff400640 10 client.18204 handle_command_reply: tid=0 2024-03-26T14:14:44.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 client.18204 shutdown 2024-03-26T14:14:44.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 msgr2=0x565123fe52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe52f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8218010f20 tx=0x7f8218004a40 comp rx=0 tx=0).stop 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 shutdown_connections 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565123fe9c80 0x565123fec0a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd4870 0x565123fde2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe52f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 --2- 172.21.15.137:0/3324126552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd3e80 0x565123fddd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f81ff400640 0 client.18204 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 >> 172.21.15.137:0/3324126552 conn(0x565123f9c200 msgr2=0x565123fcea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 shutdown_connections 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.709+0000 7f8227322740 1 -- 172.21.15.137:0/3324126552 wait complete. 2024-03-26T14:14:44.715 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.710+0000 7f8227322740 20 client.18204 trim_cache size 0 max 16384 2024-03-26T14:14:44.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 Processor -- start 2024-03-26T14:14:44.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 -- start start 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 0x565123fd2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd99e0 0x565123fd2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201aea60 con 0x565123fd3e80 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201ac560 con 0x565123fd99e0 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f8227322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201aa5e0 con 0x565123fd4870 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f821de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 0x565123fd2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.727+0000 7f821de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 0x565123fd2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46770/0 (socket says 172.21.15.137:46770) 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 -- 172.21.15.137:0/3929496135 learned_addr learned my addr 172.21.15.137:0/3929496135 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821e800640 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd99e0 0x565123fd2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821d400640 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 -- 172.21.15.137:0/3929496135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 msgr2=0x565123fe52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 -- 172.21.15.137:0/3929496135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd99e0 msgr2=0x565123fd2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd99e0 0x565123fd2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 -- 172.21.15.137:0/3929496135 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565123c0d0f0 con 0x565123fd3e80 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.728+0000 7f821de00640 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 0x565123fd2a30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8220104910 tx=0x7f821002c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbe468a4d1c01eb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f820a600640 1 -- 172.21.15.137:0/3929496135 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821004e020 con 0x565123fd3e80 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f820a600640 1 -- 172.21.15.137:0/3929496135 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82100054f0 con 0x565123fd3e80 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f820a600640 1 -- 172.21.15.137:0/3929496135 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210048070 con 0x565123fd3e80 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 -- 172.21.15.137:0/3929496135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 msgr2=0x565123fd2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 0x565123fd2a30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8220104910 tx=0x7f821002c800 comp rx=0 tx=0).stop 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 -- 172.21.15.137:0/3929496135 shutdown_connections 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd99e0 0x565123fd2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 --2- 172.21.15.137:0/3929496135 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd3e80 0x565123fd2a30 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.729+0000 7f8227322740 1 -- 172.21.15.137:0/3929496135 >> 172.21.15.137:0/3929496135 conn(0x565123f9c200 msgr2=0x565123fcea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.730+0000 7f8227322740 1 -- 172.21.15.137:0/3929496135 shutdown_connections 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.730+0000 7f8227322740 1 -- 172.21.15.137:0/3929496135 wait complete. 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.730+0000 7f8227322740 1 -- start start 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.730+0000 7f8227322740 1 Processor -- start 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fe2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201aabe0 con 0x565123fd99e0 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201abf00 con 0x565123fd3e80 2024-03-26T14:14:44.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82201ab240 con 0x565123fd4870 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f8227322740 10 client.0 _ll_register_callbacks cb 0x7f8218007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46772/0 (socket says 172.21.15.137:46772) 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821e800640 1 -- 172.21.15.137:0/3521018793 learned_addr learned my addr 172.21.15.137:0/3521018793 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821de00640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fe2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821d400640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821e800640 1 -- 172.21.15.137:0/3521018793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 msgr2=0x565123fe33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821e800640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.731+0000 7f821e800640 1 -- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 msgr2=0x565123fe2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f821e800640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fe2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f821e800640 1 -- 172.21.15.137:0/3521018793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565123fe49a0 con 0x565123fd99e0 2024-03-26T14:14:44.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f821de00640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fe2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f821e800640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe43b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8218012460 tx=0x7f8218010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106ec5e8ef4bc2dc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f81ff400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f81ff400640 1 -- 172.21.15.137:0/3521018793 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218024040 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f81ff400640 1 -- 172.21.15.137:0/3521018793 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8218034440 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82100054f0 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.732+0000 7f81ff400640 1 -- 172.21.15.137:0/3521018793 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8218045050 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 10 client.18210 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x565124045930 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f81ff400640 1 -- 172.21.15.137:0/3521018793 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82100054f0 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f81ff400640 1 -- 172.21.15.137:0/3521018793 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8218032040 con 0x565123fd99e0 2024-03-26T14:14:44.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 10 client.18210 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 10 client.18210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 20 client.18210 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565124046190 0x5651240485b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 4 client.18210 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:14:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.733+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565123f40ae0 con 0x565124046190 2024-03-26T14:14:44.739 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.734+0000 7f821de00640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565124046190 0x5651240485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f821de00640 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565124046190 0x5651240485b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b977ebb5fa6ef6e server_cookie=a7eb78eb08f3ee5d in_seq=0 out_seq=0 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f81ff400640 10 client.18210 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f81ff400640 1 -- 172.21.15.137:0/3521018793 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x565123f40ae0 con 0x565124046190 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f81ff400640 10 client.18210 handle_command_reply: tid=0 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 client.18210 shutdown 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 msgr2=0x565123fe43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe43b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8218012460 tx=0x7f8218010820 comp rx=0 tx=0).stop 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 shutdown_connections 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x565124046190 0x5651240485b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x565123fd99e0 0x565123fe43b0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565123fd4870 0x565123fe33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 --2- 172.21.15.137:0/3521018793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565123fd3e80 0x565123fe2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 >> 172.21.15.137:0/3521018793 conn(0x565123f9c200 msgr2=0x565123fd5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.735+0000 7f81ff400640 0 client.18210 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 shutdown_connections 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8227322740 1 -- 172.21.15.137:0/3521018793 wait complete. 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8227322740 20 client.18210 trim_cache size 0 max 16384 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:14:44.742 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:14:44.743 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:14:44.743 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:14:44.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f82040a2880 msgr2=0x7f82040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f82040a2880 0x7f82040a4d40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x565123fd50d0 tx=0x7f8218004ab0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 msgr2=0x7f822019a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 0x7f822019a070 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f820c00b4a0 tx=0x7f820c00b960 comp rx=0 tx=0).stop 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 shutdown_connections 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f82040a2880 0x7f82040a4d40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8220105470 0x7f82201a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8220104ab0 0x7f822019a070 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 --2- 172.21.15.137:0/4114406023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f82201038c0 0x7f8220199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.736+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 >> 172.21.15.137:0/4114406023 conn(0x7f82200fee50 msgr2=0x7f8220107e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.737+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 shutdown_connections 2024-03-26T14:14:44.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.737+0000 7f8224a00640 1 -- 172.21.15.137:0/4114406023 wait complete. 2024-03-26T14:14:44.759 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:14:44.759 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:14:44.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:44 smithi137 ceph-mon[31539]: pgmap v829: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:44.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:44 smithi159 ceph-mon[35501]: pgmap v829: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:44.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:44 smithi179 ceph-mon[35472]: pgmap v829: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:14:44.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 -- 172.21.15.137:0/2768115581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901015e0 msgr2=0x7fb890101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 --2- 172.21.15.137:0/2768115581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901015e0 0x7fb890101a60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb87c009900 tx=0x7fb87c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:14:44.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 -- 172.21.15.137:0/2768115581 shutdown_connections 2024-03-26T14:14:44.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 --2- 172.21.15.137:0/2768115581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 0x7fb890108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 --2- 172.21.15.137:0/2768115581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901015e0 0x7fb890101a60 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 --2- 172.21.15.137:0/2768115581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901003f0 0x7fb890100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 -- 172.21.15.137:0/2768115581 >> 172.21.15.137:0/2768115581 conn(0x7fb8900fb980 msgr2=0x7fb8900fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:44.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.979+0000 7fb896e00640 1 -- 172.21.15.137:0/2768115581 shutdown_connections 2024-03-26T14:14:44.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.980+0000 7fb896e00640 1 -- 172.21.15.137:0/2768115581 wait complete. 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 Processor -- start 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 -- start start 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 0x7fb890197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901015e0 0x7fb890197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 0x7fb89019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89010bf80 con 0x7fb8901003f0 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89010be00 con 0x7fb8901015e0 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.981+0000 7fb896e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89010c100 con 0x7fb890101fa0 2024-03-26T14:14:44.983 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 0x7fb890197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 0x7fb890197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46792/0 (socket says 172.21.15.137:46792) 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 -- 172.21.15.137:0/1977756562 learned_addr learned my addr 172.21.15.137:0/1977756562 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895000640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901015e0 0x7fb890197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb896400640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 0x7fb89019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 -- 172.21.15.137:0/1977756562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 msgr2=0x7fb89019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 0x7fb89019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 -- 172.21.15.137:0/1977756562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901015e0 msgr2=0x7fb890197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901015e0 0x7fb890197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 -- 172.21.15.137:0/1977756562 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89019f380 con 0x7fb8901003f0 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895000640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901015e0 0x7fb890197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb896400640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 0x7fb89019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:44.984 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.982+0000 7fb895a00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 0x7fb890197760 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9c0 tx=0x7fb88400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.983+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400cd80 con 0x7fb8901003f0 2024-03-26T14:14:44.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.983+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8840035f0 con 0x7fb8901003f0 2024-03-26T14:14:44.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.983+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884010690 con 0x7fb8901003f0 2024-03-26T14:14:44.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.983+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89019f670 con 0x7fb8901003f0 2024-03-26T14:14:44.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.983+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89019fb40 con 0x7fb8901003f0 2024-03-26T14:14:44.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.984+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89019fe00 con 0x7fb8901003f0 2024-03-26T14:14:44.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.987+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb8840037a0 con 0x7fb8901003f0 2024-03-26T14:14:44.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.988+0000 7fb88e000640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8780a2880 0x7fb8780a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:44.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.988+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb8840cf290 con 0x7fb8901003f0 2024-03-26T14:14:44.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.988+0000 7fb895000640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8780a2880 0x7fb8780a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:44.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.989+0000 7fb895000640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8780a2880 0x7fb8780a4d40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb87c006010 tx=0x7fb87c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:44.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:44.994+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb88409bf60 con 0x7fb8901003f0 2024-03-26T14:14:45.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.259+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8901a0190 con 0x7fb8901003f0 2024-03-26T14:14:45.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.261+0000 7fb88e000640 1 -- 172.21.15.137:0/1977756562 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fb88409b860 con 0x7fb8901003f0 2024-03-26T14:14:45.262 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:14:45.262 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:14:45.262 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:14:45.264 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.263+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8780a2880 msgr2=0x7fb8780a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.263+0000 7fb896e00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8780a2880 0x7fb8780a4d40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb87c006010 tx=0x7fb87c02c7e0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.263+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 msgr2=0x7fb890197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.263+0000 7fb896e00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 0x7fb890197760 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb88400e9c0 tx=0x7fb88400ee80 comp rx=0 tx=0).stop 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 shutdown_connections 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb8780a2880 0x7fb8780a4d40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8901015e0 0x7fb890197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb890101fa0 0x7fb89019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 --2- 172.21.15.137:0/1977756562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb8901003f0 0x7fb890197760 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 >> 172.21.15.137:0/1977756562 conn(0x7fb8900fb980 msgr2=0x7fb8900fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 shutdown_connections 2024-03-26T14:14:45.265 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.264+0000 7fb896e00640 1 -- 172.21.15.137:0/1977756562 wait complete. 2024-03-26T14:14:45.284 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:14:45.284 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:14:45.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 -- 172.21.15.137:0/2464508600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf54105370 msgr2=0x7faf5410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 --2- 172.21.15.137:0/2464508600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf54105370 0x7faf5410bc00 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7faf4c009ca0 tx=0x7faf4c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 -- 172.21.15.137:0/2464508600 shutdown_connections 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 --2- 172.21.15.137:0/2464508600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf541049b0 0x7faf54104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 --2- 172.21.15.137:0/2464508600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf54105370 0x7faf5410bc00 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 --2- 172.21.15.137:0/2464508600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf541037c0 0x7faf54103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.562+0000 7faf58a00640 1 -- 172.21.15.137:0/2464508600 >> 172.21.15.137:0/2464508600 conn(0x7faf540fed50 msgr2=0x7faf54101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.563+0000 7faf58a00640 1 -- 172.21.15.137:0/2464508600 shutdown_connections 2024-03-26T14:14:45.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.563+0000 7faf58a00640 1 -- 172.21.15.137:0/2464508600 wait complete. 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.563+0000 7faf58a00640 1 Processor -- start 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 -- start start 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 0x7faf54199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf541049b0 0x7faf54199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf54105370 0x7faf541a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5410f2f0 con 0x7faf541037c0 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5410f170 con 0x7faf54105370 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf58a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5410f470 con 0x7faf541049b0 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf52800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 0x7faf54199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf52800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 0x7faf54199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46816/0 (socket says 172.21.15.137:46816) 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.564+0000 7faf52800640 1 -- 172.21.15.137:0/1001141868 learned_addr learned my addr 172.21.15.137:0/1001141868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf53200640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf54105370 0x7faf541a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf51e00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf541049b0 0x7faf54199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf52800640 1 -- 172.21.15.137:0/1001141868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf541049b0 msgr2=0x7faf54199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf52800640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf541049b0 0x7faf54199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf52800640 1 -- 172.21.15.137:0/1001141868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf54105370 msgr2=0x7faf541a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf52800640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf54105370 0x7faf541a0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf52800640 1 -- 172.21.15.137:0/1001141868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf541a1560 con 0x7faf541037c0 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf52800640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 0x7faf54199940 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e9f0 tx=0x7faf4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf4b400640 1 -- 172.21.15.137:0/1001141868 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400ccc0 con 0x7faf541037c0 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf4b400640 1 -- 172.21.15.137:0/1001141868 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf440054d0 con 0x7faf541037c0 2024-03-26T14:14:45.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.565+0000 7faf4b400640 1 -- 172.21.15.137:0/1001141868 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44010430 con 0x7faf541037c0 2024-03-26T14:14:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.566+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf541a1850 con 0x7faf541037c0 2024-03-26T14:14:45.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.566+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf541a1d50 con 0x7faf541037c0 2024-03-26T14:14:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.569+0000 7faf4b400640 1 -- 172.21.15.137:0/1001141868 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7faf44005020 con 0x7faf541037c0 2024-03-26T14:14:45.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.569+0000 7faf5c177740 1 Processor -- start 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 -- start start 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbeb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbeb2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbeb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dbe152c0 con 0x5608dbeb09f0 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dbdbdc90 con 0x5608dbeb1bc0 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf5c177740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dbe11360 con 0x5608dbeb2580 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf53200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbeb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf53200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbeb0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46824/0 (socket says 172.21.15.137:46824) 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.570+0000 7faf53200640 1 -- 172.21.15.137:0/3371391482 learned_addr learned my addr 172.21.15.137:0/3371391482 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf52800640 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbeb2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf4b400640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faf380a2800 0x7faf380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf51e00640 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbeb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf4b400640 1 -- 172.21.15.137:0/1001141868 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faf440cf400 con 0x7faf541037c0 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf53200640 1 -- 172.21.15.137:0/3371391482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 msgr2=0x5608dbeb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf53200640 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbeb8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf53200640 1 -- 172.21.15.137:0/3371391482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 msgr2=0x5608dbeb2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf53200640 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbeb2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.571+0000 7faf53200640 1 -- 172.21.15.137:0/3371391482 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608dbde2080 con 0x5608dbeb09f0 2024-03-26T14:14:45.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.572+0000 7faf53200640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faf380a2800 0x7faf380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.572+0000 7faf53200640 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbeb0e10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faf4c005e30 tx=0x7faf4c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10270a0ca5a68d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.572+0000 7faf3f400640 1 -- 172.21.15.137:0/3371391482 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c039040 con 0x5608dbeb09f0 2024-03-26T14:14:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.572+0000 7faf3f400640 1 -- 172.21.15.137:0/3371391482 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf4c003d30 con 0x5608dbeb09f0 2024-03-26T14:14:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.572+0000 7faf3f400640 1 -- 172.21.15.137:0/3371391482 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c034720 con 0x5608dbeb09f0 2024-03-26T14:14:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 -- 172.21.15.137:0/3371391482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 msgr2=0x5608dbeb0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbeb0e10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7faf4c005e30 tx=0x7faf4c009850 comp rx=0 tx=0).stop 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf53200640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faf380a2800 0x7faf380a4cc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5608dbeb29e0 tx=0x7faf4c044000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 -- 172.21.15.137:0/3371391482 shutdown_connections 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbeb2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbeb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 --2- 172.21.15.137:0/3371391482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbeb0e10 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 -- 172.21.15.137:0/3371391482 >> 172.21.15.137:0/3371391482 conn(0x5608dbeabde0 msgr2=0x5608dbeac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 -- 172.21.15.137:0/3371391482 shutdown_connections 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.573+0000 7faf5c177740 1 -- 172.21.15.137:0/3371391482 wait complete. 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.574+0000 7faf5c177740 1 -- start start 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.574+0000 7faf5c177740 1 Processor -- start 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb09f0 0x5608dbebaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb1bc0 0x5608dbebafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 0x5608dbec2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dbeb9590 con 0x5608dbeb2580 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dbdd10d0 con 0x5608dbeb09f0 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608dbeb9710 con 0x5608dbeb1bc0 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf5c177740 10 client.0 _ll_register_callbacks cb 0x5608dbdd5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 0x5608dbec2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 0x5608dbec2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46832/0 (socket says 172.21.15.137:46832) 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 -- 172.21.15.137:0/3201876483 learned_addr learned my addr 172.21.15.137:0/3201876483 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf52800640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb09f0 0x5608dbebaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf51e00640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb1bc0 0x5608dbebafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 -- 172.21.15.137:0/3201876483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb1bc0 msgr2=0x5608dbebafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb1bc0 0x5608dbebafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 -- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb09f0 msgr2=0x5608dbebaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb09f0 0x5608dbebaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf53200640 1 -- 172.21.15.137:0/3201876483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608dbb293f0 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.575+0000 7faf52800640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb09f0 0x5608dbebaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.576+0000 7faf53200640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 0x5608dbec2000 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7faf4c02c630 tx=0x7faf4c032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afdf304c0c638f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.576+0000 7faf33e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.576+0000 7faf33e00640 1 -- 172.21.15.137:0/3201876483 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c059060 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.576+0000 7faf33e00640 1 -- 172.21.15.137:0/3201876483 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faf4c034980 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.576+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf4c003d30 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.576+0000 7faf33e00640 1 -- 172.21.15.137:0/3201876483 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf4c060050 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 10 client.18228 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5608dbbf37d0 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf33e00640 1 -- 172.21.15.137:0/3201876483 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf4c003d30 con 0x5608dbeb2580 2024-03-26T14:14:45.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf33e00640 1 -- 172.21.15.137:0/3201876483 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faf4c053070 con 0x5608dbeb2580 2024-03-26T14:14:45.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 10 client.18228 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:45.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 10 client.18228 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:14:45.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 20 client.18228 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:45.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbec6160 0x5608dbec8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 4 client.18228 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:45.582 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.577+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608dbe22800 con 0x5608dbec6160 2024-03-26T14:14:45.583 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.578+0000 7faf52800640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbec6160 0x5608dbec8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.579+0000 7faf33e00640 10 client.18228 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.579+0000 7faf52800640 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbec6160 0x5608dbec8550 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e369373293d27b8 server_cookie=77c8682d2d3f28c4 in_seq=0 out_seq=0 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.580+0000 7faf33e00640 1 -- 172.21.15.137:0/3201876483 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5608dbe22800 con 0x5608dbec6160 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.580+0000 7faf33e00640 10 client.18228 handle_command_reply: tid=0 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 client.18228 shutdown 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 msgr2=0x5608dbec2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 0x5608dbec2000 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7faf4c02c630 tx=0x7faf4c032e10 comp rx=0 tx=0).stop 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 shutdown_connections 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbec6160 0x5608dbec8550 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb2580 0x5608dbec2000 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb1bc0 0x5608dbebafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf33e00640 0 client.18228 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:45.584 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 --2- 172.21.15.137:0/3201876483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb09f0 0x5608dbebaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 >> 172.21.15.137:0/3201876483 conn(0x5608dbeabde0 msgr2=0x5608dbeab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 shutdown_connections 2024-03-26T14:14:45.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 1 -- 172.21.15.137:0/3201876483 wait complete. 2024-03-26T14:14:45.585 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.581+0000 7faf5c177740 20 client.18228 trim_cache size 0 max 16384 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 Processor -- start 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 -- start start 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb09f0 0x5608dbeb88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 0x5608dbec1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb2580 0x5608dbeb4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541ae720 con 0x5608dbeb1bc0 2024-03-26T14:14:45.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541ac220 con 0x5608dbeb2580 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf5c177740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541aa2a0 con 0x5608dbeb09f0 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf51e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 0x5608dbec1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf51e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 0x5608dbec1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46844/0 (socket says 172.21.15.137:46844) 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf51e00640 1 -- 172.21.15.137:0/546011229 learned_addr learned my addr 172.21.15.137:0/546011229 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.595+0000 7faf53200640 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb2580 0x5608dbeb4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf52800640 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb09f0 0x5608dbeb88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf51e00640 1 -- 172.21.15.137:0/546011229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb09f0 msgr2=0x5608dbeb88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf51e00640 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb09f0 0x5608dbeb88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf51e00640 1 -- 172.21.15.137:0/546011229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb2580 msgr2=0x5608dbeb4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf51e00640 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb2580 0x5608dbeb4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf51e00640 1 -- 172.21.15.137:0/546011229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608dbec8ae0 con 0x5608dbeb1bc0 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf51e00640 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 0x5608dbec1fc0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7faf40004520 tx=0x7faf4000daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a189ff464f874f1f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf3ea00640 1 -- 172.21.15.137:0/546011229 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400037f0 con 0x5608dbeb1bc0 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf3ea00640 1 -- 172.21.15.137:0/546011229 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf40003e40 con 0x5608dbeb1bc0 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.596+0000 7faf3ea00640 1 -- 172.21.15.137:0/546011229 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400103d0 con 0x5608dbeb1bc0 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 -- 172.21.15.137:0/546011229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 msgr2=0x5608dbec1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 0x5608dbec1fc0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7faf40004520 tx=0x7faf4000daf0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 -- 172.21.15.137:0/546011229 shutdown_connections 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb2580 0x5608dbeb4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb1bc0 0x5608dbec1fc0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 --2- 172.21.15.137:0/546011229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb09f0 0x5608dbeb88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 -- 172.21.15.137:0/546011229 >> 172.21.15.137:0/546011229 conn(0x5608dbeabde0 msgr2=0x5608dbeab160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 -- 172.21.15.137:0/546011229 shutdown_connections 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.597+0000 7faf5c177740 1 -- 172.21.15.137:0/546011229 wait complete. 2024-03-26T14:14:45.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.598+0000 7faf5c177740 1 -- start start 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.598+0000 7faf5c177740 1 Processor -- start 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.598+0000 7faf5c177740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbec0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.598+0000 7faf5c177740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbec0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.598+0000 7faf5c177740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbec12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf5c177740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541aa8a0 con 0x5608dbeb09f0 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf5c177740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541abbc0 con 0x5608dbeb1bc0 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf5c177740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541aaf00 con 0x5608dbeb2580 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf5c177740 10 client.0 _ll_register_callbacks cb 0x7faf4409b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbec0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbec0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46850/0 (socket says 172.21.15.137:46850) 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 -- 172.21.15.137:0/69098637 learned_addr learned my addr 172.21.15.137:0/69098637 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf53200640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbec12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf51e00640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbec0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 -- 172.21.15.137:0/69098637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 msgr2=0x5608dbec12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbec12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 -- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 msgr2=0x5608dbec0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbec0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf52800640 1 -- 172.21.15.137:0/69098637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608dbf218d0 con 0x5608dbeb09f0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.599+0000 7faf51e00640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbec0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf52800640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbec0850 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7faf440a3c00 tx=0x7faf44004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa4f4342ea5e733 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf33e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf33e00640 1 -- 172.21.15.137:0/69098637 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44005310 con 0x5608dbeb09f0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf33e00640 1 -- 172.21.15.137:0/69098637 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7faf440ccc00 con 0x5608dbeb09f0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf40003e40 con 0x5608dbeb09f0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf33e00640 1 -- 172.21.15.137:0/69098637 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf44005680 con 0x5608dbeb09f0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf5c177740 10 client.18234 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5608dbf22c50 con 0x5608dbeb09f0 2024-03-26T14:14:45.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.600+0000 7faf33e00640 1 -- 172.21.15.137:0/69098637 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faf40003e40 con 0x5608dbeb09f0 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf33e00640 1 -- 172.21.15.137:0/69098637 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7faf44098000 con 0x5608dbeb09f0 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf5c177740 10 client.18234 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf5c177740 10 client.18234 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf5c177740 20 client.18234 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf5c177740 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbf23560 0x5608dbf25980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf5c177740 4 client.18234 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5608dbe1cb50 con 0x5608dbf23560 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.601+0000 7faf51e00640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbf23560 0x5608dbf25980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.602+0000 7faf33e00640 10 client.18234 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.602+0000 7faf51e00640 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbf23560 0x5608dbf25980 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8cb979eb97039fd server_cookie=43b2c1178a7c2355 in_seq=0 out_seq=0 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.602+0000 7faf33e00640 1 -- 172.21.15.137:0/69098637 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5608dbe1cb50 con 0x5608dbf23560 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.602+0000 7faf33e00640 10 client.18234 handle_command_reply: tid=0 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.602+0000 7faf5c177740 1 client.18234 shutdown 2024-03-26T14:14:45.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 msgr2=0x5608dbec0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbec0850 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7faf440a3c00 tx=0x7faf44004840 comp rx=0 tx=0).stop 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 shutdown_connections 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5608dbf23560 0x5608dbf25980 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608dbeb1bc0 0x5608dbec0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608dbeb2580 0x5608dbec12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 --2- 172.21.15.137:0/69098637 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5608dbeb09f0 0x5608dbec0850 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf33e00640 0 client.18234 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 >> 172.21.15.137:0/69098637 conn(0x5608dbeabde0 msgr2=0x5608dbeaf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 shutdown_connections 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 1 -- 172.21.15.137:0/69098637 wait complete. 2024-03-26T14:14:45.610 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.603+0000 7faf5c177740 20 client.18234 trim_cache size 0 max 16384 2024-03-26T14:14:45.611 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:14:45.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faf380a2800 msgr2=0x7faf380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faf380a2800 0x7faf380a4cc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5608dbeb29e0 tx=0x7faf4c044000 comp rx=0 tx=0).stop 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 msgr2=0x7faf54199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 0x7faf54199940 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e9f0 tx=0x7faf4400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 shutdown_connections 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7faf380a2800 0x7faf380a4cc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf54105370 0x7faf541a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf541049b0 0x7faf54199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 --2- 172.21.15.137:0/1001141868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf541037c0 0x7faf54199940 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 >> 172.21.15.137:0/1001141868 conn(0x7faf540fed50 msgr2=0x7faf54107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 shutdown_connections 2024-03-26T14:14:45.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.604+0000 7faf58a00640 1 -- 172.21.15.137:0/1001141868 wait complete. 2024-03-26T14:14:45.623 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:14:45.623 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:14:45.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 -- 172.21.15.137:0/4202882874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4102000 msgr2=0x7f3cf41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/4202882874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4102000 0x7f3cf41043f0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009900 tx=0x7f3ce402bc40 comp rx=0 tx=0).stop 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 -- 172.21.15.137:0/4202882874 shutdown_connections 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/4202882874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cf4104930 0x7f3cf4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/4202882874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cf4107260 0x7f3cf41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/4202882874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4102000 0x7f3cf41043f0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 -- 172.21.15.137:0/4202882874 >> 172.21.15.137:0/4202882874 conn(0x7f3cf40fba10 msgr2=0x7f3cf40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 -- 172.21.15.137:0/4202882874 shutdown_connections 2024-03-26T14:14:45.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.840+0000 7f3cf8a00640 1 -- 172.21.15.137:0/4202882874 wait complete. 2024-03-26T14:14:45.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 Processor -- start 2024-03-26T14:14:45.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 -- start start 2024-03-26T14:14:45.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cf4102000 0x7f3cf4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 0x7f3cf4199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cf4107260 0x7f3cf41a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf410d0c0 con 0x7f3cf4104930 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf410cf40 con 0x7f3cf4107260 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf8a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf410d240 con 0x7f3cf4102000 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 0x7f3cf4199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 0x7f3cf4199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46870/0 (socket says 172.21.15.137:46870) 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.842+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3914169937 learned_addr learned my addr 172.21.15.137:0/3914169937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cf4107260 0x7f3cf41a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cf4102000 0x7f3cf4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3914169937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cf4102000 msgr2=0x7f3cf4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cf4102000 0x7f3cf4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3914169937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cf4107260 msgr2=0x7f3cf41a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cf4107260 0x7f3cf41a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3914169937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf41a1570 con 0x7f3cf4104930 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 0x7f3cf4199e90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d770 tx=0x7f3ce000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.843+0000 7f3ceb400640 1 -- 172.21.15.137:0/3914169937 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0017070 con 0x7f3cf4104930 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.844+0000 7f3ceb400640 1 -- 172.21.15.137:0/3914169937 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ce0009c30 con 0x7f3cf4104930 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.844+0000 7f3ceb400640 1 -- 172.21.15.137:0/3914169937 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce000f620 con 0x7f3cf4104930 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.844+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf41a1860 con 0x7f3cf4104930 2024-03-26T14:14:45.847 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.844+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf41a1db0 con 0x7f3cf4104930 2024-03-26T14:14:45.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.847+0000 7f3ceb400640 1 -- 172.21.15.137:0/3914169937 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3ce001a070 con 0x7f3cf4104930 2024-03-26T14:14:45.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 Processor -- start 2024-03-26T14:14:45.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 -- start start 2024-03-26T14:14:45.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f6439f0 0x56344f643e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f644bc0 0x56344f645040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f64be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344f5a82c0 con 0x56344f645580 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344f550c90 con 0x56344f644bc0 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.848+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344f5a4360 con 0x56344f6439f0 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f64be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f64be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46882/0 (socket says 172.21.15.137:46882) 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 -- 172.21.15.137:0/148867054 learned_addr learned my addr 172.21.15.137:0/148867054 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf2800640 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f644bc0 0x56344f645040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf3200640 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f6439f0 0x56344f643e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3ceb400640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cd80a2800 0x7f3cd80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 -- 172.21.15.137:0/148867054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f6439f0 msgr2=0x56344f643e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f6439f0 0x56344f643e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 -- 172.21.15.137:0/148867054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f644bc0 msgr2=0x56344f645040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f644bc0 0x56344f645040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.849+0000 7f3cf1e00640 1 -- 172.21.15.137:0/148867054 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56344f575080 con 0x56344f645580 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.850+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cd80a2800 0x7f3cd80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.850+0000 7f3ceb400640 1 -- 172.21.15.137:0/3914169937 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3ce00cea20 con 0x7f3cf4104930 2024-03-26T14:14:45.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.850+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f64be10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0001ad0 tx=0x7f3ce0023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e634ec29a05c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.850+0000 7f3cdf400640 1 -- 172.21.15.137:0/148867054 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce009aaf0 con 0x56344f645580 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.850+0000 7f3cdf400640 1 -- 172.21.15.137:0/148867054 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ce0001510 con 0x56344f645580 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.850+0000 7f3cdf400640 1 -- 172.21.15.137:0/148867054 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0001700 con 0x56344f645580 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/148867054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 msgr2=0x56344f64be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cd80a2800 0x7f3cd80a4cc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x56344f644ac0 tx=0x7f3cec004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f64be10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0001ad0 tx=0x7f3ce0023000 comp rx=0 tx=0).stop 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/148867054 shutdown_connections 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f644bc0 0x56344f645040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f64be10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/148867054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f6439f0 0x56344f643e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/148867054 >> 172.21.15.137:0/148867054 conn(0x56344f63ede0 msgr2=0x56344f63f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/148867054 shutdown_connections 2024-03-26T14:14:45.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.851+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/148867054 wait complete. 2024-03-26T14:14:45.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.852+0000 7f3cfc1bf740 1 -- start start 2024-03-26T14:14:45.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.852+0000 7f3cfc1bf740 1 Processor -- start 2024-03-26T14:14:45.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.852+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f64db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.852+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 0x56344f64e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.852+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f645580 0x56344f6550c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.852+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344f64c530 con 0x56344f644bc0 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344f5640d0 con 0x56344f6439f0 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344f64c6b0 con 0x56344f645580 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cfc1bf740 10 client.0 _ll_register_callbacks cb 0x56344f568e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 0x56344f64e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 0x56344f64e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46898/0 (socket says 172.21.15.137:46898) 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3845466258 learned_addr learned my addr 172.21.15.137:0/3845466258 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f64db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f645580 0x56344f6550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3845466258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f645580 msgr2=0x56344f6550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f645580 0x56344f6550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 msgr2=0x56344f64db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f64db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf1e00640 1 -- 172.21.15.137:0/3845466258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56344f2bc3f0 con 0x56344f644bc0 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.853+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f64db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 0x56344f64e080 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00018d0 tx=0x7f3ce009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a0a10ab4d899f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3845466258 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0001900 con 0x56344f644bc0 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3845466258 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3ce00d4180 con 0x56344f644bc0 2024-03-26T14:14:45.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56344f5b5800 con 0x56344f644bc0 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3845466258 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ce00b7020 con 0x56344f644bc0 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cfc1bf740 10 client.18246 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56344f3867d0 con 0x56344f644bc0 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.854+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3845466258 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56344f5b5800 con 0x56344f644bc0 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3845466258 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3ce00cc190 con 0x56344f644bc0 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cfc1bf740 10 client.18246 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cfc1bf740 10 client.18246 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cfc1bf740 20 client.18246 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f659220 0x56344f65b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cfc1bf740 4 client.18246 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:45.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56344f5afb50 con 0x56344f659220 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.855+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f659220 0x56344f65b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.856+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f659220 0x56344f65b610 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bdff9538b5627a8 server_cookie=35b787947170312 in_seq=0 out_seq=0 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.856+0000 7f3cd3e00640 10 client.18246 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3845466258 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56344f5afb50 con 0x56344f659220 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cd3e00640 10 client.18246 handle_command_reply: tid=0 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 client.18246 shutdown 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 msgr2=0x56344f64e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 0x56344f64e080 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f3ce00018d0 tx=0x7f3ce009ae70 comp rx=0 tx=0).stop 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 shutdown_connections 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f659220 0x56344f65b610 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f645580 0x56344f6550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f644bc0 0x56344f64e080 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3845466258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f64db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 >> 172.21.15.137:0/3845466258 conn(0x56344f63ede0 msgr2=0x56344f63e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cd3e00640 0 client.18246 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 shutdown_connections 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3845466258 wait complete. 2024-03-26T14:14:45.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.858+0000 7f3cfc1bf740 20 client.18246 trim_cache size 0 max 16384 2024-03-26T14:14:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 Processor -- start 2024-03-26T14:14:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 -- start start 2024-03-26T14:14:45.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 0x56344f64d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f655080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f645580 0x56344f63dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41ae710 con 0x56344f6439f0 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41ac210 con 0x56344f645580 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.873+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41aa290 con 0x56344f644bc0 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 0x56344f64d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 0x56344f64d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46900/0 (socket says 172.21.15.137:46900) 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 -- 172.21.15.137:0/3386542027 learned_addr learned my addr 172.21.15.137:0/3386542027 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f645580 0x56344f63dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f655080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 -- 172.21.15.137:0/3386542027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 msgr2=0x56344f655080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f655080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 -- 172.21.15.137:0/3386542027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f645580 msgr2=0x56344f63dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f645580 0x56344f63dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 -- 172.21.15.137:0/3386542027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56344f644640 con 0x56344f6439f0 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f655080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.874+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 0x56344f64d160 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4000c00 tx=0x7f3ce402c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf95e15713af388 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.875+0000 7f3cdea00640 1 -- 172.21.15.137:0/3386542027 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce404d020 con 0x56344f6439f0 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.875+0000 7f3cdea00640 1 -- 172.21.15.137:0/3386542027 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ce402cc90 con 0x56344f6439f0 2024-03-26T14:14:45.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.875+0000 7f3cdea00640 1 -- 172.21.15.137:0/3386542027 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40046c0 con 0x56344f6439f0 2024-03-26T14:14:45.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3386542027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 msgr2=0x56344f64d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 0x56344f64d160 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4000c00 tx=0x7f3ce402c810 comp rx=0 tx=0).stop 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3386542027 shutdown_connections 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f645580 0x56344f63dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f655080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3386542027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f6439f0 0x56344f64d160 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3386542027 >> 172.21.15.137:0/3386542027 conn(0x56344f63ede0 msgr2=0x56344f63d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3386542027 shutdown_connections 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3386542027 wait complete. 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.876+0000 7f3cfc1bf740 1 -- start start 2024-03-26T14:14:45.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 Processor -- start 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f652c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f653150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f654160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41aa890 con 0x56344f645580 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41abbb0 con 0x56344f6439f0 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf41aaef0 con 0x56344f644bc0 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cfc1bf740 10 client.0 _ll_register_callbacks cb 0x7f3ce4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.877+0000 7f3cf3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f654160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f654160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46912/0 (socket says 172.21.15.137:46912) 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 -- 172.21.15.137:0/3676043058 learned_addr learned my addr 172.21.15.137:0/3676043058 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:45.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f652c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf1e00640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f653150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 -- 172.21.15.137:0/3676043058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 msgr2=0x56344f653150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f653150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 -- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 msgr2=0x56344f652c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f652c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 -- 172.21.15.137:0/3676043058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56344f654750 con 0x56344f645580 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f652c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cf3200640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f654160 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3cec012af0 tx=0x7f3cec0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dfbdcc90cf4a5c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.878+0000 7f3cd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3676043058 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec02c060 con 0x56344f645580 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56344f64d930 con 0x56344f645580 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3676043058 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3cec01bd80 con 0x56344f645580 2024-03-26T14:14:45.884 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3676043058 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cec033050 con 0x56344f645580 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cfc1bf740 10 client.18252 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56344f6b4ee0 con 0x56344f645580 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.879+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3676043058 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56344f64d930 con 0x56344f645580 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3676043058 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3cec026030 con 0x56344f645580 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cfc1bf740 10 client.18252 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cfc1bf740 10 client.18252 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cfc1bf740 20 client.18252 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f6b5830 0x56344f6b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cfc1bf740 4 client.18252 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56344f52d500 con 0x56344f6b5830 2024-03-26T14:14:45.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.880+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f6b5830 0x56344f6b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:45.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.881+0000 7f3cf2800640 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f6b5830 0x56344f6b7cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe75247f7fccb9 server_cookie=fedeac9f9eb79dd2 in_seq=0 out_seq=0 2024-03-26T14:14:45.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.881+0000 7f3cd3e00640 10 client.18252 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:45.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.881+0000 7f3cd3e00640 1 -- 172.21.15.137:0/3676043058 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56344f52d500 con 0x56344f6b5830 2024-03-26T14:14:45.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.881+0000 7f3cd3e00640 10 client.18252 handle_command_reply: tid=0 2024-03-26T14:14:45.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.881+0000 7f3cfc1bf740 1 client.18252 shutdown 2024-03-26T14:14:45.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 msgr2=0x56344f654160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f654160 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f3cec012af0 tx=0x7f3cec0074f0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 shutdown_connections 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x56344f6b5830 0x56344f6b7cb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56344f645580 0x56344f654160 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56344f644bc0 0x56344f653150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 --2- 172.21.15.137:0/3676043058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56344f6439f0 0x56344f652c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cd3e00640 0 client.18252 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 >> 172.21.15.137:0/3676043058 conn(0x56344f63ede0 msgr2=0x56344f642040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 shutdown_connections 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 1 -- 172.21.15.137:0/3676043058 wait complete. 2024-03-26T14:14:45.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.882+0000 7f3cfc1bf740 20 client.18252 trim_cache size 0 max 16384 2024-03-26T14:14:45.889 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:14:45.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cd80a2800 msgr2=0x7f3cd80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cd80a2800 0x7f3cd80a4cc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x56344f644ac0 tx=0x7f3cec004a40 comp rx=0 tx=0).stop 2024-03-26T14:14:45.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 msgr2=0x7f3cf4199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 0x7f3cf4199e90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d770 tx=0x7f3ce000dc30 comp rx=0 tx=0).stop 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 shutdown_connections 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3cd80a2800 0x7f3cd80a4cc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cf4107260 0x7f3cf41a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cf4104930 0x7f3cf4199e90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 --2- 172.21.15.137:0/3914169937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3cf4102000 0x7f3cf4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 >> 172.21.15.137:0/3914169937 conn(0x7f3cf40fba10 msgr2=0x7f3cf41016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 shutdown_connections 2024-03-26T14:14:45.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:45.883+0000 7f3cf8a00640 1 -- 172.21.15.137:0/3914169937 wait complete. 2024-03-26T14:14:45.900 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:45 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1977756562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:14:45.903 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:14:45.903 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:14:46.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.100+0000 7f7e71e00640 1 -- 172.21.15.137:0/2151751853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c0756e0 msgr2=0x7f7e6c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.100+0000 7f7e71e00640 1 --2- 172.21.15.137:0/2151751853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c073b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009930 tx=0x7f7e6002bc30 comp rx=0 tx=0).stop 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 -- 172.21.15.137:0/2151751853 shutdown_connections 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 --2- 172.21.15.137:0/2151751853 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c10db70 0x7f7e6c10ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 --2- 172.21.15.137:0/2151751853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c074100 0x7f7e6c074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 --2- 172.21.15.137:0/2151751853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c073b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 -- 172.21.15.137:0/2151751853 >> 172.21.15.137:0/2151751853 conn(0x7f7e6c0fcb10 msgr2=0x7f7e6c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 -- 172.21.15.137:0/2151751853 shutdown_connections 2024-03-26T14:14:46.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.101+0000 7f7e71e00640 1 -- 172.21.15.137:0/2151751853 wait complete. 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 Processor -- start 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 -- start start 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 0x7f7e6c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c10db70 0x7f7e6c1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1136a0 con 0x7f7e6c074100 2024-03-26T14:14:46.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c113520 con 0x7f7e6c10db70 2024-03-26T14:14:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.102+0000 7f7e71e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c113820 con 0x7f7e6c0756e0 2024-03-26T14:14:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 0x7f7e6c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 0x7f7e6c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46926/0 (socket says 172.21.15.137:46926) 2024-03-26T14:14:46.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 -- 172.21.15.137:0/3990381600 learned_addr learned my addr 172.21.15.137:0/3990381600 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e71400640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c10db70 0x7f7e6c1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e6be00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 -- 172.21.15.137:0/3990381600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c0756e0 msgr2=0x7f7e6c19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 -- 172.21.15.137:0/3990381600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c10db70 msgr2=0x7f7e6c1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c10db70 0x7f7e6c1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e70a00640 1 -- 172.21.15.137:0/3990381600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c1a5a40 con 0x7f7e6c074100 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.103+0000 7f7e6be00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.104+0000 7f7e70a00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 0x7f7e6c19de20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009230 tx=0x7f7e60005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.104+0000 7f7e69600640 1 -- 172.21.15.137:0/3990381600 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60043020 con 0x7f7e6c074100 2024-03-26T14:14:46.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.104+0000 7f7e69600640 1 -- 172.21.15.137:0/3990381600 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e600043d0 con 0x7f7e6c074100 2024-03-26T14:14:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.104+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c1a5cd0 con 0x7f7e6c074100 2024-03-26T14:14:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.104+0000 7f7e69600640 1 -- 172.21.15.137:0/3990381600 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e60003af0 con 0x7f7e6c074100 2024-03-26T14:14:46.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.104+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c1a61c0 con 0x7f7e6c074100 2024-03-26T14:14:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 Processor -- start 2024-03-26T14:14:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e69600640 1 -- 172.21.15.137:0/3990381600 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7e60046020 con 0x7f7e6c074100 2024-03-26T14:14:46.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 -- start start 2024-03-26T14:14:46.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bc6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8b232c0 con 0x5603c8bbe9f0 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8acbc90 con 0x5603c8bc0580 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.108+0000 7f7e76038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8b1f360 con 0x5603c8bbfbc0 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bbee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46934/0 (socket says 172.21.15.137:46934) 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 -- 172.21.15.137:0/10029778 learned_addr learned my addr 172.21.15.137:0/10029778 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e6be00640 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bc6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e70a00640 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 -- 172.21.15.137:0/10029778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbfbc0 msgr2=0x5603c8bc0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 -- 172.21.15.137:0/10029778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 msgr2=0x5603c8bc6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.111 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bc6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e71400640 1 -- 172.21.15.137:0/10029778 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c8af0080 con 0x5603c8bbe9f0 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.109+0000 7f7e6be00640 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bc6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e71400640 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bbee10 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009fd0 tx=0x7f7e64010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33073c17f3d85e94 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e69600640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e540a2880 0x7f7e540a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e5dc00640 1 -- 172.21.15.137:0/10029778 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64011700 con 0x5603c8bbe9f0 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e69600640 1 -- 172.21.15.137:0/3990381600 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7e600f1120 con 0x7f7e6c074100 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e5dc00640 1 -- 172.21.15.137:0/10029778 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e64011d50 con 0x5603c8bbe9f0 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e5dc00640 1 -- 172.21.15.137:0/10029778 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6401ad60 con 0x5603c8bbe9f0 2024-03-26T14:14:46.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.110+0000 7f7e70a00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e540a2880 0x7f7e540a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 -- 172.21.15.137:0/10029778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 msgr2=0x5603c8bbee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bbee10 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009fd0 tx=0x7f7e64010860 comp rx=0 tx=0).stop 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e70a00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e540a2880 0x7f7e540a4d40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e600b9b10 tx=0x7f7e60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 -- 172.21.15.137:0/10029778 shutdown_connections 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bc6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 --2- 172.21.15.137:0/10029778 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bbee10 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 -- 172.21.15.137:0/10029778 >> 172.21.15.137:0/10029778 conn(0x5603c8bb9de0 msgr2=0x5603c8bba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 -- 172.21.15.137:0/10029778 shutdown_connections 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.111+0000 7f7e76038740 1 -- 172.21.15.137:0/10029778 wait complete. 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.112+0000 7f7e76038740 1 -- start start 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.112+0000 7f7e76038740 1 Processor -- start 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.112+0000 7f7e76038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bc8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.112+0000 7f7e76038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e76038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 0x5603c8bd0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e76038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8bc7590 con 0x5603c8bc0580 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e76038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8adf0d0 con 0x5603c8bbfbc0 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e76038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603c8bc7710 con 0x5603c8bbe9f0 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e76038740 10 client.0 _ll_register_callbacks cb 0x5603c8ae3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e71400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 0x5603c8bd0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e71400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 0x5603c8bd0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46942/0 (socket says 172.21.15.137:46942) 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e71400640 1 -- 172.21.15.137:0/2273670108 learned_addr learned my addr 172.21.15.137:0/2273670108 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e71400640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bc8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e6be00640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e71400640 1 -- 172.21.15.137:0/2273670108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 msgr2=0x5603c8bc8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.113+0000 7f7e71400640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bc8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e71400640 1 -- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 msgr2=0x5603c8bc91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e71400640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e71400640 1 -- 172.21.15.137:0/2273670108 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603c88373f0 con 0x5603c8bc0580 2024-03-26T14:14:46.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e71400640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 0x5603c8bd0210 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7e6401ad60 tx=0x7f7e64013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3522078fc5ea6d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:46.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e52000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:46.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e52000640 1 -- 172.21.15.137:0/2273670108 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64004a40 con 0x5603c8bc0580 2024-03-26T14:14:46.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e52000640 1 -- 172.21.15.137:0/2273670108 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7e64004be0 con 0x5603c8bc0580 2024-03-26T14:14:46.117 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e64011d50 con 0x5603c8bc0580 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e52000640 1 -- 172.21.15.137:0/2273670108 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e64030050 con 0x5603c8bc0580 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.114+0000 7f7e76038740 10 client.18264 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5603c89017d0 con 0x5603c8bc0580 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e52000640 1 -- 172.21.15.137:0/2273670108 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e64011d50 con 0x5603c8bc0580 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e52000640 1 -- 172.21.15.137:0/2273670108 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7e64005170 con 0x5603c8bc0580 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 10 client.18264 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 20 client.18264 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8bd6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 4 client.18264 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.115+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603c8b30800 con 0x5603c8bd4560 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.116+0000 7f7e6be00640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8bd6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.117+0000 7f7e6be00640 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8bd6950 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6a3ef12aecd89e server_cookie=1d94a6b21e45e718 in_seq=0 out_seq=0 2024-03-26T14:14:46.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.117+0000 7f7e52000640 10 client.18264 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:46.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e52000640 1 -- 172.21.15.137:0/2273670108 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5603c8b30800 con 0x5603c8bd4560 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e52000640 10 client.18264 handle_command_reply: tid=0 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 client.18264 shutdown 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 msgr2=0x5603c8bd0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 0x5603c8bd0210 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f7e6401ad60 tx=0x7f7e64013040 comp rx=0 tx=0).stop 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 shutdown_connections 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8bd6950 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bc91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e52000640 0 client.18264 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bc0580 0x5603c8bd0210 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 --2- 172.21.15.137:0/2273670108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bc8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 >> 172.21.15.137:0/2273670108 conn(0x5603c8bb9de0 msgr2=0x5603c8bb9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.118+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 shutdown_connections 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.119+0000 7f7e76038740 1 -- 172.21.15.137:0/2273670108 wait complete. 2024-03-26T14:14:46.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.119+0000 7f7e76038740 20 client.18264 trim_cache size 0 max 16384 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 Processor -- start 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 -- start start 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bdb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 0x5603c8bd01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bdb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1b2b10 con 0x5603c8bbfbc0 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.136+0000 7f7e76038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1b0610 con 0x5603c8bc0580 2024-03-26T14:14:46.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e76038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1ae690 con 0x5603c8bbe9f0 2024-03-26T14:14:46.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 0x5603c8bd01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 0x5603c8bd01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46958/0 (socket says 172.21.15.137:46958) 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 -- 172.21.15.137:0/2071253938 learned_addr learned my addr 172.21.15.137:0/2071253938 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e71400640 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bdb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e71400640 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bdb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 -- 172.21.15.137:0/2071253938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 msgr2=0x5603c8bdb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bdb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 -- 172.21.15.137:0/2071253938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 msgr2=0x5603c8bdb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bdb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e6be00640 1 -- 172.21.15.137:0/2071253938 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603c8bd6f30 con 0x5603c8bbfbc0 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.137+0000 7f7e71400640 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bdb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.138+0000 7f7e6be00640 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 0x5603c8bd01a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c19f330 tx=0x7f7e580044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3432545bd7b82f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.138+0000 7f7e5d200640 1 -- 172.21.15.137:0/2071253938 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58022020 con 0x5603c8bbfbc0 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.138+0000 7f7e5d200640 1 -- 172.21.15.137:0/2071253938 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e58003da0 con 0x5603c8bbfbc0 2024-03-26T14:14:46.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.138+0000 7f7e5d200640 1 -- 172.21.15.137:0/2071253938 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58012a10 con 0x5603c8bbfbc0 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 -- 172.21.15.137:0/2071253938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 msgr2=0x5603c8bd01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 0x5603c8bd01a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c19f330 tx=0x7f7e580044d0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 -- 172.21.15.137:0/2071253938 shutdown_connections 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bc0580 0x5603c8bdb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbfbc0 0x5603c8bd01a0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 --2- 172.21.15.137:0/2071253938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bbe9f0 0x5603c8bdb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 -- 172.21.15.137:0/2071253938 >> 172.21.15.137:0/2071253938 conn(0x5603c8bb9de0 msgr2=0x5603c8bbd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:46.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 -- 172.21.15.137:0/2071253938 shutdown_connections 2024-03-26T14:14:46.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 -- 172.21.15.137:0/2071253938 wait complete. 2024-03-26T14:14:46.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.139+0000 7f7e76038740 1 -- start start 2024-03-26T14:14:46.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 Processor -- start 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bcbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bcc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bc0580 0x5603c8bcf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1aec90 con 0x5603c8bbe9f0 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1affb0 con 0x5603c8bbfbc0 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1af2f0 con 0x5603c8bc0580 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e76038740 10 client.0 _ll_register_callbacks cb 0x7f7e64009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e71400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bcbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e71400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bcbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46964/0 (socket says 172.21.15.137:46964) 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.140+0000 7f7e71400640 1 -- 172.21.15.137:0/3514549072 learned_addr learned my addr 172.21.15.137:0/3514549072 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e6be00640 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bcc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e71400640 1 -- 172.21.15.137:0/3514549072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bc0580 msgr2=0x5603c8bcf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e71400640 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bc0580 0x5603c8bcf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e71400640 1 -- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 msgr2=0x5603c8bcc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e71400640 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bcc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e71400640 1 -- 172.21.15.137:0/3514549072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603c8bcf900 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e71400640 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bcbdb0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7e64005610 tx=0x7f7e64009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17e5fe1599a8ce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e52000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e52000640 1 -- 172.21.15.137:0/3514549072 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640059b0 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.141+0000 7f7e52000640 1 -- 172.21.15.137:0/3514549072 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7e64027040 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e58003da0 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e52000640 1 -- 172.21.15.137:0/3514549072 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e64044050 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 10 client.18270 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5603c8c30290 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e52000640 1 -- 172.21.15.137:0/3514549072 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e58003da0 con 0x5603c8bbe9f0 2024-03-26T14:14:46.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e52000640 1 -- 172.21.15.137:0/3514549072 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f7e64004f70 con 0x5603c8bbe9f0 2024-03-26T14:14:46.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 10 client.18270 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:46.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:46.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 20 client.18270 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:46.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8c32c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:46.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 4 client.18270 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:14:46.145 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.142+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5603c8b2ab50 con 0x5603c8bd4560 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.143+0000 7f7e6be00640 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8c32c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e52000640 10 client.18270 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e6be00640 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8c32c50 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e543a484e2c1191a server_cookie=53508136c765daf5 in_seq=0 out_seq=0 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e52000640 1 -- 172.21.15.137:0/3514549072 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5603c8b2ab50 con 0x5603c8bd4560 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e52000640 10 client.18270 handle_command_reply: tid=0 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e76038740 1 client.18270 shutdown 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 msgr2=0x5603c8bcbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.146 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.144+0000 7f7e76038740 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bcbdb0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7e64005610 tx=0x7f7e64009ae0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 shutdown_connections 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5603c8bd4560 0x5603c8c32c50 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603c8bbfbc0 0x5603c8bcc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603c8bc0580 0x5603c8bcf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 --2- 172.21.15.137:0/3514549072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5603c8bbe9f0 0x5603c8bcbdb0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 >> 172.21.15.137:0/3514549072 conn(0x5603c8bb9de0 msgr2=0x5603c8bc2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e52000640 0 client.18270 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 shutdown_connections 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 1 -- 172.21.15.137:0/3514549072 wait complete. 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.145+0000 7f7e76038740 20 client.18270 trim_cache size 0 max 16384 2024-03-26T14:14:46.147 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e540a2880 msgr2=0x7f7e540a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e540a2880 0x7f7e540a4d40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7e600b9b10 tx=0x7f7e60009260 comp rx=0 tx=0).stop 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 msgr2=0x7f7e6c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 0x7f7e6c19de20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009230 tx=0x7f7e60005e40 comp rx=0 tx=0).stop 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 shutdown_connections 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e540a2880 0x7f7e540a4d40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e6c10db70 0x7f7e6c1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e6c0756e0 0x7f7e6c19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 --2- 172.21.15.137:0/3990381600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e6c074100 0x7f7e6c19de20 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 >> 172.21.15.137:0/3990381600 conn(0x7f7e6c0fcb10 msgr2=0x7f7e6c100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 shutdown_connections 2024-03-26T14:14:46.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:46.146+0000 7f7e71e00640 1 -- 172.21.15.137:0/3990381600 wait complete. 2024-03-26T14:14:46.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:45 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1977756562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:14:46.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:45 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1977756562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:14:47.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:46 smithi137 ceph-mon[31539]: pgmap v830: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:47.169 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:14:47.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:14:47.170 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:14:47.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:46 smithi179 ceph-mon[35472]: pgmap v830: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:47.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:46 smithi159 ceph-mon[35501]: pgmap v830: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:47.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 -- 172.21.15.137:0/1685482976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 msgr2=0x7f71a8104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 --2- 172.21.15.137:0/1685482976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8104510 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7198009900 tx=0x7f719802bc30 comp rx=0 tx=0).stop 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 -- 172.21.15.137:0/1685482976 shutdown_connections 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 --2- 172.21.15.137:0/1685482976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a8104a50 0x7f71a8106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 --2- 172.21.15.137:0/1685482976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8107380 0x7f71a81097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 --2- 172.21.15.137:0/1685482976 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8104510 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 -- 172.21.15.137:0/1685482976 >> 172.21.15.137:0/1685482976 conn(0x7f71a80fbb10 msgr2=0x7f71a80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 -- 172.21.15.137:0/1685482976 shutdown_connections 2024-03-26T14:14:47.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.429+0000 7f71aec00640 1 -- 172.21.15.137:0/1685482976 wait complete. 2024-03-26T14:14:47.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 Processor -- start 2024-03-26T14:14:47.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 -- start start 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8104a50 0x7f71a819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a8107380 0x7f71a81a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a810d230 con 0x7f71a8102120 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a810d0b0 con 0x7f71a8107380 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71aec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a810d3b0 con 0x7f71a8104a50 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.431+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8199b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:46978/0 (socket says 172.21.15.137:46978) 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 -- 172.21.15.137:0/1719369101 learned_addr learned my addr 172.21.15.137:0/1719369101 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71acc00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a8107380 0x7f71a81a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7400640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8104a50 0x7f71a819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 -- 172.21.15.137:0/1719369101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8104a50 msgr2=0x7f71a819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8104a50 0x7f71a819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 -- 172.21.15.137:0/1719369101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a8107380 msgr2=0x7f71a81a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a8107380 0x7f71a81a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 -- 172.21.15.137:0/1719369101 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a81a17a0 con 0x7f71a8102120 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7400640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8104a50 0x7f71a819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.432+0000 7f71a7e00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8199b80 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7198000c00 tx=0x7f71980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.433+0000 7f71a4c00640 1 -- 172.21.15.137:0/1719369101 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198043020 con 0x7f71a8102120 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.433+0000 7f71a4c00640 1 -- 172.21.15.137:0/1719369101 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7198004aa0 con 0x7f71a8102120 2024-03-26T14:14:47.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.433+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a81a1a30 con 0x7f71a8102120 2024-03-26T14:14:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.433+0000 7f71a4c00640 1 -- 172.21.15.137:0/1719369101 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198003d70 con 0x7f71a8102120 2024-03-26T14:14:47.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.433+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a81a1f80 con 0x7f71a8102120 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.436+0000 7f71b19ab740 1 Processor -- start 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 -- start start 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5d84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5dd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435f53deb0 con 0x56435f5d8a90 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435f4e6a60 con 0x56435f5ddc00 2024-03-26T14:14:47.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71b19ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435f539f50 con 0x56435f5d80a0 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:56138/0 (socket says 172.21.15.137:56138) 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.437+0000 7f71a7400640 1 -- 172.21.15.137:0/2808602722 learned_addr learned my addr 172.21.15.137:0/2808602722 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71acc00640 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5d84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71a4c00640 1 -- 172.21.15.137:0/1719369101 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7198046020 con 0x7f71a8102120 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71a7400640 1 -- 172.21.15.137:0/2808602722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 msgr2=0x56435f5d84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71a7400640 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5d84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71a7400640 1 -- 172.21.15.137:0/2808602722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 msgr2=0x56435f5dd590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71a7400640 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5dd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.438+0000 7f71a7400640 1 -- 172.21.15.137:0/2808602722 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435f31a7e0 con 0x56435f5ddc00 2024-03-26T14:14:47.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.439+0000 7f71a7400640 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e0090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f7194010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7ea06af4b17cc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.439+0000 7f719d200640 1 -- 172.21.15.137:0/2808602722 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194004030 con 0x56435f5ddc00 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.439+0000 7f719d200640 1 -- 172.21.15.137:0/2808602722 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71940035f0 con 0x56435f5ddc00 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.439+0000 7f719d200640 1 -- 172.21.15.137:0/2808602722 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71940126b0 con 0x56435f5ddc00 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71a4c00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f718c0a2800 0x7f718c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 -- 172.21.15.137:0/2808602722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 msgr2=0x56435f5e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e0090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f7194010740 comp rx=0 tx=0).stop 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71a4c00640 1 -- 172.21.15.137:0/1719369101 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f71980f11b0 con 0x7f71a8102120 2024-03-26T14:14:47.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71acc00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f718c0a2800 0x7f718c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 -- 172.21.15.137:0/2808602722 shutdown_connections 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e0090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5dd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2808602722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5d84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 -- 172.21.15.137:0/2808602722 >> 172.21.15.137:0/2808602722 conn(0x56435f5d3610 msgr2=0x56435f5d3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.440+0000 7f71b19ab740 1 -- 172.21.15.137:0/2808602722 shutdown_connections 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.441+0000 7f71b19ab740 1 -- 172.21.15.137:0/2808602722 wait complete. 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.441+0000 7f71acc00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f718c0a2800 0x7f718c0a4cc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x56435f5d92f0 tx=0x7f71a0009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.441+0000 7f71b19ab740 1 -- start start 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.441+0000 7f71b19ab740 1 Processor -- start 2024-03-26T14:14:47.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435f5e0bb0 con 0x56435f5d80a0 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435f4f9eb0 con 0x56435f5ddc00 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435f5e0d30 con 0x56435f5d8a90 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71b19ab740 10 client.0 _ll_register_callbacks cb 0x56435f4ee150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5e2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47000/0 (socket says 172.21.15.137:47000) 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71a7e00640 1 -- 172.21.15.137:0/2927017621 learned_addr learned my addr 172.21.15.137:0/2927017621 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.442+0000 7f71acc00640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7400640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7e00640 1 -- 172.21.15.137:0/2927017621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 msgr2=0x56435f5e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7e00640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7e00640 1 -- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 msgr2=0x56435f5e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7e00640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7e00640 1 -- 172.21.15.137:0/2927017621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56435f37af70 con 0x56435f5d80a0 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71acc00640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71a7e00640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5e2310 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f71980f1490 tx=0x7f71980bd2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd46da14781a713 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f7191600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f7191600640 1 -- 172.21.15.137:0/2927017621 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71980d6060 con 0x56435f5d80a0 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f7191600640 1 -- 172.21.15.137:0/2927017621 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f71980bebd0 con 0x56435f5d80a0 2024-03-26T14:14:47.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.443+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71940035f0 con 0x56435f5d80a0 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f7191600640 1 -- 172.21.15.137:0/2927017621 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71980dc170 con 0x56435f5d80a0 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 10 client.18279 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56435f212170 con 0x56435f5d80a0 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f7191600640 1 -- 172.21.15.137:0/2927017621 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71940035f0 con 0x56435f5d80a0 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f7191600640 1 -- 172.21.15.137:0/2927017621 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f71980c5bb0 con 0x56435f5d80a0 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 10 client.18279 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 20 client.18279 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f5ee070 0x56435f5f0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.450 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 4 client.18279 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.444+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56435f54b3f0 con 0x56435f5ee070 2024-03-26T14:14:47.451 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.445+0000 7f71a7400640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f5ee070 0x56435f5f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.446+0000 7f71a7400640 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f5ee070 0x56435f5f0480 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ab5ec0fd087f8c server_cookie=34c2c3d4ea8353fd in_seq=0 out_seq=0 2024-03-26T14:14:47.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.446+0000 7f7191600640 10 client.18279 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.447+0000 7f7191600640 1 -- 172.21.15.137:0/2927017621 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56435f54b3f0 con 0x56435f5ee070 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.447+0000 7f7191600640 10 client.18279 handle_command_reply: tid=0 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.447+0000 7f71b19ab740 1 client.18279 shutdown 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 msgr2=0x56435f5e2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5e2310 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f71980f1490 tx=0x7f71980bd2c0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 shutdown_connections 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f5ee070 0x56435f5f0480 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f7191600640 0 client.18279 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.453 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2927017621 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5e2310 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 >> 172.21.15.137:0/2927017621 conn(0x56435f5d3610 msgr2=0x56435f5d2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 shutdown_connections 2024-03-26T14:14:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 1 -- 172.21.15.137:0/2927017621 wait complete. 2024-03-26T14:14:47.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.448+0000 7f71b19ab740 20 client.18279 trim_cache size 0 max 16384 2024-03-26T14:14:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 Processor -- start 2024-03-26T14:14:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 -- start start 2024-03-26T14:14:47.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81ae930 con 0x56435f5d80a0 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81ac430 con 0x56435f5ddc00 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.472+0000 7f71b19ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81aa4b0 con 0x56435f5d8a90 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71acc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47006/0 (socket says 172.21.15.137:47006) 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71acc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5d2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:56162/0 (socket says 172.21.15.137:56162) 2024-03-26T14:14:47.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 -- 172.21.15.137:0/2904299540 learned_addr learned my addr 172.21.15.137:0/2904299540 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7400640 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 -- 172.21.15.137:0/2904299540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 msgr2=0x56435f5d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 -- 172.21.15.137:0/2904299540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 msgr2=0x56435f5d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7e00640 1 -- 172.21.15.137:0/2904299540 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435f5cec20 con 0x56435f5d80a0 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.473+0000 7f71a7400640 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.474+0000 7f71a7e00640 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5d1fa0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f71980ba270 tx=0x7f71980bd330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d094eb887f965566 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.474+0000 7f7193e00640 1 -- 172.21.15.137:0/2904299540 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198086020 con 0x56435f5d80a0 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.474+0000 7f7193e00640 1 -- 172.21.15.137:0/2904299540 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71980dc170 con 0x56435f5d80a0 2024-03-26T14:14:47.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.474+0000 7f7193e00640 1 -- 172.21.15.137:0/2904299540 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71980c6040 con 0x56435f5d80a0 2024-03-26T14:14:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 -- 172.21.15.137:0/2904299540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 msgr2=0x56435f5d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5d1fa0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f71980ba270 tx=0x7f71980bd330 comp rx=0 tx=0).stop 2024-03-26T14:14:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 -- 172.21.15.137:0/2904299540 shutdown_connections 2024-03-26T14:14:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d8a90 0x56435f5d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 --2- 172.21.15.137:0/2904299540 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d80a0 0x56435f5d1fa0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 -- 172.21.15.137:0/2904299540 >> 172.21.15.137:0/2904299540 conn(0x56435f5d3610 msgr2=0x56435f5d7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 -- 172.21.15.137:0/2904299540 shutdown_connections 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 -- 172.21.15.137:0/2904299540 wait complete. 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 -- start start 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.475+0000 7f71b19ab740 1 Processor -- start 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.481 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81aaab0 con 0x56435f5d8a90 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81abdd0 con 0x56435f5ddc00 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81ab110 con 0x56435f5d80a0 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71b19ab740 10 client.0 _ll_register_callbacks cb 0x7f71980bd6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71a7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71a7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5e8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47018/0 (socket says 172.21.15.137:47018) 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71a7400640 1 -- 172.21.15.137:0/3447404746 learned_addr learned my addr 172.21.15.137:0/3447404746 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71a7e00640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.476+0000 7f71acc00640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71a7400640 1 -- 172.21.15.137:0/3447404746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 msgr2=0x56435f5e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71a7400640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.482 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71a7400640 1 -- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 msgr2=0x56435f5e70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71a7400640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71a7400640 1 -- 172.21.15.137:0/3447404746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56435f5e7920 con 0x56435f5d8a90 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71a7400640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5e8f60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f7194005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11553a961f9e04ca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f7191600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f7191600640 1 -- 172.21.15.137:0/3447404746 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194009730 con 0x56435f5d8a90 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71980dc170 con 0x56435f5d8a90 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.477+0000 7f7191600640 1 -- 172.21.15.137:0/3447404746 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f71940050b0 con 0x56435f5d8a90 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f7191600640 1 -- 172.21.15.137:0/3447404746 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f719403a050 con 0x56435f5d8a90 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 10 client.18285 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56435f64a570 con 0x56435f5d8a90 2024-03-26T14:14:47.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f7191600640 1 -- 172.21.15.137:0/3447404746 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71980dc170 con 0x56435f5d8a90 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f7191600640 1 -- 172.21.15.137:0/3447404746 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f719400f040 con 0x56435f5d8a90 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 10 client.18285 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 10 client.18285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 20 client.18285 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f64ac70 0x56435f64d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 4 client.18285 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.478+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56435f545740 con 0x56435f64ac70 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.479+0000 7f71a7e00640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f64ac70 0x56435f64d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.479+0000 7f71a7e00640 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f64ac70 0x56435f64d090 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb37658276f8f30b server_cookie=fa8390f23d37d0b3 in_seq=0 out_seq=0 2024-03-26T14:14:47.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.480+0000 7f7191600640 10 client.18285 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f7191600640 1 -- 172.21.15.137:0/3447404746 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56435f545740 con 0x56435f64ac70 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f7191600640 10 client.18285 handle_command_reply: tid=0 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 client.18285 shutdown 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 msgr2=0x56435f5e8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5e8f60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f7194009ca0 tx=0x7f7194005d90 comp rx=0 tx=0).stop 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 shutdown_connections 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x56435f64ac70 0x56435f64d090 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.486 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56435f5ddc00 0x56435f5e70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x56435f5d8a90 0x56435f5e8f60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 --2- 172.21.15.137:0/3447404746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56435f5d80a0 0x56435f5e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f7191600640 0 client.18285 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 >> 172.21.15.137:0/3447404746 conn(0x56435f5d3610 msgr2=0x56435f5d9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.482+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 shutdown_connections 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.483+0000 7f71b19ab740 1 -- 172.21.15.137:0/3447404746 wait complete. 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.483+0000 7f71b19ab740 20 client.18285 trim_cache size 0 max 16384 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:14:47.487 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:14:47.488 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "4c99c61e-ff7f-433f-ae9e-63f030ff0386", 2024-03-26T14:14:47.488 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:14:47.488 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.483+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f718c0a2800 msgr2=0x7f718c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.483+0000 7f71aec00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f718c0a2800 0x7f718c0a4cc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x56435f5d92f0 tx=0x7f71a0009600 comp rx=0 tx=0).stop 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.483+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 msgr2=0x7f71a8199b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.483+0000 7f71aec00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8199b80 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7198000c00 tx=0x7f71980042f0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 shutdown_connections 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f718c0a2800 0x7f718c0a4cc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71a8107380 0x7f71a81a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71a8104a50 0x7f71a819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.489 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 --2- 172.21.15.137:0/1719369101 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f71a8102120 0x7f71a8199b80 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 >> 172.21.15.137:0/1719369101 conn(0x7f71a80fbb10 msgr2=0x7f71a8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 shutdown_connections 2024-03-26T14:14:47.490 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.484+0000 7f71aec00640 1 -- 172.21.15.137:0/1719369101 wait complete. 2024-03-26T14:14:47.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:14:47.502 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 -- 172.21.15.137:0/1661579874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 msgr2=0x7fc6d4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1661579874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d4100d30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009900 tx=0x7fc6c002bc50 comp rx=0 tx=0).stop 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 -- 172.21.15.137:0/1661579874 shutdown_connections 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1661579874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6d4101270 0x7fc6d41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1661579874 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d4100d30 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1661579874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d40fff70 0x7fc6d4100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 -- 172.21.15.137:0/1661579874 >> 172.21.15.137:0/1661579874 conn(0x7fc6d40fbb30 msgr2=0x7fc6d40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 -- 172.21.15.137:0/1661579874 shutdown_connections 2024-03-26T14:14:47.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.725+0000 7fc6dba00640 1 -- 172.21.15.137:0/1661579874 wait complete. 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.726+0000 7fc6dba00640 1 Processor -- start 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.726+0000 7fc6dba00640 1 -- start start 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6dba00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d40fff70 0x7fc6d4199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6dba00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6dba00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6d4101270 0x7fc6d41a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6dba00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410f3e0 con 0x7fc6d41008d0 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6dba00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410f260 con 0x7fc6d40fff70 2024-03-26T14:14:47.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6dba00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d410f560 con 0x7fc6d4101270 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d9000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d40fff70 0x7fc6d4199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47036/0 (socket says 172.21.15.137:47036) 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 -- 172.21.15.137:0/1005190288 learned_addr learned my addr 172.21.15.137:0/1005190288 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6d4101270 0x7fc6d41a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 -- 172.21.15.137:0/1005190288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6d4101270 msgr2=0x7fc6d41a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6d4101270 0x7fc6d41a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 -- 172.21.15.137:0/1005190288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d40fff70 msgr2=0x7fc6d4199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d40fff70 0x7fc6d4199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.727+0000 7fc6d3e00640 1 -- 172.21.15.137:0/1005190288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d41a1710 con 0x7fc6d41008d0 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.728+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d419a030 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000c00 tx=0x7fc6c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.728+0000 7fc6d1600640 1 -- 172.21.15.137:0/1005190288 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0043020 con 0x7fc6d41008d0 2024-03-26T14:14:47.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.728+0000 7fc6d1600640 1 -- 172.21.15.137:0/1005190288 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6c0004aa0 con 0x7fc6d41008d0 2024-03-26T14:14:47.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.728+0000 7fc6d1600640 1 -- 172.21.15.137:0/1005190288 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0003d00 con 0x7fc6d41008d0 2024-03-26T14:14:47.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.728+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d41a19a0 con 0x7fc6d41008d0 2024-03-26T14:14:47.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.728+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d41a1f40 con 0x7fc6d41008d0 2024-03-26T14:14:47.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.732+0000 7fc6de781740 1 Processor -- start 2024-03-26T14:14:47.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.732+0000 7fc6de781740 1 -- start start 2024-03-26T14:14:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6de781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23de2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6de781740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6de781740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23e39e0 0x5618b23e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6de781740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b2343250 con 0x5618b23de870 2024-03-26T14:14:47.735 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6de781740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b22eba30 con 0x5618b23e39e0 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6de781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b233f2f0 con 0x5618b23dde80 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6d1600640 1 -- 172.21.15.137:0/1005190288 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc6c0046020 con 0x7fc6d41008d0 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6d9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23e3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47040/0 (socket says 172.21.15.137:47040) 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.733+0000 7fc6d9000640 1 -- 172.21.15.137:0/3240701285 learned_addr learned my addr 172.21.15.137:0/3240701285 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23de2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23e39e0 0x5618b23e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9000640 1 -- 172.21.15.137:0/3240701285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 msgr2=0x5618b23de2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9000640 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23de2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9000640 1 -- 172.21.15.137:0/3240701285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23e39e0 msgr2=0x5618b23e5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9000640 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23e39e0 0x5618b23e5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9000640 1 -- 172.21.15.137:0/3240701285 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b22d6690 con 0x5618b23de870 2024-03-26T14:14:47.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6d9000640 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23e3370 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009cf0 tx=0x7fc6c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaaf6a6f9a47b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6c6000640 1 -- 172.21.15.137:0/3240701285 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8004230 con 0x5618b23de870 2024-03-26T14:14:47.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6c6000640 1 -- 172.21.15.137:0/3240701285 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6c80043d0 con 0x5618b23de870 2024-03-26T14:14:47.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.734+0000 7fc6c6000640 1 -- 172.21.15.137:0/3240701285 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8019890 con 0x5618b23de870 2024-03-26T14:14:47.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6d1600640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc6bc0a2880 0x7fc6bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 -- 172.21.15.137:0/3240701285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 msgr2=0x5618b23e3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23e3370 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009cf0 tx=0x7fc6c8011810 comp rx=0 tx=0).stop 2024-03-26T14:14:47.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 -- 172.21.15.137:0/3240701285 shutdown_connections 2024-03-26T14:14:47.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23e39e0 0x5618b23e5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6d1600640 1 -- 172.21.15.137:0/1005190288 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc6c00f1120 con 0x7fc6d41008d0 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23e3370 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 --2- 172.21.15.137:0/3240701285 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23de2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.735+0000 7fc6de781740 1 -- 172.21.15.137:0/3240701285 >> 172.21.15.137:0/3240701285 conn(0x5618b23a6200 msgr2=0x5618b23d97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.736+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc6bc0a2880 0x7fc6bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.736+0000 7fc6de781740 1 -- 172.21.15.137:0/3240701285 shutdown_connections 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.736+0000 7fc6de781740 1 -- 172.21.15.137:0/3240701285 wait complete. 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.736+0000 7fc6de781740 1 -- start start 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.736+0000 7fc6de781740 1 Processor -- start 2024-03-26T14:14:47.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.736+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc6bc0a2880 0x7fc6bc0a4d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5618b23df0d0 tx=0x7fc6cc009580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23e7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b23e6990 con 0x5618b23e39e0 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b22fee80 con 0x5618b23de870 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618b23e6b10 con 0x5618b23dde80 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6de781740 10 client.0 _ll_register_callbacks cb 0x5618b2311130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6d9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6d9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47056/0 (socket says 172.21.15.137:47056) 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6d9a00640 1 -- 172.21.15.137:0/2220991979 learned_addr learned my addr 172.21.15.137:0/2220991979 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.742 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.737+0000 7fc6d9000640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23e7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6d9a00640 1 -- 172.21.15.137:0/2220991979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 msgr2=0x5618b23e7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23e7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6d9a00640 1 -- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 msgr2=0x5618b23e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6d9a00640 1 -- 172.21.15.137:0/2220991979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618b21217d0 con 0x5618b23e39e0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ef3d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc012870 tx=0x7fc6cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea182a0708817d48 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6baa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6baa00640 1 -- 172.21.15.137:0/2220991979 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00ead0 con 0x5618b23e39e0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6c80043d0 con 0x5618b23e39e0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.738+0000 7fc6baa00640 1 -- 172.21.15.137:0/2220991979 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc6cc00ec70 con 0x5618b23e39e0 2024-03-26T14:14:47.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6baa00640 1 -- 172.21.15.137:0/2220991979 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6cc030050 con 0x5618b23e39e0 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6de781740 10 client.18297 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5618b20573f0 con 0x5618b23e39e0 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6baa00640 1 -- 172.21.15.137:0/2220991979 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6c80043d0 con 0x5618b23e39e0 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6baa00640 1 -- 172.21.15.137:0/2220991979 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc6cc00f040 con 0x5618b23e39e0 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6de781740 10 client.18297 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.739+0000 7fc6de781740 10 client.18297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.740+0000 7fc6de781740 20 client.18297 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.740+0000 7fc6de781740 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b23f3cb0 0x5618b23f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.740+0000 7fc6de781740 4 client.18297 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:14:47.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.740+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618b2350790 con 0x5618b23f3cb0 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.740+0000 7fc6d9000640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b23f3cb0 0x5618b23f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.741+0000 7fc6d9000640 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b23f3cb0 0x5618b23f60d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236e7e8a48b1d988 server_cookie=e8601871d84ce5cd in_seq=0 out_seq=0 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.741+0000 7fc6baa00640 10 client.18297 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6baa00640 1 -- 172.21.15.137:0/2220991979 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5618b2350790 con 0x5618b23f3cb0 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6baa00640 10 client.18297 handle_command_reply: tid=0 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 client.18297 shutdown 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 msgr2=0x5618b23ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ef3d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc012870 tx=0x7fc6cc004a40 comp rx=0 tx=0).stop 2024-03-26T14:14:47.746 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 shutdown_connections 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b23f3cb0 0x5618b23f60d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ef3d0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 --2- 172.21.15.137:0/2220991979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23e7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6baa00640 0 client.18297 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 >> 172.21.15.137:0/2220991979 conn(0x5618b23a6200 msgr2=0x5618b23d8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 shutdown_connections 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 1 -- 172.21.15.137:0/2220991979 wait complete. 2024-03-26T14:14:47.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.743+0000 7fc6de781740 20 client.18297 trim_cache size 0 max 16384 2024-03-26T14:14:47.765 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.763+0000 7fc6de781740 1 Processor -- start 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.763+0000 7fc6de781740 1 -- start start 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6de781740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23dde80 0x5618b23e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6de781740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6de781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23e39e0 0x5618b23dfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6de781740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41ae940 con 0x5618b23de870 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6de781740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41ac440 con 0x5618b23dde80 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6de781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41aa4c0 con 0x5618b23e39e0 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6d3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6d3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23ef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47066/0 (socket says 172.21.15.137:47066) 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6d3e00640 1 -- 172.21.15.137:0/657999652 learned_addr learned my addr 172.21.15.137:0/657999652 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6d9000640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23dde80 0x5618b23e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23e39e0 0x5618b23dfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.764+0000 7fc6d3e00640 1 -- 172.21.15.137:0/657999652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23e39e0 msgr2=0x5618b23dfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23e39e0 0x5618b23dfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d3e00640 1 -- 172.21.15.137:0/657999652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23dde80 msgr2=0x5618b23e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23dde80 0x5618b23e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d3e00640 1 -- 172.21.15.137:0/657999652 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618b20170f0 con 0x5618b23de870 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d9000640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23dde80 0x5618b23e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23e39e0 0x5618b23dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23ef390 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00ba1a0 tx=0x7fc6c00f3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd30fdde8215b131 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6c5600640 1 -- 172.21.15.137:0/657999652 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00bd130 con 0x5618b23de870 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6c5600640 1 -- 172.21.15.137:0/657999652 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6c0004c50 con 0x5618b23de870 2024-03-26T14:14:47.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.765+0000 7fc6c5600640 1 -- 172.21.15.137:0/657999652 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00044e0 con 0x5618b23de870 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 -- 172.21.15.137:0/657999652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 msgr2=0x5618b23ef390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23ef390 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00ba1a0 tx=0x7fc6c00f3040 comp rx=0 tx=0).stop 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 -- 172.21.15.137:0/657999652 shutdown_connections 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23e39e0 0x5618b23dfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23de870 0x5618b23ef390 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 --2- 172.21.15.137:0/657999652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23dde80 0x5618b23e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 -- 172.21.15.137:0/657999652 >> 172.21.15.137:0/657999652 conn(0x5618b23a6200 msgr2=0x5618b23df310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 -- 172.21.15.137:0/657999652 shutdown_connections 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.766+0000 7fc6de781740 1 -- 172.21.15.137:0/657999652 wait complete. 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 -- start start 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 Processor -- start 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ed290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41aaac0 con 0x5618b23e39e0 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41abde0 con 0x5618b23de870 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41ab120 con 0x5618b23dde80 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.767+0000 7fc6de781740 10 client.0 _ll_register_callbacks cb 0x7fc6cc009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:14:47.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ed290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47070/0 (socket says 172.21.15.137:47070) 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 -- 172.21.15.137:0/1044828868 learned_addr learned my addr 172.21.15.137:0/1044828868 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d3e00640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9000640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 -- 172.21.15.137:0/1044828868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 msgr2=0x5618b23eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 -- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 msgr2=0x5618b23ef1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23ef1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 -- 172.21.15.137:0/1044828868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618b23ed980 con 0x5618b23e39e0 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6d9a00640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ed290 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc004af0 tx=0x7fc6cc009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c513796ff268a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.768+0000 7fc6baa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:14:47.773 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6baa00640 1 -- 172.21.15.137:0/1044828868 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc03a030 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6baa00640 1 -- 172.21.15.137:0/1044828868 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc6cc00e930 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6c0004c50 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6baa00640 1 -- 172.21.15.137:0/1044828868 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6cc047050 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6de781740 10 client.18303 fetch_fsmap learned FSMap version 24 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5618b244ff10 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.769+0000 7fc6baa00640 1 -- 172.21.15.137:0/1044828868 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6c0004c50 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6baa00640 1 -- 172.21.15.137:0/1044828868 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc6cc033050 con 0x5618b23e39e0 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6de781740 10 client.18303 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6de781740 10 client.18303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6de781740 20 client.18303 populate_metadata read hostname 'smithi137' 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6de781740 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b2450840 0x5618b2452c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6de781740 4 client.18303 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:14:47.774 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5618b234aae0 con 0x5618b2450840 2024-03-26T14:14:47.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.770+0000 7fc6d9000640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b2450840 0x5618b2452c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.771+0000 7fc6baa00640 10 client.18303 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.771+0000 7fc6d9000640 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b2450840 0x5618b2452c60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d15e8731e7715c server_cookie=168b3666c114c78d in_seq=0 out_seq=0 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6baa00640 1 -- 172.21.15.137:0/1044828868 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5618b234aae0 con 0x5618b2450840 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6baa00640 10 client.18303 handle_command_reply: tid=0 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 client.18303 shutdown 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 msgr2=0x5618b23ed290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ed290 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc004af0 tx=0x7fc6cc009d30 comp rx=0 tx=0).stop 2024-03-26T14:14:47.776 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 shutdown_connections 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5618b2450840 0x5618b2452c60 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618b23de870 0x5618b23ef1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5618b23e39e0 0x5618b23ed290 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 --2- 172.21.15.137:0/1044828868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5618b23dde80 0x5618b23eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 >> 172.21.15.137:0/1044828868 conn(0x5618b23a6200 msgr2=0x5618b23df310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6baa00640 0 client.18303 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 shutdown_connections 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 1 -- 172.21.15.137:0/1044828868 wait complete. 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.772+0000 7fc6de781740 20 client.18303 trim_cache size 0 max 16384 2024-03-26T14:14:47.777 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: "4c99c61e-ff7f-433f-ae9e-63f030ff0386": { 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "4c99c61e-ff7f-433f-ae9e-63f030ff0386", 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:14:47.778 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc6bc0a2880 msgr2=0x7fc6bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc6bc0a2880 0x7fc6bc0a4d40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5618b23df0d0 tx=0x7fc6cc009580 comp rx=0 tx=0).stop 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 msgr2=0x7fc6d419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d419a030 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0000c00 tx=0x7fc6c00042f0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 shutdown_connections 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc6bc0a2880 0x7fc6bc0a4d40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc6d4101270 0x7fc6d41a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc6d41008d0 0x7fc6d419a030 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 --2- 172.21.15.137:0/1005190288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d40fff70 0x7fc6d4199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:14:47.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 >> 172.21.15.137:0/1005190288 conn(0x7fc6d40fbb30 msgr2=0x7fc6d40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:14:47.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 shutdown_connections 2024-03-26T14:14:47.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:14:47.773+0000 7fc6dba00640 1 -- 172.21.15.137:0/1005190288 wait complete. 2024-03-26T14:14:47.791 INFO:tasks.cephfs.filesystem:scrub status for tag:4c99c61e-ff7f-433f-ae9e-63f030ff0386 - {'path': '/', 'tag': '4c99c61e-ff7f-433f-ae9e-63f030ff0386', 'options': 'recursive,force'} 2024-03-26T14:14:49.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:48 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:14:49.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:48 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:49.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:48 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:14:49.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:48 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:49.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:48 smithi137 ceph-mon[31539]: pgmap v831: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:14:49.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:48 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:14:49.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:48 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:49.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:48 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:14:49.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:48 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:49.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:48 smithi179 ceph-mon[35472]: pgmap v831: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:14:49.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:48 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:14:49.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:48 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:49.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:48 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:14:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:48 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:14:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:48 smithi159 ceph-mon[35501]: pgmap v831: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:14:51.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:50 smithi137 ceph-mon[31539]: pgmap v832: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:51.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:50 smithi179 ceph-mon[35472]: pgmap v832: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:51.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:50 smithi159 ceph-mon[35501]: pgmap v832: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:14:53.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:52 smithi137 ceph-mon[31539]: pgmap v833: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:14:53.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:52 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:14:53.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:52 smithi179 ceph-mon[35472]: pgmap v833: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:14:53.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:52 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:14:53.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:52 smithi159 ceph-mon[35501]: pgmap v833: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:14:53.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:52 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:14:55.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:54 smithi137 ceph-mon[31539]: pgmap v834: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:14:55.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:54 smithi179 ceph-mon[35472]: pgmap v834: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:14:55.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:54 smithi159 ceph-mon[35501]: pgmap v834: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:14:57.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:56 smithi137 ceph-mon[31539]: pgmap v835: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:57.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:56 smithi179 ceph-mon[35472]: pgmap v835: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:57.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:56 smithi159 ceph-mon[35501]: pgmap v835: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:14:59.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:14:58 smithi137 ceph-mon[31539]: pgmap v836: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:59.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:14:58 smithi179 ceph-mon[35472]: pgmap v836: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:14:59.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:14:58 smithi159 ceph-mon[35501]: pgmap v836: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:15:01.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:00 smithi137 ceph-mon[31539]: pgmap v837: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:15:01.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:00 smithi179 ceph-mon[35472]: pgmap v837: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:15:01.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:00 smithi159 ceph-mon[35501]: pgmap v837: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:15:03.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:02 smithi137 ceph-mon[31539]: pgmap v838: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:15:03.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:02 smithi179 ceph-mon[35472]: pgmap v838: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:15:03.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:02 smithi159 ceph-mon[35501]: pgmap v838: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:15:05.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:04 smithi137 ceph-mon[31539]: pgmap v839: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:05.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:04 smithi179 ceph-mon[35472]: pgmap v839: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:05.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:04 smithi159 ceph-mon[35501]: pgmap v839: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:07.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:06 smithi137 ceph-mon[31539]: pgmap v840: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:07.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:06 smithi179 ceph-mon[35472]: pgmap v840: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:06 smithi159 ceph-mon[35501]: pgmap v840: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:09.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:08 smithi137 ceph-mon[31539]: pgmap v841: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:09.174 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:08 smithi179 ceph-mon[35472]: pgmap v841: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:09.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:08 smithi159 ceph-mon[35501]: pgmap v841: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:11.037 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:10 smithi137 ceph-mon[31539]: pgmap v842: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:11.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:10 smithi179 ceph-mon[35472]: pgmap v842: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:11.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:10 smithi159 ceph-mon[35501]: pgmap v842: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:13.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:12 smithi137 ceph-mon[31539]: pgmap v843: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:13.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:12 smithi179 ceph-mon[35472]: pgmap v843: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:13.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:12 smithi159 ceph-mon[35501]: pgmap v843: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:15.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:14 smithi179 ceph-mon[35472]: pgmap v844: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:15.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:14 smithi159 ceph-mon[35501]: pgmap v844: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:15.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:14 smithi137 ceph-mon[31539]: pgmap v844: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:17.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:16 smithi179 ceph-mon[35472]: pgmap v845: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:17.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:16 smithi159 ceph-mon[35501]: pgmap v845: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:17.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:16 smithi137 ceph-mon[31539]: pgmap v845: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:17.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:15:17.793 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:15:18.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 -- 172.21.15.137:0/2686478831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1580105470 msgr2=0x7f158010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 --2- 172.21.15.137:0/2686478831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1580105470 0x7f158010bd00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1578009900 tx=0x7f157802bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 -- 172.21.15.137:0/2686478831 shutdown_connections 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 --2- 172.21.15.137:0/2686478831 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1580105470 0x7f158010bd00 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 --2- 172.21.15.137:0/2686478831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1580104ab0 0x7f1580104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 --2- 172.21.15.137:0/2686478831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15801038c0 0x7f1580103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 -- 172.21.15.137:0/2686478831 >> 172.21.15.137:0/2686478831 conn(0x7f15800fee50 msgr2=0x7f1580101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 -- 172.21.15.137:0/2686478831 shutdown_connections 2024-03-26T14:15:18.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.025+0000 7f1587600640 1 -- 172.21.15.137:0/2686478831 wait complete. 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 Processor -- start 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 -- start start 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 0x7f1580199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1580104ab0 0x7f1580199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580105470 0x7f15801a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158010f3d0 con 0x7f15801038c0 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158010f250 con 0x7f1580105470 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.026+0000 7f1587600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158010f550 con 0x7f1580104ab0 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 0x7f1580199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 0x7f1580199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41354/0 (socket says 172.21.15.137:41354) 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 -- 172.21.15.137:0/2912487495 learned_addr learned my addr 172.21.15.137:0/2912487495 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1585600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580105470 0x7f15801a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1577e00640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1580104ab0 0x7f1580199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 -- 172.21.15.137:0/2912487495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1580104ab0 msgr2=0x7f1580199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1580104ab0 0x7f1580199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 -- 172.21.15.137:0/2912487495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580105470 msgr2=0x7f15801a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580105470 0x7f15801a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 -- 172.21.15.137:0/2912487495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15801a16a0 con 0x7f15801038c0 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1585600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580105470 0x7f15801a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.027+0000 7f1584c00640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 0x7f1580199a80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f157000ebc0 tx=0x7f157000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.028+0000 7f157e000640 1 -- 172.21.15.137:0/2912487495 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570003b70 con 0x7f15801038c0 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.028+0000 7f157e000640 1 -- 172.21.15.137:0/2912487495 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1570003d10 con 0x7f15801038c0 2024-03-26T14:15:18.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.028+0000 7f157e000640 1 -- 172.21.15.137:0/2912487495 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570019840 con 0x7f15801038c0 2024-03-26T14:15:18.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.028+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15801a1990 con 0x7f15801038c0 2024-03-26T14:15:18.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.028+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15801a1e90 con 0x7f15801038c0 2024-03-26T14:15:18.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.031+0000 7f157e000640 1 -- 172.21.15.137:0/2912487495 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f15700199e0 con 0x7f15801038c0 2024-03-26T14:15:18.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.031+0000 7f158a25f740 1 Processor -- start 2024-03-26T14:15:18.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 -- start start 2024-03-26T14:15:18.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 0x564a40c12370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c14e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a40b72250 con 0x564a40c0d870 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a40b1aa30 con 0x564a40c129e0 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f158a25f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a40b6e2f0 con 0x564a40c0ce80 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 0x564a40c12370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 0x564a40c12370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41360/0 (socket says 172.21.15.137:41360) 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f1584c00640 1 -- 172.21.15.137:0/2260010247 learned_addr learned my addr 172.21.15.137:0/2260010247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.032+0000 7f1577e00640 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c14e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1585600640 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f157e000640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15640a2800 0x7f15640a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f157e000640 1 -- 172.21.15.137:0/2912487495 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f15700cf840 con 0x7f15801038c0 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1584c00640 1 -- 172.21.15.137:0/2260010247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 msgr2=0x564a40c0d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1584c00640 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c0d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1584c00640 1 -- 172.21.15.137:0/2260010247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 msgr2=0x564a40c14e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1584c00640 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c14e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1584c00640 1 -- 172.21.15.137:0/2260010247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a40b05690 con 0x564a40c0d870 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1577e00640 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c14e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:15:18.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1585600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15640a2800 0x7f15640a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.033+0000 7f1584c00640 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 0x564a40c12370 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1570098230 tx=0x7f157000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232891f26c69ad50 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.034+0000 7f1575c00640 1 -- 172.21.15.137:0/2260010247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157009b910 con 0x564a40c0d870 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.034+0000 7f1575c00640 1 -- 172.21.15.137:0/2260010247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f157009c4d0 con 0x564a40c0d870 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.034+0000 7f1575c00640 1 -- 172.21.15.137:0/2260010247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157009b730 con 0x564a40c0d870 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 -- 172.21.15.137:0/2260010247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 msgr2=0x564a40c12370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 0x564a40c12370 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1570098230 tx=0x7f157000cf90 comp rx=0 tx=0).stop 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f1585600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15640a2800 0x7f15640a4cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x564a40c0e0d0 tx=0x7f1578009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 -- 172.21.15.137:0/2260010247 shutdown_connections 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c14e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0d870 0x564a40c12370 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 --2- 172.21.15.137:0/2260010247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c0d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 -- 172.21.15.137:0/2260010247 >> 172.21.15.137:0/2260010247 conn(0x564a40bd5200 msgr2=0x564a40c087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 -- 172.21.15.137:0/2260010247 shutdown_connections 2024-03-26T14:15:18.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 -- 172.21.15.137:0/2260010247 wait complete. 2024-03-26T14:15:18.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.035+0000 7f158a25f740 1 -- start start 2024-03-26T14:15:18.038 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 Processor -- start 2024-03-26T14:15:18.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c16de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a40c15920 con 0x564a40c0ce80 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a40b2de80 con 0x564a40c129e0 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a40c15aa0 con 0x564a40c0d870 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f158a25f740 10 client.0 _ll_register_callbacks cb 0x564a40b40130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c16de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c16de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41372/0 (socket says 172.21.15.137:41372) 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.036+0000 7f1584c00640 1 -- 172.21.15.137:0/676198998 learned_addr learned my addr 172.21.15.137:0/676198998 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1577e00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c17320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1585600640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1584c00640 1 -- 172.21.15.137:0/676198998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 msgr2=0x564a40c17320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1584c00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c17320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1584c00640 1 -- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 msgr2=0x564a40c1e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1584c00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1584c00640 1 -- 172.21.15.137:0/676198998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a409507d0 con 0x564a40c0ce80 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1577e00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f1584c00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c16de0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f157009bad0 tx=0x7f157009b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5fb5bdc3ade260 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f156e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f156e000640 1 -- 172.21.15.137:0/676198998 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157009b910 con 0x564a40c0ce80 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.037+0000 7f156e000640 1 -- 172.21.15.137:0/676198998 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f15700a40e0 con 0x564a40c0ce80 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f157009c4d0 con 0x564a40c0ce80 2024-03-26T14:15:18.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f156e000640 1 -- 172.21.15.137:0/676198998 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15700a4570 con 0x564a40c0ce80 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f158a25f740 10 client.18315 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564a408863f0 con 0x564a40c0ce80 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f156e000640 1 -- 172.21.15.137:0/676198998 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f157009c4d0 con 0x564a40c0ce80 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f156e000640 1 -- 172.21.15.137:0/676198998 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1570097b20 con 0x564a40c0ce80 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f158a25f740 10 client.18315 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.038+0000 7f158a25f740 10 client.18315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.039+0000 7f158a25f740 20 client.18315 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.039+0000 7f158a25f740 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c22850 0x564a40c24c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.039+0000 7f158a25f740 4 client.18315 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.039+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a40b7f790 con 0x564a40c22850 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.039+0000 7f1577e00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c22850 0x564a40c24c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.040+0000 7f156e000640 10 client.18315 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:18.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.040+0000 7f1577e00640 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c22850 0x564a40c24c40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b094c5f72c5160b7 server_cookie=ea67d38b63474b67 in_seq=0 out_seq=0 2024-03-26T14:15:18.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f156e000640 1 -- 172.21.15.137:0/676198998 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564a40b7f790 con 0x564a40c22850 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f156e000640 10 client.18315 handle_command_reply: tid=0 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 client.18315 shutdown 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 msgr2=0x564a40c16de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c16de0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f157009bad0 tx=0x7f157009b730 comp rx=0 tx=0).stop 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 shutdown_connections 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c22850 0x564a40c24c40 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 --2- 172.21.15.137:0/676198998 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c16de0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 >> 172.21.15.137:0/676198998 conn(0x564a40bd5200 msgr2=0x564a40c07a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f156e000640 0 client.18315 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 shutdown_connections 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.042+0000 7f158a25f740 1 -- 172.21.15.137:0/676198998 wait complete. 2024-03-26T14:15:18.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.043+0000 7f158a25f740 20 client.18315 trim_cache size 0 max 16384 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 Processor -- start 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 -- start start 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c17280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 0x564a40c1e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c129e0 0x564a40c07910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801ae900 con 0x564a40c129e0 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801ac400 con 0x564a40c0d870 2024-03-26T14:15:18.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.065+0000 7f158a25f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801aa480 con 0x564a40c0ce80 2024-03-26T14:15:18.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 0x564a40c1e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 0x564a40c1e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:41962/0 (socket says 172.21.15.137:41962) 2024-03-26T14:15:18.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 -- 172.21.15.137:0/3140719272 learned_addr learned my addr 172.21.15.137:0/3140719272 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.067 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1585600640 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c129e0 0x564a40c07910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1584c00640 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c17280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 -- 172.21.15.137:0/3140719272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 msgr2=0x564a40c17280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c17280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 -- 172.21.15.137:0/3140719272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c129e0 msgr2=0x564a40c07910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c129e0 0x564a40c07910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.066+0000 7f1577e00640 1 -- 172.21.15.137:0/3140719272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a408460f0 con 0x564a40c0d870 2024-03-26T14:15:18.068 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.067+0000 7f1577e00640 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 0x564a40c1e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f156800bc30 tx=0x7f156800b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f92f023515f0aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.067+0000 7f1575200640 1 -- 172.21.15.137:0/3140719272 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156801f020 con 0x564a40c0d870 2024-03-26T14:15:18.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.067+0000 7f1575200640 1 -- 172.21.15.137:0/3140719272 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f156801b2c0 con 0x564a40c0d870 2024-03-26T14:15:18.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.067+0000 7f1575200640 1 -- 172.21.15.137:0/3140719272 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568012740 con 0x564a40c0d870 2024-03-26T14:15:18.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 -- 172.21.15.137:0/3140719272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 msgr2=0x564a40c1e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 0x564a40c1e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f156800bc30 tx=0x7f156800b4d0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 -- 172.21.15.137:0/3140719272 shutdown_connections 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c0d870 0x564a40c1e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c129e0 0x564a40c07910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 --2- 172.21.15.137:0/3140719272 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0ce80 0x564a40c17280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 -- 172.21.15.137:0/3140719272 >> 172.21.15.137:0/3140719272 conn(0x564a40bd5200 msgr2=0x564a40c07200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 -- 172.21.15.137:0/3140719272 shutdown_connections 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.068+0000 7f158a25f740 1 -- 172.21.15.137:0/3140719272 wait complete. 2024-03-26T14:15:18.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 -- start start 2024-03-26T14:15:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 Processor -- start 2024-03-26T14:15:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c1be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c1c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801aaa80 con 0x564a40c0ce80 2024-03-26T14:15:18.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801abda0 con 0x564a40c129e0 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.069+0000 7f158a25f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15801ab0e0 con 0x564a40c0d870 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f158a25f740 10 client.0 _ll_register_callbacks cb 0x7f156800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c1be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c1be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41392/0 (socket says 172.21.15.137:41392) 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 -- 172.21.15.137:0/1858079507 learned_addr learned my addr 172.21.15.137:0/1858079507 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1585600640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1577e00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c1c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 -- 172.21.15.137:0/1858079507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 msgr2=0x564a40c1c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c1c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 -- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 msgr2=0x564a40c1d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1584c00640 1 -- 172.21.15.137:0/1858079507 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a40c1d8a0 con 0x564a40c0ce80 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.070+0000 7f1577e00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c1c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:18.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f1584c00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c1be40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f1570098230 tx=0x7f15700cfac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b531784c30deb2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f156e000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f156e000640 1 -- 172.21.15.137:0/1858079507 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157009b910 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f156e000640 1 -- 172.21.15.137:0/1858079507 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f15700a42e0 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f156801b2c0 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f156e000640 1 -- 172.21.15.137:0/1858079507 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f157009c4d0 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f158a25f740 10 client.18321 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564a40c1d8a0 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.071+0000 7f156e000640 1 -- 172.21.15.137:0/1858079507 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f156801b2c0 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f156e000640 1 -- 172.21.15.137:0/1858079507 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1570004b60 con 0x564a40c0ce80 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f158a25f740 10 client.18321 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f158a25f740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f158a25f740 20 client.18321 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f158a25f740 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c7e770 0x564a40c80b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f158a25f740 4 client.18321 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:15:18.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a40b79ae0 con 0x564a40c7e770 2024-03-26T14:15:18.076 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.072+0000 7f1577e00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c7e770 0x564a40c80b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.073+0000 7f156e000640 10 client.18321 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.073+0000 7f1577e00640 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c7e770 0x564a40c80b90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=402cb4e4ed7e7ff9 server_cookie=c0994acf1d9ae2e2 in_seq=0 out_seq=0 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.073+0000 7f156e000640 1 -- 172.21.15.137:0/1858079507 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564a40b79ae0 con 0x564a40c7e770 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.073+0000 7f156e000640 10 client.18321 handle_command_reply: tid=0 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 client.18321 shutdown 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 msgr2=0x564a40c1be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c1be40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f1570098230 tx=0x7f15700cfac0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 shutdown_connections 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564a40c7e770 0x564a40c80b90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a40c129e0 0x564a40c1d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.077 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a40c0d870 0x564a40c1c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f156e000640 0 client.18321 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 --2- 172.21.15.137:0/1858079507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564a40c0ce80 0x564a40c1be40 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 >> 172.21.15.137:0/1858079507 conn(0x564a40bd5200 msgr2=0x564a40c170b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 shutdown_connections 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 1 -- 172.21.15.137:0/1858079507 wait complete. 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.074+0000 7f158a25f740 20 client.18321 trim_cache size 0 max 16384 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:15:18.078 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15640a2800 msgr2=0x7f15640a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15640a2800 0x7f15640a4cc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x564a40c0e0d0 tx=0x7f1578009260 comp rx=0 tx=0).stop 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 msgr2=0x7f1580199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 0x7f1580199a80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f157000ebc0 tx=0x7f157000c6a0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 shutdown_connections 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f15640a2800 0x7f15640a4cc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580105470 0x7f15801a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1580104ab0 0x7f1580199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 --2- 172.21.15.137:0/2912487495 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f15801038c0 0x7f1580199a80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 >> 172.21.15.137:0/2912487495 conn(0x7f15800fee50 msgr2=0x7f1580107db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 shutdown_connections 2024-03-26T14:15:18.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.075+0000 7f1587600640 1 -- 172.21.15.137:0/2912487495 wait complete. 2024-03-26T14:15:18.093 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:15:18.093 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:15:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.306+0000 7f79aea00640 1 -- 172.21.15.137:0/2647824503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8104ad0 msgr2=0x7f79a8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.307 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.306+0000 7f79aea00640 1 --2- 172.21.15.137:0/2647824503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8104ad0 0x7f79a8104f50 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f7990009900 tx=0x7f799002bc50 comp rx=0 tx=0).stop 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 -- 172.21.15.137:0/2647824503 shutdown_connections 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 --2- 172.21.15.137:0/2647824503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a8105490 0x7f79a810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 --2- 172.21.15.137:0/2647824503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8104ad0 0x7f79a8104f50 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 --2- 172.21.15.137:0/2647824503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79a81038e0 0x7f79a8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 -- 172.21.15.137:0/2647824503 >> 172.21.15.137:0/2647824503 conn(0x7f79a80fee90 msgr2=0x7f79a81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 -- 172.21.15.137:0/2647824503 shutdown_connections 2024-03-26T14:15:18.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.307+0000 7f79aea00640 1 -- 172.21.15.137:0/2647824503 wait complete. 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 Processor -- start 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 -- start start 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79a81038e0 0x7f79a8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a8104ad0 0x7f79a819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 0x7f79a81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a810f3f0 con 0x7f79a8105490 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a810f270 con 0x7f79a8104ad0 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.308+0000 7f79aea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a810f570 con 0x7f79a81038e0 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 0x7f79a81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 0x7f79a81a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41404/0 (socket says 172.21.15.137:41404) 2024-03-26T14:15:18.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 -- 172.21.15.137:0/3040797051 learned_addr learned my addr 172.21.15.137:0/3040797051 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79a7400640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a8104ad0 0x7f79a819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79a7e00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79a81038e0 0x7f79a8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 -- 172.21.15.137:0/3040797051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79a81038e0 msgr2=0x7f79a8199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79a81038e0 0x7f79a8199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 -- 172.21.15.137:0/3040797051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a8104ad0 msgr2=0x7f79a819a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a8104ad0 0x7f79a819a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.309+0000 7f79aca00640 1 -- 172.21.15.137:0/3040797051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a81a16f0 con 0x7f79a8105490 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.310+0000 7f79aca00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 0x7f79a81a1080 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f79a000d750 tx=0x7f79a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.310+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0017070 con 0x7f79a8105490 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.310+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f79a00054d0 con 0x7f79a8105490 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.310+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a81a19e0 con 0x7f79a8105490 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.310+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004c60 con 0x7f79a8105490 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.310+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a81a1f00 con 0x7f79a8105490 2024-03-26T14:15:18.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.312+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a810ac80 con 0x7f79a8105490 2024-03-26T14:15:18.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.314+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f79a0004e00 con 0x7f79a8105490 2024-03-26T14:15:18.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.316+0000 7f79a4c00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f798c0a2880 0x7f798c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.316+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f79a00ce710 con 0x7f79a8105490 2024-03-26T14:15:18.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.316+0000 7f79a7e00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f798c0a2880 0x7f798c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.317 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.317+0000 7f79a7e00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f798c0a2880 0x7f798c0a4d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f79a8104930 tx=0x7f7998009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.321+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f79a009ad50 con 0x7f79a8105490 2024-03-26T14:15:18.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.598+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79a80622a0 con 0x7f79a8105490 2024-03-26T14:15:18.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.600+0000 7f79a4c00640 1 -- 172.21.15.137:0/3040797051 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f79a009edf0 con 0x7f79a8105490 2024-03-26T14:15:18.601 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:15:18.602 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:15:18.602 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:15:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f798c0a2880 msgr2=0x7f798c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f798c0a2880 0x7f798c0a4d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f79a8104930 tx=0x7f7998009670 comp rx=0 tx=0).stop 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 msgr2=0x7f79a81a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 0x7f79a81a1080 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f79a000d750 tx=0x7f79a000dc10 comp rx=0 tx=0).stop 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 shutdown_connections 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f798c0a2880 0x7f798c0a4d40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79a8104ad0 0x7f79a819a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f79a8105490 0x7f79a81a1080 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 --2- 172.21.15.137:0/3040797051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79a81038e0 0x7f79a8199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 >> 172.21.15.137:0/3040797051 conn(0x7f79a80fee90 msgr2=0x7f79a8107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 shutdown_connections 2024-03-26T14:15:18.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.603+0000 7f79aea00640 1 -- 172.21.15.137:0/3040797051 wait complete. 2024-03-26T14:15:18.624 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:15:18.624 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:15:18.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 -- 172.21.15.137:0/3831546691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 msgr2=0x7f957c1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 --2- 172.21.15.137:0/3831546691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1096b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9574009ca0 tx=0x7f957402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 -- 172.21.15.137:0/3831546691 shutdown_connections 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 --2- 172.21.15.137:0/3831546691 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1096b0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 --2- 172.21.15.137:0/3831546691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957c104930 0x7f957c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 --2- 172.21.15.137:0/3831546691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f957c102000 0x7f957c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 -- 172.21.15.137:0/3831546691 >> 172.21.15.137:0/3831546691 conn(0x7f957c0fba10 msgr2=0x7f957c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 -- 172.21.15.137:0/3831546691 shutdown_connections 2024-03-26T14:15:18.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.844+0000 7f9581600640 1 -- 172.21.15.137:0/3831546691 wait complete. 2024-03-26T14:15:18.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.845+0000 7f9581600640 1 Processor -- start 2024-03-26T14:15:18.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 -- start start 2024-03-26T14:15:18.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f957c102000 0x7f957c101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957c104930 0x7f957c0fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c10d0c0 con 0x7f957c107260 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c10cf40 con 0x7f957c102000 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9581600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c10d240 con 0x7f957c104930 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9580c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9580c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41420/0 (socket says 172.21.15.137:41420) 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f9580c00640 1 -- 172.21.15.137:0/4061116107 learned_addr learned my addr 172.21.15.137:0/4061116107 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.846+0000 7f957b400640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957c104930 0x7f957c0fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f957be00640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f957c102000 0x7f957c101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9580c00640 1 -- 172.21.15.137:0/4061116107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957c104930 msgr2=0x7f957c0fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9580c00640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957c104930 0x7f957c0fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9580c00640 1 -- 172.21.15.137:0/4061116107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f957c102000 msgr2=0x7f957c101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9580c00640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f957c102000 0x7f957c101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9580c00640 1 -- 172.21.15.137:0/4061116107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c100d80 con 0x7f957c107260 2024-03-26T14:15:18.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9580c00640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1004f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f957402c700 tx=0x7f95740328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9578c00640 1 -- 172.21.15.137:0/4061116107 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574041020 con 0x7f957c107260 2024-03-26T14:15:18.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9578c00640 1 -- 172.21.15.137:0/4061116107 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9574003e20 con 0x7f957c107260 2024-03-26T14:15:18.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.847+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c1a1850 con 0x7f957c107260 2024-03-26T14:15:18.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.848+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c1a1da0 con 0x7f957c107260 2024-03-26T14:15:18.850 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.848+0000 7f9578c00640 1 -- 172.21.15.137:0/4061116107 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574039040 con 0x7f957c107260 2024-03-26T14:15:18.852 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 Processor -- start 2024-03-26T14:15:18.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9578c00640 1 -- 172.21.15.137:0/4061116107 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9574044020 con 0x7f957c107260 2024-03-26T14:15:18.853 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 -- start start 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56401850f9f0 0x56401850fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018511040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564018511580 0x564018517e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640184742c0 con 0x564018510bc0 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x56401841cc90 con 0x56401850f9f0 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.852+0000 7f9585806740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564018470360 con 0x564018511580 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018511040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018511040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41428/0 (socket says 172.21.15.137:41428) 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 -- 172.21.15.137:0/2484694160 learned_addr learned my addr 172.21.15.137:0/2484694160 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957b400640 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56401850f9f0 0x56401850fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957b400640 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564018511580 0x564018517e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 -- 172.21.15.137:0/2484694160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564018511580 msgr2=0x564018517e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564018511580 0x564018517e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 -- 172.21.15.137:0/2484694160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56401850f9f0 msgr2=0x56401850fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56401850f9f0 0x56401850fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957be00640 1 -- 172.21.15.137:0/2484694160 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564018441080 con 0x564018510bc0 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.853+0000 7f957b400640 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56401850f9f0 0x56401850fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f957be00640 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018511040 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f956c00bcb0 tx=0x7f956c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47904a6985cacf00 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f9578c00640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f95600a2800 0x7f95600a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f9578c00640 1 -- 172.21.15.137:0/4061116107 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f95740f1340 con 0x7f957c107260 2024-03-26T14:15:18.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f9571200640 1 -- 172.21.15.137:0/2484694160 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c015070 con 0x564018510bc0 2024-03-26T14:15:18.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f957b400640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f95600a2800 0x7f95600a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f9571200640 1 -- 172.21.15.137:0/2484694160 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f956c01a910 con 0x564018510bc0 2024-03-26T14:15:18.857 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.854+0000 7f9571200640 1 -- 172.21.15.137:0/2484694160 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c011990 con 0x564018510bc0 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 -- 172.21.15.137:0/2484694160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 msgr2=0x564018511040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f957b400640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f95600a2800 0x7f95600a4cc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x564018510ac0 tx=0x7f9568004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018511040 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f956c00bcb0 tx=0x7f956c00f730 comp rx=0 tx=0).stop 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 -- 172.21.15.137:0/2484694160 shutdown_connections 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564018511580 0x564018517e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018511040 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 --2- 172.21.15.137:0/2484694160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56401850f9f0 0x56401850fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 -- 172.21.15.137:0/2484694160 >> 172.21.15.137:0/2484694160 conn(0x56401850ade0 msgr2=0x56401850b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 -- 172.21.15.137:0/2484694160 shutdown_connections 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.855+0000 7f9585806740 1 -- 172.21.15.137:0/2484694160 wait complete. 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 -- start start 2024-03-26T14:15:18.862 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 Processor -- start 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018519c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018510bc0 0x56401851a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 0x5640185211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564018518590 con 0x564018511580 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640184300d0 con 0x564018510bc0 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.856+0000 7f9585806740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564018518710 con 0x56401850f9f0 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9585806740 10 client.0 _ll_register_callbacks cb 0x564018434e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 0x5640185211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 0x5640185211a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41430/0 (socket says 172.21.15.137:41430) 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 -- 172.21.15.137:0/2171661946 learned_addr learned my addr 172.21.15.137:0/2171661946 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f957b400640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018510bc0 0x56401851a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.863 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f957be00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018519c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 -- 172.21.15.137:0/2171661946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 msgr2=0x564018519c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018519c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 -- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018510bc0 msgr2=0x56401851a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018510bc0 0x56401851a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f9580c00640 1 -- 172.21.15.137:0/2171661946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640181883f0 con 0x564018511580 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f957be00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018519c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.857+0000 7f957b400640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018510bc0 0x56401851a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9580c00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 0x5640185211a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f95740b9d70 tx=0x7f95740bd510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c419fa4dfab4bfc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9565600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9565600640 1 -- 172.21.15.137:0/2171661946 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740bd910 con 0x564018511580 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9565600640 1 -- 172.21.15.137:0/2171661946 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f95740bdb70 con 0x564018511580 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f956c01a910 con 0x564018511580 2024-03-26T14:15:18.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9565600640 1 -- 172.21.15.137:0/2171661946 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95740c1a60 con 0x564018511580 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9585806740 10 client.18339 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.858+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5640182527d0 con 0x564018511580 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9565600640 1 -- 172.21.15.137:0/2171661946 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f956c01a910 con 0x564018511580 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9565600640 1 -- 172.21.15.137:0/2171661946 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f957403c030 con 0x564018511580 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9585806740 10 client.18339 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9585806740 10 client.18339 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9585806740 20 client.18339 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9585806740 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5640185256c0 0x564018527b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9585806740 4 client.18339 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564018481800 con 0x5640185256c0 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.859+0000 7f957be00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5640185256c0 0x564018527b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.860+0000 7f957be00640 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5640185256c0 0x564018527b00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=143ab7dd650fd4ee server_cookie=8a4866bb0582817 in_seq=0 out_seq=0 2024-03-26T14:15:18.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.860+0000 7f9565600640 10 client.18339 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.862+0000 7f9565600640 1 -- 172.21.15.137:0/2171661946 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564018481800 con 0x5640185256c0 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.862+0000 7f9565600640 10 client.18339 handle_command_reply: tid=0 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.862+0000 7f9585806740 1 client.18339 shutdown 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.862+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 msgr2=0x5640185211a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.862+0000 7f9585806740 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 0x5640185211a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f95740b9d70 tx=0x7f95740bd510 comp rx=0 tx=0).stop 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 shutdown_connections 2024-03-26T14:15:18.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5640185256c0 0x564018527b00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018510bc0 0x56401851a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018511580 0x5640185211a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 --2- 172.21.15.137:0/2171661946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018519c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 >> 172.21.15.137:0/2171661946 conn(0x56401850ade0 msgr2=0x56401850a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9565600640 0 client.18339 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 shutdown_connections 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 1 -- 172.21.15.137:0/2171661946 wait complete. 2024-03-26T14:15:18.868 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.863+0000 7f9585806740 20 client.18339 trim_cache size 0 max 16384 2024-03-26T14:15:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.878+0000 7f9585806740 1 Processor -- start 2024-03-26T14:15:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 -- start start 2024-03-26T14:15:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018505ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018521160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x564018513c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1ae710 con 0x564018510bc0 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1ac210 con 0x564018511580 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f9585806740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1aa290 con 0x56401850f9f0 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f957b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018521160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f957b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018521160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41434/0 (socket says 172.21.15.137:41434) 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.879+0000 7f957b400640 1 -- 172.21.15.137:0/2446428168 learned_addr learned my addr 172.21.15.137:0/2446428168 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957be00640 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018505ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f9580c00640 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x564018513c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957b400640 1 -- 172.21.15.137:0/2446428168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 msgr2=0x564018505ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957b400640 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018505ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957b400640 1 -- 172.21.15.137:0/2446428168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 msgr2=0x564018513c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957b400640 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x564018513c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957b400640 1 -- 172.21.15.137:0/2446428168 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564018510640 con 0x564018510bc0 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f957b400640 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018521160 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f9568011840 tx=0x7f9568016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57134bea0b6e0f8e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f9567e00640 1 -- 172.21.15.137:0/2446428168 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800f9a0 con 0x564018510bc0 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f9567e00640 1 -- 172.21.15.137:0/2446428168 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9568026070 con 0x564018510bc0 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.880+0000 7f9567e00640 1 -- 172.21.15.137:0/2446428168 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956802a410 con 0x564018510bc0 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 -- 172.21.15.137:0/2446428168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 msgr2=0x564018521160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018521160 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f9568011840 tx=0x7f9568016000 comp rx=0 tx=0).stop 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 -- 172.21.15.137:0/2446428168 shutdown_connections 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x564018513c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018521160 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 --2- 172.21.15.137:0/2446428168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x564018505ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 -- 172.21.15.137:0/2446428168 >> 172.21.15.137:0/2446428168 conn(0x56401850ade0 msgr2=0x56401850e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 -- 172.21.15.137:0/2446428168 shutdown_connections 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.881+0000 7f9585806740 1 -- 172.21.15.137:0/2446428168 wait complete. 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 -- start start 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 Processor -- start 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x5640185209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018520ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x56401851d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1aa890 con 0x564018510bc0 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1abbb0 con 0x564018511580 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.882+0000 7f9585806740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1aaef0 con 0x56401850f9f0 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f9585806740 10 client.0 _ll_register_callbacks cb 0x7f9574034790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:18.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018520ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018520ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41438/0 (socket says 172.21.15.137:41438) 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 -- 172.21.15.137:0/40715705 learned_addr learned my addr 172.21.15.137:0/40715705 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f9580c00640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x56401851d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957be00640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x5640185209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 -- 172.21.15.137:0/40715705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 msgr2=0x5640185209a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x5640185209a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 -- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 msgr2=0x56401851d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x56401851d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.883+0000 7f957b400640 1 -- 172.21.15.137:0/40715705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56401851dae0 con 0x564018510bc0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f957b400640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018520ee0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f956800cbe0 tx=0x7f9568011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0532ab166e6a266 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9565600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9565600640 1 -- 172.21.15.137:0/40715705 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800fb30 con 0x564018510bc0 2024-03-26T14:15:18.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9565600640 1 -- 172.21.15.137:0/40715705 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f956800fcd0 con 0x564018510bc0 2024-03-26T14:15:18.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9568026070 con 0x564018510bc0 2024-03-26T14:15:18.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9565600640 1 -- 172.21.15.137:0/40715705 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9568034420 con 0x564018510bc0 2024-03-26T14:15:18.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9585806740 10 client.18345 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:18.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564018581bb0 con 0x564018510bc0 2024-03-26T14:15:18.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.884+0000 7f9565600640 1 -- 172.21.15.137:0/40715705 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9568026070 con 0x564018510bc0 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9565600640 1 -- 172.21.15.137:0/40715705 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f956802a510 con 0x564018510bc0 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9585806740 10 client.18345 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9585806740 10 client.18345 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9585806740 20 client.18345 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9585806740 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x564018582480 0x5640185848a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9585806740 4 client.18345 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56401847bb50 con 0x564018582480 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.885+0000 7f957be00640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x564018582480 0x5640185848a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:18.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.886+0000 7f957be00640 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x564018582480 0x5640185848a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d84a1451edd3440d server_cookie=3b824d3f61329332 in_seq=0 out_seq=0 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.886+0000 7f9565600640 10 client.18345 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9565600640 1 -- 172.21.15.137:0/40715705 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56401847bb50 con 0x564018582480 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9565600640 10 client.18345 handle_command_reply: tid=0 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 client.18345 shutdown 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 msgr2=0x564018520ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018520ee0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f956800cbe0 tx=0x7f9568011f30 comp rx=0 tx=0).stop 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 shutdown_connections 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x564018582480 0x5640185848a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564018511580 0x56401851d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564018510bc0 0x564018520ee0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 --2- 172.21.15.137:0/40715705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56401850f9f0 0x5640185209a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 >> 172.21.15.137:0/40715705 conn(0x56401850ade0 msgr2=0x5640185098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9565600640 0 client.18345 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:18.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 shutdown_connections 2024-03-26T14:15:18.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.887+0000 7f9585806740 1 -- 172.21.15.137:0/40715705 wait complete. 2024-03-26T14:15:18.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9585806740 20 client.18345 trim_cache size 0 max 16384 2024-03-26T14:15:18.892 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f95600a2800 msgr2=0x7f95600a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f95600a2800 0x7f95600a4cc0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x564018510ac0 tx=0x7f9568004730 comp rx=0 tx=0).stop 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 msgr2=0x7f957c1004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1004f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f957402c700 tx=0x7f95740328a0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 shutdown_connections 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f95600a2800 0x7f95600a4cc0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f957c107260 0x7f957c1004f0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f957c104930 0x7f957c0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 --2- 172.21.15.137:0/4061116107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f957c102000 0x7f957c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.888+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 >> 172.21.15.137:0/4061116107 conn(0x7f957c0fba10 msgr2=0x7f957c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:18.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.889+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 shutdown_connections 2024-03-26T14:15:18.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:18.889+0000 7f9581600640 1 -- 172.21.15.137:0/4061116107 wait complete. 2024-03-26T14:15:18.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:18 smithi137 ceph-mon[31539]: pgmap v846: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:18.904 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:18 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3040797051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:15:18.907 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:15:18.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:15:19.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.103+0000 7fe219000640 1 -- 172.21.15.137:0/236725690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214107380 msgr2=0x7fe2141097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.103+0000 7fe219000640 1 --2- 172.21.15.137:0/236725690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214107380 0x7fe2141097d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009ca0 tx=0x7fe20c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 -- 172.21.15.137:0/236725690 shutdown_connections 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 --2- 172.21.15.137:0/236725690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe214104a50 0x7fe214106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 --2- 172.21.15.137:0/236725690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214107380 0x7fe2141097d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 --2- 172.21.15.137:0/236725690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 0x7fe214104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 -- 172.21.15.137:0/236725690 >> 172.21.15.137:0/236725690 conn(0x7fe2140fbb30 msgr2=0x7fe2140fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 -- 172.21.15.137:0/236725690 shutdown_connections 2024-03-26T14:15:19.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.104+0000 7fe219000640 1 -- 172.21.15.137:0/236725690 wait complete. 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.105+0000 7fe219000640 1 Processor -- start 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.105+0000 7fe219000640 1 -- start start 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.105+0000 7fe219000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 0x7fe214199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.105+0000 7fe219000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 0x7fe21419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.105+0000 7fe219000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe214107380 0x7fe2141a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe219000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410d1d0 con 0x7fe214104a50 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe219000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410d050 con 0x7fe214107380 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe219000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21410d350 con 0x7fe214102120 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 0x7fe21419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 0x7fe21419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41456/0 (socket says 172.21.15.137:41456) 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 -- 172.21.15.137:0/1617228944 learned_addr learned my addr 172.21.15.137:0/1617228944 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe212800640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe214107380 0x7fe2141a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211e00640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 0x7fe214199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 -- 172.21.15.137:0/1617228944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 msgr2=0x7fe214199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 0x7fe214199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 -- 172.21.15.137:0/1617228944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe214107380 msgr2=0x7fe2141a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe214107380 0x7fe2141a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211400640 1 -- 172.21.15.137:0/1617228944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2141a16f0 con 0x7fe214104a50 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.106+0000 7fe211e00640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 0x7fe214199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.107+0000 7fe211400640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 0x7fe21419a010 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe20000e9c0 tx=0x7fe20000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.107+0000 7fe20aa00640 1 -- 172.21.15.137:0/1617228944 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20000cd80 con 0x7fe214104a50 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.107+0000 7fe20aa00640 1 -- 172.21.15.137:0/1617228944 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe2000054d0 con 0x7fe214104a50 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.107+0000 7fe20aa00640 1 -- 172.21.15.137:0/1617228944 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200010430 con 0x7fe214104a50 2024-03-26T14:15:19.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.107+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2141a19e0 con 0x7fe214104a50 2024-03-26T14:15:19.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.107+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141a1ee0 con 0x7fe214104a50 2024-03-26T14:15:19.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.111+0000 7fe20aa00640 1 -- 172.21.15.137:0/1617228944 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe200004fd0 con 0x7fe214104a50 2024-03-26T14:15:19.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.111+0000 7fe21bd6a740 1 Processor -- start 2024-03-26T14:15:19.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.111+0000 7fe21bd6a740 1 -- start start 2024-03-26T14:15:19.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.111+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfc7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfc8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfcfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88bf2c2c0 con 0x55f88bfc93a0 2024-03-26T14:15:19.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88bed4c90 con 0x55f88bfc7810 2024-03-26T14:15:19.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88bf28360 con 0x55f88bfc89e0 2024-03-26T14:15:19.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfcfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfcfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41470/0 (socket says 172.21.15.137:41470) 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe211400640 1 -- 172.21.15.137:0/3163195754 learned_addr learned my addr 172.21.15.137:0/3163195754 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe212800640 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfc7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.112+0000 7fe211e00640 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfc8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe211400640 1 -- 172.21.15.137:0/3163195754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 msgr2=0x55f88bfc8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe211400640 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfc8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe211400640 1 -- 172.21.15.137:0/3163195754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 msgr2=0x55f88bfc7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe211400640 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfc7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe211400640 1 -- 172.21.15.137:0/3163195754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f88bef9080 con 0x55f88bfc93a0 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe20aa00640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2800 0x7fe1f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe20aa00640 1 -- 172.21.15.137:0/1617228944 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe2000cf2b0 con 0x7fe214104a50 2024-03-26T14:15:19.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe212800640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2800 0x7fe1f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.113+0000 7fe211400640 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfcfc30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe2000d7320 tx=0x7fe2000d7410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c34460566f27344 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.114+0000 7fe1ff400640 1 -- 172.21.15.137:0/3163195754 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20009b650 con 0x55f88bfc93a0 2024-03-26T14:15:19.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.114+0000 7fe1ff400640 1 -- 172.21.15.137:0/3163195754 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe20009bf80 con 0x55f88bfc93a0 2024-03-26T14:15:19.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.114+0000 7fe1ff400640 1 -- 172.21.15.137:0/3163195754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2000afc60 con 0x55f88bfc93a0 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.114+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3163195754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 msgr2=0x55f88bfcfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.114+0000 7fe212800640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2800 0x7fe1f80a4cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02c630 tx=0x7fe20c009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.114+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfcfc30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe2000d7320 tx=0x7fe2000d7410 comp rx=0 tx=0).stop 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3163195754 shutdown_connections 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfcfc30 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfc8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3163195754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfc7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3163195754 >> 172.21.15.137:0/3163195754 conn(0x55f88bfc2c00 msgr2=0x55f88bfc3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3163195754 shutdown_connections 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3163195754 wait complete. 2024-03-26T14:15:19.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 -- start start 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.115+0000 7fe21bd6a740 1 Processor -- start 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfd1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfd9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88bfd03b0 con 0x55f88bfc89e0 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88bee80d0 con 0x55f88bfc93a0 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88bfd0530 con 0x55f88bfc7810 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe21bd6a740 10 client.0 _ll_register_callbacks cb 0x55f88beece80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41480/0 (socket says 172.21.15.137:41480) 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe211400640 1 -- 172.21.15.137:0/3436650508 learned_addr learned my addr 172.21.15.137:0/3436650508 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe212800640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfd9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.116+0000 7fe211e00640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfd1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe211400640 1 -- 172.21.15.137:0/3436650508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 msgr2=0x55f88bfd1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe211400640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfd1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe211400640 1 -- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 msgr2=0x55f88bfd9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe211400640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfd9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe211400640 1 -- 172.21.15.137:0/3436650508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f88bc403f0 con 0x55f88bfc89e0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe211400640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd1fd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe2000d72f0 tx=0x7fe2000cf5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84b74ae42877e1d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe1f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe1f3e00640 1 -- 172.21.15.137:0/3436650508 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20009b1e0 con 0x55f88bfc89e0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe1f3e00640 1 -- 172.21.15.137:0/3436650508 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe2000860c0 con 0x55f88bfc89e0 2024-03-26T14:15:19.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.117+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe20009bf80 con 0x55f88bfc89e0 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe1f3e00640 1 -- 172.21.15.137:0/3436650508 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20009b3c0 con 0x55f88bfc89e0 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 10 client.18357 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f88bd0a7d0 con 0x55f88bfc89e0 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe1f3e00640 1 -- 172.21.15.137:0/3436650508 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe20009bf80 con 0x55f88bfc89e0 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe1f3e00640 1 -- 172.21.15.137:0/3436650508 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe20009bae0 con 0x55f88bfc89e0 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 10 client.18357 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 10 client.18357 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 20 client.18357 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88bfdd170 0x55f88bfdf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 4 client.18357 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.118+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f88bf39800 con 0x55f88bfdd170 2024-03-26T14:15:19.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.119+0000 7fe211e00640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88bfdd170 0x55f88bfdf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.120+0000 7fe211e00640 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88bfdd170 0x55f88bfdf560 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc118e96d90d4ac0 server_cookie=e1a2c6460486dbd1 in_seq=0 out_seq=0 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.120+0000 7fe1f3e00640 10 client.18357 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.121+0000 7fe1f3e00640 1 -- 172.21.15.137:0/3436650508 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f88bf39800 con 0x55f88bfdd170 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.121+0000 7fe1f3e00640 10 client.18357 handle_command_reply: tid=0 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 client.18357 shutdown 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 msgr2=0x55f88bfd1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd1fd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe2000d72f0 tx=0x7fe2000cf5a0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 shutdown_connections 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88bfdd170 0x55f88bfdf560 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfd9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd1fd0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/3436650508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfd1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 >> 172.21.15.137:0/3436650508 conn(0x55f88bfc2c00 msgr2=0x55f88bfc1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe1f3e00640 0 client.18357 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:19.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 shutdown_connections 2024-03-26T14:15:19.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 1 -- 172.21.15.137:0/3436650508 wait complete. 2024-03-26T14:15:19.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.122+0000 7fe21bd6a740 20 client.18357 trim_cache size 0 max 16384 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 Processor -- start 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 -- start start 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfcc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfcc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141ae990 con 0x55f88bfc89e0 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141ac490 con 0x55f88bfc93a0 2024-03-26T14:15:19.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141aa510 con 0x55f88bfc7810 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe211400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41496/0 (socket says 172.21.15.137:41496) 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe211400640 1 -- 172.21.15.137:0/1794078026 learned_addr learned my addr 172.21.15.137:0/1794078026 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.138+0000 7fe212800640 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfcc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211e00640 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfcc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211400640 1 -- 172.21.15.137:0/1794078026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 msgr2=0x55f88bfcc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211400640 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfcc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211400640 1 -- 172.21.15.137:0/1794078026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 msgr2=0x55f88bfcc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211400640 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfcc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211400640 1 -- 172.21.15.137:0/1794078026 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f88bfc3670 con 0x55f88bfc89e0 2024-03-26T14:15:19.141 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe211400640 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8fd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe200097ce0 tx=0x7fe200099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955843c3563305bf server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe1fea00640 1 -- 172.21.15.137:0/1794078026 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20009b1e0 con 0x55f88bfc89e0 2024-03-26T14:15:19.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe1fea00640 1 -- 172.21.15.137:0/1794078026 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe20009bf80 con 0x55f88bfc89e0 2024-03-26T14:15:19.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.139+0000 7fe1fea00640 1 -- 172.21.15.137:0/1794078026 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20009b650 con 0x55f88bfc89e0 2024-03-26T14:15:19.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1794078026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 msgr2=0x55f88bfd8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8fd0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe200097ce0 tx=0x7fe200099d90 comp rx=0 tx=0).stop 2024-03-26T14:15:19.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1794078026 shutdown_connections 2024-03-26T14:15:19.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc93a0 0x55f88bfcc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8fd0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1794078026 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc7810 0x55f88bfcc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1794078026 >> 172.21.15.137:0/1794078026 conn(0x55f88bfc2c00 msgr2=0x55f88bfcba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1794078026 shutdown_connections 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.140+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1794078026 wait complete. 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 -- start start 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 Processor -- start 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfd8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfd5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141aab10 con 0x55f88bfc93a0 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141abe30 con 0x55f88bfc7810 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.141+0000 7fe21bd6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2141ab170 con 0x55f88bfc89e0 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe21bd6a740 10 client.0 _ll_register_callbacks cb 0x7fe200005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfd5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfd5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41510/0 (socket says 172.21.15.137:41510) 2024-03-26T14:15:19.148 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 -- 172.21.15.137:0/1314138098 learned_addr learned my addr 172.21.15.137:0/1314138098 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe211e00640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfd8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe211400640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 -- 172.21.15.137:0/1314138098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 msgr2=0x55f88bfd8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 -- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 msgr2=0x55f88bfd8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfd8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.142+0000 7fe212800640 1 -- 172.21.15.137:0/1314138098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f88bfd5ca0 con 0x55f88bfc93a0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe212800640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfd5470 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009a60 tx=0x7fe20c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9420fedab3f61fcb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe1f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe1f3e00640 1 -- 172.21.15.137:0/1314138098 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c039040 con 0x55f88bfc93a0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe1f3e00640 1 -- 172.21.15.137:0/1314138098 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe20c04f440 con 0x55f88bfc93a0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe20009bf80 con 0x55f88bfc93a0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe1f3e00640 1 -- 172.21.15.137:0/1314138098 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe20c04f640 con 0x55f88bfc93a0 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe21bd6a740 10 client.18363 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:19.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.143+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f88c039560 con 0x55f88bfc93a0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe1f3e00640 1 -- 172.21.15.137:0/1314138098 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe20009bf80 con 0x55f88bfc93a0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe1f3e00640 1 -- 172.21.15.137:0/1314138098 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe20c034980 con 0x55f88bfc93a0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe21bd6a740 10 client.18363 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe21bd6a740 10 client.18363 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe21bd6a740 20 client.18363 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88c039ef0 0x55f88c03c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe21bd6a740 4 client.18363 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f88bf33b50 con 0x55f88c039ef0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.144+0000 7fe211e00640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88c039ef0 0x55f88c03c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.145+0000 7fe211e00640 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88c039ef0 0x55f88c03c310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5f75089e7b751db server_cookie=9aab2c1d4d104339 in_seq=0 out_seq=0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.145+0000 7fe1f3e00640 10 client.18363 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.145+0000 7fe1f3e00640 1 -- 172.21.15.137:0/1314138098 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f88bf33b50 con 0x55f88c039ef0 2024-03-26T14:15:19.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.145+0000 7fe1f3e00640 10 client.18363 handle_command_reply: tid=0 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.145+0000 7fe21bd6a740 1 client.18363 shutdown 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 msgr2=0x55f88bfd5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfd5470 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009a60 tx=0x7fe20c0095c0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 shutdown_connections 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55f88c039ef0 0x55f88c03c310 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f88bfc93a0 0x55f88bfd5470 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f88bfc89e0 0x55f88bfd8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 --2- 172.21.15.137:0/1314138098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f88bfc7810 0x55f88bfd8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 >> 172.21.15.137:0/1314138098 conn(0x55f88bfc2c00 msgr2=0x55f88bfe44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe1f3e00640 0 client.18363 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 shutdown_connections 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 1 -- 172.21.15.137:0/1314138098 wait complete. 2024-03-26T14:15:19.151 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.146+0000 7fe21bd6a740 20 client.18363 trim_cache size 0 max 16384 2024-03-26T14:15:19.152 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:15:19.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2800 msgr2=0x7fe1f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2800 0x7fe1f80a4cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe20c02c630 tx=0x7fe20c009360 comp rx=0 tx=0).stop 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 msgr2=0x7fe21419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 0x7fe21419a010 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fe20000e9c0 tx=0x7fe20000ee80 comp rx=0 tx=0).stop 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 shutdown_connections 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe1f80a2800 0x7fe1f80a4cc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe214107380 0x7fe2141a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe214104a50 0x7fe21419a010 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 --2- 172.21.15.137:0/1617228944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe214102120 0x7fe214199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 >> 172.21.15.137:0/1617228944 conn(0x7fe2140fbb30 msgr2=0x7fe214101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 shutdown_connections 2024-03-26T14:15:19.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.147+0000 7fe219000640 1 -- 172.21.15.137:0/1617228944 wait complete. 2024-03-26T14:15:19.166 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:15:19.166 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:15:19.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:18 smithi179 ceph-mon[35472]: pgmap v846: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:19.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:18 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3040797051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:15:19.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:18 smithi159 ceph-mon[35501]: pgmap v846: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:19.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:18 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3040797051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:15:19.380 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.379+0000 7f8a3be00640 1 -- 172.21.15.137:0/4181693196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c102760 msgr2=0x7f8a3c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.379+0000 7f8a3be00640 1 --2- 172.21.15.137:0/4181693196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c102760 0x7f8a3c104bd0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009900 tx=0x7f8a2802bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.379+0000 7f8a3be00640 1 -- 172.21.15.137:0/4181693196 shutdown_connections 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.379+0000 7f8a3be00640 1 --2- 172.21.15.137:0/4181693196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3c105110 0x7f8a3c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.380+0000 7f8a3be00640 1 --2- 172.21.15.137:0/4181693196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c102760 0x7f8a3c104bd0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.380+0000 7f8a3be00640 1 --2- 172.21.15.137:0/4181693196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a3c0ffdf0 0x7f8a3c102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.380+0000 7f8a3be00640 1 -- 172.21.15.137:0/4181693196 >> 172.21.15.137:0/4181693196 conn(0x7f8a3c0fba10 msgr2=0x7f8a3c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.380+0000 7f8a3be00640 1 -- 172.21.15.137:0/4181693196 shutdown_connections 2024-03-26T14:15:19.381 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.380+0000 7f8a3be00640 1 -- 172.21.15.137:0/4181693196 wait complete. 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 Processor -- start 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 -- start start 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3c0ffdf0 0x7f8a3c19dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a3c102760 0x7f8a3c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 0x7f8a3c1a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c113680 con 0x7f8a3c105110 2024-03-26T14:15:19.382 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c113500 con 0x7f8a3c102760 2024-03-26T14:15:19.383 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.381+0000 7f8a3be00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c113800 con 0x7f8a3c0ffdf0 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 0x7f8a3c1a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 0x7f8a3c1a5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41518/0 (socket says 172.21.15.137:41518) 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 -- 172.21.15.137:0/848515847 learned_addr learned my addr 172.21.15.137:0/848515847 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3a000640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a3c102760 0x7f8a3c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3c0ffdf0 0x7f8a3c19dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 -- 172.21.15.137:0/848515847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3c0ffdf0 msgr2=0x7f8a3c19dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3c0ffdf0 0x7f8a3c19dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 -- 172.21.15.137:0/848515847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a3c102760 msgr2=0x7f8a3c19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a3c102760 0x7f8a3c19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.382+0000 7f8a3b400640 1 -- 172.21.15.137:0/848515847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c1a5970 con 0x7f8a3c105110 2024-03-26T14:15:19.384 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.383+0000 7f8a3b400640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 0x7f8a3c1a5300 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400eaa0 tx=0x7f8a3400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.383+0000 7f8a33400640 1 -- 172.21.15.137:0/848515847 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3400cdb0 con 0x7f8a3c105110 2024-03-26T14:15:19.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.383+0000 7f8a33400640 1 -- 172.21.15.137:0/848515847 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a34005020 con 0x7f8a3c105110 2024-03-26T14:15:19.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.383+0000 7f8a33400640 1 -- 172.21.15.137:0/848515847 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a340116d0 con 0x7f8a3c105110 2024-03-26T14:15:19.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.383+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c1a5c60 con 0x7f8a3c105110 2024-03-26T14:15:19.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.383+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c1a6130 con 0x7f8a3c105110 2024-03-26T14:15:19.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a33400640 1 -- 172.21.15.137:0/848515847 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8a340051d0 con 0x7f8a3c105110 2024-03-26T14:15:19.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 Processor -- start 2024-03-26T14:15:19.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 -- start start 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 0x55e58072ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072bbc0 0x55e58072c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e580732e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58068f2c0 con 0x55e58072bbc0 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e580637c90 con 0x55e58072a9f0 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.388+0000 7f8a443b0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58068b360 con 0x55e58072c580 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 0x55e58072ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 0x55e58072ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:42120/0 (socket says 172.21.15.137:42120) 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 -- 172.21.15.137:0/2431224440 learned_addr learned my addr 172.21.15.137:0/2431224440 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072bbc0 0x55e58072c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e580732e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 -- 172.21.15.137:0/2431224440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 msgr2=0x55e580732e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e580732e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 -- 172.21.15.137:0/2431224440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072bbc0 msgr2=0x55e58072c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072bbc0 0x55e58072c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3a000640 1 -- 172.21.15.137:0/2431224440 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e58065c080 con 0x55e58072a9f0 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.389+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072bbc0 0x55e58072c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a33400640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a200a2950 0x7f8a200a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a33400640 1 -- 172.21.15.137:0/848515847 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8a340cef50 con 0x7f8a3c105110 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a3a000640 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 0x55e58072ae10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a2802c840 tx=0x7f8a280095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9adf2de979042c66 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a200a2950 0x7f8a200a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a27400640 1 -- 172.21.15.137:0/2431224440 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28004030 con 0x55e58072a9f0 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a27400640 1 -- 172.21.15.137:0/2431224440 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a28048070 con 0x55e58072a9f0 2024-03-26T14:15:19.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.390+0000 7f8a27400640 1 -- 172.21.15.137:0/2431224440 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28004920 con 0x55e58072a9f0 2024-03-26T14:15:19.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 -- 172.21.15.137:0/2431224440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 msgr2=0x55e58072ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a200a2950 0x7f8a200a4e10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009ec0 tx=0x7f8a2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 0x55e58072ae10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8a2802c840 tx=0x7f8a280095a0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 -- 172.21.15.137:0/2431224440 shutdown_connections 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e580732e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072bbc0 0x55e58072c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2431224440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072a9f0 0x55e58072ae10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 -- 172.21.15.137:0/2431224440 >> 172.21.15.137:0/2431224440 conn(0x55e580725de0 msgr2=0x55e5807261e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.391+0000 7f8a443b0740 1 -- 172.21.15.137:0/2431224440 shutdown_connections 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.392+0000 7f8a443b0740 1 -- 172.21.15.137:0/2431224440 wait complete. 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.392+0000 7f8a443b0740 1 -- start start 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.392+0000 7f8a443b0740 1 Processor -- start 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e580734c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e580735150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e58073c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e580733530 con 0x55e58072a9f0 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e58064b0d0 con 0x55e58072bbc0 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5807336b0 con 0x55e58072c580 2024-03-26T14:15:19.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a443b0740 10 client.0 _ll_register_callbacks cb 0x55e58064fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e580734c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e580734c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41542/0 (socket says 172.21.15.137:41542) 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 -- 172.21.15.137:0/4228409999 learned_addr learned my addr 172.21.15.137:0/4228409999 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3a000640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e580735150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3b400640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e58073c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 -- 172.21.15.137:0/4228409999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 msgr2=0x55e58073c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e58073c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 -- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 msgr2=0x55e580735150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.393+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e580735150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.394+0000 7f8a3aa00640 1 -- 172.21.15.137:0/4228409999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5803a33f0 con 0x55e58072a9f0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.394+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e580734c10 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c020a10 tx=0x7f8a2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3477097794419c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.394+0000 7f8a1be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.394+0000 7f8a1be00640 1 -- 172.21.15.137:0/4228409999 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c009b10 con 0x55e58072a9f0 2024-03-26T14:15:19.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.394+0000 7f8a1be00640 1 -- 172.21.15.137:0/4228409999 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8a2c004260 con 0x55e58072a9f0 2024-03-26T14:15:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.394+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a28048070 con 0x55e58072a9f0 2024-03-26T14:15:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a1be00640 1 -- 172.21.15.137:0/4228409999 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a2c009760 con 0x55e58072a9f0 2024-03-26T14:15:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 10 client.18375 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e58046d7d0 con 0x55e58072a9f0 2024-03-26T14:15:19.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a1be00640 1 -- 172.21.15.137:0/4228409999 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a28048070 con 0x55e58072a9f0 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a1be00640 1 -- 172.21.15.137:0/4228409999 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8a2c027070 con 0x55e58072a9f0 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 10 client.18375 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 10 client.18375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 20 client.18375 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e580740750 0x55e580742bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 4 client.18375 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.395+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e58069c800 con 0x55e580740750 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.396+0000 7f8a3a000640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e580740750 0x55e580742bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.396+0000 7f8a3a000640 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e580740750 0x55e580742bb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ceed05f31a989d9 server_cookie=cf4a350ae49b5b4c in_seq=0 out_seq=0 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.397+0000 7f8a1be00640 10 client.18375 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.398+0000 7f8a1be00640 1 -- 172.21.15.137:0/4228409999 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55e58069c800 con 0x55e580740750 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.398+0000 7f8a1be00640 10 client.18375 handle_command_reply: tid=0 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.398+0000 7f8a443b0740 1 client.18375 shutdown 2024-03-26T14:15:19.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 msgr2=0x55e580734c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e580734c10 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c020a10 tx=0x7f8a2c004730 comp rx=0 tx=0).stop 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 shutdown_connections 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e580740750 0x55e580742bb0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e580735150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072c580 0x55e58073c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a1be00640 0 client.18375 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 --2- 172.21.15.137:0/4228409999 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e580734c10 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 >> 172.21.15.137:0/4228409999 conn(0x55e580725de0 msgr2=0x55e580725160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 shutdown_connections 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 1 -- 172.21.15.137:0/4228409999 wait complete. 2024-03-26T14:15:19.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.399+0000 7f8a443b0740 20 client.18375 trim_cache size 0 max 16384 2024-03-26T14:15:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.415+0000 7f8a443b0740 1 Processor -- start 2024-03-26T14:15:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.415+0000 7f8a443b0740 1 -- start start 2024-03-26T14:15:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.415+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e5807350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.415+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072bbc0 0x55e58073c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072c580 0x55e580729750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.417 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a443b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1b29b0 con 0x55e58072a9f0 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a443b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1b04b0 con 0x55e58072c580 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a443b0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1ae530 con 0x55e58072bbc0 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e5807350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e5807350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41546/0 (socket says 172.21.15.137:41546) 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3aa00640 1 -- 172.21.15.137:0/3435611483 learned_addr learned my addr 172.21.15.137:0/3435611483 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3b400640 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072c580 0x55e580729750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3a000640 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072bbc0 0x55e58073c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3b400640 1 -- 172.21.15.137:0/3435611483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072bbc0 msgr2=0x55e58073c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3b400640 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072bbc0 0x55e58073c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3b400640 1 -- 172.21.15.137:0/3435611483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 msgr2=0x55e5807350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3b400640 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e5807350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.418 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3b400640 1 -- 172.21.15.137:0/3435611483 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e580740530 con 0x55e58072c580 2024-03-26T14:15:19.419 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.416+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e5807350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.417+0000 7f8a3b400640 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072c580 0x55e580729750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400efd0 tx=0x7f8a340978b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee6618a7283cb89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.417+0000 7f8a26a00640 1 -- 172.21.15.137:0/3435611483 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3401f040 con 0x55e58072c580 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.417+0000 7f8a26a00640 1 -- 172.21.15.137:0/3435611483 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a3409f630 con 0x55e58072c580 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.417+0000 7f8a26a00640 1 -- 172.21.15.137:0/3435611483 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3409f170 con 0x55e58072c580 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 -- 172.21.15.137:0/3435611483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072c580 msgr2=0x55e580729750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072c580 0x55e580729750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400efd0 tx=0x7f8a340978b0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 -- 172.21.15.137:0/3435611483 shutdown_connections 2024-03-26T14:15:19.423 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072c580 0x55e580729750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072bbc0 0x55e58073c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 --2- 172.21.15.137:0/3435611483 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072a9f0 0x55e5807350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 -- 172.21.15.137:0/3435611483 >> 172.21.15.137:0/3435611483 conn(0x55e580725de0 msgr2=0x55e580729040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 -- 172.21.15.137:0/3435611483 shutdown_connections 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 -- 172.21.15.137:0/3435611483 wait complete. 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 -- start start 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.418+0000 7f8a443b0740 1 Processor -- start 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072a9f0 0x55e58073b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e58073be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 0x55e580738380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1aeb30 con 0x55e58072c580 2024-03-26T14:15:19.424 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1afe50 con 0x55e58072bbc0 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c1af190 con 0x55e58072a9f0 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a443b0740 10 client.0 _ll_register_callbacks cb 0x7f8a2c009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 0x55e580738380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a3b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 0x55e580738380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41560/0 (socket says 172.21.15.137:41560) 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a3b400640 1 -- 172.21.15.137:0/2609388247 learned_addr learned my addr 172.21.15.137:0/2609388247 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.419+0000 7f8a3a000640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e58073be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072a9f0 0x55e58073b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3b400640 1 -- 172.21.15.137:0/2609388247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072a9f0 msgr2=0x55e58073b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3b400640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072a9f0 0x55e58073b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3b400640 1 -- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 msgr2=0x55e58073be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3b400640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e58073be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3b400640 1 -- 172.21.15.137:0/2609388247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e580738970 con 0x55e58072c580 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072a9f0 0x55e58073b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:19.425 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a3b400640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 0x55e580738380 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8a34097eb0 tx=0x7f8a340055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a49dd42c5bcd40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a1be00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a1be00640 1 -- 172.21.15.137:0/2609388247 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3401f070 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a3409f630 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a1be00640 1 -- 172.21.15.137:0/2609388247 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8a3409f170 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a1be00640 1 -- 172.21.15.137:0/2609388247 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a3409f370 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a443b0740 10 client.18381 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.420+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e58079cbd0 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a1be00640 1 -- 172.21.15.137:0/2609388247 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8a3409f630 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a1be00640 1 -- 172.21.15.137:0/2609388247 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8a3400f040 con 0x55e58072c580 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a443b0740 10 client.18381 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a443b0740 10 client.18381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a443b0740 20 client.18381 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e58079d420 0x55e58079f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a443b0740 4 client.18381 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:15:19.426 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.421+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e580696b50 con 0x55e58079d420 2024-03-26T14:15:19.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.422+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e58079d420 0x55e58079f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:19.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.422+0000 7f8a1be00640 10 client.18381 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:19.427 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.422+0000 7f8a3aa00640 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e58079d420 0x55e58079f840 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd5fdc70eb86eed1 server_cookie=dc9ed6590d9d352a in_seq=0 out_seq=0 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a1be00640 1 -- 172.21.15.137:0/2609388247 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e580696b50 con 0x55e58079d420 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a1be00640 10 client.18381 handle_command_reply: tid=0 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 client.18381 shutdown 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 msgr2=0x55e580738380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 0x55e580738380 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8a34097eb0 tx=0x7f8a340055d0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 shutdown_connections 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55e58079d420 0x55e58079f840 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e58072bbc0 0x55e58073be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55e58072c580 0x55e580738380 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 --2- 172.21.15.137:0/2609388247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e58072a9f0 0x55e58073b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a1be00640 0 client.18381 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 >> 172.21.15.137:0/2609388247 conn(0x55e580725de0 msgr2=0x55e580734f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 shutdown_connections 2024-03-26T14:15:19.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 1 -- 172.21.15.137:0/2609388247 wait complete. 2024-03-26T14:15:19.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.423+0000 7f8a443b0740 20 client.18381 trim_cache size 0 max 16384 2024-03-26T14:15:19.429 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:15:19.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a200a2950 msgr2=0x7f8a200a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a200a2950 0x7f8a200a4e10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009ec0 tx=0x7f8a2c005f90 comp rx=0 tx=0).stop 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 msgr2=0x7f8a3c1a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 0x7f8a3c1a5300 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400eaa0 tx=0x7f8a3400ef60 comp rx=0 tx=0).stop 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 shutdown_connections 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a200a2950 0x7f8a200a4e10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a3c102760 0x7f8a3c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8a3c105110 0x7f8a3c1a5300 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 --2- 172.21.15.137:0/848515847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a3c0ffdf0 0x7f8a3c19dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 >> 172.21.15.137:0/848515847 conn(0x7f8a3c0fba10 msgr2=0x7f8a3c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 shutdown_connections 2024-03-26T14:15:19.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:19.424+0000 7f8a3be00640 1 -- 172.21.15.137:0/848515847 wait complete. 2024-03-26T14:15:20.444 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:15:20.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:15:20.445 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:15:20.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 -- 172.21.15.137:0/2618899011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 msgr2=0x7f2ed0104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2618899011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0104e10 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009900 tx=0x7f2ebc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:20.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 -- 172.21.15.137:0/2618899011 shutdown_connections 2024-03-26T14:15:20.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2618899011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 0x7f2ed010bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2618899011 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0104e10 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2618899011 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ed01037a0 0x7f2ed0103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 -- 172.21.15.137:0/2618899011 >> 172.21.15.137:0/2618899011 conn(0x7f2ed00fed10 msgr2=0x7f2ed0101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.676+0000 7f2ed6200640 1 -- 172.21.15.137:0/2618899011 shutdown_connections 2024-03-26T14:15:20.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.677+0000 7f2ed6200640 1 -- 172.21.15.137:0/2618899011 wait complete. 2024-03-26T14:15:20.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 Processor -- start 2024-03-26T14:15:20.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 -- start start 2024-03-26T14:15:20.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ed01037a0 0x7f2ed01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.679 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 0x7f2ed01a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed010f320 con 0x7f2ed0104990 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed010f1a0 con 0x7f2ed0105350 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.678+0000 7f2ed6200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed010f4a0 con 0x7f2ed01037a0 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41566/0 (socket says 172.21.15.137:41566) 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 -- 172.21.15.137:0/2009282354 learned_addr learned my addr 172.21.15.137:0/2009282354 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 0x7f2ed01a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ed01037a0 0x7f2ed01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 -- 172.21.15.137:0/2009282354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ed01037a0 msgr2=0x7f2ed01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ed01037a0 0x7f2ed01999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 -- 172.21.15.137:0/2009282354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 msgr2=0x7f2ed01a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 0x7f2ed01a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ecfe00640 1 -- 172.21.15.137:0/2009282354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed01a1610 con 0x7f2ed0104990 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.679+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 0x7f2ed01a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:20.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.680+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0199f30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc000c00 tx=0x7f2ebc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.680+0000 7f2ecd600640 1 -- 172.21.15.137:0/2009282354 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc043020 con 0x7f2ed0104990 2024-03-26T14:15:20.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.680+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed01a18a0 con 0x7f2ed0104990 2024-03-26T14:15:20.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.680+0000 7f2ecd600640 1 -- 172.21.15.137:0/2009282354 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ebc0054c0 con 0x7f2ed0104990 2024-03-26T14:15:20.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.680+0000 7f2ecd600640 1 -- 172.21.15.137:0/2009282354 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc003d40 con 0x7f2ed0104990 2024-03-26T14:15:20.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.680+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed01a1e40 con 0x7f2ed0104990 2024-03-26T14:15:20.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 Processor -- start 2024-03-26T14:15:20.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2ecd600640 1 -- 172.21.15.137:0/2009282354 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2ebc046020 con 0x7f2ed0104990 2024-03-26T14:15:20.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 -- start start 2024-03-26T14:15:20.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 0x55d56a8846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a8897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a88c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a7e9eb0 con 0x55d56a889e10 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a792a60 con 0x55d56a8842b0 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.684+0000 7f2eda563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a7e5f50 con 0x55d56a884ca0 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ed5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 0x55d56a8846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a88c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ed5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 0x55d56a8846d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:42178/0 (socket says 172.21.15.137:42178) 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ed5800640 1 -- 172.21.15.137:0/1009835889 learned_addr learned my addr 172.21.15.137:0/1009835889 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a8897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ecfe00640 1 -- 172.21.15.137:0/1009835889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 msgr2=0x55d56a8897a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a8897a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ecfe00640 1 -- 172.21.15.137:0/1009835889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 msgr2=0x55d56a8846d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 0x55d56a8846d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ecfe00640 1 -- 172.21.15.137:0/1009835889 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a5c67e0 con 0x55d56a889e10 2024-03-26T14:15:20.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a8897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.685+0000 7f2ed5800640 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 0x55d56a8846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a88c2a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0ee9a0 tx=0x7f2ebc0f93b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496e6d113bbc8bd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ecd600640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2eb80a2800 0x7f2eb80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ecd600640 1 -- 172.21.15.137:0/2009282354 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2ebc0f1310 con 0x7f2ed0104990 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2eb80a2800 0x7f2eb80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ec2000640 1 -- 172.21.15.137:0/1009835889 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0b9c60 con 0x55d56a889e10 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ec2000640 1 -- 172.21.15.137:0/1009835889 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ebc0fe030 con 0x55d56a889e10 2024-03-26T14:15:20.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.686+0000 7f2ec2000640 1 -- 172.21.15.137:0/1009835889 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0bd320 con 0x55d56a889e10 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 -- 172.21.15.137:0/1009835889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 msgr2=0x55d56a88c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2eb80a2800 0x7f2eb80a4cc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009fd0 tx=0x7f2ec80098a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a88c2a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0ee9a0 tx=0x7f2ebc0f93b0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 -- 172.21.15.137:0/1009835889 shutdown_connections 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a88c2a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a8897a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 --2- 172.21.15.137:0/1009835889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a8842b0 0x55d56a8846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.687+0000 7f2eda563740 1 -- 172.21.15.137:0/1009835889 >> 172.21.15.137:0/1009835889 conn(0x55d56a87f820 msgr2=0x55d56a87fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.688+0000 7f2eda563740 1 -- 172.21.15.137:0/1009835889 shutdown_connections 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.688+0000 7f2eda563740 1 -- 172.21.15.137:0/1009835889 wait complete. 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.688+0000 7f2eda563740 1 -- start start 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.688+0000 7f2eda563740 1 Processor -- start 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a88e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a88e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a889e10 0x55d56a895760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a88cb80 con 0x55d56a8842b0 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a7a5eb0 con 0x55d56a889e10 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a88cd00 con 0x55d56a884ca0 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2eda563740 10 client.0 _ll_register_callbacks cb 0x55d56a79a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ed4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a88e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ed4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a88e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41584/0 (socket says 172.21.15.137:41584) 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ed4e00640 1 -- 172.21.15.137:0/2632899567 learned_addr learned my addr 172.21.15.137:0/2632899567 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a889e10 0x55d56a895760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a88e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ed4e00640 1 -- 172.21.15.137:0/2632899567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 msgr2=0x55d56a88e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.689+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a88e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2ed4e00640 1 -- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a889e10 msgr2=0x55d56a895760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a889e10 0x55d56a895760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2ed4e00640 1 -- 172.21.15.137:0/2632899567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d56a626f70 con 0x55d56a8842b0 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a88e1e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400bea0 tx=0x7f2ec400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d802f5897483c44f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2632899567 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4012650 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2632899567 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2ec40127f0 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ebc0fe030 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2632899567 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ec40129f0 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eda563740 10 client.18393 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.690+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d56a4be170 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2632899567 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ebc0fe030 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2632899567 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2ec401bc80 con 0x55d56a8842b0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eda563740 10 client.18393 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eda563740 10 client.18393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eda563740 20 client.18393 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eda563740 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a89a0d0 0x55d56a89c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eda563740 4 client.18393 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.691+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d56a7f73f0 con 0x55d56a89a0d0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.692+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a89a0d0 0x55d56a89c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.693+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a89a0d0 0x55d56a89c570 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ff469ee8eddb795 server_cookie=8eec69dfca8698e6 in_seq=0 out_seq=0 2024-03-26T14:15:20.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.693+0000 7f2eb6a00640 10 client.18393 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.694+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2632899567 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d56a7f73f0 con 0x55d56a89a0d0 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.694+0000 7f2eb6a00640 10 client.18393 handle_command_reply: tid=0 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.694+0000 7f2eda563740 1 client.18393 shutdown 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 msgr2=0x55d56a88e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a88e1e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400bea0 tx=0x7f2ec400bf90 comp rx=0 tx=0).stop 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 shutdown_connections 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a89a0d0 0x55d56a89c570 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a889e10 0x55d56a895760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a884ca0 0x55d56a88e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 --2- 172.21.15.137:0/2632899567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a88e1e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 >> 172.21.15.137:0/2632899567 conn(0x55d56a87f820 msgr2=0x55d56a87ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eb6a00640 0 client.18393 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 shutdown_connections 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 1 -- 172.21.15.137:0/2632899567 wait complete. 2024-03-26T14:15:20.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.695+0000 7f2eda563740 20 client.18393 trim_cache size 0 max 16384 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 Processor -- start 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 -- start start 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a87e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a8a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a889e10 0x55d56a8a1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed01ae760 con 0x55d56a8842b0 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed01ac260 con 0x55d56a884ca0 2024-03-26T14:15:20.718 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.717+0000 7f2eda563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed01aa2e0 con 0x55d56a889e10 2024-03-26T14:15:20.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a87e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a87e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41590/0 (socket says 172.21.15.137:41590) 2024-03-26T14:15:20.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 -- 172.21.15.137:0/780357992 learned_addr learned my addr 172.21.15.137:0/780357992 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a8a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed5800640 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a889e10 0x55d56a8a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.719 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 -- 172.21.15.137:0/780357992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a889e10 msgr2=0x55d56a8a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a889e10 0x55d56a8a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 -- 172.21.15.137:0/780357992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 msgr2=0x55d56a8a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a8a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.718+0000 7f2ed4e00640 1 -- 172.21.15.137:0/780357992 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a87f650 con 0x55d56a8842b0 2024-03-26T14:15:20.720 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.719+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a87e900 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4000c00 tx=0x7f2ec400b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0e6760ef12e24c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.719+0000 7f2ec1600640 1 -- 172.21.15.137:0/780357992 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec402e030 con 0x55d56a8842b0 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.719+0000 7f2ec1600640 1 -- 172.21.15.137:0/780357992 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ec4012b90 con 0x55d56a8842b0 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.719+0000 7f2ec1600640 1 -- 172.21.15.137:0/780357992 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec4000e60 con 0x55d56a8842b0 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 -- 172.21.15.137:0/780357992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 msgr2=0x55d56a87e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a87e900 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4000c00 tx=0x7f2ec400b530 comp rx=0 tx=0).stop 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 -- 172.21.15.137:0/780357992 shutdown_connections 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a8a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a889e10 0x55d56a8a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 --2- 172.21.15.137:0/780357992 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a8842b0 0x55d56a87e900 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 -- 172.21.15.137:0/780357992 >> 172.21.15.137:0/780357992 conn(0x55d56a87f820 msgr2=0x55d56a87e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 -- 172.21.15.137:0/780357992 shutdown_connections 2024-03-26T14:15:20.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 -- 172.21.15.137:0/780357992 wait complete. 2024-03-26T14:15:20.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.720+0000 7f2eda563740 1 -- start start 2024-03-26T14:15:20.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 Processor -- start 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a8842b0 0x55d56a892ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a893220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a894230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed01aa8e0 con 0x55d56a889e10 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed01abc00 con 0x55d56a884ca0 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed01aaf40 con 0x55d56a8842b0 2024-03-26T14:15:20.724 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2eda563740 10 client.0 _ll_register_callbacks cb 0x7f2ec400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2ed5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a894230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2ed5800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a894230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41596/0 (socket says 172.21.15.137:41596) 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.721+0000 7f2ed5800640 1 -- 172.21.15.137:0/2613538807 learned_addr learned my addr 172.21.15.137:0/2613538807 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a893220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a8842b0 0x55d56a892ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed5800640 1 -- 172.21.15.137:0/2613538807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a8842b0 msgr2=0x55d56a892ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a8842b0 0x55d56a892ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.725 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed5800640 1 -- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 msgr2=0x55d56a893220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a893220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed5800640 1 -- 172.21.15.137:0/2613538807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d56a894820 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ecfe00640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a893220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2ed5800640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a894230 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8011ef0 tx=0x7f2ec8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca46469e6eab774 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.722+0000 7f2eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2613538807 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec802e060 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2613538807 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2ec8020b30 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ec4012b90 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2613538807 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ec8035050 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eda563740 10 client.18399 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d56a8f61f0 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2613538807 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2ec4012b90 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.723+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2613538807 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2ec8028070 con 0x55d56a889e10 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2eda563740 10 client.18399 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2eda563740 10 client.18399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:20.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2eda563740 20 client.18399 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:20.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2eda563740 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a8f6c00 0x55d56a8f9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2eda563740 4 client.18399 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:15:20.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d56a7f1740 con 0x55d56a8f6c00 2024-03-26T14:15:20.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.724+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a8f6c00 0x55d56a8f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.725+0000 7f2ed4e00640 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a8f6c00 0x55d56a8f9020 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdb5027c76a01ca server_cookie=eb29823b37a79968 in_seq=0 out_seq=0 2024-03-26T14:15:20.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.725+0000 7f2eb6a00640 10 client.18399 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:20.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.727+0000 7f2eb6a00640 1 -- 172.21.15.137:0/2613538807 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d56a7f1740 con 0x55d56a8f6c00 2024-03-26T14:15:20.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.727+0000 7f2eb6a00640 10 client.18399 handle_command_reply: tid=0 2024-03-26T14:15:20.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 client.18399 shutdown 2024-03-26T14:15:20.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 msgr2=0x55d56a894230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a894230 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8011ef0 tx=0x7f2ec8004a40 comp rx=0 tx=0).stop 2024-03-26T14:15:20.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 shutdown_connections 2024-03-26T14:15:20.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d56a8f6c00 0x55d56a8f9020 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d56a884ca0 0x55d56a893220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d56a889e10 0x55d56a894230 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 --2- 172.21.15.137:0/2613538807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d56a8842b0 0x55d56a892ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eb6a00640 0 client.18399 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 >> 172.21.15.137:0/2613538807 conn(0x55d56a87f820 msgr2=0x55d56a87ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 shutdown_connections 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 1 -- 172.21.15.137:0/2613538807 wait complete. 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.728+0000 7f2eda563740 20 client.18399 trim_cache size 0 max 16384 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "25187228-cbfe-42ca-a7cf-771a10ceea80", 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:15:20.730 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2eb80a2800 msgr2=0x7f2eb80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2eb80a2800 0x7f2eb80a4cc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009fd0 tx=0x7f2ec80098a0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 msgr2=0x7f2ed0199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0199f30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc000c00 tx=0x7f2ebc0042f0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 shutdown_connections 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2eb80a2800 0x7f2eb80a4cc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ed0105350 0x7f2ed01a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ed0104990 0x7f2ed0199f30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 --2- 172.21.15.137:0/2009282354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ed01037a0 0x7f2ed01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 >> 172.21.15.137:0/2009282354 conn(0x7f2ed00fed10 msgr2=0x7f2ed0107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 shutdown_connections 2024-03-26T14:15:20.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.729+0000 7f2ed6200640 1 -- 172.21.15.137:0/2009282354 wait complete. 2024-03-26T14:15:20.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:15:20.753 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:15:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 -- 172.21.15.137:0/2024224004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358108db0 msgr2=0x7f2358109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 --2- 172.21.15.137:0/2024224004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358108db0 0x7f2358109230 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2344009900 tx=0x7f234402bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:20.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 -- 172.21.15.137:0/2024224004 shutdown_connections 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 --2- 172.21.15.137:0/2024224004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2358109770 0x7f2358110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 --2- 172.21.15.137:0/2024224004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358108db0 0x7f2358109230 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 --2- 172.21.15.137:0/2024224004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358107bc0 0x7f2358107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.961+0000 7f235da00640 1 -- 172.21.15.137:0/2024224004 >> 172.21.15.137:0/2024224004 conn(0x7f2358076b70 msgr2=0x7f2358078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.962+0000 7f235da00640 1 -- 172.21.15.137:0/2024224004 shutdown_connections 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.962+0000 7f235da00640 1 -- 172.21.15.137:0/2024224004 wait complete. 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 Processor -- start 2024-03-26T14:15:20.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 -- start start 2024-03-26T14:15:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2358107bc0 0x7f235819dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358108db0 0x7f235819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 0x7f23581a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358113740 con 0x7f2358109770 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581135c0 con 0x7f2358107bc0 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.963+0000 7f235da00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581138c0 con 0x7f2358108db0 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 0x7f23581a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f2357e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2358107bc0 0x7f235819dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 0x7f23581a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41618/0 (socket says 172.21.15.137:41618) 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 -- 172.21.15.137:0/401165851 learned_addr learned my addr 172.21.15.137:0/401165851 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f2357400640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358108db0 0x7f235819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 -- 172.21.15.137:0/401165851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358108db0 msgr2=0x7f235819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358108db0 0x7f235819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 -- 172.21.15.137:0/401165851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2358107bc0 msgr2=0x7f235819dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2358107bc0 0x7f235819dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f235d000640 1 -- 172.21.15.137:0/401165851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23581a5a00 con 0x7f2358109770 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.964+0000 7f2357400640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358108db0 0x7f235819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.965+0000 7f235d000640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 0x7f23581a5390 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f235000eb10 tx=0x7f235000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.965+0000 7f2354c00640 1 -- 172.21.15.137:0/401165851 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350004b20 con 0x7f2358109770 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.965+0000 7f2354c00640 1 -- 172.21.15.137:0/401165851 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2350004cc0 con 0x7f2358109770 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.965+0000 7f2354c00640 1 -- 172.21.15.137:0/401165851 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350010690 con 0x7f2358109770 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.965+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23581a5cf0 con 0x7f2358109770 2024-03-26T14:15:20.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.965+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23581a61c0 con 0x7f2358109770 2024-03-26T14:15:20.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.969+0000 7f2354c00640 1 -- 172.21.15.137:0/401165851 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2350005020 con 0x7f2358109770 2024-03-26T14:15:20.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.969+0000 7f2361def740 1 Processor -- start 2024-03-26T14:15:20.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.969+0000 7f2361def740 1 -- start start 2024-03-26T14:15:20.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2361def740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c2050 0x55b5b36c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2361def740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2361def740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c7bb0 0x55b5b36ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2361def740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b3627250 con 0x55b5b36c2a40 2024-03-26T14:15:20.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2361def740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b35cfa30 con 0x55b5b36c7bb0 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2361def740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b36232f0 con 0x55b5b36c2050 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2357e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2357e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41628/0 (socket says 172.21.15.137:41628) 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2357e00640 1 -- 172.21.15.137:0/1657559045 learned_addr learned my addr 172.21.15.137:0/1657559045 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2357400640 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c7bb0 0x55b5b36ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.970+0000 7f2357400640 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c2050 0x55b5b36c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357e00640 1 -- 172.21.15.137:0/1657559045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c2050 msgr2=0x55b5b36c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357e00640 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c2050 0x55b5b36c2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357e00640 1 -- 172.21.15.137:0/1657559045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c7bb0 msgr2=0x55b5b36ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357e00640 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c7bb0 0x55b5b36ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357e00640 1 -- 172.21.15.137:0/1657559045 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5b35ba690 con 0x55b5b36c2a40 2024-03-26T14:15:20.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357400640 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c7bb0 0x55b5b36ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2354c00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f233c0a2880 0x7f233c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357e00640 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36c7540 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f234800bcb0 tx=0x7f23480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bb661baf6ce17f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2354c00640 1 -- 172.21.15.137:0/401165851 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f23500cf430 con 0x7f2358109770 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f2357400640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f233c0a2880 0x7f233c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f234d600640 1 -- 172.21.15.137:0/1657559045 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005020 con 0x55b5b36c2a40 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f234d600640 1 -- 172.21.15.137:0/1657559045 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23480047a0 con 0x55b5b36c2a40 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.971+0000 7f234d600640 1 -- 172.21.15.137:0/1657559045 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480116a0 con 0x55b5b36c2a40 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.972+0000 7f2361def740 1 -- 172.21.15.137:0/1657559045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 msgr2=0x55b5b36c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.972+0000 7f2357400640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f233c0a2880 0x7f233c0a4d40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2344006010 tx=0x7f234402c5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.972+0000 7f2361def740 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36c7540 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f234800bcb0 tx=0x7f23480107a0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 -- 172.21.15.137:0/1657559045 shutdown_connections 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c7bb0 0x55b5b36ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36c7540 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 --2- 172.21.15.137:0/1657559045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c2050 0x55b5b36c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 -- 172.21.15.137:0/1657559045 >> 172.21.15.137:0/1657559045 conn(0x55b5b368a200 msgr2=0x55b5b36bd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 -- 172.21.15.137:0/1657559045 shutdown_connections 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 -- 172.21.15.137:0/1657559045 wait complete. 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 -- start start 2024-03-26T14:15:20.975 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.973+0000 7f2361def740 1 Processor -- start 2024-03-26T14:15:20.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.977 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b36cab60 con 0x55b5b36c2050 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b35e2e80 con 0x55b5b36c2a40 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b36cace0 con 0x55b5b36c7bb0 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2361def740 10 client.0 _ll_register_callbacks cb 0x55b5b35f5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2357e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2357e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41634/0 (socket says 172.21.15.137:41634) 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2357e00640 1 -- 172.21.15.137:0/4172255863 learned_addr learned my addr 172.21.15.137:0/4172255863 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.974+0000 7f2357400640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f235d000640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2357e00640 1 -- 172.21.15.137:0/4172255863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 msgr2=0x55b5b36d3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.978 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2357e00640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2357e00640 1 -- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 msgr2=0x55b5b36cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2357e00640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2357e00640 1 -- 172.21.15.137:0/4172255863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5b34057d0 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2357e00640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36cc000 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f234800bc80 tx=0x7f2348004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4aa7735d4498c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.975+0000 7f2342000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2342000640 1 -- 172.21.15.137:0/4172255863 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005170 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2342000640 1 -- 172.21.15.137:0/4172255863 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2348005310 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23480047a0 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2342000640 1 -- 172.21.15.137:0/4172255863 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2348033050 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2361def740 10 client.18411 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b5b333b3f0 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.976+0000 7f2342000640 1 -- 172.21.15.137:0/4172255863 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23480047a0 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2342000640 1 -- 172.21.15.137:0/4172255863 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f234802e030 con 0x55b5b36c2050 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2361def740 10 client.18411 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2361def740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:20.979 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2361def740 20 client.18411 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:20.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2361def740 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b36d7b20 0x55b5b36d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2361def740 4 client.18411 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:20.980 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5b3634790 con 0x55b5b36d7b20 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.977+0000 7f2357400640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b36d7b20 0x55b5b36d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.978+0000 7f2357400640 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b36d7b20 0x55b5b36d9f10 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8790ad1338651717 server_cookie=777ed34170c55840 in_seq=0 out_seq=0 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.978+0000 7f2342000640 10 client.18411 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2342000640 1 -- 172.21.15.137:0/4172255863 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b5b3634790 con 0x55b5b36d7b20 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2342000640 10 client.18411 handle_command_reply: tid=0 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 client.18411 shutdown 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 msgr2=0x55b5b36cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36cc000 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f234800bc80 tx=0x7f2348004950 comp rx=0 tx=0).stop 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 shutdown_connections 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b36d7b20 0x55b5b36d9f10 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.981 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 --2- 172.21.15.137:0/4172255863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36cc000 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2342000640 0 client.18411 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.979+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 >> 172.21.15.137:0/4172255863 conn(0x55b5b368a200 msgr2=0x55b5b36bcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.980+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 shutdown_connections 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.980+0000 7f2361def740 1 -- 172.21.15.137:0/4172255863 wait complete. 2024-03-26T14:15:20.982 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.980+0000 7f2361def740 20 client.18411 trim_cache size 0 max 16384 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 Processor -- start 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 -- start start 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36bc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581b2b80 con 0x55b5b36c2050 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581b0680 con 0x55b5b36c2a40 2024-03-26T14:15:20.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2361def740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581ae700 con 0x55b5b36c7bb0 2024-03-26T14:15:21.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2357e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36bc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.998+0000 7f2357e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36bc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41636/0 (socket says 172.21.15.137:41636) 2024-03-26T14:15:21.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 -- 172.21.15.137:0/806198543 learned_addr learned my addr 172.21.15.137:0/806198543 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:21.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357400640 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f235d000640 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 -- 172.21.15.137:0/806198543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 msgr2=0x55b5b36de6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36de6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 -- 172.21.15.137:0/806198543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 msgr2=0x55b5b36d3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36d3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 -- 172.21.15.137:0/806198543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5b32fb0f0 con 0x55b5b36c2050 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f235d000640 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f2357e00640 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36bc330 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f234800bbd0 tx=0x7f2348005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68bfe7607b14c42 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:20.999+0000 7f234cc00640 1 -- 172.21.15.137:0/806198543 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234803b020 con 0x55b5b36c2050 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f234cc00640 1 -- 172.21.15.137:0/806198543 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2348005870 con 0x55b5b36c2050 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f234cc00640 1 -- 172.21.15.137:0/806198543 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005310 con 0x55b5b36c2050 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 -- 172.21.15.137:0/806198543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 msgr2=0x55b5b36bc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:21.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36bc330 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f234800bbd0 tx=0x7f2348005f30 comp rx=0 tx=0).stop 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 -- 172.21.15.137:0/806198543 shutdown_connections 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2a40 0x55b5b36d3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36de6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 --2- 172.21.15.137:0/806198543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2050 0x55b5b36bc330 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.000+0000 7f2361def740 1 -- 172.21.15.137:0/806198543 >> 172.21.15.137:0/806198543 conn(0x55b5b368a200 msgr2=0x55b5b36c3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.001+0000 7f2361def740 1 -- 172.21.15.137:0/806198543 shutdown_connections 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.001+0000 7f2361def740 1 -- 172.21.15.137:0/806198543 wait complete. 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.001+0000 7f2361def740 1 -- start start 2024-03-26T14:15:21.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.001+0000 7f2361def740 1 Processor -- start 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2050 0x55b5b36d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581aed00 con 0x55b5b36c2a40 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581b0020 con 0x55b5b36c2050 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581af360 con 0x55b5b36c7bb0 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2361def740 10 client.0 _ll_register_callbacks cb 0x7f234800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2050 0x55b5b36d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2050 0x55b5b36d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:42258/0 (socket says 172.21.15.137:42258) 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36d0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41642/0 (socket says 172.21.15.137:41642) 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357e00640 1 -- 172.21.15.137:0/2701978613 learned_addr learned my addr 172.21.15.137:0/2701978613 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f235d000640 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357400640 1 -- 172.21.15.137:0/2701978613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 msgr2=0x55b5b36d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.002+0000 7f2357400640 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2357400640 1 -- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2050 msgr2=0x55b5b36d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2357400640 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2050 0x55b5b36d08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2357400640 1 -- 172.21.15.137:0/2701978613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5b36d2400 con 0x55b5b36c2a40 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2357400640 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36d0e00 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f234402c170 tx=0x7f234402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4216b163b7567ba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2342000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2342000640 1 -- 172.21.15.137:0/2701978613 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2344048040 con 0x55b5b36c2a40 2024-03-26T14:15:21.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2342000640 1 -- 172.21.15.137:0/2701978613 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f23440046c0 con 0x55b5b36c2a40 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2348005870 con 0x55b5b36c2a40 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.003+0000 7f2342000640 1 -- 172.21.15.137:0/2701978613 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23440048c0 con 0x55b5b36c2a40 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 10 client.18417 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b5b36d36f0 con 0x55b5b36c2a40 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2342000640 1 -- 172.21.15.137:0/2701978613 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2348005870 con 0x55b5b36c2a40 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2342000640 1 -- 172.21.15.137:0/2701978613 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f234405b070 con 0x55b5b36c2a40 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 10 client.18417 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 10 client.18417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 20 client.18417 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b37338d0 0x55b5b3735cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 4 client.18417 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.004+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5b362eae0 con 0x55b5b37338d0 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.005+0000 7f2357e00640 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b37338d0 0x55b5b3735cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.005+0000 7f2357e00640 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b37338d0 0x55b5b3735cf0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6ecfa8784f02d0 server_cookie=a0595bc193cb3c1a in_seq=0 out_seq=0 2024-03-26T14:15:21.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.005+0000 7f2342000640 10 client.18417 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2342000640 1 -- 172.21.15.137:0/2701978613 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b5b362eae0 con 0x55b5b37338d0 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2342000640 10 client.18417 handle_command_reply: tid=0 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 client.18417 shutdown 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 msgr2=0x55b5b36d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36d0e00 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f234402c170 tx=0x7f234402bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 shutdown_connections 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55b5b37338d0 0x55b5b3735cf0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5b36c7bb0 0x55b5b36d1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b5b36c2a40 0x55b5b36d0e00 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 --2- 172.21.15.137:0/2701978613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5b36c2050 0x55b5b36d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 >> 172.21.15.137:0/2701978613 conn(0x55b5b368a200 msgr2=0x55b5b36c3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2342000640 0 client.18417 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 shutdown_connections 2024-03-26T14:15:21.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.006+0000 7f2361def740 1 -- 172.21.15.137:0/2701978613 wait complete. 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f2361def740 20 client.18417 trim_cache size 0 max 16384 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "25187228-cbfe-42ca-a7cf-771a10ceea80": { 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "25187228-cbfe-42ca-a7cf-771a10ceea80", 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:15:21.011 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:15:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f233c0a2880 msgr2=0x7f233c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f233c0a2880 0x7f233c0a4d40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2344006010 tx=0x7f234402c5f0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 msgr2=0x7f23581a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 0x7f23581a5390 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f235000eb10 tx=0x7f235000c2d0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 shutdown_connections 2024-03-26T14:15:21.012 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f233c0a2880 0x7f233c0a4d40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2358109770 0x7f23581a5390 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2358108db0 0x7f235819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 --2- 172.21.15.137:0/401165851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2358107bc0 0x7f235819dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.007+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 >> 172.21.15.137:0/401165851 conn(0x7f2358076b70 msgr2=0x7f235810c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.008+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 shutdown_connections 2024-03-26T14:15:21.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:21.008+0000 7f235da00640 1 -- 172.21.15.137:0/401165851 wait complete. 2024-03-26T14:15:21.026 INFO:tasks.cephfs.filesystem:scrub status for tag:25187228-cbfe-42ca-a7cf-771a10ceea80 - {'path': '/', 'tag': '25187228-cbfe-42ca-a7cf-771a10ceea80', 'options': 'recursive,force'} 2024-03-26T14:15:21.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:20 smithi179 ceph-mon[35472]: pgmap v847: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:21.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:20 smithi159 ceph-mon[35501]: pgmap v847: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:21.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:20 smithi137 ceph-mon[31539]: pgmap v847: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:22.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:21 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:15:22.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:21 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:22.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:21 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:15:22.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:21 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:22.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:21 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:15:22.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:21 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:22.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:21 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:15:22.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:21 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:22.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:21 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:15:22.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:21 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:22.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:21 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:15:22.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:21 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:23.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:22 smithi179 ceph-mon[35472]: pgmap v848: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:15:23.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:22 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:15:23.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:22 smithi159 ceph-mon[35501]: pgmap v848: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:15:23.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:22 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:15:23.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:22 smithi137 ceph-mon[31539]: pgmap v848: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:15:23.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:22 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:15:25.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:25 smithi159 ceph-mon[35501]: pgmap v849: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:15:25.535 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:25 smithi137 ceph-mon[31539]: pgmap v849: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:15:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:25 smithi179 ceph-mon[35472]: pgmap v849: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-03-26T14:15:27.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:27 smithi159 ceph-mon[35501]: pgmap v850: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:15:27.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:27 smithi137 ceph-mon[31539]: pgmap v850: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:15:27.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:27 smithi179 ceph-mon[35472]: pgmap v850: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:15:29.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:29 smithi159 ceph-mon[35501]: pgmap v851: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:15:29.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:29 smithi137 ceph-mon[31539]: pgmap v851: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:15:29.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:29 smithi179 ceph-mon[35472]: pgmap v851: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-03-26T14:15:31.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:31 smithi137 ceph-mon[31539]: pgmap v852: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:15:31.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:31 smithi179 ceph-mon[35472]: pgmap v852: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:15:31.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:31 smithi159 ceph-mon[35501]: pgmap v852: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-26T14:15:33.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:33 smithi137 ceph-mon[31539]: pgmap v853: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:15:33.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:33 smithi179 ceph-mon[35472]: pgmap v853: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:15:33.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:33 smithi159 ceph-mon[35501]: pgmap v853: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-03-26T14:15:35.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:35 smithi137 ceph-mon[31539]: pgmap v854: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:15:35.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:35 smithi179 ceph-mon[35472]: pgmap v854: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:15:35.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:35 smithi159 ceph-mon[35501]: pgmap v854: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-03-26T14:15:37.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:37 smithi137 ceph-mon[31539]: pgmap v855: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:37.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:37 smithi179 ceph-mon[35472]: pgmap v855: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:37.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:37 smithi159 ceph-mon[35501]: pgmap v855: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:39.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:39 smithi137 ceph-mon[31539]: pgmap v856: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:39.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:39 smithi179 ceph-mon[35472]: pgmap v856: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:39.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:39 smithi159 ceph-mon[35501]: pgmap v856: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:41.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:41 smithi137 ceph-mon[31539]: pgmap v857: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:41.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:41 smithi179 ceph-mon[35472]: pgmap v857: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:41.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:41 smithi159 ceph-mon[35501]: pgmap v857: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:43.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:43 smithi137 ceph-mon[31539]: pgmap v858: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:43.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:43 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:15:43.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:43 smithi179 ceph-mon[35472]: pgmap v858: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:43.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:43 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:15:43.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:43 smithi159 ceph-mon[35501]: pgmap v858: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:43.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:43 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:15:44.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:15:44.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:15:44.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:15:44.577 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:15:44.577 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:15:44.577 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:15:44.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:15:44.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:15:44.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:15:45.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:45 smithi137 ceph-mon[31539]: pgmap v859: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:45.605 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:45 smithi159 ceph-mon[35501]: pgmap v859: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:45.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:45 smithi179 ceph-mon[35472]: pgmap v859: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:47.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:47 smithi137 ceph-mon[31539]: pgmap v860: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:47.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:47 smithi179 ceph-mon[35472]: pgmap v860: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:47.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:47 smithi159 ceph-mon[35501]: pgmap v860: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:49.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:49 smithi137 ceph-mon[31539]: pgmap v861: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:49.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:49 smithi179 ceph-mon[35472]: pgmap v861: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:49.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:49 smithi159 ceph-mon[35501]: pgmap v861: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:15:51.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:15:51.028 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:15:51.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 -- 172.21.15.137:0/4101218278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e94103150 msgr2=0x7f2e941099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/4101218278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e94103150 0x7f2e941099e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009900 tx=0x7f2e8c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 -- 172.21.15.137:0/4101218278 shutdown_connections 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/4101218278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e94102790 0x7f2e94102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/4101218278 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e94103150 0x7f2e941099e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/4101218278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e941015a0 0x7f2e941019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 -- 172.21.15.137:0/4101218278 >> 172.21.15.137:0/4101218278 conn(0x7f2e940fcb10 msgr2=0x7f2e940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 -- 172.21.15.137:0/4101218278 shutdown_connections 2024-03-26T14:15:51.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.261+0000 7f2e9ac00640 1 -- 172.21.15.137:0/4101218278 wait complete. 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.262+0000 7f2e9ac00640 1 Processor -- start 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.262+0000 7f2e9ac00640 1 -- start start 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e9ac00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 0x7f2e94197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e9ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e94102790 0x7f2e94197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e9ac00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e94103150 0x7f2e9419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e9ac00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9410d0b0 con 0x7f2e941015a0 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e9ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9410cf30 con 0x7f2e94102790 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e9ac00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9410d230 con 0x7f2e94103150 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e99800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 0x7f2e94197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e99800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 0x7f2e94197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48680/0 (socket says 172.21.15.137:48680) 2024-03-26T14:15:51.266 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e99800640 1 -- 172.21.15.137:0/2257246053 learned_addr learned my addr 172.21.15.137:0/2257246053 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.263+0000 7f2e98e00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e94102790 0x7f2e94197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e9a200640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e94103150 0x7f2e9419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e99800640 1 -- 172.21.15.137:0/2257246053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e94103150 msgr2=0x7f2e9419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e99800640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e94103150 0x7f2e9419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e99800640 1 -- 172.21.15.137:0/2257246053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e94102790 msgr2=0x7f2e94197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e99800640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e94102790 0x7f2e94197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e99800640 1 -- 172.21.15.137:0/2257246053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9419f380 con 0x7f2e941015a0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e99800640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 0x7f2e94197760 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800ccb0 tx=0x7f2e8800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e92000640 1 -- 172.21.15.137:0/2257246053 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88003990 con 0x7f2e941015a0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9419f670 con 0x7f2e941015a0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e92000640 1 -- 172.21.15.137:0/2257246053 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e88004020 con 0x7f2e941015a0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.264+0000 7f2e92000640 1 -- 172.21.15.137:0/2257246053 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800f6b0 con 0x7f2e941015a0 2024-03-26T14:15:51.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.265+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9419fc10 con 0x7f2e941015a0 2024-03-26T14:15:51.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.268+0000 7f2e9ef8c740 1 Processor -- start 2024-03-26T14:15:51.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.268+0000 7f2e92000640 1 -- 172.21.15.137:0/2257246053 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2e88003b30 con 0x7f2e941015a0 2024-03-26T14:15:51.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.268+0000 7f2e9ef8c740 1 -- start start 2024-03-26T14:15:51.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058767880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x55905876c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x55905876f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590586c8250 con 0x55905876cfc0 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559058670a30 con 0x559058767e50 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590586c42f0 con 0x559058767460 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x55905876f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x55905876f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48686/0 (socket says 172.21.15.137:48686) 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 -- 172.21.15.137:0/4276537963 learned_addr learned my addr 172.21.15.137:0/4276537963 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e99800640 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x55905876c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e9a200640 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058767880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 -- 172.21.15.137:0/4276537963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 msgr2=0x559058767880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058767880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 -- 172.21.15.137:0/4276537963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 msgr2=0x55905876c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.269+0000 7f2e98e00640 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x55905876c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.272 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e98e00640 1 -- 172.21.15.137:0/4276537963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55905865b690 con 0x55905876cfc0 2024-03-26T14:15:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e98e00640 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x55905876f450 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000bc80 tx=0x7f2e8000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88671256ac2fe7f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e92000640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e7c0a2800 0x7f2e7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e92000640 1 -- 172.21.15.137:0/2257246053 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2e880cf2e0 con 0x7f2e941015a0 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e86a00640 1 -- 172.21.15.137:0/4276537963 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80015070 con 0x55905876cfc0 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e86a00640 1 -- 172.21.15.137:0/4276537963 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e8001a910 con 0x55905876cfc0 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e86a00640 1 -- 172.21.15.137:0/4276537963 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800118d0 con 0x55905876cfc0 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.270+0000 7f2e9a200640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e7c0a2800 0x7f2e7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/4276537963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 msgr2=0x55905876f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x55905876f450 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000bc80 tx=0x7f2e8000f730 comp rx=0 tx=0).stop 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/4276537963 shutdown_connections 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x55905876c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x55905876f450 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/4276537963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058767880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/4276537963 >> 172.21.15.137:0/4276537963 conn(0x55905872b200 msgr2=0x55905875e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9a200640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e7c0a2800 0x7f2e7c0a4cc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006010 tx=0x7f2e8c02c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.271+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/4276537963 shutdown_connections 2024-03-26T14:15:51.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.272+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/4276537963 wait complete. 2024-03-26T14:15:51.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.272+0000 7f2e9ef8c740 1 -- start start 2024-03-26T14:15:51.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.272+0000 7f2e9ef8c740 1 Processor -- start 2024-03-26T14:15:51.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058771460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x5590587789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55905876ff70 con 0x55905876cfc0 2024-03-26T14:15:51.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559058683e80 con 0x559058767e50 2024-03-26T14:15:51.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590587700f0 con 0x559058767460 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9ef8c740 10 client.0 _ll_register_callbacks cb 0x559058696130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x5590587789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.273+0000 7f2e9a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x5590587789e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48696/0 (socket says 172.21.15.137:48696) 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e9a200640 1 -- 172.21.15.137:0/311578661 learned_addr learned my addr 172.21.15.137:0/311578661 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e99800640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058771460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e98e00640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e9a200640 1 -- 172.21.15.137:0/311578661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 msgr2=0x559058771460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e9a200640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058771460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e9a200640 1 -- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 msgr2=0x5590587719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e9a200640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.274+0000 7f2e9a200640 1 -- 172.21.15.137:0/311578661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590584a67d0 con 0x55905876cfc0 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.275+0000 7f2e9a200640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x5590587789e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009900 tx=0x7f2e8c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439a896b1efec5df server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.275+0000 7f2e7b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:51.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.278+0000 7f2e7b400640 1 -- 172.21.15.137:0/311578661 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c04e020 con 0x55905876cfc0 2024-03-26T14:15:51.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.278+0000 7f2e7b400640 1 -- 172.21.15.137:0/311578661 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2e8c004ad0 con 0x55905876cfc0 2024-03-26T14:15:51.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.278+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e8001a910 con 0x55905876cfc0 2024-03-26T14:15:51.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e7b400640 1 -- 172.21.15.137:0/311578661 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e8c004cd0 con 0x55905876cfc0 2024-03-26T14:15:51.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 10 client.18429 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5590583dc3f0 con 0x55905876cfc0 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e7b400640 1 -- 172.21.15.137:0/311578661 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e8001a910 con 0x55905876cfc0 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e7b400640 1 -- 172.21.15.137:0/311578661 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2e8c034440 con 0x55905876cfc0 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 10 client.18429 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 10 client.18429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 20 client.18429 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55905877d360 0x55905877f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 4 client.18429 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:51.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.279+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590586d5790 con 0x55905877d360 2024-03-26T14:15:51.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.280+0000 7f2e99800640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55905877d360 0x55905877f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.281+0000 7f2e7b400640 10 client.18429 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:51.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.281+0000 7f2e99800640 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55905877d360 0x55905877f7a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a371a9e6e8cec4f8 server_cookie=58bf1984f14dd113 in_seq=0 out_seq=0 2024-03-26T14:15:51.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e7b400640 1 -- 172.21.15.137:0/311578661 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5590586d5790 con 0x55905877d360 2024-03-26T14:15:51.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e7b400640 10 client.18429 handle_command_reply: tid=0 2024-03-26T14:15:51.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 client.18429 shutdown 2024-03-26T14:15:51.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 msgr2=0x5590587789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x5590587789e0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009900 tx=0x7f2e8c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 shutdown_connections 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55905877d360 0x55905877f7a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x5590587789e0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/311578661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058771460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 >> 172.21.15.137:0/311578661 conn(0x55905872b200 msgr2=0x55905875dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e7b400640 0 client.18429 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 shutdown_connections 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/311578661 wait complete. 2024-03-26T14:15:51.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.283+0000 7f2e9ef8c740 20 client.18429 trim_cache size 0 max 16384 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 Processor -- start 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 -- start start 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058759920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x559058769000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941aa250 con 0x55905876cfc0 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941a7d50 con 0x559058767e50 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.301+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941a5dd0 con 0x559058767460 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x559058769000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.303 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x559058769000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48704/0 (socket says 172.21.15.137:48704) 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 -- 172.21.15.137:0/2989336580 learned_addr learned my addr 172.21.15.137:0/2989336580 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e98e00640 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e99800640 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058759920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 -- 172.21.15.137:0/2989336580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 msgr2=0x559058759920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058759920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 -- 172.21.15.137:0/2989336580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 msgr2=0x5590587789a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587789a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 -- 172.21.15.137:0/2989336580 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55905839c0f0 con 0x55905876cfc0 2024-03-26T14:15:51.304 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e98e00640 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.302+0000 7f2e9a200640 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x559058769000 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c02c900 tx=0x7f2e8c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1d9bfc1164bfca server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.303+0000 7f2e86000640 1 -- 172.21.15.137:0/2989336580 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c04e070 con 0x55905876cfc0 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.303+0000 7f2e86000640 1 -- 172.21.15.137:0/2989336580 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e8c034e50 con 0x55905876cfc0 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.303+0000 7f2e86000640 1 -- 172.21.15.137:0/2989336580 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c004b30 con 0x55905876cfc0 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/2989336580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 msgr2=0x559058769000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x559058769000 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c02c900 tx=0x7f2e8c004040 comp rx=0 tx=0).stop 2024-03-26T14:15:51.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/2989336580 shutdown_connections 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559058767e50 0x5590587789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55905876cfc0 0x559058769000 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/2989336580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058759920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/2989336580 >> 172.21.15.137:0/2989336580 conn(0x55905872b200 msgr2=0x5590587688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/2989336580 shutdown_connections 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/2989336580 wait complete. 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 -- start start 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.304+0000 7f2e9ef8c740 1 Processor -- start 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058778300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 0x559058778840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.309 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55905876cfc0 0x559058776910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941a63d0 con 0x559058767e50 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941a76f0 con 0x55905876cfc0 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e941a6a30 con 0x559058767460 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9ef8c740 10 client.0 _ll_register_callbacks cb 0x7f2e8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 0x559058778840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e98e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 0x559058778840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48710/0 (socket says 172.21.15.137:48710) 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e98e00640 1 -- 172.21.15.137:0/202012964 learned_addr learned my addr 172.21.15.137:0/202012964 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.305+0000 7f2e9a200640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55905876cfc0 0x559058776910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e99800640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058778300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e98e00640 1 -- 172.21.15.137:0/202012964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 msgr2=0x559058778300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e98e00640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058778300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.310 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e98e00640 1 -- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55905876cfc0 msgr2=0x559058776910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e98e00640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55905876cfc0 0x559058776910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e98e00640 1 -- 172.21.15.137:0/202012964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559058777140 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e99800640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058778300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e98e00640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 0x559058778840 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000bc80 tx=0x7f2e80005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b2e70ea5f53a5d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e7b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e7b400640 1 -- 172.21.15.137:0/202012964 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e80028060 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e7b400640 1 -- 172.21.15.137:0/202012964 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2e80004440 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.306+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e8c034e50 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e7b400640 1 -- 172.21.15.137:0/202012964 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e8002f050 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 10 client.18435 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5590587d9710 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e7b400640 1 -- 172.21.15.137:0/202012964 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e8c034e50 con 0x559058767e50 2024-03-26T14:15:51.311 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e7b400640 1 -- 172.21.15.137:0/202012964 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2e80004bc0 con 0x559058767e50 2024-03-26T14:15:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 10 client.18435 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 20 client.18435 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590587d9de0 0x5590587dc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 4 client.18435 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:15:51.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.307+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590586cfae0 con 0x5590587d9de0 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.308+0000 7f2e99800640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590587d9de0 0x5590587dc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.308+0000 7f2e99800640 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590587d9de0 0x5590587dc1d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba3de69b830a1c6 server_cookie=c1b3039c189b3645 in_seq=0 out_seq=0 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.308+0000 7f2e7b400640 10 client.18435 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e7b400640 1 -- 172.21.15.137:0/202012964 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590586cfae0 con 0x5590587d9de0 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e7b400640 10 client.18435 handle_command_reply: tid=0 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 client.18435 shutdown 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 msgr2=0x559058778840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 0x559058778840 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000bc80 tx=0x7f2e80005f90 comp rx=0 tx=0).stop 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 shutdown_connections 2024-03-26T14:15:51.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5590587d9de0 0x5590587dc1d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55905876cfc0 0x559058776910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559058767e50 0x559058778840 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 --2- 172.21.15.137:0/202012964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559058767460 0x559058778300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e7b400640 0 client.18435 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 >> 172.21.15.137:0/202012964 conn(0x55905872b200 msgr2=0x559058769180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 shutdown_connections 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 1 -- 172.21.15.137:0/202012964 wait complete. 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.309+0000 7f2e9ef8c740 20 client.18435 trim_cache size 0 max 16384 2024-03-26T14:15:51.314 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:15:51.315 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:15:51.315 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:15:51.315 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:15:51.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e7c0a2800 msgr2=0x7f2e7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e7c0a2800 0x7f2e7c0a4cc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c006010 tx=0x7f2e8c02c7e0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.315 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 msgr2=0x7f2e94197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 0x7f2e94197760 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800ccb0 tx=0x7f2e8800a620 comp rx=0 tx=0).stop 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 shutdown_connections 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2e7c0a2800 0x7f2e7c0a4cc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e94102790 0x7f2e94197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e94103150 0x7f2e9419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 --2- 172.21.15.137:0/2257246053 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e941015a0 0x7f2e94197760 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.310+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 >> 172.21.15.137:0/2257246053 conn(0x7f2e940fcb10 msgr2=0x7f2e94105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.311+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 shutdown_connections 2024-03-26T14:15:51.316 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.311+0000 7f2e9ac00640 1 -- 172.21.15.137:0/2257246053 wait complete. 2024-03-26T14:15:51.330 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:15:51.330 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:15:51.404 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:51 smithi137 ceph-mon[31539]: pgmap v862: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:51.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.532+0000 7f32be400640 1 -- 172.21.15.137:0/2044175520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8102160 msgr2=0x7f32b8104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.532+0000 7f32be400640 1 --2- 172.21.15.137:0/2044175520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8102160 0x7f32b8104550 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009900 tx=0x7f32a802bc50 comp rx=0 tx=0).stop 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 -- 172.21.15.137:0/2044175520 shutdown_connections 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 --2- 172.21.15.137:0/2044175520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b81073c0 0x7f32b8109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 --2- 172.21.15.137:0/2044175520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b8104a90 0x7f32b8106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 --2- 172.21.15.137:0/2044175520 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8102160 0x7f32b8104550 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 -- 172.21.15.137:0/2044175520 >> 172.21.15.137:0/2044175520 conn(0x7f32b80fbb50 msgr2=0x7f32b80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 -- 172.21.15.137:0/2044175520 shutdown_connections 2024-03-26T14:15:51.534 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.533+0000 7f32be400640 1 -- 172.21.15.137:0/2044175520 wait complete. 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 Processor -- start 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 -- start start 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8102160 0x7f32b8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 0x7f32b819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b81073c0 0x7f32b81a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810d260 con 0x7f32b8104a90 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810d0e0 con 0x7f32b8102160 2024-03-26T14:15:51.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.534+0000 7f32be400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b810d3e0 con 0x7f32b81073c0 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 0x7f32b819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 0x7f32b819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48728/0 (socket says 172.21.15.137:48728) 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 -- 172.21.15.137:0/169049398 learned_addr learned my addr 172.21.15.137:0/169049398 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b7400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8102160 0x7f32b8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b7e00640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b81073c0 0x7f32b81a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 -- 172.21.15.137:0/169049398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b81073c0 msgr2=0x7f32b81a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b81073c0 0x7f32b81a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 -- 172.21.15.137:0/169049398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8102160 msgr2=0x7f32b8199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8102160 0x7f32b8199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 -- 172.21.15.137:0/169049398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b81a16f0 con 0x7f32b8104a90 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b7400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8102160 0x7f32b8199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.535+0000 7f32b6a00640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 0x7f32b819a010 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f32a400c970 tx=0x7f32a400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.536+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4014070 con 0x7f32b8104a90 2024-03-26T14:15:51.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.536+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32a4004440 con 0x7f32b8104a90 2024-03-26T14:15:51.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.536+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b81a19e0 con 0x7f32b8104a90 2024-03-26T14:15:51.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.536+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4003e50 con 0x7f32b8104a90 2024-03-26T14:15:51.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.536+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b81a1f80 con 0x7f32b8104a90 2024-03-26T14:15:51.539 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.537+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8108f10 con 0x7f32b8104a90 2024-03-26T14:15:51.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.540+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f32a4017070 con 0x7f32b8104a90 2024-03-26T14:15:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.542+0000 7f32afe00640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f329c0a2a20 0x7f329c0a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.542+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f32a40cea30 con 0x7f32b8104a90 2024-03-26T14:15:51.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.543+0000 7f32b7400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f329c0a2a20 0x7f329c0a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:51.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.543+0000 7f32b7400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f329c0a2a20 0x7f329c0a4ee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006010 tx=0x7f32a802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:51.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.548+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f32a401e030 con 0x7f32b8104a90 2024-03-26T14:15:51.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:51 smithi179 ceph-mon[35472]: pgmap v862: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:51.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:51 smithi159 ceph-mon[35501]: pgmap v862: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:51.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.812+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32b80622a0 con 0x7f32b8104a90 2024-03-26T14:15:51.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.814+0000 7f32afe00640 1 -- 172.21.15.137:0/169049398 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f32a409b6c0 con 0x7f32b8104a90 2024-03-26T14:15:51.815 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:15:51.815 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:15:51.816 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:15:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f329c0a2a20 msgr2=0x7f329c0a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f329c0a2a20 0x7f329c0a4ee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f32a8006010 tx=0x7f32a802c800 comp rx=0 tx=0).stop 2024-03-26T14:15:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 msgr2=0x7f32b819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:51.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 0x7f32b819a010 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f32a400c970 tx=0x7f32a400ce30 comp rx=0 tx=0).stop 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 shutdown_connections 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f329c0a2a20 0x7f329c0a4ee0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32b81073c0 0x7f32b81a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32b8104a90 0x7f32b819a010 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 --2- 172.21.15.137:0/169049398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32b8102160 0x7f32b8199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.817+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 >> 172.21.15.137:0/169049398 conn(0x7f32b80fbb50 msgr2=0x7f32b8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.818+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 shutdown_connections 2024-03-26T14:15:51.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:51.818+0000 7f32be400640 1 -- 172.21.15.137:0/169049398 wait complete. 2024-03-26T14:15:51.837 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:15:51.837 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 -- 172.21.15.137:0/1685466539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 msgr2=0x7f5fd8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 --2- 172.21.15.137:0/1685466539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd8104f50 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009900 tx=0x7f5fc402bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 -- 172.21.15.137:0/1685466539 shutdown_connections 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 --2- 172.21.15.137:0/1685466539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fd8105490 0x7f5fd810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 --2- 172.21.15.137:0/1685466539 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd8104f50 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 --2- 172.21.15.137:0/1685466539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81038e0 0x7f5fd8103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 -- 172.21.15.137:0/1685466539 >> 172.21.15.137:0/1685466539 conn(0x7f5fd80fee50 msgr2=0x7f5fd8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 -- 172.21.15.137:0/1685466539 shutdown_connections 2024-03-26T14:15:52.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.059+0000 7f5fdea00640 1 -- 172.21.15.137:0/1685466539 wait complete. 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 Processor -- start 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 -- start start 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fd81038e0 0x7f5fd8199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd8105490 0x7f5fd81a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810f440 con 0x7f5fd8104ad0 2024-03-26T14:15:52.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810f2c0 con 0x7f5fd8105490 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.060+0000 7f5fdea00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810f5c0 con 0x7f5fd81038e0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd819a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48742/0 (socket says 172.21.15.137:48742) 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 -- 172.21.15.137:0/2063313635 learned_addr learned my addr 172.21.15.137:0/2063313635 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fdca00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd8105490 0x7f5fd81a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fd81038e0 0x7f5fd8199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 -- 172.21.15.137:0/2063313635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fd81038e0 msgr2=0x7f5fd8199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fd81038e0 0x7f5fd8199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 -- 172.21.15.137:0/2063313635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd8105490 msgr2=0x7f5fd81a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd8105490 0x7f5fd81a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.061+0000 7f5fd7400640 1 -- 172.21.15.137:0/2063313635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd81a1740 con 0x7f5fd8104ad0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.062+0000 7f5fd7400640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd819a060 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4000c00 tx=0x7f5fc40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.062+0000 7f5fd4c00640 1 -- 172.21.15.137:0/2063313635 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4043020 con 0x7f5fd8104ad0 2024-03-26T14:15:52.064 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.062+0000 7f5fd4c00640 1 -- 172.21.15.137:0/2063313635 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fc4004aa0 con 0x7f5fd8104ad0 2024-03-26T14:15:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.062+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd81a19d0 con 0x7f5fd8104ad0 2024-03-26T14:15:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.062+0000 7f5fd4c00640 1 -- 172.21.15.137:0/2063313635 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4003d00 con 0x7f5fd8104ad0 2024-03-26T14:15:52.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.062+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd81a1f70 con 0x7f5fd8104ad0 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 Processor -- start 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 -- start start 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fd4c00640 1 -- 172.21.15.137:0/2063313635 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5fc4046020 con 0x7f5fd8104ad0 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a01e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff929652c0 con 0x55ff92a00810 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff9290dc90 con 0x55ff92a023a0 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.067+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92961360 con 0x55ff92a019e0 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a00c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48752/0 (socket says 172.21.15.137:48752) 2024-03-26T14:15:52.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 -- 172.21.15.137:0/2604435060 learned_addr learned my addr 172.21.15.137:0/2604435060 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fd7400640 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a01e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 -- 172.21.15.137:0/2604435060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 msgr2=0x55ff92a01e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a01e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 -- 172.21.15.137:0/2604435060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 msgr2=0x55ff92a08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.070 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.068+0000 7f5fdca00640 1 -- 172.21.15.137:0/2604435060 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff92932080 con 0x55ff92a00810 2024-03-26T14:15:52.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fdca00640 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a00c30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0010370 tx=0x7f5fd0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2155bcfd027ebda0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fd4c00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5fbc0a2880 0x7f5fbc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fcd200640 1 -- 172.21.15.137:0/2604435060 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0016070 con 0x55ff92a00810 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fcd200640 1 -- 172.21.15.137:0/2604435060 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fd0004d20 con 0x55ff92a00810 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fd4c00640 1 -- 172.21.15.137:0/2063313635 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5fc40f1030 con 0x7f5fd8104ad0 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fcd200640 1 -- 172.21.15.137:0/2604435060 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0005330 con 0x55ff92a00810 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.069+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5fbc0a2880 0x7f5fbc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5fbc0a2880 0x7f5fbc0a4d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8104930 tx=0x7f5fc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 -- 172.21.15.137:0/2604435060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 msgr2=0x55ff92a00c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a00c30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0010370 tx=0x7f5fd0010830 comp rx=0 tx=0).stop 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 -- 172.21.15.137:0/2604435060 shutdown_connections 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.073 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a01e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/2604435060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a00c30 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 -- 172.21.15.137:0/2604435060 >> 172.21.15.137:0/2604435060 conn(0x55ff929fbc00 msgr2=0x55ff929fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 -- 172.21.15.137:0/2604435060 shutdown_connections 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.070+0000 7f5fe17a9740 1 -- 172.21.15.137:0/2604435060 wait complete. 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.071+0000 7f5fe17a9740 1 -- start start 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.071+0000 7f5fe17a9740 1 Processor -- start 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.071+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a0aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.071+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a0afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.071+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a12000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92a093b0 con 0x55ff92a00810 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff929210d0 con 0x55ff92a023a0 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff92a09530 con 0x55ff92a019e0 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fe17a9740 10 client.0 _ll_register_callbacks cb 0x55ff92925e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:52.074 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a0aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a0aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48756/0 (socket says 172.21.15.137:48756) 2024-03-26T14:15:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1024040175 learned_addr learned my addr 172.21.15.137:0/1024040175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fdca00640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a12000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.075 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a0afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1024040175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 msgr2=0x55ff92a0afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a0afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 msgr2=0x55ff92a12000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a12000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1024040175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff926793f0 con 0x55ff92a00810 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.072+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:52.078 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.073+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a0aa80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5fc801ea00 tx=0x7f5fc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d684769bcb1fd96 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.073+0000 7f5fc1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.073+0000 7f5fc1600640 1 -- 172.21.15.137:0/1024040175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc802e060 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.073+0000 7f5fc1600640 1 -- 172.21.15.137:0/1024040175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5fc8018030 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.073+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fd0004d20 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fc1600640 1 -- 172.21.15.137:0/1024040175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc8035050 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 10 client.18453 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff927437d0 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fc1600640 1 -- 172.21.15.137:0/1024040175 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fd0004d20 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fc1600640 1 -- 172.21.15.137:0/1024040175 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5fc800f3c0 con 0x55ff92a00810 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 10 client.18453 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 10 client.18453 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 20 client.18453 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a16570 0x55ff92a189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.079 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 4 client.18453 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.074+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff92972800 con 0x55ff92a16570 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.075+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a16570 0x55ff92a189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.075+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a16570 0x55ff92a189d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c20d0b204479eec9 server_cookie=d36d703d1e1511fe in_seq=0 out_seq=0 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.075+0000 7f5fc1600640 10 client.18453 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fc1600640 1 -- 172.21.15.137:0/1024040175 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ff92972800 con 0x55ff92a16570 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fc1600640 10 client.18453 handle_command_reply: tid=0 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 client.18453 shutdown 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 msgr2=0x55ff92a0aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a0aa80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5fc801ea00 tx=0x7f5fc8004730 comp rx=0 tx=0).stop 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 shutdown_connections 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a16570 0x55ff92a189d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a12000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a0afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.080 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1024040175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a0aa80 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 >> 172.21.15.137:0/1024040175 conn(0x55ff929fbc00 msgr2=0x55ff929faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fc1600640 0 client.18453 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 shutdown_connections 2024-03-26T14:15:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1024040175 wait complete. 2024-03-26T14:15:52.081 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.077+0000 7f5fe17a9740 20 client.18453 trim_cache size 0 max 16384 2024-03-26T14:15:52.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 Processor -- start 2024-03-26T14:15:52.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 -- start start 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a00810 0x55ff92a04c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 0x55ff92a11fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81ae940 con 0x55ff92a019e0 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81ac440 con 0x55ff92a023a0 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81aa4c0 con 0x55ff92a00810 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 0x55ff92a11fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 0x55ff92a11fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48762/0 (socket says 172.21.15.137:48762) 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.091+0000 7f5fd7400640 1 -- 172.21.15.137:0/224878582 learned_addr learned my addr 172.21.15.137:0/224878582 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fdca00640 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a00810 0x55ff92a04c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7400640 1 -- 172.21.15.137:0/224878582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a00810 msgr2=0x55ff92a04c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7400640 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a00810 0x55ff92a04c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7400640 1 -- 172.21.15.137:0/224878582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 msgr2=0x55ff92a05150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7400640 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a05150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7400640 1 -- 172.21.15.137:0/224878582 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff929fb970 con 0x55ff92a019e0 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fd7400640 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 0x55ff92a11fc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5fc402bc70 tx=0x7f5fc40ee630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a59acc822e474 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.092+0000 7f5fc3e00640 1 -- 172.21.15.137:0/224878582 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40bd260 con 0x55ff92a019e0 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fc3e00640 1 -- 172.21.15.137:0/224878582 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fc4004c50 con 0x55ff92a019e0 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fc3e00640 1 -- 172.21.15.137:0/224878582 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40044e0 con 0x55ff92a019e0 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 -- 172.21.15.137:0/224878582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 msgr2=0x55ff92a11fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 0x55ff92a11fc0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f5fc402bc70 tx=0x7f5fc40ee630 comp rx=0 tx=0).stop 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 -- 172.21.15.137:0/224878582 shutdown_connections 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a019e0 0x55ff92a11fc0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/224878582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a00810 0x55ff92a04c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 -- 172.21.15.137:0/224878582 >> 172.21.15.137:0/224878582 conn(0x55ff929fbc00 msgr2=0x55ff929ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.093+0000 7f5fe17a9740 1 -- 172.21.15.137:0/224878582 shutdown_connections 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 -- 172.21.15.137:0/224878582 wait complete. 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 -- start start 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 Processor -- start 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a11d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a0fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.094+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81aaac0 con 0x55ff92a00810 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81abde0 con 0x55ff92a023a0 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fe17a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81ab120 con 0x55ff92a019e0 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fe17a9740 10 client.0 _ll_register_callbacks cb 0x7f5fc80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48764/0 (socket says 172.21.15.137:48764) 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1778970493 learned_addr learned my addr 172.21.15.137:0/1778970493 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a11d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fdca00640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a0fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1778970493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 msgr2=0x55ff92a11d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a11d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 msgr2=0x55ff92a0fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a0fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7e00640 1 -- 172.21.15.137:0/1778970493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff92a10500 con 0x55ff92a00810 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.095+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a11d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fd7e00640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a117e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5fc801e9d0 tx=0x7f5fc8009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcf015c4b897cf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fc1600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fc1600640 1 -- 172.21.15.137:0/1778970493 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc803d020 con 0x55ff92a00810 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fc1600640 1 -- 172.21.15.137:0/1778970493 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5fc800f3c0 con 0x55ff92a00810 2024-03-26T14:15:52.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fc4004c50 con 0x55ff92a00810 2024-03-26T14:15:52.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fc1600640 1 -- 172.21.15.137:0/1778970493 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fc8045050 con 0x55ff92a00810 2024-03-26T14:15:52.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fe17a9740 10 client.18459 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:52.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ff92a72840 con 0x55ff92a00810 2024-03-26T14:15:52.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.096+0000 7f5fc1600640 1 -- 172.21.15.137:0/1778970493 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5fc4004c50 con 0x55ff92a00810 2024-03-26T14:15:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fc1600640 1 -- 172.21.15.137:0/1778970493 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f5fc8005230 con 0x55ff92a00810 2024-03-26T14:15:52.101 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fe17a9740 10 client.18459 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fe17a9740 10 client.18459 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fe17a9740 20 client.18459 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a731c0 0x55ff92a755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fe17a9740 4 client.18459 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff9296cb50 con 0x55ff92a731c0 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.097+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a731c0 0x55ff92a755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.098+0000 7f5fd7400640 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a731c0 0x55ff92a755e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8af5d5cbf7798b0d server_cookie=de1ff81591346cb2 in_seq=0 out_seq=0 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.098+0000 7f5fc1600640 10 client.18459 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fc1600640 1 -- 172.21.15.137:0/1778970493 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff9296cb50 con 0x55ff92a731c0 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fc1600640 10 client.18459 handle_command_reply: tid=0 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 client.18459 shutdown 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 msgr2=0x55ff92a117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a117e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5fc801e9d0 tx=0x7f5fc8009ae0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 shutdown_connections 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ff92a731c0 0x55ff92a755e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff92a023a0 0x55ff92a0fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff92a019e0 0x55ff92a11d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 --2- 172.21.15.137:0/1778970493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ff92a00810 0x55ff92a117e0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 >> 172.21.15.137:0/1778970493 conn(0x55ff929fbc00 msgr2=0x55ff929fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fc1600640 0 client.18459 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 shutdown_connections 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 1 -- 172.21.15.137:0/1778970493 wait complete. 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.099+0000 7f5fe17a9740 20 client.18459 trim_cache size 0 max 16384 2024-03-26T14:15:52.103 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:15:52.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5fbc0a2880 msgr2=0x7f5fbc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5fbc0a2880 0x7f5fbc0a4d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8104930 tx=0x7f5fc8005f90 comp rx=0 tx=0).stop 2024-03-26T14:15:52.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 msgr2=0x7f5fd819a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd819a060 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4000c00 tx=0x7f5fc40042f0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 shutdown_connections 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5fbc0a2880 0x7f5fbc0a4d40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd8105490 0x7f5fd81a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5fd8104ad0 0x7f5fd819a060 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 --2- 172.21.15.137:0/2063313635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5fd81038e0 0x7f5fd8199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 >> 172.21.15.137:0/2063313635 conn(0x7f5fd80fee50 msgr2=0x7f5fd8107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 shutdown_connections 2024-03-26T14:15:52.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.100+0000 7f5fdea00640 1 -- 172.21.15.137:0/2063313635 wait complete. 2024-03-26T14:15:52.120 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:15:52.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:15:52.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.355+0000 7f6d65800640 1 -- 172.21.15.137:0/1775406797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 msgr2=0x7f6d60102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.355+0000 7f6d65800640 1 --2- 172.21.15.137:0/1775406797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60102c10 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c009900 tx=0x7f6d4c02bc50 comp rx=0 tx=0).stop 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 -- 172.21.15.137:0/1775406797 shutdown_connections 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 --2- 172.21.15.137:0/1775406797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d60103150 0x7f6d601099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 --2- 172.21.15.137:0/1775406797 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60102c10 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 --2- 172.21.15.137:0/1775406797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d601015a0 0x7f6d601019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 -- 172.21.15.137:0/1775406797 >> 172.21.15.137:0/1775406797 conn(0x7f6d600fcb10 msgr2=0x7f6d600fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 -- 172.21.15.137:0/1775406797 shutdown_connections 2024-03-26T14:15:52.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.356+0000 7f6d65800640 1 -- 172.21.15.137:0/1775406797 wait complete. 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 Processor -- start 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 -- start start 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d601015a0 0x7f6d60199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d60103150 0x7f6d601a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6010d0e0 con 0x7f6d60102790 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6010cf60 con 0x7f6d60103150 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.357+0000 7f6d65800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6010d260 con 0x7f6d601015a0 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48792/0 (socket says 172.21.15.137:48792) 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 -- 172.21.15.137:0/3424170930 learned_addr learned my addr 172.21.15.137:0/3424170930 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d64e00640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d60103150 0x7f6d601a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d601015a0 0x7f6d60199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 -- 172.21.15.137:0/3424170930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d601015a0 msgr2=0x7f6d60199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d601015a0 0x7f6d60199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 -- 172.21.15.137:0/3424170930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d60103150 msgr2=0x7f6d601a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d60103150 0x7f6d601a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 -- 172.21.15.137:0/3424170930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d601a15a0 con 0x7f6d60102790 2024-03-26T14:15:52.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.358+0000 7f6d5f400640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60199ec0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c006010 tx=0x7f6d4c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.359+0000 7f6d5cc00640 1 -- 172.21.15.137:0/3424170930 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c043020 con 0x7f6d60102790 2024-03-26T14:15:52.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.359+0000 7f6d5cc00640 1 -- 172.21.15.137:0/3424170930 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d4c004440 con 0x7f6d60102790 2024-03-26T14:15:52.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.359+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d601a1830 con 0x7f6d60102790 2024-03-26T14:15:52.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.359+0000 7f6d5cc00640 1 -- 172.21.15.137:0/3424170930 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c003bf0 con 0x7f6d60102790 2024-03-26T14:15:52.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.359+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d601a1d80 con 0x7f6d60102790 2024-03-26T14:15:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 Processor -- start 2024-03-26T14:15:52.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 -- start start 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d5cc00640 1 -- 172.21.15.137:0/3424170930 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6d4c046020 con 0x7f6d60102790 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8a6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0f8092c0 con 0x559e0f8a49f0 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0f7b1c90 con 0x559e0f8a5bc0 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0f805360 con 0x559e0f8a6580 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.363+0000 7f6d64e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48802/0 (socket says 172.21.15.137:48802) 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 -- 172.21.15.137:0/288603876 learned_addr learned my addr 172.21.15.137:0/288603876 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8a6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d5f400640 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 -- 172.21.15.137:0/288603876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 msgr2=0x559e0f8ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 -- 172.21.15.137:0/288603876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 msgr2=0x559e0f8a6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8a6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 -- 172.21.15.137:0/288603876 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e0f7d6080 con 0x559e0f8a49f0 2024-03-26T14:15:52.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.364+0000 7f6d64e00640 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a4e10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009ee0 tx=0x7f6d58010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91246dff9907b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d55600640 1 -- 172.21.15.137:0/288603876 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58016070 con 0x559e0f8a49f0 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d55600640 1 -- 172.21.15.137:0/288603876 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d58004d20 con 0x559e0f8a49f0 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d55600640 1 -- 172.21.15.137:0/288603876 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58012740 con 0x559e0f8a49f0 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d5cc00640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d440a2800 0x7f6d440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d5cc00640 1 -- 172.21.15.137:0/3424170930 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d4c0f1300 con 0x7f6d60102790 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d440a2800 0x7f6d440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d69bf3740 1 -- 172.21.15.137:0/288603876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 msgr2=0x559e0f8a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.365+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a4e10 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009ee0 tx=0x7f6d58010740 comp rx=0 tx=0).stop 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 -- 172.21.15.137:0/288603876 shutdown_connections 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8a6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/288603876 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a4e10 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 -- 172.21.15.137:0/288603876 >> 172.21.15.137:0/288603876 conn(0x559e0f89fde0 msgr2=0x559e0f8a01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 -- 172.21.15.137:0/288603876 shutdown_connections 2024-03-26T14:15:52.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 -- 172.21.15.137:0/288603876 wait complete. 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d440a2800 0x7f6d440a4cc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d601025f0 tx=0x7f6d50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.366+0000 7f6d69bf3740 1 -- start start 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 Processor -- start 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a49f0 0x559e0f8aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 0x559e0f8af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0f8ad590 con 0x559e0f8a5bc0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0f7c50d0 con 0x559e0f8a49f0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0f8ad710 con 0x559e0f8a6580 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d69bf3740 10 client.0 _ll_register_callbacks cb 0x559e0f7c9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.367+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 0x559e0f8af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 0x559e0f8af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48816/0 (socket says 172.21.15.137:48816) 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 -- 172.21.15.137:0/848882839 learned_addr learned my addr 172.21.15.137:0/848882839 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a49f0 0x559e0f8aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d64e00640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 -- 172.21.15.137:0/848882839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 msgr2=0x559e0f8b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 -- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a49f0 msgr2=0x559e0f8aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a49f0 0x559e0f8aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 -- 172.21.15.137:0/848882839 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e0f51d3f0 con 0x559e0f8a5bc0 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a49f0 0x559e0f8aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d5f400640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 0x559e0f8af1a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0bd860 tx=0x7f6d4c0bd6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e749389953bf09 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.368+0000 7f6d4a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d4a000640 1 -- 172.21.15.137:0/848882839 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0a6e60 con 0x559e0f8a5bc0 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d4a000640 1 -- 172.21.15.137:0/848882839 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d4c0a80b0 con 0x559e0f8a5bc0 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d58004d20 con 0x559e0f8a5bc0 2024-03-26T14:15:52.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d4a000640 1 -- 172.21.15.137:0/848882839 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d4c005560 con 0x559e0f8a5bc0 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d69bf3740 10 client.18471 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559e0f5e77d0 con 0x559e0f8a5bc0 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d4a000640 1 -- 172.21.15.137:0/848882839 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d58004d20 con 0x559e0f8a5bc0 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d4a000640 1 -- 172.21.15.137:0/848882839 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6d4c0c5be0 con 0x559e0f8a5bc0 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d69bf3740 10 client.18471 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.369+0000 7f6d69bf3740 10 client.18471 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.370+0000 7f6d69bf3740 20 client.18471 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.370+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f8ba6c0 0x559e0f8bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.370+0000 7f6d69bf3740 4 client.18471 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.370+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e0f816800 con 0x559e0f8ba6c0 2024-03-26T14:15:52.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.370+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f8ba6c0 0x559e0f8bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.371+0000 7f6d4a000640 10 client.18471 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:52.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.371+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f8ba6c0 0x559e0f8bcaf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ee23be5b20c63d1 server_cookie=fd33e6a04008a1c1 in_seq=0 out_seq=0 2024-03-26T14:15:52.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.372+0000 7f6d4a000640 1 -- 172.21.15.137:0/848882839 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559e0f816800 con 0x559e0f8ba6c0 2024-03-26T14:15:52.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.372+0000 7f6d4a000640 10 client.18471 handle_command_reply: tid=0 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.372+0000 7f6d69bf3740 1 client.18471 shutdown 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 msgr2=0x559e0f8af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 0x559e0f8af1a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0bd860 tx=0x7f6d4c0bd6a0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 shutdown_connections 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f8ba6c0 0x559e0f8bcaf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d4a000640 0 client.18471 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a5bc0 0x559e0f8af1a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/848882839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a49f0 0x559e0f8aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 >> 172.21.15.137:0/848882839 conn(0x559e0f89fde0 msgr2=0x559e0f89f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 shutdown_connections 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 1 -- 172.21.15.137:0/848882839 wait complete. 2024-03-26T14:15:52.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.373+0000 7f6d69bf3740 20 client.18471 trim_cache size 0 max 16384 2024-03-26T14:15:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.391+0000 7f6d69bf3740 1 Processor -- start 2024-03-26T14:15:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.391+0000 7f6d69bf3740 1 -- start start 2024-03-26T14:15:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8a9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601ae690 con 0x559e0f8a49f0 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601ac190 con 0x559e0f8a5bc0 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601aa210 con 0x559e0f8a6580 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48826/0 (socket says 172.21.15.137:48826) 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d5fe00640 1 -- 172.21.15.137:0/3820786296 learned_addr learned my addr 172.21.15.137:0/3820786296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d5f400640 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.392+0000 7f6d64e00640 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d5fe00640 1 -- 172.21.15.137:0/3820786296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 msgr2=0x559e0f8a9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8a9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d5fe00640 1 -- 172.21.15.137:0/3820786296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 msgr2=0x559e0f8b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.394 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d5fe00640 1 -- 172.21.15.137:0/3820786296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e0f8bd170 con 0x559e0f8a49f0 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a8df0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x559e0f8bd3a0 tx=0x7f6d50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebeb398b9074c8f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d54c00640 1 -- 172.21.15.137:0/3820786296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50020cc0 con 0x559e0f8a49f0 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d54c00640 1 -- 172.21.15.137:0/3820786296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d5002c030 con 0x559e0f8a49f0 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.393+0000 7f6d54c00640 1 -- 172.21.15.137:0/3820786296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d500116b0 con 0x559e0f8a49f0 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 -- 172.21.15.137:0/3820786296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 msgr2=0x559e0f8a8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a8df0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x559e0f8bd3a0 tx=0x7f6d50019000 comp rx=0 tx=0).stop 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 -- 172.21.15.137:0/3820786296 shutdown_connections 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8a9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/3820786296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8a8df0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 -- 172.21.15.137:0/3820786296 >> 172.21.15.137:0/3820786296 conn(0x559e0f89fde0 msgr2=0x559e0f8a38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 -- 172.21.15.137:0/3820786296 shutdown_connections 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.394+0000 7f6d69bf3740 1 -- 172.21.15.137:0/3820786296 wait complete. 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 -- start start 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 Processor -- start 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601aa810 con 0x559e0f8a49f0 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601abb30 con 0x559e0f8a5bc0 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.395+0000 7f6d69bf3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601aae70 con 0x559e0f8a6580 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d69bf3740 10 client.0 _ll_register_callbacks cb 0x7f6d4c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48830/0 (socket says 172.21.15.137:48830) 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 -- 172.21.15.137:0/2539624940 learned_addr learned my addr 172.21.15.137:0/2539624940 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5f400640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d64e00640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 -- 172.21.15.137:0/2539624940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 msgr2=0x559e0f8b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 -- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 msgr2=0x559e0f8b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 -- 172.21.15.137:0/2539624940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e0f8b4ea0 con 0x559e0f8a49f0 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d5fe00640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8b5d30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6d50020d30 tx=0x7f6d50020d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4666c91cf4496f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.396+0000 7f6d4a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:52.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d4a000640 1 -- 172.21.15.137:0/2539624940 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50005240 con 0x559e0f8a49f0 2024-03-26T14:15:52.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d4a000640 1 -- 172.21.15.137:0/2539624940 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6d500053e0 con 0x559e0f8a49f0 2024-03-26T14:15:52.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d5002c030 con 0x559e0f8a49f0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d4a000640 1 -- 172.21.15.137:0/2539624940 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d500055e0 con 0x559e0f8a49f0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d69bf3740 10 client.18477 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559e0f916ca0 con 0x559e0f8a49f0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.397+0000 7f6d4a000640 1 -- 172.21.15.137:0/2539624940 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d5002c030 con 0x559e0f8a49f0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d4a000640 1 -- 172.21.15.137:0/2539624940 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6d50038070 con 0x559e0f8a49f0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d69bf3740 10 client.18477 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d69bf3740 10 client.18477 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d69bf3740 20 client.18477 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f917600 0x559e0f919a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d69bf3740 4 client.18477 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e0f810b50 con 0x559e0f917600 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.398+0000 7f6d5f400640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f917600 0x559e0f919a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d5f400640 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f917600 0x559e0f919a20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6e62e8b085e5429 server_cookie=dff24dcbb9dd572c in_seq=0 out_seq=0 2024-03-26T14:15:52.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d4a000640 10 client.18477 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d4a000640 1 -- 172.21.15.137:0/2539624940 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e0f810b50 con 0x559e0f917600 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d4a000640 10 client.18477 handle_command_reply: tid=0 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d69bf3740 1 client.18477 shutdown 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 msgr2=0x559e0f8b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.399+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8b5d30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6d50020d30 tx=0x7f6d50020d60 comp rx=0 tx=0).stop 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 shutdown_connections 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x559e0f917600 0x559e0f919a20 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e0f8a5bc0 0x559e0f8b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559e0f8a6580 0x559e0f8b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d4a000640 0 client.18477 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 --2- 172.21.15.137:0/2539624940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x559e0f8a49f0 0x559e0f8b5d30 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 >> 172.21.15.137:0/2539624940 conn(0x559e0f89fde0 msgr2=0x559e0f89e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 shutdown_connections 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 1 -- 172.21.15.137:0/2539624940 wait complete. 2024-03-26T14:15:52.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.400+0000 7f6d69bf3740 20 client.18477 trim_cache size 0 max 16384 2024-03-26T14:15:52.404 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:15:52.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d440a2800 msgr2=0x7f6d440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d440a2800 0x7f6d440a4cc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d601025f0 tx=0x7f6d50005f90 comp rx=0 tx=0).stop 2024-03-26T14:15:52.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 msgr2=0x7f6d60199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.404 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60199ec0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c006010 tx=0x7f6d4c004040 comp rx=0 tx=0).stop 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 shutdown_connections 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6d440a2800 0x7f6d440a4cc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d60103150 0x7f6d601a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6d60102790 0x7f6d60199ec0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 --2- 172.21.15.137:0/3424170930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d601015a0 0x7f6d60199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 >> 172.21.15.137:0/3424170930 conn(0x7f6d600fcb10 msgr2=0x7f6d60107f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 shutdown_connections 2024-03-26T14:15:52.405 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.401+0000 7f6d65800640 1 -- 172.21.15.137:0/3424170930 wait complete. 2024-03-26T14:15:52.418 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:52 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/169049398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:15:52.423 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:15:52.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:15:52.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 -- 172.21.15.137:0/3554153866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584108eb0 msgr2=0x7f2584109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 --2- 172.21.15.137:0/3554153866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584108eb0 0x7f2584109330 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f2570009900 tx=0x7f257002bc30 comp rx=0 tx=0).stop 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 -- 172.21.15.137:0/3554153866 shutdown_connections 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 --2- 172.21.15.137:0/3554153866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584109870 0x7f2584110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 --2- 172.21.15.137:0/3554153866 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584108eb0 0x7f2584109330 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 --2- 172.21.15.137:0/3554153866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 0x7f25841080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 -- 172.21.15.137:0/3554153866 >> 172.21.15.137:0/3554153866 conn(0x7f2584076c60 msgr2=0x7f25840790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.639+0000 7f258b400640 1 -- 172.21.15.137:0/3554153866 shutdown_connections 2024-03-26T14:15:52.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.640+0000 7f258b400640 1 -- 172.21.15.137:0/3554153866 wait complete. 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.640+0000 7f258b400640 1 Processor -- start 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 -- start start 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 0x7f258419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584108eb0 0x7f258419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 0x7f25841a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841137f0 con 0x7f2584109870 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584113670 con 0x7f2584107cc0 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f258b400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584113970 con 0x7f2584108eb0 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f2589400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 0x7f25841a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f2589400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 0x7f25841a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48850/0 (socket says 172.21.15.137:48850) 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f2589400640 1 -- 172.21.15.137:0/4140991347 learned_addr learned my addr 172.21.15.137:0/4140991347 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.641+0000 7f2588a00640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 0x7f258419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2583e00640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584108eb0 0x7f258419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2589400640 1 -- 172.21.15.137:0/4140991347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584108eb0 msgr2=0x7f258419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2589400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584108eb0 0x7f258419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2589400640 1 -- 172.21.15.137:0/4140991347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 msgr2=0x7f258419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2589400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 0x7f258419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2589400640 1 -- 172.21.15.137:0/4140991347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25841a5b10 con 0x7f2584109870 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2583e00640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584108eb0 0x7f258419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2588a00640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 0x7f258419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2589400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 0x7f25841a54a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f257c00eaa0 tx=0x7f257c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2581600640 1 -- 172.21.15.137:0/4140991347 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00cdb0 con 0x7f2584109870 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f2581600640 1 -- 172.21.15.137:0/4140991347 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f257c005020 con 0x7f2584109870 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.642+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25841a5e00 con 0x7f2584109870 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.643+0000 7f2581600640 1 -- 172.21.15.137:0/4140991347 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0116d0 con 0x7f2584109870 2024-03-26T14:15:52.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.643+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841a62c0 con 0x7f2584109870 2024-03-26T14:15:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.646+0000 7f2581600640 1 -- 172.21.15.137:0/4140991347 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f257c0051d0 con 0x7f2584109870 2024-03-26T14:15:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.646+0000 7f258e019740 1 Processor -- start 2024-03-26T14:15:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 -- start start 2024-03-26T14:15:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88129c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8812ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb88131c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8808e2c0 con 0x55eb88129810 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb88036c90 con 0x55eb8812a9e0 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f258e019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8808a360 con 0x55eb8812b3a0 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f2583e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88129c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f2583e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88129c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48856/0 (socket says 172.21.15.137:48856) 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f2583e00640 1 -- 172.21.15.137:0/4262903024 learned_addr learned my addr 172.21.15.137:0/4262903024 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.647+0000 7f2588a00640 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8812ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.648+0000 7f2583e00640 1 -- 172.21.15.137:0/4262903024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 msgr2=0x55eb88131c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.648+0000 7f2583e00640 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb88131c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.648+0000 7f2583e00640 1 -- 172.21.15.137:0/4262903024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 msgr2=0x55eb8812ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.648+0000 7f2583e00640 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8812ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.648+0000 7f2583e00640 1 -- 172.21.15.137:0/4262903024 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb8805b080 con 0x55eb88129810 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.648+0000 7f2583e00640 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88129c30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f257002c6a0 tx=0x7f2570005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f069bdf39ae89b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f2575c00640 1 -- 172.21.15.137:0/4262903024 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257002cc20 con 0x55eb88129810 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f2575c00640 1 -- 172.21.15.137:0/4262903024 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2570044070 con 0x55eb88129810 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f2575c00640 1 -- 172.21.15.137:0/4262903024 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700046c0 con 0x55eb88129810 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f2581600640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f256c0a2800 0x7f256c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f2581600640 1 -- 172.21.15.137:0/4140991347 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f257c0cf0e0 con 0x7f2584109870 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f2588a00640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f256c0a2800 0x7f256c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f258e019740 1 -- 172.21.15.137:0/4262903024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 msgr2=0x55eb88129c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.649+0000 7f258e019740 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88129c30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f257002c6a0 tx=0x7f2570005d50 comp rx=0 tx=0).stop 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 -- 172.21.15.137:0/4262903024 shutdown_connections 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8812ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb88131c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 --2- 172.21.15.137:0/4262903024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88129c30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 -- 172.21.15.137:0/4262903024 >> 172.21.15.137:0/4262903024 conn(0x55eb88124c00 msgr2=0x55eb88125000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 -- 172.21.15.137:0/4262903024 shutdown_connections 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 -- 172.21.15.137:0/4262903024 wait complete. 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f2588a00640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f256c0a2800 0x7f256c0a4cc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2584108d10 tx=0x7f2578005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.650+0000 7f258e019740 1 -- start start 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 Processor -- start 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88133ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb88132350 con 0x55eb88129810 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb8804a0d0 con 0x55eb8812a9e0 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb881324d0 con 0x55eb8812b3a0 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f258e019740 10 client.0 _ll_register_callbacks cb 0x55eb8804ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f2588a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f2588a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88133ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48862/0 (socket says 172.21.15.137:48862) 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.651+0000 7f2588a00640 1 -- 172.21.15.137:0/4108737895 learned_addr learned my addr 172.21.15.137:0/4108737895 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2583e00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88133ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2588a00640 1 -- 172.21.15.137:0/4108737895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 msgr2=0x55eb8813afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2588a00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2588a00640 1 -- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 msgr2=0x55eb88133ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2588a00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88133ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2588a00640 1 -- 172.21.15.137:0/4108737895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb87da23f0 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2583e00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88133ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f2588a00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88133ab0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2578004610 tx=0x7f2578004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23441441080bc543 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f256a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.652+0000 7f256a000640 1 -- 172.21.15.137:0/4108737895 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578003d80 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.653+0000 7f256a000640 1 -- 172.21.15.137:0/4108737895 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f257802a420 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.653+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2570044070 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.653+0000 7f256a000640 1 -- 172.21.15.137:0/4108737895 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2578032050 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.653+0000 7f258e019740 10 client.18489 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.653+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eb87e6c7d0 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.653+0000 7f256a000640 1 -- 172.21.15.137:0/4108737895 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2570044070 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f256a000640 1 -- 172.21.15.137:0/4108737895 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f25780205d0 con 0x55eb88129810 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f258e019740 10 client.18489 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f258e019740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:52.658 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f258e019740 20 client.18489 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f258e019740 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8813f610 0x55eb88141a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f258e019740 4 client.18489 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb8809b800 con 0x55eb8813f610 2024-03-26T14:15:52.659 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.654+0000 7f2583e00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8813f610 0x55eb88141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.655+0000 7f2583e00640 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8813f610 0x55eb88141a80 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8520277befbef8a server_cookie=81a175ae97e6a074 in_seq=0 out_seq=0 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.655+0000 7f256a000640 10 client.18489 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.656+0000 7f256a000640 1 -- 172.21.15.137:0/4108737895 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eb8809b800 con 0x55eb8813f610 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.656+0000 7f256a000640 10 client.18489 handle_command_reply: tid=0 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.656+0000 7f258e019740 1 client.18489 shutdown 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 msgr2=0x55eb88133ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88133ab0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f2578004610 tx=0x7f2578004730 comp rx=0 tx=0).stop 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 shutdown_connections 2024-03-26T14:15:52.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8813f610 0x55eb88141a80 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88133ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 --2- 172.21.15.137:0/4108737895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88133ab0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f256a000640 0 client.18489 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 >> 172.21.15.137:0/4108737895 conn(0x55eb88124c00 msgr2=0x55eb88123eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 shutdown_connections 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 1 -- 172.21.15.137:0/4108737895 wait complete. 2024-03-26T14:15:52.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.657+0000 7f258e019740 20 client.18489 trim_cache size 0 max 16384 2024-03-26T14:15:52.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:52 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/169049398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:15:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 Processor -- start 2024-03-26T14:15:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 -- start start 2024-03-26T14:15:52.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb881318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8813af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8812da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841b2d50 con 0x55eb88129810 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841b0850 con 0x55eb8812a9e0 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.674+0000 7f258e019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841ae8d0 con 0x55eb8812b3a0 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb881318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb881318f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48866/0 (socket says 172.21.15.137:48866) 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 -- 172.21.15.137:0/1406084375 learned_addr learned my addr 172.21.15.137:0/1406084375 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2583e00640 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2589400640 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8812da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 -- 172.21.15.137:0/1406084375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 msgr2=0x55eb8812da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8812da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 -- 172.21.15.137:0/1406084375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 msgr2=0x55eb8813af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8813af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 -- 172.21.15.137:0/1406084375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb88131570 con 0x55eb88129810 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2589400640 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8812da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.675+0000 7f2588a00640 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb881318f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2578021010 tx=0x7f2578005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a386ee7d515f29 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.676+0000 7f2575200640 1 -- 172.21.15.137:0/1406084375 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257803a020 con 0x55eb88129810 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.676+0000 7f2575200640 1 -- 172.21.15.137:0/1406084375 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f257802f040 con 0x55eb88129810 2024-03-26T14:15:52.678 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.676+0000 7f2575200640 1 -- 172.21.15.137:0/1406084375 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578036690 con 0x55eb88129810 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.676+0000 7f258e019740 1 -- 172.21.15.137:0/1406084375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 msgr2=0x55eb881318f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.676+0000 7f258e019740 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb881318f0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2578021010 tx=0x7f2578005a70 comp rx=0 tx=0).stop 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 -- 172.21.15.137:0/1406084375 shutdown_connections 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb8813af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8812da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 --2- 172.21.15.137:0/1406084375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb881318f0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 -- 172.21.15.137:0/1406084375 >> 172.21.15.137:0/1406084375 conn(0x55eb88124c00 msgr2=0x55eb881286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 -- 172.21.15.137:0/1406084375 shutdown_connections 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 -- 172.21.15.137:0/1406084375 wait complete. 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 -- start start 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.677+0000 7f258e019740 1 Processor -- start 2024-03-26T14:15:52.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88137030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88137570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841aeed0 con 0x55eb88129810 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841b01f0 con 0x55eb8812a9e0 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841af530 con 0x55eb8812b3a0 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f258e019740 10 client.0 _ll_register_callbacks cb 0x7f2578009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f2588a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88137030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f2588a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88137030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48880/0 (socket says 172.21.15.137:48880) 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f2588a00640 1 -- 172.21.15.137:0/4114663869 learned_addr learned my addr 172.21.15.137:0/4114663869 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.678+0000 7f2583e00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2589400640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2588a00640 1 -- 172.21.15.137:0/4114663869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 msgr2=0x55eb8813a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2588a00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2588a00640 1 -- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 msgr2=0x55eb88137570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2588a00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88137570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2588a00640 1 -- 172.21.15.137:0/4114663869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb8813ab80 con 0x55eb88129810 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2583e00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f2588a00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88137030 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2578020a40 tx=0x7f25780039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f4ab680a70715f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f256a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f256a000640 1 -- 172.21.15.137:0/4114663869 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578043070 con 0x55eb88129810 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f256a000640 1 -- 172.21.15.137:0/4114663869 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2578020410 con 0x55eb88129810 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.679+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f257802f040 con 0x55eb88129810 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f256a000640 1 -- 172.21.15.137:0/4114663869 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257804f050 con 0x55eb88129810 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f258e019740 10 client.18495 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:52.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eb8819b800 con 0x55eb88129810 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f256a000640 1 -- 172.21.15.137:0/4114663869 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f257802f040 con 0x55eb88129810 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f256a000640 1 -- 172.21.15.137:0/4114663869 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2578036430 con 0x55eb88129810 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f258e019740 10 client.18495 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f258e019740 10 client.18495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f258e019740 20 client.18495 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.680+0000 7f258e019740 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8819c040 0x55eb8819e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.681+0000 7f258e019740 4 client.18495 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.681+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eb88095b50 con 0x55eb8819c040 2024-03-26T14:15:52.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.681+0000 7f2583e00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8819c040 0x55eb8819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:52.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f2583e00640 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8819c040 0x55eb8819e460 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea88c4ccc8414b52 server_cookie=e1217804341a7638 in_seq=0 out_seq=0 2024-03-26T14:15:52.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f256a000640 10 client.18495 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:52.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f256a000640 1 -- 172.21.15.137:0/4114663869 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eb88095b50 con 0x55eb8819c040 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f256a000640 10 client.18495 handle_command_reply: tid=0 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 client.18495 shutdown 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 msgr2=0x55eb88137030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88137030 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2578020a40 tx=0x7f25780039c0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 shutdown_connections 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55eb8819c040 0x55eb8819e460 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb8812a9e0 0x55eb88137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f256a000640 0 client.18495 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55eb8812b3a0 0x55eb8813a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 --2- 172.21.15.137:0/4114663869 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55eb88129810 0x55eb88137030 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.682+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 >> 172.21.15.137:0/4114663869 conn(0x55eb88124c00 msgr2=0x55eb881286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 shutdown_connections 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258e019740 1 -- 172.21.15.137:0/4114663869 wait complete. 2024-03-26T14:15:52.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258e019740 20 client.18495 trim_cache size 0 max 16384 2024-03-26T14:15:52.688 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:15:52.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f256c0a2800 msgr2=0x7f256c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258b400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f256c0a2800 0x7f256c0a4cc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2584108d10 tx=0x7f2578005f90 comp rx=0 tx=0).stop 2024-03-26T14:15:52.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 msgr2=0x7f25841a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:52.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.683+0000 7f258b400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 0x7f25841a54a0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f257c00eaa0 tx=0x7f257c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 shutdown_connections 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f256c0a2800 0x7f256c0a4cc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2584109870 0x7f25841a54a0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2584108eb0 0x7f258419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 --2- 172.21.15.137:0/4140991347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2584107cc0 0x7f258419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 >> 172.21.15.137:0/4140991347 conn(0x7f2584076c60 msgr2=0x7f258410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 shutdown_connections 2024-03-26T14:15:52.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:52.684+0000 7f258b400640 1 -- 172.21.15.137:0/4140991347 wait complete. 2024-03-26T14:15:52.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:52 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/169049398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:15:53.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:53 smithi179 ceph-mon[35472]: pgmap v863: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:53.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:53 smithi159 ceph-mon[35501]: pgmap v863: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:53.705 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:15:53.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:15:53.705 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:15:53.785 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:53 smithi137 ceph-mon[31539]: pgmap v863: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 -- 172.21.15.137:0/1265029352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c00740a0 msgr2=0x7f41c0074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 --2- 172.21.15.137:0/1265029352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c00740a0 0x7f41c0074500 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009900 tx=0x7f41ac02bcc0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 -- 172.21.15.137:0/1265029352 shutdown_connections 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 --2- 172.21.15.137:0/1265029352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41c0109430 0x7f41c010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 --2- 172.21.15.137:0/1265029352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c00740a0 0x7f41c0074500 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 --2- 172.21.15.137:0/1265029352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 0x7f41c0073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 -- 172.21.15.137:0/1265029352 >> 172.21.15.137:0/1265029352 conn(0x7f41c00fcb10 msgr2=0x7f41c00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 -- 172.21.15.137:0/1265029352 shutdown_connections 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.952+0000 7f41c6800640 1 -- 172.21.15.137:0/1265029352 wait complete. 2024-03-26T14:15:53.955 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 Processor -- start 2024-03-26T14:15:53.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 -- start start 2024-03-26T14:15:53.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41c00740a0 0x7f41c0199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 0x7f41c0199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 0x7f41c01a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c010f2b0 con 0x7f41c0109430 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.953+0000 7f41c6800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c010f130 con 0x7f41c0075680 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c6800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c010f430 con 0x7f41c00740a0 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 0x7f41c01a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 0x7f41c01a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48894/0 (socket says 172.21.15.137:48894) 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 -- 172.21.15.137:0/2847794152 learned_addr learned my addr 172.21.15.137:0/2847794152 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5400640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41c00740a0 0x7f41c0199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 0x7f41c0199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 -- 172.21.15.137:0/2847794152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41c00740a0 msgr2=0x7f41c0199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41c00740a0 0x7f41c0199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 -- 172.21.15.137:0/2847794152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 msgr2=0x7f41c0199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 0x7f41c0199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c5e00640 1 -- 172.21.15.137:0/2847794152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c01a1630 con 0x7f41c0109430 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.954+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 0x7f41c0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.955+0000 7f41c5e00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 0x7f41c01a0fc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eaa0 tx=0x7f41b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.955+0000 7f41be000640 1 -- 172.21.15.137:0/2847794152 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b800cdb0 con 0x7f41c0109430 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.955+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c01a1920 con 0x7f41c0109430 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.955+0000 7f41be000640 1 -- 172.21.15.137:0/2847794152 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41b8005020 con 0x7f41c0109430 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.955+0000 7f41be000640 1 -- 172.21.15.137:0/2847794152 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b80116d0 con 0x7f41c0109430 2024-03-26T14:15:53.959 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.955+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c01a1e70 con 0x7f41c0109430 2024-03-26T14:15:53.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.959+0000 7f41be000640 1 -- 172.21.15.137:0/2847794152 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f41b80051d0 con 0x7f41c0109430 2024-03-26T14:15:53.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.959+0000 7f41cab70740 1 Processor -- start 2024-03-26T14:15:53.961 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 -- start start 2024-03-26T14:15:53.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d89c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89cca0 0x562a9d8a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 0x562a9d8a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9d801eb0 con 0x562a9d8a1e10 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9d7aaa60 con 0x562a9d89cca0 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41cab70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9d7fdf50 con 0x562a9d89c2b0 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41c4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 0x562a9d8a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41c4a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 0x562a9d8a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48908/0 (socket says 172.21.15.137:48908) 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.960+0000 7f41c4a00640 1 -- 172.21.15.137:0/2677029019 learned_addr learned my addr 172.21.15.137:0/2677029019 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c5400640 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89cca0 0x562a9d8a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41be000640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41a80a2800 0x7f41a80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d89c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41be000640 1 -- 172.21.15.137:0/2847794152 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f41b80cf050 con 0x7f41c0109430 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 -- 172.21.15.137:0/2677029019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 msgr2=0x562a9d89c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:53.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d89c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 -- 172.21.15.137:0/2677029019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89cca0 msgr2=0x562a9d8a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89cca0 0x562a9d8a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 -- 172.21.15.137:0/2677029019 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9d5de7e0 con 0x562a9d8a1e10 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c5e00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41a80a2800 0x7f41a80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.961+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 0x562a9d8a42a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f41ac005940 tx=0x7f41ac0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c373aa7ab37af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.962+0000 7f41b2600640 1 -- 172.21.15.137:0/2677029019 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac049070 con 0x562a9d8a1e10 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.962+0000 7f41b2600640 1 -- 172.21.15.137:0/2677029019 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac02cb70 con 0x562a9d8a1e10 2024-03-26T14:15:53.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.962+0000 7f41b2600640 1 -- 172.21.15.137:0/2677029019 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac003e30 con 0x562a9d8a1e10 2024-03-26T14:15:53.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 -- 172.21.15.137:0/2677029019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 msgr2=0x562a9d8a42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41c5e00640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41a80a2800 0x7f41a80a4cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f41b800efd0 tx=0x7f41b800cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 0x562a9d8a42a0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f41ac005940 tx=0x7f41ac0058b0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 -- 172.21.15.137:0/2677029019 shutdown_connections 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89cca0 0x562a9d8a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d8a1e10 0x562a9d8a42a0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 --2- 172.21.15.137:0/2677029019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d89c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 -- 172.21.15.137:0/2677029019 >> 172.21.15.137:0/2677029019 conn(0x562a9d897820 msgr2=0x562a9d897c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 -- 172.21.15.137:0/2677029019 shutdown_connections 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 -- 172.21.15.137:0/2677029019 wait complete. 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.963+0000 7f41cab70740 1 -- start start 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 Processor -- start 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.969 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9d8a4d60 con 0x562a9d89cca0 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9d7bdeb0 con 0x562a9d8a1e10 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9d8a4ee0 con 0x562a9d89c2b0 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.964+0000 7f41cab70740 10 client.0 _ll_register_callbacks cb 0x562a9d7b2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48920/0 (socket says 172.21.15.137:48920) 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 -- 172.21.15.137:0/3145172296 learned_addr learned my addr 172.21.15.137:0/3145172296 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c4a00640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c4a00640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 -- 172.21.15.137:0/3145172296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 msgr2=0x562a9d8a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:53.970 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 -- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 msgr2=0x562a9d8ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 -- 172.21.15.137:0/3145172296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9d63ef70 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41c5400640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8a6a00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f41b40126e0 tx=0x7f41b4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc05dcce99d6134 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.965+0000 7f41a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41a6a00640 1 -- 172.21.15.137:0/3145172296 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4012190 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41ac02cb70 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41a6a00640 1 -- 172.21.15.137:0/3145172296 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f41b400f8c0 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41a6a00640 1 -- 172.21.15.137:0/3145172296 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b400fac0 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41cab70740 10 client.18507 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562a9d4d6170 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.966+0000 7f41a6a00640 1 -- 172.21.15.137:0/3145172296 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41ac02cb70 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41a6a00640 1 -- 172.21.15.137:0/3145172296 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f41b4010c10 con 0x562a9d89cca0 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41cab70740 10 client.18507 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:53.971 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41cab70740 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:53.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41cab70740 20 client.18507 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:53.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41cab70740 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d8b2470 0x562a9d8b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41cab70740 4 client.18507 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:53.972 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a9d80f3f0 con 0x562a9d8b2470 2024-03-26T14:15:53.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.967+0000 7f41c4a00640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d8b2470 0x562a9d8b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.968+0000 7f41c4a00640 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d8b2470 0x562a9d8b48b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d67b18d8fd66375 server_cookie=5aebc9ef707f5197 in_seq=0 out_seq=0 2024-03-26T14:15:53.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.968+0000 7f41a6a00640 10 client.18507 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:53.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.969+0000 7f41a6a00640 1 -- 172.21.15.137:0/3145172296 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562a9d80f3f0 con 0x562a9d8b2470 2024-03-26T14:15:53.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.969+0000 7f41a6a00640 10 client.18507 handle_command_reply: tid=0 2024-03-26T14:15:53.973 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.969+0000 7f41cab70740 1 client.18507 shutdown 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 msgr2=0x562a9d8a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8a6a00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f41b40126e0 tx=0x7f41b4012ba0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 shutdown_connections 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d8b2470 0x562a9d8b48b0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8a6a00 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 --2- 172.21.15.137:0/3145172296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 >> 172.21.15.137:0/3145172296 conn(0x562a9d897820 msgr2=0x562a9d896a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41a6a00640 0 client.18507 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 shutdown_connections 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 1 -- 172.21.15.137:0/3145172296 wait complete. 2024-03-26T14:15:53.974 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.970+0000 7f41cab70740 20 client.18507 trim_cache size 0 max 16384 2024-03-26T14:15:53.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.993+0000 7f41cab70740 1 Processor -- start 2024-03-26T14:15:53.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.993+0000 7f41cab70740 1 -- start start 2024-03-26T14:15:53.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.993+0000 7f41cab70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89c2b0 0x562a9d8b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.993+0000 7f41cab70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.993+0000 7f41cab70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d8a1e10 0x562a9d8b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:53.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41cab70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01ae6b0 con 0x562a9d89cca0 2024-03-26T14:15:53.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41cab70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01ac1b0 con 0x562a9d89c2b0 2024-03-26T14:15:53.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41cab70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01aa230 con 0x562a9d8a1e10 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48932/0 (socket says 172.21.15.137:48932) 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 -- 172.21.15.137:0/2864100671 learned_addr learned my addr 172.21.15.137:0/2864100671 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89c2b0 0x562a9d8b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c4a00640 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d8a1e10 0x562a9d8b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 -- 172.21.15.137:0/2864100671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d8a1e10 msgr2=0x562a9d8b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:53.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d8a1e10 0x562a9d8b96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 -- 172.21.15.137:0/2864100671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89c2b0 msgr2=0x562a9d8b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89c2b0 0x562a9d8b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.994+0000 7f41c5400640 1 -- 172.21.15.137:0/2864100671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9d898590 con 0x562a9d89cca0 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.995+0000 7f41c5400640 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ada00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f41b40123d0 tx=0x7f41b4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1983b1e1c6e73a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.995+0000 7f41b1c00640 1 -- 172.21.15.137:0/2864100671 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b401c6e0 con 0x562a9d89cca0 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.995+0000 7f41b1c00640 1 -- 172.21.15.137:0/2864100671 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41b400fc60 con 0x562a9d89cca0 2024-03-26T14:15:53.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.995+0000 7f41b1c00640 1 -- 172.21.15.137:0/2864100671 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b40035f0 con 0x562a9d89cca0 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 -- 172.21.15.137:0/2864100671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 msgr2=0x562a9d8ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ada00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f41b40123d0 tx=0x7f41b4009660 comp rx=0 tx=0).stop 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 -- 172.21.15.137:0/2864100671 shutdown_connections 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d8a1e10 0x562a9d8b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ada00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 --2- 172.21.15.137:0/2864100671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d89c2b0 0x562a9d8b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 -- 172.21.15.137:0/2864100671 >> 172.21.15.137:0/2864100671 conn(0x562a9d897820 msgr2=0x562a9d89b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 -- 172.21.15.137:0/2864100671 shutdown_connections 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.996+0000 7f41cab70740 1 -- 172.21.15.137:0/2864100671 wait complete. 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 -- start start 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 Processor -- start 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01aa830 con 0x562a9d89cca0 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01abb50 con 0x562a9d8a1e10 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01aae90 con 0x562a9d89c2b0 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41cab70740 10 client.0 _ll_register_callbacks cb 0x7f41b4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.997+0000 7f41c5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48948/0 (socket says 172.21.15.137:48948) 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 -- 172.21.15.137:0/92663690 learned_addr learned my addr 172.21.15.137:0/92663690 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c4a00640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c4a00640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 -- 172.21.15.137:0/92663690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 msgr2=0x562a9d8ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 -- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 msgr2=0x562a9d8ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 -- 172.21.15.137:0/92663690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9d8ac0c0 con 0x562a9d89cca0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41c5400640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ad8e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f41b4000c00 tx=0x7f41b40121b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef0c04984009681 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.998+0000 7f41a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41a6a00640 1 -- 172.21.15.137:0/92663690 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b401c690 con 0x562a9d89cca0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41a6a00640 1 -- 172.21.15.137:0/92663690 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f41b401c830 con 0x562a9d89cca0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41b400fc60 con 0x562a9d89cca0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41a6a00640 1 -- 172.21.15.137:0/92663690 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b403c050 con 0x562a9d89cca0 2024-03-26T14:15:54.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41cab70740 10 client.18513 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x562a9d90e7c0 con 0x562a9d89cca0 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41a6a00640 1 -- 172.21.15.137:0/92663690 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f41b400fc60 con 0x562a9d89cca0 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:53.999+0000 7f41a6a00640 1 -- 172.21.15.137:0/92663690 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f41b4031020 con 0x562a9d89cca0 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41cab70740 10 client.18513 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41cab70740 10 client.18513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41cab70740 20 client.18513 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41cab70740 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d90f090 0x562a9d9114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41cab70740 4 client.18513 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:15:54.004 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562a9d809740 con 0x562a9d90f090 2024-03-26T14:15:54.005 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.000+0000 7f41c4a00640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d90f090 0x562a9d9114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.001+0000 7f41c4a00640 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d90f090 0x562a9d9114b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7590af5305be6ca server_cookie=b6c24302bc74b086 in_seq=0 out_seq=0 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.001+0000 7f41a6a00640 10 client.18513 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41a6a00640 1 -- 172.21.15.137:0/92663690 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562a9d809740 con 0x562a9d90f090 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41a6a00640 10 client.18513 handle_command_reply: tid=0 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 client.18513 shutdown 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 msgr2=0x562a9d8ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ad8e0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f41b4000c00 tx=0x7f41b40121b0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 shutdown_connections 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x562a9d90f090 0x562a9d9114b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a9d8a1e10 0x562a9d8ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562a9d89cca0 0x562a9d8ad8e0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 --2- 172.21.15.137:0/92663690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a9d89c2b0 0x562a9d8ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.006 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 >> 172.21.15.137:0/92663690 conn(0x562a9d897820 msgr2=0x562a9d8961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.003+0000 7f41a6a00640 0 client.18513 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.004+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 shutdown_connections 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.004+0000 7f41cab70740 1 -- 172.21.15.137:0/92663690 wait complete. 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.004+0000 7f41cab70740 20 client.18513 trim_cache size 0 max 16384 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "a9360623-046f-4b97-857b-e63bacbb92e0", 2024-03-26T14:15:54.007 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41a80a2800 msgr2=0x7f41a80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41a80a2800 0x7f41a80a4cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f41b800efd0 tx=0x7f41b800cf50 comp rx=0 tx=0).stop 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 msgr2=0x7f41c01a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 0x7f41c01a0fc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f41b800eaa0 tx=0x7f41b800ef60 comp rx=0 tx=0).stop 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 shutdown_connections 2024-03-26T14:15:54.008 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41a80a2800 0x7f41a80a4cc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41c0075680 0x7f41c0199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41c0109430 0x7f41c01a0fc0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 --2- 172.21.15.137:0/2847794152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41c00740a0 0x7f41c0199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 >> 172.21.15.137:0/2847794152 conn(0x7f41c00fcb10 msgr2=0x7f41c01053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 shutdown_connections 2024-03-26T14:15:54.009 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.005+0000 7f41c6800640 1 -- 172.21.15.137:0/2847794152 wait complete. 2024-03-26T14:15:54.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:15:54.025 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:15:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 -- 172.21.15.137:0/329475037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 msgr2=0x7f34781096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 --2- 172.21.15.137:0/329475037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781096b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f3470009ca0 tx=0x7f347002c100 comp rx=0 tx=0).stop 2024-03-26T14:15:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 -- 172.21.15.137:0/329475037 shutdown_connections 2024-03-26T14:15:54.244 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 --2- 172.21.15.137:0/329475037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478104930 0x7f3478106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 --2- 172.21.15.137:0/329475037 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781096b0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 --2- 172.21.15.137:0/329475037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3478102000 0x7f34781043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 -- 172.21.15.137:0/329475037 >> 172.21.15.137:0/329475037 conn(0x7f34780fba10 msgr2=0x7f34780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 -- 172.21.15.137:0/329475037 shutdown_connections 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.238+0000 7f3477200640 1 -- 172.21.15.137:0/329475037 wait complete. 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.239+0000 7f3477200640 1 Processor -- start 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.239+0000 7f3477200640 1 -- start start 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3477200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102000 0x7f3478199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3477200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3478104930 0x7f3478199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3477200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3477200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347810d0c0 con 0x7f3478107260 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3477200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347810cf40 con 0x7f3478102000 2024-03-26T14:15:54.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3477200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347810d240 con 0x7f3478104930 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3475e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102000 0x7f3478199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48962/0 (socket says 172.21.15.137:48962) 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3476800640 1 -- 172.21.15.137:0/2000493280 learned_addr learned my addr 172.21.15.137:0/2000493280 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3475e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102000 0x7f3478199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37346/0 (socket says 172.21.15.137:37346) 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.240+0000 7f3475400640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3478104930 0x7f3478199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3476800640 1 -- 172.21.15.137:0/2000493280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3478104930 msgr2=0x7f3478199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3476800640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3478104930 0x7f3478199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3476800640 1 -- 172.21.15.137:0/2000493280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102000 msgr2=0x7f3478199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3476800640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102000 0x7f3478199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3476800640 1 -- 172.21.15.137:0/2000493280 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34781a1530 con 0x7f3478107260 2024-03-26T14:15:54.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3476800640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781a0ec0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3470032520 tx=0x7f3470035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f346ea00640 1 -- 172.21.15.137:0/2000493280 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470039040 con 0x7f3478107260 2024-03-26T14:15:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f346ea00640 1 -- 172.21.15.137:0/2000493280 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3470044070 con 0x7f3478107260 2024-03-26T14:15:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f346ea00640 1 -- 172.21.15.137:0/2000493280 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470004330 con 0x7f3478107260 2024-03-26T14:15:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34781a17c0 con 0x7f3478107260 2024-03-26T14:15:54.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.241+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34781a1d00 con 0x7f3478107260 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.245+0000 7f347fb95740 1 Processor -- start 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.245+0000 7f346ea00640 1 -- 172.21.15.137:0/2000493280 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f34700044d0 con 0x7f3478107260 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 -- start start 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bc2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bc7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc7bb0 0x556c12bca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12b27250 con 0x556c12bc2050 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12acfa30 con 0x556c12bc7bb0 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f347fb95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12b232f0 con 0x556c12bc2a40 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.246+0000 7f3475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bc2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bc2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48972/0 (socket says 172.21.15.137:48972) 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 -- 172.21.15.137:0/30749713 learned_addr learned my addr 172.21.15.137:0/30749713 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc7bb0 0x556c12bca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475e00640 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bc7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 -- 172.21.15.137:0/30749713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 msgr2=0x556c12bc7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bc7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 -- 172.21.15.137:0/30749713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc7bb0 msgr2=0x556c12bca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc7bb0 0x556c12bca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 -- 172.21.15.137:0/30749713 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c12aba690 con 0x556c12bc2050 2024-03-26T14:15:54.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f346ea00640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f345c0a2880 0x7f345c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.247+0000 7f3475400640 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bc2470 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3464012750 tx=0x7f34640096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef73ab2f680f228 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.248+0000 7f346ea00640 1 -- 172.21.15.137:0/2000493280 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f34700f1370 con 0x7f3478107260 2024-03-26T14:15:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.248+0000 7f3475e00640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f345c0a2880 0x7f345c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.248+0000 7f3463400640 1 -- 172.21.15.137:0/30749713 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464012cd0 con 0x556c12bc2050 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.248+0000 7f3463400640 1 -- 172.21.15.137:0/30749713 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3464010c80 con 0x556c12bc2050 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.248+0000 7f3463400640 1 -- 172.21.15.137:0/30749713 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346401eb30 con 0x556c12bc2050 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 -- 172.21.15.137:0/30749713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 msgr2=0x556c12bc2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f3475e00640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f345c0a2880 0x7f345c0a4d40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x556c12bc2e40 tx=0x7f3468005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bc2470 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3464012750 tx=0x7f34640096c0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 -- 172.21.15.137:0/30749713 shutdown_connections 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc7bb0 0x556c12bca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bc7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 --2- 172.21.15.137:0/30749713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bc2470 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 -- 172.21.15.137:0/30749713 >> 172.21.15.137:0/30749713 conn(0x556c12b8a200 msgr2=0x556c12bbd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 -- 172.21.15.137:0/30749713 shutdown_connections 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.249+0000 7f347fb95740 1 -- 172.21.15.137:0/30749713 wait complete. 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 -- start start 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 Processor -- start 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bcc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bcc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc7bb0 0x556c12bd36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12bcab60 con 0x556c12bc2050 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12ae2e80 con 0x556c12bc2a40 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.250+0000 7f347fb95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c12bcace0 con 0x556c12bc7bb0 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f347fb95740 10 client.0 _ll_register_callbacks cb 0x556c12af5130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bcc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bcc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:48986/0 (socket says 172.21.15.137:48986) 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 -- 172.21.15.137:0/2240178668 learned_addr learned my addr 172.21.15.137:0/2240178668 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:54.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475400640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bcc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3476800640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc7bb0 0x556c12bd36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 -- 172.21.15.137:0/2240178668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc7bb0 msgr2=0x556c12bd36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc7bb0 0x556c12bd36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 -- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 msgr2=0x556c12bcc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bcc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 -- 172.21.15.137:0/2240178668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c129057d0 con 0x556c12bc2050 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3475e00640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bcc120 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3468009560 tx=0x7f3468004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be80ccb679ff60da server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.251+0000 7f3457e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:54.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.252+0000 7f3457e00640 1 -- 172.21.15.137:0/2240178668 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346801e680 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.252+0000 7f3457e00640 1 -- 172.21.15.137:0/2240178668 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f346801e820 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.252+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3464010c80 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.252+0000 7f3457e00640 1 -- 172.21.15.137:0/2240178668 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f346801ea20 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.252+0000 7f347fb95740 10 client.18525 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.252+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556c1283b3f0 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f3457e00640 1 -- 172.21.15.137:0/2240178668 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3464010c80 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f3457e00640 1 -- 172.21.15.137:0/2240178668 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3468027070 con 0x556c12bc2050 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f347fb95740 10 client.18525 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f347fb95740 10 client.18525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f347fb95740 20 client.18525 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:54.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f347fb95740 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12bd8010 0x556c12bda490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f347fb95740 4 client.18525 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c12b34790 con 0x556c12bd8010 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.253+0000 7f3475400640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12bd8010 0x556c12bda490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.254+0000 7f3457e00640 10 client.18525 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.254+0000 7f3475400640 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12bd8010 0x556c12bda490 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef391bef5ff3c63 server_cookie=83cc77b7ba5fd447 in_seq=0 out_seq=0 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f3457e00640 1 -- 172.21.15.137:0/2240178668 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556c12b34790 con 0x556c12bd8010 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f3457e00640 10 client.18525 handle_command_reply: tid=0 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 client.18525 shutdown 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 msgr2=0x556c12bcc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bcc120 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3468009560 tx=0x7f3468004730 comp rx=0 tx=0).stop 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 shutdown_connections 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12bd8010 0x556c12bda490 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bcc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc7bb0 0x556c12bd36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 --2- 172.21.15.137:0/2240178668 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc2050 0x556c12bcc120 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 >> 172.21.15.137:0/2240178668 conn(0x556c12b8a200 msgr2=0x556c12bbcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f3457e00640 0 client.18525 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.255+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 shutdown_connections 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.256+0000 7f347fb95740 1 -- 172.21.15.137:0/2240178668 wait complete. 2024-03-26T14:15:54.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.256+0000 7f347fb95740 20 client.18525 trim_cache size 0 max 16384 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 Processor -- start 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 -- start start 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2050 0x556c12bcc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bd3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bc1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781ae710 con 0x556c12bc7bb0 2024-03-26T14:15:54.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781ac210 con 0x556c12bc2050 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.278+0000 7f347fb95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781aa290 con 0x556c12bc2a40 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3475e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2050 0x556c12bcc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bc1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3475e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2050 0x556c12bcc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:37370/0 (socket says 172.21.15.137:37370) 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3475e00640 1 -- 172.21.15.137:0/4121757964 learned_addr learned my addr 172.21.15.137:0/4121757964 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3475400640 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bd3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3476800640 1 -- 172.21.15.137:0/4121757964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 msgr2=0x556c12bd3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3476800640 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bd3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3476800640 1 -- 172.21.15.137:0/4121757964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2050 msgr2=0x556c12bcc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3476800640 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2050 0x556c12bcc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.279+0000 7f3476800640 1 -- 172.21.15.137:0/4121757964 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c127fb0f0 con 0x556c12bc7bb0 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.280+0000 7f3476800640 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bc1310 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f34700eea60 tx=0x7f34700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c42e9fb29151ce8 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.280+0000 7f3462a00640 1 -- 172.21.15.137:0/4121757964 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700327c0 con 0x556c12bc7bb0 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.280+0000 7f3462a00640 1 -- 172.21.15.137:0/4121757964 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34700be000 con 0x556c12bc7bb0 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.280+0000 7f3462a00640 1 -- 172.21.15.137:0/4121757964 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700bbc00 con 0x556c12bc7bb0 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 -- 172.21.15.137:0/4121757964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 msgr2=0x556c12bc1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bc1310 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f34700eea60 tx=0x7f34700095c0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 -- 172.21.15.137:0/4121757964 shutdown_connections 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bc1310 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2a40 0x556c12bd3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 --2- 172.21.15.137:0/4121757964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2050 0x556c12bcc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 -- 172.21.15.137:0/4121757964 >> 172.21.15.137:0/4121757964 conn(0x556c12b8a200 msgr2=0x556c12bc0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 -- 172.21.15.137:0/4121757964 shutdown_connections 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 -- 172.21.15.137:0/4121757964 wait complete. 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.281+0000 7f347fb95740 1 -- start start 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 Processor -- start 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2050 0x556c12bd0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bd1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bd22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781aa890 con 0x556c12bc7bb0 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781abbb0 con 0x556c12bc2a40 2024-03-26T14:15:54.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781aaef0 con 0x556c12bc2050 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f347fb95740 10 client.0 _ll_register_callbacks cb 0x7f3468009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f3476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bd22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f3476800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bd22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:49010/0 (socket says 172.21.15.137:49010) 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f3476800640 1 -- 172.21.15.137:0/1808419485 learned_addr learned my addr 172.21.15.137:0/1808419485 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.282+0000 7f3475400640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bd1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3475e00640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2050 0x556c12bd0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3476800640 1 -- 172.21.15.137:0/1808419485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2050 msgr2=0x556c12bd0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3476800640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2050 0x556c12bd0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3476800640 1 -- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 msgr2=0x556c12bd1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3476800640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bd1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3476800640 1 -- 172.21.15.137:0/1808419485 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c12bd28b0 con 0x556c12bc7bb0 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3475e00640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2050 0x556c12bd0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:15:54.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3476800640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bd22c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3470032380 tx=0x7f34700bbaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb2651f49542096 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3457e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3457e00640 1 -- 172.21.15.137:0/1808419485 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700bd900 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f3457e00640 1 -- 172.21.15.137:0/1808419485 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f34700bdb60 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.283+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34700be000 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f3457e00640 1 -- 172.21.15.137:0/1808419485 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34700c60a0 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 10 client.18531 fetch_fsmap learned FSMap version 24 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x556c12c33980 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f3457e00640 1 -- 172.21.15.137:0/1808419485 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34700be000 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f3457e00640 1 -- 172.21.15.137:0/1808419485 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f34700d1ce0 con 0x556c12bc7bb0 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 10 client.18531 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 10 client.18531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 20 client.18531 populate_metadata read hostname 'smithi137' 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12c33fe0 0x556c12c36400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:15:54.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 4 client.18531 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:15:54.290 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.284+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556c12b2eae0 con 0x556c12c33fe0 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.285+0000 7f3475e00640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12c33fe0 0x556c12c36400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.285+0000 7f3475e00640 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12c33fe0 0x556c12c36400 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0c231caf7d72aa server_cookie=f01ba91a8f8b3996 in_seq=0 out_seq=0 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.285+0000 7f3457e00640 10 client.18531 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f3457e00640 1 -- 172.21.15.137:0/1808419485 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556c12b2eae0 con 0x556c12c33fe0 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f3457e00640 10 client.18531 handle_command_reply: tid=0 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 client.18531 shutdown 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 msgr2=0x556c12bd22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bd22c0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3470032380 tx=0x7f34700bbaa0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 shutdown_connections 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x556c12c33fe0 0x556c12c36400 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c12bc2a40 0x556c12bd1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x556c12bc7bb0 0x556c12bd22c0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f3457e00640 0 client.18531 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 --2- 172.21.15.137:0/1808419485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c12bc2050 0x556c12bd0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 >> 172.21.15.137:0/1808419485 conn(0x556c12b8a200 msgr2=0x556c12bcc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 shutdown_connections 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 1 -- 172.21.15.137:0/1808419485 wait complete. 2024-03-26T14:15:54.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.286+0000 7f347fb95740 20 client.18531 trim_cache size 0 max 16384 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: "a9360623-046f-4b97-857b-e63bacbb92e0": { 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "a9360623-046f-4b97-857b-e63bacbb92e0", 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:15:54.293 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:15:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f345c0a2880 msgr2=0x7f345c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f345c0a2880 0x7f345c0a4d40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x556c12bc2e40 tx=0x7f3468005f90 comp rx=0 tx=0).stop 2024-03-26T14:15:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 msgr2=0x7f34781a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:15:54.294 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781a0ec0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f3470032520 tx=0x7f3470035ce0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 shutdown_connections 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f345c0a2880 0x7f345c0a4d40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3478107260 0x7f34781a0ec0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3478104930 0x7f3478199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 --2- 172.21.15.137:0/2000493280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102000 0x7f3478199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 >> 172.21.15.137:0/2000493280 conn(0x7f34780fba10 msgr2=0x7f3478101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 shutdown_connections 2024-03-26T14:15:54.295 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:15:54.287+0000 7f3477200640 1 -- 172.21.15.137:0/2000493280 wait complete. 2024-03-26T14:15:54.308 INFO:tasks.cephfs.filesystem:scrub status for tag:a9360623-046f-4b97-857b-e63bacbb92e0 - {'path': '/', 'tag': 'a9360623-046f-4b97-857b-e63bacbb92e0', 'options': 'recursive,force'} 2024-03-26T14:15:54.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:54 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:15:54.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:54 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:54.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:54 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:15:54.537 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:54 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:54.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:54 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:15:54.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:54 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:54.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:54 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:15:54.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:54 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:54.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:54 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:15:54.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:54 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:54.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:54 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:15:54.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:54 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:15:55.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:55 smithi179 ceph-mon[35472]: pgmap v864: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:55.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:55 smithi159 ceph-mon[35501]: pgmap v864: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:55.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:55 smithi137 ceph-mon[31539]: pgmap v864: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:15:57.588 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:57 smithi137 ceph-mon[31539]: pgmap v865: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:57.589 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:57 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:15:57.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:57 smithi179 ceph-mon[35472]: pgmap v865: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:57.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:57 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:15:57.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:57 smithi159 ceph-mon[35501]: pgmap v865: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:15:57.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:57 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:15:59.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:15:59 smithi179 ceph-mon[35472]: pgmap v866: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:15:59.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:15:59 smithi159 ceph-mon[35501]: pgmap v866: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:15:59.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:15:59 smithi137 ceph-mon[31539]: pgmap v866: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:01.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:01 smithi179 ceph-mon[35472]: pgmap v867: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:01 smithi159 ceph-mon[35501]: pgmap v867: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:01.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:01 smithi137 ceph-mon[31539]: pgmap v867: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:03.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:03 smithi179 ceph-mon[35472]: pgmap v868: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:03.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:03 smithi159 ceph-mon[35501]: pgmap v868: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:03.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:03 smithi137 ceph-mon[31539]: pgmap v868: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:05.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:05 smithi179 ceph-mon[35472]: pgmap v869: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:16:05.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:05 smithi159 ceph-mon[35501]: pgmap v869: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:16:05.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:05 smithi137 ceph-mon[31539]: pgmap v869: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:16:07.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:07 smithi179 ceph-mon[35472]: pgmap v870: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:07.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:07 smithi159 ceph-mon[35501]: pgmap v870: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:07.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:07 smithi137 ceph-mon[31539]: pgmap v870: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:09.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:09 smithi179 ceph-mon[35472]: pgmap v871: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:09.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:09 smithi159 ceph-mon[35501]: pgmap v871: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:09.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:09 smithi137 ceph-mon[31539]: pgmap v871: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:11.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:11 smithi179 ceph-mon[35472]: pgmap v872: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:11.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:11 smithi159 ceph-mon[35501]: pgmap v872: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:11.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:11 smithi137 ceph-mon[31539]: pgmap v872: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:13.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:13 smithi179 ceph-mon[35472]: pgmap v873: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:16:13.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:13 smithi159 ceph-mon[35501]: pgmap v873: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:16:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:13 smithi137 ceph-mon[31539]: pgmap v873: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:16:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:15 smithi179 ceph-mon[35472]: pgmap v874: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:15 smithi159 ceph-mon[35501]: pgmap v874: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:15 smithi137 ceph-mon[31539]: pgmap v874: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:17.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:17 smithi179 ceph-mon[35472]: pgmap v875: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:17.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:17 smithi159 ceph-mon[35501]: pgmap v875: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:17.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:17 smithi137 ceph-mon[31539]: pgmap v875: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:19.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:19 smithi179 ceph-mon[35472]: pgmap v876: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:19.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:19 smithi159 ceph-mon[35501]: pgmap v876: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:19.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:19 smithi137 ceph-mon[31539]: pgmap v876: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:21.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:21 smithi179 ceph-mon[35472]: pgmap v877: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:21.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:21 smithi159 ceph-mon[35501]: pgmap v877: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:21.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:21 smithi137 ceph-mon[31539]: pgmap v877: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:23.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:23 smithi179 ceph-mon[35472]: pgmap v878: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:16:23.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:23 smithi159 ceph-mon[35501]: pgmap v878: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:16:23.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:23 smithi137 ceph-mon[31539]: pgmap v878: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:16:24.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:16:24.310 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:16:24.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 -- 172.21.15.137:0/1183139513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198101670 msgr2=0x7fd198101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 --2- 172.21.15.137:0/1183139513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198101670 0x7fd198101af0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fd184009900 tx=0x7fd18402bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:24.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 -- 172.21.15.137:0/1183139513 shutdown_connections 2024-03-26T14:16:24.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 --2- 172.21.15.137:0/1183139513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd198102030 0x7fd1981088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 --2- 172.21.15.137:0/1183139513 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198101670 0x7fd198101af0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 --2- 172.21.15.137:0/1183139513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 0x7fd1981008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 -- 172.21.15.137:0/1183139513 >> 172.21.15.137:0/1183139513 conn(0x7fd1980fba10 msgr2=0x7fd1980fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 -- 172.21.15.137:0/1183139513 shutdown_connections 2024-03-26T14:16:24.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.550+0000 7fd19e600640 1 -- 172.21.15.137:0/1183139513 wait complete. 2024-03-26T14:16:24.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.551+0000 7fd19e600640 1 Processor -- start 2024-03-26T14:16:24.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 -- start start 2024-03-26T14:16:24.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 0x7fd198197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd198101670 0x7fd198197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 0x7fd19819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810bfc0 con 0x7fd198102030 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810be40 con 0x7fd198101670 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19e600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19810c140 con 0x7fd198100480 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 0x7fd19819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 0x7fd19819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51058/0 (socket says 172.21.15.137:51058) 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.552+0000 7fd19dc00640 1 -- 172.21.15.137:0/2910802306 learned_addr learned my addr 172.21.15.137:0/2910802306 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd197e00640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd198101670 0x7fd198197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19d200640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 0x7fd198197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19dc00640 1 -- 172.21.15.137:0/2910802306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 msgr2=0x7fd198197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19dc00640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 0x7fd198197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19dc00640 1 -- 172.21.15.137:0/2910802306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd198101670 msgr2=0x7fd198197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19dc00640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd198101670 0x7fd198197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19dc00640 1 -- 172.21.15.137:0/2910802306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19819f3f0 con 0x7fd198102030 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19d200640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 0x7fd198197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.553+0000 7fd19dc00640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 0x7fd19819ed80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd19000eaa0 tx=0x7fd19000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.554+0000 7fd195600640 1 -- 172.21.15.137:0/2910802306 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19000cdb0 con 0x7fd198102030 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.554+0000 7fd195600640 1 -- 172.21.15.137:0/2910802306 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd190005020 con 0x7fd198102030 2024-03-26T14:16:24.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.554+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19819f6e0 con 0x7fd198102030 2024-03-26T14:16:24.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.554+0000 7fd195600640 1 -- 172.21.15.137:0/2910802306 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900116d0 con 0x7fd198102030 2024-03-26T14:16:24.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.554+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19819fc00 con 0x7fd198102030 2024-03-26T14:16:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.557+0000 7fd1a2a2b740 1 Processor -- start 2024-03-26T14:16:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd195600640 1 -- 172.21.15.137:0/2910802306 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd1900051d0 con 0x7fd198102030 2024-03-26T14:16:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 -- start start 2024-03-26T14:16:24.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bc460 0x564e408bc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bce50 0x564e408c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408c4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e4081d250 con 0x564e408c1fc0 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e407c5a30 con 0x564e408bce50 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.558+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e408192f0 con 0x564e408bc460 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.559+0000 7fd197e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.559+0000 7fd197e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51060/0 (socket says 172.21.15.137:51060) 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.559+0000 7fd19d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bce50 0x564e408c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.559+0000 7fd197e00640 1 -- 172.21.15.137:0/4229584908 learned_addr learned my addr 172.21.15.137:0/4229584908 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.559+0000 7fd197e00640 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bc460 0x564e408bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.559+0000 7fd195600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1800a2880 0x7fd1800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd197e00640 1 -- 172.21.15.137:0/4229584908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bc460 msgr2=0x564e408bc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd197e00640 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bc460 0x564e408bc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd197e00640 1 -- 172.21.15.137:0/4229584908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bce50 msgr2=0x564e408c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd197e00640 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bce50 0x564e408c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd197e00640 1 -- 172.21.15.137:0/4229584908 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e407b0690 con 0x564e408c1fc0 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd19d200640 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bce50 0x564e408c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd195600640 1 -- 172.21.15.137:0/2910802306 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd1900ceff0 con 0x7fd198102030 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd19dc00640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1800a2880 0x7fd1800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd197e00640 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408c4450 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd1840096c0 tx=0x7fd18402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3b91b164c9c6ac server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd18a000640 1 -- 172.21.15.137:0/4229584908 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184004180 con 0x564e408c1fc0 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd18a000640 1 -- 172.21.15.137:0/4229584908 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd184005500 con 0x564e408c1fc0 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.560+0000 7fd18a000640 1 -- 172.21.15.137:0/4229584908 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184004ac0 con 0x564e408c1fc0 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd19dc00640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1800a2880 0x7fd1800a4d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd19000efd0 tx=0x7fd19000cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/4229584908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 msgr2=0x564e408c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408c4450 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd1840096c0 tx=0x7fd18402bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/4229584908 shutdown_connections 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bce50 0x564e408c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408c4450 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/4229584908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bc460 0x564e408bc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/4229584908 >> 172.21.15.137:0/4229584908 conn(0x564e40880200 msgr2=0x564e408b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/4229584908 shutdown_connections 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.561+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/4229584908 wait complete. 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.562+0000 7fd1a2a2b740 1 -- start start 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.562+0000 7fd1a2a2b740 1 Processor -- start 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 0x564e408b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408c1fc0 0x564e408b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e408c4f10 con 0x564e408bc460 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e407d8e80 con 0x564e408c1fc0 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e408c5090 con 0x564e408bce50 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd1a2a2b740 10 client.0 _ll_register_callbacks cb 0x564e407eb130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:24.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 0x564e408b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 0x564e408b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51064/0 (socket says 172.21.15.137:51064) 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 -- 172.21.15.137:0/2347899941 learned_addr learned my addr 172.21.15.137:0/2347899941 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd19d200640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408c1fc0 0x564e408b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 -- 172.21.15.137:0/2347899941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 msgr2=0x564e408b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 -- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408c1fc0 msgr2=0x564e408b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408c1fc0 0x564e408b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.563+0000 7fd197e00640 1 -- 172.21.15.137:0/2347899941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e405fb7d0 con 0x564e408bc460 2024-03-26T14:16:24.570 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd197e00640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 0x564e408b6cc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd184005850 tx=0x7fd1840056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2436bb5e7d8b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd17ea00640 1 -- 172.21.15.137:0/2347899941 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184003c30 con 0x564e408bc460 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd184005500 con 0x564e408bc460 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd17ea00640 1 -- 172.21.15.137:0/2347899941 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd184003dd0 con 0x564e408bc460 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd17ea00640 1 -- 172.21.15.137:0/2347899941 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18405d050 con 0x564e408bc460 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd1a2a2b740 10 client.18543 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:24.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.564+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564e405313f0 con 0x564e408bc460 2024-03-26T14:16:24.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd17ea00640 1 -- 172.21.15.137:0/2347899941 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd184005500 con 0x564e408bc460 2024-03-26T14:16:24.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd17ea00640 1 -- 172.21.15.137:0/2347899941 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd1840344d0 con 0x564e408bc460 2024-03-26T14:16:24.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd1a2a2b740 10 client.18543 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:24.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd1a2a2b740 10 client.18543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:24.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd1a2a2b740 20 client.18543 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e408d62b0 0x564e408d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd1a2a2b740 4 client.18543 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.565+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e4082a790 con 0x564e408d62b0 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.566+0000 7fd19d200640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e408d62b0 0x564e408d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.567+0000 7fd19d200640 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e408d62b0 0x564e408d86a0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd84b0e22f1f643 server_cookie=786126be02ea30a5 in_seq=0 out_seq=0 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.567+0000 7fd17ea00640 10 client.18543 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd17ea00640 1 -- 172.21.15.137:0/2347899941 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x564e4082a790 con 0x564e408d62b0 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd17ea00640 10 client.18543 handle_command_reply: tid=0 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 client.18543 shutdown 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 msgr2=0x564e408b6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 0x564e408b6cc0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd184005850 tx=0x7fd1840056b0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 shutdown_connections 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e408d62b0 0x564e408d86a0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408c1fc0 0x564e408b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd17ea00640 0 client.18543 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2347899941 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bc460 0x564e408b6cc0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 >> 172.21.15.137:0/2347899941 conn(0x564e40880200 msgr2=0x564e408b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 shutdown_connections 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2347899941 wait complete. 2024-03-26T14:16:24.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.569+0000 7fd1a2a2b740 20 client.18543 trim_cache size 0 max 16384 2024-03-26T14:16:24.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.592+0000 7fd1a2a2b740 1 Processor -- start 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.592+0000 7fd1a2a2b740 1 -- start start 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.592+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.592+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408dce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.592+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408dd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981aa2c0 con 0x564e408c1fc0 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a7dc0 con 0x564e408bc460 2024-03-26T14:16:24.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a5e40 con 0x564e408bce50 2024-03-26T14:16:24.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd197e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd197e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408dd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51068/0 (socket says 172.21.15.137:51068) 2024-03-26T14:16:24.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd197e00640 1 -- 172.21.15.137:0/2864186225 learned_addr learned my addr 172.21.15.137:0/2864186225 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:24.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd19d200640 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408dce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.593+0000 7fd197e00640 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd197e00640 1 -- 172.21.15.137:0/2864186225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 msgr2=0x564e408dce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd197e00640 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408dce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd197e00640 1 -- 172.21.15.137:0/2864186225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 msgr2=0x564e408be000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd197e00640 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408be000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd197e00640 1 -- 172.21.15.137:0/2864186225 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e404f10f0 con 0x564e408c1fc0 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd197e00640 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408dd340 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd18402ca60 tx=0x7fd184000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7f88fa8bf676cd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd189600640 1 -- 172.21.15.137:0/2864186225 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184005d50 con 0x564e408c1fc0 2024-03-26T14:16:24.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd189600640 1 -- 172.21.15.137:0/2864186225 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd184054070 con 0x564e408c1fc0 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.594+0000 7fd189600640 1 -- 172.21.15.137:0/2864186225 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184043740 con 0x564e408c1fc0 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2864186225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 msgr2=0x564e408dd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408dd340 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd18402ca60 tx=0x7fd184000bc0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2864186225 shutdown_connections 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408c1fc0 0x564e408dd340 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408bce50 0x564e408dce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2864186225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408be000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2864186225 >> 172.21.15.137:0/2864186225 conn(0x564e40880200 msgr2=0x564e408bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2864186225 shutdown_connections 2024-03-26T14:16:24.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.595+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2864186225 wait complete. 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 -- start start 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 Processor -- start 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408e65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 0x564e408e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408c1fc0 0x564e408e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a6440 con 0x564e408bce50 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a7760 con 0x564e408bc460 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1981a6aa0 con 0x564e408c1fc0 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.596+0000 7fd1a2a2b740 10 client.0 _ll_register_callbacks cb 0x7fd18c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:24.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 0x564e408e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd197e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408e65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 0x564e408e6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51076/0 (socket says 172.21.15.137:51076) 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 -- 172.21.15.137:0/2219684423 learned_addr learned my addr 172.21.15.137:0/2219684423 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd197e00640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408c1fc0 0x564e408e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 -- 172.21.15.137:0/2219684423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408c1fc0 msgr2=0x564e408e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408c1fc0 0x564e408e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 -- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 msgr2=0x564e408e65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408e65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd19d200640 1 -- 172.21.15.137:0/2219684423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e408e8070 con 0x564e408bce50 2024-03-26T14:16:24.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.597+0000 7fd197e00640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd19d200640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 0x564e408e6b10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c01da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152bb7c3fdacc206 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd17ea00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd17ea00640 1 -- 172.21.15.137:0/2219684423 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c004230 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd17ea00640 1 -- 172.21.15.137:0/2219684423 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd18c0043d0 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd184054070 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd17ea00640 1 -- 172.21.15.137:0/2219684423 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd18c02d050 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd1a2a2b740 10 client.18549 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x564e409316a0 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.598+0000 7fd17ea00640 1 -- 172.21.15.137:0/2219684423 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd184054070 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd17ea00640 1 -- 172.21.15.137:0/2219684423 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd18c0138a0 con 0x564e408bce50 2024-03-26T14:16:24.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd1a2a2b740 10 client.18549 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:24.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd1a2a2b740 10 client.18549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:24.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd1a2a2b740 20 client.18549 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:24.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e40931fd0 0x564e409343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd1a2a2b740 4 client.18549 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:16:24.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e40824ae0 con 0x564e40931fd0 2024-03-26T14:16:24.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.599+0000 7fd197e00640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e40931fd0 0x564e409343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd17ea00640 10 client.18549 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd197e00640 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e40931fd0 0x564e409343f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f7c5c0c9d0f5fd6 server_cookie=4ef449aed453312c in_seq=0 out_seq=0 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd17ea00640 1 -- 172.21.15.137:0/2219684423 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564e40824ae0 con 0x564e40931fd0 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd17ea00640 10 client.18549 handle_command_reply: tid=0 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 client.18549 shutdown 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 msgr2=0x564e408e6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 0x564e408e6b10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd18c005fd0 tx=0x7fd18c01da50 comp rx=0 tx=0).stop 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 shutdown_connections 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x564e40931fd0 0x564e409343f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.608 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e408c1fc0 0x564e408e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x564e408bce50 0x564e408e6b10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd17ea00640 0 client.18549 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 --2- 172.21.15.137:0/2219684423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e408bc460 0x564e408e65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.600+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 >> 172.21.15.137:0/2219684423 conn(0x564e40880200 msgr2=0x564e408be180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 shutdown_connections 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd1a2a2b740 1 -- 172.21.15.137:0/2219684423 wait complete. 2024-03-26T14:16:24.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd1a2a2b740 20 client.18549 trim_cache size 0 max 16384 2024-03-26T14:16:24.610 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:16:24.610 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:16:24.610 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:16:24.610 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1800a2880 msgr2=0x7fd1800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd19e600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1800a2880 0x7fd1800a4d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd19000efd0 tx=0x7fd19000cf50 comp rx=0 tx=0).stop 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 msgr2=0x7fd19819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd19e600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 0x7fd19819ed80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fd19000eaa0 tx=0x7fd19000ef60 comp rx=0 tx=0).stop 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 shutdown_connections 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.601+0000 7fd19e600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd1800a2880 0x7fd1800a4d40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.602+0000 7fd19e600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd198101670 0x7fd198197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.611 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.602+0000 7fd19e600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd198102030 0x7fd19819ed80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.602+0000 7fd19e600640 1 --2- 172.21.15.137:0/2910802306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd198100480 0x7fd198197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.602+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 >> 172.21.15.137:0/2910802306 conn(0x7fd1980fba10 msgr2=0x7fd1980fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.602+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 shutdown_connections 2024-03-26T14:16:24.612 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.602+0000 7fd19e600640 1 -- 172.21.15.137:0/2910802306 wait complete. 2024-03-26T14:16:24.620 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:16:24.620 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:16:24.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 -- 172.21.15.137:0/1216029858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35741015e0 msgr2=0x7f3574101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 --2- 172.21.15.137:0/1216029858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35741015e0 0x7f3574101a60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3560009860 tx=0x7f356002bb70 comp rx=0 tx=0).stop 2024-03-26T14:16:24.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 -- 172.21.15.137:0/1216029858 shutdown_connections 2024-03-26T14:16:24.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 --2- 172.21.15.137:0/1216029858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3574101fa0 0x7f3574108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 --2- 172.21.15.137:0/1216029858 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f35741015e0 0x7f3574101a60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 --2- 172.21.15.137:0/1216029858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35741003f0 0x7f3574100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 -- 172.21.15.137:0/1216029858 >> 172.21.15.137:0/1216029858 conn(0x7f35740fb980 msgr2=0x7f35740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:24.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 -- 172.21.15.137:0/1216029858 shutdown_connections 2024-03-26T14:16:24.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.839+0000 7f3579200640 1 -- 172.21.15.137:0/1216029858 wait complete. 2024-03-26T14:16:24.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.840+0000 7f3579200640 1 Processor -- start 2024-03-26T14:16:24.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 -- start start 2024-03-26T14:16:24.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35741003f0 0x7f35741998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35741015e0 0x7f3574199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 0x7f35741a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357410bf30 con 0x7f3574101fa0 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357410bdb0 con 0x7f35741015e0 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3579200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357410c0b0 con 0x7f35741003f0 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3573e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 0x7f35741a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3573e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 0x7f35741a0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51084/0 (socket says 172.21.15.137:51084) 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3573e00640 1 -- 172.21.15.137:0/256062881 learned_addr learned my addr 172.21.15.137:0/256062881 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3572a00640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35741015e0 0x7f3574199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.841+0000 7f3573400640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35741003f0 0x7f35741998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573e00640 1 -- 172.21.15.137:0/256062881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35741003f0 msgr2=0x7f35741998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.844 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573e00640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35741003f0 0x7f35741998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573e00640 1 -- 172.21.15.137:0/256062881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35741015e0 msgr2=0x7f3574199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573e00640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35741015e0 0x7f3574199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573e00640 1 -- 172.21.15.137:0/256062881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35741a1500 con 0x7f3574101fa0 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573400640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35741003f0 0x7f35741998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3573e00640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 0x7f35741a0e90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f356c00eaa0 tx=0x7f356c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00cdb0 con 0x7f3574101fa0 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f356c005020 con 0x7f3574101fa0 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c0116d0 con 0x7f3574101fa0 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.842+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35741a17f0 con 0x7f3574101fa0 2024-03-26T14:16:24.845 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.843+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35741a1d60 con 0x7f3574101fa0 2024-03-26T14:16:24.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.844+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35741a1fc0 con 0x7f3574101fa0 2024-03-26T14:16:24.846 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.846+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f356c0051d0 con 0x7f3574101fa0 2024-03-26T14:16:24.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.847+0000 7f356be00640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35580a2880 0x7f35580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:24.848 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.848+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f356c0cefc0 con 0x7f3574101fa0 2024-03-26T14:16:24.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.848+0000 7f3573400640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35580a2880 0x7f35580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:24.849 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.849+0000 7f3573400640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35580a2880 0x7f35580a4d40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3574101440 tx=0x7f3564009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:24.854 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:24.853+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f356c016030 con 0x7f3574101fa0 2024-03-26T14:16:25.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.112+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35741a2190 con 0x7f3574101fa0 2024-03-26T14:16:25.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.115+0000 7f356be00640 1 -- 172.21.15.137:0/256062881 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f356c09bc50 con 0x7f3574101fa0 2024-03-26T14:16:25.117 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:16:25.117 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:16:25.117 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:16:25.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35580a2880 msgr2=0x7f35580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35580a2880 0x7f35580a4d40 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3574101440 tx=0x7f3564009260 comp rx=0 tx=0).stop 2024-03-26T14:16:25.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 msgr2=0x7f35741a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 0x7f35741a0e90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f356c00eaa0 tx=0x7f356c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 shutdown_connections 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f35580a2880 0x7f35580a4d40 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35741015e0 0x7f3574199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3574101fa0 0x7f35741a0e90 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 --2- 172.21.15.137:0/256062881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f35741003f0 0x7f35741998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 >> 172.21.15.137:0/256062881 conn(0x7f35740fb980 msgr2=0x7f35740fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 shutdown_connections 2024-03-26T14:16:25.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.118+0000 7f3579200640 1 -- 172.21.15.137:0/256062881 wait complete. 2024-03-26T14:16:25.137 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:16:25.137 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:16:25.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 -- 172.21.15.137:0/2098224294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 msgr2=0x7f9e70101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 --2- 172.21.15.137:0/2098224294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e70101710 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009ca0 tx=0x7f9e6802c190 comp rx=0 tx=0).stop 2024-03-26T14:16:25.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 -- 172.21.15.137:0/2098224294 shutdown_connections 2024-03-26T14:16:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 --2- 172.21.15.137:0/2098224294 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e70101710 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 --2- 172.21.15.137:0/2098224294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e701008f0 0x7f9e70100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 --2- 172.21.15.137:0/2098224294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e700fff90 0x7f9e701003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 -- 172.21.15.137:0/2098224294 >> 172.21.15.137:0/2098224294 conn(0x7f9e700fbb30 msgr2=0x7f9e700fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 -- 172.21.15.137:0/2098224294 shutdown_connections 2024-03-26T14:16:25.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.356+0000 7f9e75400640 1 -- 172.21.15.137:0/2098224294 wait complete. 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 Processor -- start 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 -- start start 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e700fff90 0x7f9e70199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e701008f0 0x7f9e70199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e701a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7010f3d0 con 0x7f9e70101290 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7010f250 con 0x7f9e701008f0 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.358+0000 7f9e75400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7010f550 con 0x7f9e700fff90 2024-03-26T14:16:25.360 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e701a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e701a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51104/0 (socket says 172.21.15.137:51104) 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 -- 172.21.15.137:0/2419873947 learned_addr learned my addr 172.21.15.137:0/2419873947 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6de00640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e701008f0 0x7f9e70199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6e800640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e700fff90 0x7f9e70199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 -- 172.21.15.137:0/2419873947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e700fff90 msgr2=0x7f9e70199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e700fff90 0x7f9e70199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 -- 172.21.15.137:0/2419873947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e701008f0 msgr2=0x7f9e70199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e701008f0 0x7f9e70199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6f200640 1 -- 172.21.15.137:0/2419873947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e701a16c0 con 0x7f9e70101290 2024-03-26T14:16:25.361 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.359+0000 7f9e6de00640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e701008f0 0x7f9e70199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.360+0000 7f9e6f200640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e701a1050 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9e68037040 tx=0x7f9e68003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.360+0000 7f9e67400640 1 -- 172.21.15.137:0/2419873947 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68039040 con 0x7f9e70101290 2024-03-26T14:16:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.360+0000 7f9e67400640 1 -- 172.21.15.137:0/2419873947 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e68044070 con 0x7f9e70101290 2024-03-26T14:16:25.362 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.360+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e701a1950 con 0x7f9e70101290 2024-03-26T14:16:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.360+0000 7f9e67400640 1 -- 172.21.15.137:0/2419873947 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68004280 con 0x7f9e70101290 2024-03-26T14:16:25.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.360+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e701a1e40 con 0x7f9e70101290 2024-03-26T14:16:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.364+0000 7f9e7801f740 1 Processor -- start 2024-03-26T14:16:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.364+0000 7f9e67400640 1 -- 172.21.15.137:0/2419873947 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9e680328f0 con 0x7f9e70101290 2024-03-26T14:16:25.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.364+0000 7f9e7801f740 1 -- start start 2024-03-26T14:16:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c71c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c72e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c79c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e7801f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba16bd62c0 con 0x55ba16c733a0 2024-03-26T14:16:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e7801f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba16b7ec90 con 0x55ba16c729e0 2024-03-26T14:16:25.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e7801f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba16bd2360 con 0x55ba16c71810 2024-03-26T14:16:25.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c79c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c79c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51114/0 (socket says 172.21.15.137:51114) 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e6de00640 1 -- 172.21.15.137:0/1176213104 learned_addr learned my addr 172.21.15.137:0/1176213104 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c72e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c71c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.365+0000 7f9e67400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e540a2800 0x7f9e540a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6de00640 1 -- 172.21.15.137:0/1176213104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 msgr2=0x55ba16c71c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c71c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e67400640 1 -- 172.21.15.137:0/2419873947 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9e680f1530 con 0x7f9e70101290 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6de00640 1 -- 172.21.15.137:0/1176213104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 msgr2=0x55ba16c72e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c72e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6de00640 1 -- 172.21.15.137:0/1176213104 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba16ba3080 con 0x55ba16c733a0 2024-03-26T14:16:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6f200640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e540a2800 0x7f9e540a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c79c30 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c012770 tx=0x7f9e5c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ccbb2352f3b54c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.366+0000 7f9e5b400640 1 -- 172.21.15.137:0/1176213104 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c012cf0 con 0x55ba16c733a0 2024-03-26T14:16:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.367+0000 7f9e6f200640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e540a2800 0x7f9e540a4cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9e680bda90 tx=0x7f9e680095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.367+0000 7f9e5b400640 1 -- 172.21.15.137:0/1176213104 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e5c010c80 con 0x55ba16c733a0 2024-03-26T14:16:25.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.367+0000 7f9e5b400640 1 -- 172.21.15.137:0/1176213104 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c01eab0 con 0x55ba16c733a0 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.367+0000 7f9e7801f740 1 -- 172.21.15.137:0/1176213104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 msgr2=0x55ba16c79c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.367+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c79c30 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c012770 tx=0x7f9e5c0096a0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 -- 172.21.15.137:0/1176213104 shutdown_connections 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c72e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c79c30 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1176213104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c71c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 -- 172.21.15.137:0/1176213104 >> 172.21.15.137:0/1176213104 conn(0x55ba16c6cc00 msgr2=0x55ba16c6d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 -- 172.21.15.137:0/1176213104 shutdown_connections 2024-03-26T14:16:25.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 -- 172.21.15.137:0/1176213104 wait complete. 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 -- start start 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.368+0000 7f9e7801f740 1 Processor -- start 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c7ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c7bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba16c7a3b0 con 0x55ba16c733a0 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba16b920d0 con 0x55ba16c729e0 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba16c7a530 con 0x55ba16c71810 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e7801f740 10 client.0 _ll_register_callbacks cb 0x55ba16b96e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c83000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51124/0 (socket says 172.21.15.137:51124) 2024-03-26T14:16:25.374 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e6de00640 1 -- 172.21.15.137:0/180088608 learned_addr learned my addr 172.21.15.137:0/180088608 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.369+0000 7f9e6e800640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c7bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c7ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 -- 172.21.15.137:0/180088608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 msgr2=0x55ba16c7ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c7ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 -- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 msgr2=0x55ba16c7bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c7bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 -- 172.21.15.137:0/180088608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba168ea3f0 con 0x55ba16c733a0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e6de00640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c83000 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001eb0 tx=0x7f9e5c01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e0ed43c46eb5a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.370+0000 7f9e4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e4fe00640 1 -- 172.21.15.137:0/180088608 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0109e0 con 0x55ba16c733a0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e4fe00640 1 -- 172.21.15.137:0/180088608 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9e5c01e8b0 con 0x55ba16c733a0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e5c010c80 con 0x55ba16c733a0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e4fe00640 1 -- 172.21.15.137:0/180088608 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5c01eab0 con 0x55ba16c733a0 2024-03-26T14:16:25.375 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e7801f740 10 client.18567 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ba169b47d0 con 0x55ba16c733a0 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e4fe00640 1 -- 172.21.15.137:0/180088608 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e5c010c80 con 0x55ba16c733a0 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.371+0000 7f9e4fe00640 1 -- 172.21.15.137:0/180088608 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9e5c02a040 con 0x55ba16c733a0 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e7801f740 10 client.18567 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e7801f740 10 client.18567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e7801f740 20 client.18567 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e7801f740 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16c87160 0x55ba16c89550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e7801f740 4 client.18567 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba16be3800 con 0x55ba16c87160 2024-03-26T14:16:25.376 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.372+0000 7f9e6e800640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16c87160 0x55ba16c89550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.373+0000 7f9e4fe00640 10 client.18567 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:25.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.373+0000 7f9e6e800640 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16c87160 0x55ba16c89550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=13703e3a8e4a4200 server_cookie=66ef2a9da3f29bdb in_seq=0 out_seq=0 2024-03-26T14:16:25.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e4fe00640 1 -- 172.21.15.137:0/180088608 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ba16be3800 con 0x55ba16c87160 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e4fe00640 10 client.18567 handle_command_reply: tid=0 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 client.18567 shutdown 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 msgr2=0x55ba16c83000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c83000 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c001eb0 tx=0x7f9e5c01b040 comp rx=0 tx=0).stop 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 shutdown_connections 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16c87160 0x55ba16c89550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c729e0 0x55ba16c7bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c733a0 0x55ba16c83000 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 --2- 172.21.15.137:0/180088608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c71810 0x55ba16c7ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e4fe00640 0 client.18567 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.374+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 >> 172.21.15.137:0/180088608 conn(0x55ba16c6cc00 msgr2=0x55ba16c6beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.375+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 shutdown_connections 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.375+0000 7f9e7801f740 1 -- 172.21.15.137:0/180088608 wait complete. 2024-03-26T14:16:25.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.375+0000 7f9e7801f740 20 client.18567 trim_cache size 0 max 16384 2024-03-26T14:16:25.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.388+0000 7f9e7801f740 1 Processor -- start 2024-03-26T14:16:25.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.388+0000 7f9e7801f740 1 -- start start 2024-03-26T14:16:25.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.388+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c71810 0x55ba16c8de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 0x55ba16c82f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c733a0 0x55ba16c8e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e7801f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701ae890 con 0x55ba16c729e0 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e7801f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701ac390 con 0x55ba16c71810 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e7801f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701aa410 con 0x55ba16c733a0 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 0x55ba16c82f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 0x55ba16c82f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51138/0 (socket says 172.21.15.137:51138) 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 -- 172.21.15.137:0/1966858190 learned_addr learned my addr 172.21.15.137:0/1966858190 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c71810 0x55ba16c8de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 -- 172.21.15.137:0/1966858190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c733a0 msgr2=0x55ba16c8e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c733a0 0x55ba16c8e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 -- 172.21.15.137:0/1966858190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c71810 msgr2=0x55ba16c8de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c71810 0x55ba16c8de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.389+0000 7f9e6e800640 1 -- 172.21.15.137:0/1966858190 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba16c89bb0 con 0x55ba16c729e0 2024-03-26T14:16:25.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.390+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 0x55ba16c82f90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x55ba16c6db70 tx=0x7f9e600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58703ad2fe8de54 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.390+0000 7f9e5aa00640 1 -- 172.21.15.137:0/1966858190 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e600040f0 con 0x55ba16c729e0 2024-03-26T14:16:25.395 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.390+0000 7f9e5aa00640 1 -- 172.21.15.137:0/1966858190 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e600054d0 con 0x55ba16c729e0 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.390+0000 7f9e5aa00640 1 -- 172.21.15.137:0/1966858190 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60004de0 con 0x55ba16c729e0 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 -- 172.21.15.137:0/1966858190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 msgr2=0x55ba16c82f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 0x55ba16c82f90 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x55ba16c6db70 tx=0x7f9e600118d0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 -- 172.21.15.137:0/1966858190 shutdown_connections 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c733a0 0x55ba16c8e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c729e0 0x55ba16c82f90 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1966858190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c71810 0x55ba16c8de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 -- 172.21.15.137:0/1966858190 >> 172.21.15.137:0/1966858190 conn(0x55ba16c6cc00 msgr2=0x55ba16c706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 -- 172.21.15.137:0/1966858190 shutdown_connections 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 -- 172.21.15.137:0/1966858190 wait complete. 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.391+0000 7f9e7801f740 1 -- start start 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 Processor -- start 2024-03-26T14:16:25.396 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 0x55ba16c80cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c729e0 0x55ba16c81200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c733a0 0x55ba16c82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701aaa10 con 0x55ba16c71810 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701abd30 con 0x55ba16c733a0 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701ab070 con 0x55ba16c729e0 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e7801f740 10 client.0 _ll_register_callbacks cb 0x7f9e6000d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 0x55ba16c80cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.392+0000 7f9e6de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 0x55ba16c80cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51150/0 (socket says 172.21.15.137:51150) 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 -- 172.21.15.137:0/1837557169 learned_addr learned my addr 172.21.15.137:0/1837557169 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c729e0 0x55ba16c81200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 -- 172.21.15.137:0/1837557169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c729e0 msgr2=0x55ba16c81200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c729e0 0x55ba16c81200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 -- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c733a0 msgr2=0x55ba16c82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c733a0 0x55ba16c82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 -- 172.21.15.137:0/1837557169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba16c82800 con 0x55ba16c71810 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c729e0 0x55ba16c81200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e6de00640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 0x55ba16c80cc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x55ba16c682c0 tx=0x7f9e5c01fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134adb8461f8973e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.393+0000 7f9e4fe00640 1 -- 172.21.15.137:0/1837557169 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c009bc0 con 0x55ba16c71810 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e600054d0 con 0x55ba16c71810 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e4fe00640 1 -- 172.21.15.137:0/1837557169 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f9e5c009d60 con 0x55ba16c71810 2024-03-26T14:16:25.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e4fe00640 1 -- 172.21.15.137:0/1837557169 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e5c045050 con 0x55ba16c71810 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e7801f740 10 client.18573 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ba16ce3070 con 0x55ba16c71810 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e4fe00640 1 -- 172.21.15.137:0/1837557169 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e600054d0 con 0x55ba16c71810 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e4fe00640 1 -- 172.21.15.137:0/1837557169 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f9e5c005040 con 0x55ba16c71810 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e7801f740 10 client.18573 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e7801f740 10 client.18573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.394+0000 7f9e7801f740 20 client.18573 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.395+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16ce3990 0x55ba16ce5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.395+0000 7f9e7801f740 4 client.18573 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:16:25.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.395+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ba16bddb50 con 0x55ba16ce3990 2024-03-26T14:16:25.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.395+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16ce3990 0x55ba16ce5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.396+0000 7f9e4fe00640 10 client.18573 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.396+0000 7f9e6e800640 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16ce3990 0x55ba16ce5db0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df5844243a65d5a3 server_cookie=3785b685f90e10fc in_seq=0 out_seq=0 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.396+0000 7f9e4fe00640 1 -- 172.21.15.137:0/1837557169 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ba16bddb50 con 0x55ba16ce3990 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.396+0000 7f9e4fe00640 10 client.18573 handle_command_reply: tid=0 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 client.18573 shutdown 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 msgr2=0x55ba16c80cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 0x55ba16c80cc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x55ba16c682c0 tx=0x7f9e5c01fce0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 shutdown_connections 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55ba16ce3990 0x55ba16ce5db0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ba16c733a0 0x55ba16c82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba16c729e0 0x55ba16c81200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 --2- 172.21.15.137:0/1837557169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ba16c71810 0x55ba16c80cc0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 >> 172.21.15.137:0/1837557169 conn(0x55ba16c6cc00 msgr2=0x55ba16c75c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e4fe00640 0 client.18573 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:25.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 shutdown_connections 2024-03-26T14:16:25.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 1 -- 172.21.15.137:0/1837557169 wait complete. 2024-03-26T14:16:25.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.397+0000 7f9e7801f740 20 client.18573 trim_cache size 0 max 16384 2024-03-26T14:16:25.402 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e540a2800 msgr2=0x7f9e540a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e540a2800 0x7f9e540a4cc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9e680bda90 tx=0x7f9e680095c0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 msgr2=0x7f9e701a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e701a1050 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f9e68037040 tx=0x7f9e68003c20 comp rx=0 tx=0).stop 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 shutdown_connections 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f9e540a2800 0x7f9e540a4cc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e701008f0 0x7f9e70199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9e70101290 0x7f9e701a1050 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 --2- 172.21.15.137:0/2419873947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e700fff90 0x7f9e70199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 >> 172.21.15.137:0/2419873947 conn(0x7f9e700fbb30 msgr2=0x7f9e700fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 shutdown_connections 2024-03-26T14:16:25.403 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.398+0000 7f9e75400640 1 -- 172.21.15.137:0/2419873947 wait complete. 2024-03-26T14:16:25.417 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:16:25.417 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 -- 172.21.15.137:0/2655246296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 msgr2=0x7f8824101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 --2- 172.21.15.137:0/2655246296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824101c30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8810009900 tx=0x7f881002bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 -- 172.21.15.137:0/2655246296 shutdown_connections 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 --2- 172.21.15.137:0/2655246296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824101c30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 --2- 172.21.15.137:0/2655246296 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8824102170 0x7f8824108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 --2- 172.21.15.137:0/2655246296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88241005c0 0x7f88241009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 -- 172.21.15.137:0/2655246296 >> 172.21.15.137:0/2655246296 conn(0x7f88240fbb50 msgr2=0x7f88240fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 -- 172.21.15.137:0/2655246296 shutdown_connections 2024-03-26T14:16:25.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.635+0000 7f8829600640 1 -- 172.21.15.137:0/2655246296 wait complete. 2024-03-26T14:16:25.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 Processor -- start 2024-03-26T14:16:25.637 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 -- start start 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 0x7f8824104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8824102170 0x7f88241030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882410c150 con 0x7f88241005c0 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882410bfd0 con 0x7f88241017b0 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.637+0000 7f8829600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882410c2d0 con 0x7f8824102170 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 0x7f8824104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 0x7f8824104550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51164/0 (socket says 172.21.15.137:51164) 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 -- 172.21.15.137:0/1669252769 learned_addr learned my addr 172.21.15.137:0/1669252769 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822000640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8823400640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8824102170 0x7f88241030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 -- 172.21.15.137:0/1669252769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8824102170 msgr2=0x7f88241030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.640 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8824102170 0x7f88241030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 -- 172.21.15.137:0/1669252769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 msgr2=0x7f8824102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 -- 172.21.15.137:0/1669252769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824103960 con 0x7f88241005c0 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8823400640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8824102170 0x7f88241030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822000640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.638+0000 7f8822a00640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 0x7f8824104550 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f881400eaa0 tx=0x7f881400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.639+0000 7f881b400640 1 -- 172.21.15.137:0/1669252769 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814016070 con 0x7f88241005c0 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.639+0000 7f881b400640 1 -- 172.21.15.137:0/1669252769 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8814003e20 con 0x7f88241005c0 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.639+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88241a1af0 con 0x7f88241005c0 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.639+0000 7f881b400640 1 -- 172.21.15.137:0/1669252769 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88140116d0 con 0x7f88241005c0 2024-03-26T14:16:25.641 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.639+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88241a2010 con 0x7f88241005c0 2024-03-26T14:16:25.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.642+0000 7f882c380740 1 Processor -- start 2024-03-26T14:16:25.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.642+0000 7f881b400640 1 -- 172.21.15.137:0/1669252769 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8814019070 con 0x7f88241005c0 2024-03-26T14:16:25.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.642+0000 7f882c380740 1 -- start start 2024-03-26T14:16:25.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f882c380740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4ee9fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f882c380740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 0x55a4ee9fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f882c380740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea03c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f882c380740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ee9602c0 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f882c380740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ee908c90 con 0x55a4ee9fb810 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f882c380740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ee95c360 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f8822a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 0x55a4ee9fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f8822a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 0x55a4ee9fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51168/0 (socket says 172.21.15.137:51168) 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f8822a00640 1 -- 172.21.15.137:0/281139536 learned_addr learned my addr 172.21.15.137:0/281139536 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f8823400640 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4ee9fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.643+0000 7f8822000640 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea03c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8822a00640 1 -- 172.21.15.137:0/281139536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fd3a0 msgr2=0x55a4eea03c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8822a00640 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea03c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8822a00640 1 -- 172.21.15.137:0/281139536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 msgr2=0x55a4ee9fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8822a00640 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4ee9fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8822a00640 1 -- 172.21.15.137:0/281139536 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4ee92d080 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8823400640 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4ee9fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f8822a00640 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 0x55a4ee9fce60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f88140d7160 tx=0x7f88140d7620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31abd669ca64eee4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f881b400640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f88080a2880 0x7f88080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.644+0000 7f880f400640 1 -- 172.21.15.137:0/281139536 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88140e2070 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.645+0000 7f880f400640 1 -- 172.21.15.137:0/281139536 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88140d7e50 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.645+0000 7f880f400640 1 -- 172.21.15.137:0/281139536 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88140047f0 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.645+0000 7f881b400640 1 -- 172.21.15.137:0/1669252769 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f88140cefe0 con 0x7f88241005c0 2024-03-26T14:16:25.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.645+0000 7f8823400640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f88080a2880 0x7f88080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 -- 172.21.15.137:0/281139536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 msgr2=0x55a4ee9fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 0x55a4ee9fce60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f88140d7160 tx=0x7f88140d7620 comp rx=0 tx=0).stop 2024-03-26T14:16:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f8823400640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f88080a2880 0x7f88080a4d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55a4ee9fc8e0 tx=0x7f881c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 -- 172.21.15.137:0/281139536 shutdown_connections 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea03c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fc9e0 0x55a4ee9fce60 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 --2- 172.21.15.137:0/281139536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4ee9fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 -- 172.21.15.137:0/281139536 >> 172.21.15.137:0/281139536 conn(0x55a4ee9f6c00 msgr2=0x55a4ee9f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 -- 172.21.15.137:0/281139536 shutdown_connections 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 -- 172.21.15.137:0/281139536 wait complete. 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.646+0000 7f882c380740 1 -- start start 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 Processor -- start 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4eea05a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea05f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eea043b0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4ee91c0d0 con 0x55a4ee9fb810 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4eea04530 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.647+0000 7f882c380740 10 client.0 _ll_register_callbacks cb 0x55a4ee920e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8822a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4eea05a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51182/0 (socket says 172.21.15.137:51182) 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8822a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4eea05a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:50116/0 (socket says 172.21.15.137:50116) 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 -- 172.21.15.137:0/1891061752 learned_addr learned my addr 172.21.15.137:0/1891061752 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8822000640 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea05f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 -- 172.21.15.137:0/1891061752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fc9e0 msgr2=0x55a4eea05f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea05f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 -- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 msgr2=0x55a4eea05a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4eea05a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.648+0000 7f8823400640 1 -- 172.21.15.137:0/1891061752 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4ee6743f0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f8823400640 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0cfd0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f881c01db60 tx=0x7f881c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2542fdc28d388b3e server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f8803e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f8803e00640 1 -- 172.21.15.137:0/1891061752 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c02a060 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88140d7e50 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f8803e00640 1 -- 172.21.15.137:0/1891061752 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f881c010d10 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f8803e00640 1 -- 172.21.15.137:0/1891061752 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f881c031050 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.649+0000 7f882c380740 10 client.18585 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a4ee73e7d0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f8803e00640 1 -- 172.21.15.137:0/1891061752 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88140d7e50 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f8803e00640 1 -- 172.21.15.137:0/1891061752 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f881c019040 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 10 client.18585 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 10 client.18585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:16:25.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 20 client.18585 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea11500 0x55a4eea13980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 4 client.18585 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.650+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4ee96d800 con 0x55a4eea11500 2024-03-26T14:16:25.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.651+0000 7f8822a00640 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea11500 0x55a4eea13980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.652+0000 7f8803e00640 10 client.18585 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.652+0000 7f8822a00640 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea11500 0x55a4eea13980 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a838e16ebbcf6f2f server_cookie=ca4a7755149a5d8f in_seq=0 out_seq=0 2024-03-26T14:16:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f8803e00640 1 -- 172.21.15.137:0/1891061752 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a4ee96d800 con 0x55a4eea11500 2024-03-26T14:16:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f8803e00640 10 client.18585 handle_command_reply: tid=0 2024-03-26T14:16:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 client.18585 shutdown 2024-03-26T14:16:25.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 msgr2=0x55a4eea0cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0cfd0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f881c01db60 tx=0x7f881c0046c0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 shutdown_connections 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea11500 0x55a4eea13980 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0cfd0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea05f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 --2- 172.21.15.137:0/1891061752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fb810 0x55a4eea05a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 >> 172.21.15.137:0/1891061752 conn(0x55a4ee9f6c00 msgr2=0x55a4ee9f5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f8803e00640 0 client.18585 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 shutdown_connections 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.653+0000 7f882c380740 1 -- 172.21.15.137:0/1891061752 wait complete. 2024-03-26T14:16:25.657 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.654+0000 7f882c380740 20 client.18585 trim_cache size 0 max 16384 2024-03-26T14:16:25.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 Processor -- start 2024-03-26T14:16:25.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 -- start start 2024-03-26T14:16:25.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4ee9f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea0cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4ee9f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88241aea70 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88241ac570 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.670+0000 7f882c380740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88241aa5f0 con 0x55a4ee9fb810 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4ee9f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4ee9f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51196/0 (socket says 172.21.15.137:51196) 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 -- 172.21.15.137:0/853298585 learned_addr learned my addr 172.21.15.137:0/853298585 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8822000640 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea0cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8822a00640 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4ee9f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 -- 172.21.15.137:0/853298585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 msgr2=0x55a4ee9f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4ee9f5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 -- 172.21.15.137:0/853298585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 msgr2=0x55a4eea0cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea0cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.671+0000 7f8823400640 1 -- 172.21.15.137:0/853298585 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4eea034f0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.672+0000 7f8823400640 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4ee9f5b60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f881c000c00 tx=0x7f881c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852c620eb50417d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.672+0000 7f880ea00640 1 -- 172.21.15.137:0/853298585 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c03a070 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.672+0000 7f880ea00640 1 -- 172.21.15.137:0/853298585 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f881c0054f0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.672+0000 7f880ea00640 1 -- 172.21.15.137:0/853298585 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c004de0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 -- 172.21.15.137:0/853298585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 msgr2=0x55a4ee9f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4ee9f5b60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f881c000c00 tx=0x7f881c005c50 comp rx=0 tx=0).stop 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 -- 172.21.15.137:0/853298585 shutdown_connections 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea0cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4ee9f5b60 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 --2- 172.21.15.137:0/853298585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4ee9f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 -- 172.21.15.137:0/853298585 >> 172.21.15.137:0/853298585 conn(0x55a4ee9f6c00 msgr2=0x55a4eea05a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 -- 172.21.15.137:0/853298585 shutdown_connections 2024-03-26T14:16:25.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 -- 172.21.15.137:0/853298585 wait complete. 2024-03-26T14:16:25.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:25 smithi179 ceph-mon[35472]: pgmap v879: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:25.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:25 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/256062881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 -- start start 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.673+0000 7f882c380740 1 Processor -- start 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4eea08920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea08e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88241aabf0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88241abf10 con 0x55a4ee9fc9e0 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88241ab250 con 0x55a4ee9fb810 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f882c380740 10 client.0 _ll_register_callbacks cb 0x7f881c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f8823400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.680 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f8823400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51212/0 (socket says 172.21.15.137:51212) 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f8823400640 1 -- 172.21.15.137:0/1436701862 learned_addr learned my addr 172.21.15.137:0/1436701862 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.674+0000 7f8822a00640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4eea08920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8822000640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea08e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8823400640 1 -- 172.21.15.137:0/1436701862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 msgr2=0x55a4eea08920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8823400640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4eea08920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8823400640 1 -- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 msgr2=0x55a4eea08e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8823400640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea08e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8823400640 1 -- 172.21.15.137:0/1436701862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4eea0c470 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8822000640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea08e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8823400640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0be80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f881c038040 tx=0x7f881c005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c4c930d1da0b73 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8803e00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8803e00640 1 -- 172.21.15.137:0/1436701862 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c044020 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f8803e00640 1 -- 172.21.15.137:0/1436701862 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f881c004b60 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.675+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f881c0054f0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.681 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f8803e00640 1 -- 172.21.15.137:0/1436701862 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f881c04c050 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 10 client.18591 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a4eea6d020 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f8803e00640 1 -- 172.21.15.137:0/1436701862 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f881c0054f0 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f8803e00640 1 -- 172.21.15.137:0/1436701862 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f881c03d030 con 0x55a4ee9fd3a0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 10 client.18591 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 10 client.18591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 20 client.18591 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea6d8e0 0x55a4eea6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 4 client.18591 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.676+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a4ee967b50 con 0x55a4eea6d8e0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.677+0000 7f8822a00640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea6d8e0 0x55a4eea6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f8803e00640 10 client.18591 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f8822a00640 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea6d8e0 0x55a4eea6fd00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c09564569d8a9a89 server_cookie=6b7fc46cf4744505 in_seq=0 out_seq=0 2024-03-26T14:16:25.682 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f8803e00640 1 -- 172.21.15.137:0/1436701862 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a4ee967b50 con 0x55a4eea6d8e0 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f8803e00640 10 client.18591 handle_command_reply: tid=0 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 client.18591 shutdown 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 msgr2=0x55a4eea0be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0be80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f881c038040 tx=0x7f881c005d40 comp rx=0 tx=0).stop 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 shutdown_connections 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55a4eea6d8e0 0x55a4eea6fd00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a4ee9fc9e0 0x55a4eea08e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a4ee9fd3a0 0x55a4eea0be80 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 --2- 172.21.15.137:0/1436701862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a4ee9fb810 0x55a4eea08920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f8803e00640 0 client.18591 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.678+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 >> 172.21.15.137:0/1436701862 conn(0x55a4ee9f6c00 msgr2=0x55a4ee9f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 shutdown_connections 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f882c380740 1 -- 172.21.15.137:0/1436701862 wait complete. 2024-03-26T14:16:25.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f882c380740 20 client.18591 trim_cache size 0 max 16384 2024-03-26T14:16:25.684 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:16:25.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f88080a2880 msgr2=0x7f88080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f8829600640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f88080a2880 0x7f88080a4d40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55a4ee9fc8e0 tx=0x7f881c005f90 comp rx=0 tx=0).stop 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 msgr2=0x7f8824104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.679+0000 7f8829600640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 0x7f8824104550 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f881400eaa0 tx=0x7f881400ef60 comp rx=0 tx=0).stop 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 shutdown_connections 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f88080a2880 0x7f88080a4d40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88241017b0 0x7f8824102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8824102170 0x7f88241030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 --2- 172.21.15.137:0/1669252769 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88241005c0 0x7f8824104550 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 >> 172.21.15.137:0/1669252769 conn(0x7f88240fbb50 msgr2=0x7f88240fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 shutdown_connections 2024-03-26T14:16:25.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.680+0000 7f8829600640 1 -- 172.21.15.137:0/1669252769 wait complete. 2024-03-26T14:16:25.697 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:25 smithi137 ceph-mon[31539]: pgmap v879: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:25.697 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:25 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/256062881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:16:25.699 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:16:25.700 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:16:25.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:25 smithi159 ceph-mon[35501]: pgmap v879: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:25.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:25 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/256062881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:16:25.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 -- 172.21.15.137:0/1376222399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10104a60 msgr2=0x7f2c10106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 --2- 172.21.15.137:0/1376222399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10104a60 0x7f2c10106e50 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009900 tx=0x7f2bfc02bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:25.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 -- 172.21.15.137:0/1376222399 shutdown_connections 2024-03-26T14:16:25.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 --2- 172.21.15.137:0/1376222399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c10107390 0x7f2c101097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 --2- 172.21.15.137:0/1376222399 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10104a60 0x7f2c10106e50 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 --2- 172.21.15.137:0/1376222399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c10102130 0x7f2c10104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 -- 172.21.15.137:0/1376222399 >> 172.21.15.137:0/1376222399 conn(0x7f2c100fbb40 msgr2=0x7f2c100fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 -- 172.21.15.137:0/1376222399 shutdown_connections 2024-03-26T14:16:25.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.890+0000 7f2c17200640 1 -- 172.21.15.137:0/1376222399 wait complete. 2024-03-26T14:16:25.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.891+0000 7f2c17200640 1 Processor -- start 2024-03-26T14:16:25.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.891+0000 7f2c17200640 1 -- start start 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c17200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 0x7f2c10199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c17200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c10104a60 0x7f2c1019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c17200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c10107390 0x7f2c101a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c17200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010d230 con 0x7f2c10102130 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c17200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010d0b0 con 0x7f2c10107390 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c17200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1010d3b0 con 0x7f2c10104a60 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 0x7f2c10199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 0x7f2c10199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51224/0 (socket says 172.21.15.137:51224) 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c0fe00640 1 -- 172.21.15.137:0/4025107508 learned_addr learned my addr 172.21.15.137:0/4025107508 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c0f400640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c10104a60 0x7f2c1019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.892+0000 7f2c15200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c10107390 0x7f2c101a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0fe00640 1 -- 172.21.15.137:0/4025107508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c10104a60 msgr2=0x7f2c1019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c10104a60 0x7f2c1019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0fe00640 1 -- 172.21.15.137:0/4025107508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c10107390 msgr2=0x7f2c101a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c10107390 0x7f2c101a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0fe00640 1 -- 172.21.15.137:0/4025107508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c101a1750 con 0x7f2c10102130 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 0x7f2c10199b30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000ebc0 tx=0x7f2c0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0cc00640 1 -- 172.21.15.137:0/4025107508 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00010630 con 0x7f2c10102130 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0cc00640 1 -- 172.21.15.137:0/4025107508 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c00010c80 con 0x7f2c10102130 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c101a1a40 con 0x7f2c10102130 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.893+0000 7f2c0cc00640 1 -- 172.21.15.137:0/4025107508 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00018b30 con 0x7f2c10102130 2024-03-26T14:16:25.896 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.894+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c101a1f60 con 0x7f2c10102130 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.897+0000 7f2c19fe1740 1 Processor -- start 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.897+0000 7f2c0cc00640 1 -- 172.21.15.137:0/4025107508 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2c00018cd0 con 0x7f2c10102130 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 -- start start 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f6ff9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f6ffae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffb3a0 0x5651f7001c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f6f5e2c0 con 0x5651f6ffa9e0 2024-03-26T14:16:25.899 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f6f06c90 con 0x5651f6ff9810 2024-03-26T14:16:25.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f6f5a360 con 0x5651f6ffb3a0 2024-03-26T14:16:25.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f6ffae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f6ffae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51226/0 (socket says 172.21.15.137:51226) 2024-03-26T14:16:25.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c0fe00640 1 -- 172.21.15.137:0/451599754 learned_addr learned my addr 172.21.15.137:0/451599754 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.900 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.898+0000 7f2c15200640 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f6ff9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0f400640 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffb3a0 0x5651f7001c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0fe00640 1 -- 172.21.15.137:0/451599754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffb3a0 msgr2=0x5651f7001c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffb3a0 0x5651f7001c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0fe00640 1 -- 172.21.15.137:0/451599754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 msgr2=0x5651f6ff9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f6ff9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.901 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0fe00640 1 -- 172.21.15.137:0/451599754 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651f6f2b080 con 0x5651f6ffa9e0 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f6ffae60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2c000d5660 tx=0x7f2c000d5b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7d9f3310c7cca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.899+0000 7f2c0cc00640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2bf40a2800 0x7f2bf40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.900+0000 7f2c0cc00640 1 -- 172.21.15.137:0/4025107508 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c000cf560 con 0x7f2c10102130 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.900+0000 7f2c15200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2bf40a2800 0x7f2bf40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.900+0000 7f2c05600640 1 -- 172.21.15.137:0/451599754 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c000e2070 con 0x5651f6ffa9e0 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.900+0000 7f2c05600640 1 -- 172.21.15.137:0/451599754 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c0009c1f0 con 0x5651f6ffa9e0 2024-03-26T14:16:25.902 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.900+0000 7f2c05600640 1 -- 172.21.15.137:0/451599754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00097d90 con 0x5651f6ffa9e0 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c15200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2bf40a2800 0x7f2bf40a4cc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5651f6ffa8e0 tx=0x7f2c0800b450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 -- 172.21.15.137:0/451599754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 msgr2=0x5651f6ffae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f6ffae60 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f2c000d5660 tx=0x7f2c000d5b20 comp rx=0 tx=0).stop 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 -- 172.21.15.137:0/451599754 shutdown_connections 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffb3a0 0x5651f7001c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f6ffae60 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/451599754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f6ff9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.906 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 -- 172.21.15.137:0/451599754 >> 172.21.15.137:0/451599754 conn(0x5651f6ff4c00 msgr2=0x5651f6ff5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 -- 172.21.15.137:0/451599754 shutdown_connections 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 -- 172.21.15.137:0/451599754 wait complete. 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.901+0000 7f2c19fe1740 1 -- start start 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 Processor -- start 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f7003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 0x5651f700afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f70023b0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f6f1a0d0 con 0x5651f6ff9810 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f7002530 con 0x5651f6ffa9e0 2024-03-26T14:16:25.907 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c19fe1740 10 client.0 _ll_register_callbacks cb 0x5651f6f1ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c15200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 0x5651f700afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.902+0000 7f2c15200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 0x5651f700afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51234/0 (socket says 172.21.15.137:51234) 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 -- 172.21.15.137:0/1865653100 learned_addr learned my addr 172.21.15.137:0/1865653100 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f7003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c0f400640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 -- 172.21.15.137:0/1865653100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 msgr2=0x5651f7003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 -- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 msgr2=0x5651f7003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f7003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 -- 172.21.15.137:0/1865653100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651f6c723f0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2c15200640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 0x5651f700afb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2c08013fd0 tx=0x7f2c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8f1ebb2a1ca7d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.903+0000 7f2bfa000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:25.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2bfa000640 1 -- 172.21.15.137:0/1865653100 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08029680 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2bfa000640 1 -- 172.21.15.137:0/1865653100 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c08029820 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c0009c1f0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2bfa000640 1 -- 172.21.15.137:0/1865653100 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0801de40 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2c19fe1740 10 client.18603 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5651f6d3c7d0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.904+0000 7f2bfa000640 1 -- 172.21.15.137:0/1865653100 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2c0009c1f0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2bfa000640 1 -- 172.21.15.137:0/1865653100 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2c0801eba0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c19fe1740 10 client.18603 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c19fe1740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:25.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c19fe1740 20 client.18603 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f700f500 0x5651f7011980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c19fe1740 4 client.18603 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651f6f6b800 con 0x5651f700f500 2024-03-26T14:16:25.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.905+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f700f500 0x5651f7011980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.906+0000 7f2bfa000640 10 client.18603 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.906+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f700f500 0x5651f7011980 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203c69f63bb02779 server_cookie=97c4e62920b3ea12 in_seq=0 out_seq=0 2024-03-26T14:16:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.907+0000 7f2bfa000640 1 -- 172.21.15.137:0/1865653100 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5651f6f6b800 con 0x5651f700f500 2024-03-26T14:16:25.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.907+0000 7f2bfa000640 10 client.18603 handle_command_reply: tid=0 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 client.18603 shutdown 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 msgr2=0x5651f700afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 0x5651f700afb0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f2c08013fd0 tx=0x7f2c080046c0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 shutdown_connections 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f700f500 0x5651f7011980 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffb3a0 0x5651f700afb0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/1865653100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ff9810 0x5651f7003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2bfa000640 0 client.18603 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 >> 172.21.15.137:0/1865653100 conn(0x5651f6ff4c00 msgr2=0x5651f6ff3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 shutdown_connections 2024-03-26T14:16:25.912 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 1 -- 172.21.15.137:0/1865653100 wait complete. 2024-03-26T14:16:25.913 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.908+0000 7f2c19fe1740 20 client.18603 trim_cache size 0 max 16384 2024-03-26T14:16:25.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 Processor -- start 2024-03-26T14:16:25.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 -- start start 2024-03-26T14:16:25.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ff9810 0x5651f6ff3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f700af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f7016240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.925 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101ae9f0 con 0x5651f6ffa9e0 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101ac4f0 con 0x5651f6ffb3a0 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.923+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101aa570 con 0x5651f6ff9810 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f700af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f700af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51246/0 (socket says 172.21.15.137:51246) 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 -- 172.21.15.137:0/993955472 learned_addr learned my addr 172.21.15.137:0/993955472 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c15200640 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f7016240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ff9810 0x5651f6ff3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 -- 172.21.15.137:0/993955472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ff9810 msgr2=0x5651f6ff3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ff9810 0x5651f6ff3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 -- 172.21.15.137:0/993955472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 msgr2=0x5651f7016240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f7016240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c0f400640 1 -- 172.21.15.137:0/993955472 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651f7002b00 con 0x5651f6ffa9e0 2024-03-26T14:16:25.926 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.924+0000 7f2c15200640 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f7016240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:25.927 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.925+0000 7f2c0f400640 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f700af70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2c1006a7d0 tx=0x7f2bfc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aecdffaff168cd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.925+0000 7f2c04c00640 1 -- 172.21.15.137:0/993955472 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc02cc20 con 0x5651f6ffa9e0 2024-03-26T14:16:25.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.925+0000 7f2c04c00640 1 -- 172.21.15.137:0/993955472 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc005050 con 0x5651f6ffa9e0 2024-03-26T14:16:25.929 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.925+0000 7f2c04c00640 1 -- 172.21.15.137:0/993955472 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0048e0 con 0x5651f6ffa9e0 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 -- 172.21.15.137:0/993955472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 msgr2=0x5651f700af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f700af70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2c1006a7d0 tx=0x7f2bfc005e00 comp rx=0 tx=0).stop 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 -- 172.21.15.137:0/993955472 shutdown_connections 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f7016240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ffa9e0 0x5651f700af70 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/993955472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ff9810 0x5651f6ff3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 -- 172.21.15.137:0/993955472 >> 172.21.15.137:0/993955472 conn(0x5651f6ff4c00 msgr2=0x5651f6ff3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 -- 172.21.15.137:0/993955472 shutdown_connections 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.926+0000 7f2c19fe1740 1 -- 172.21.15.137:0/993955472 wait complete. 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 -- start start 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 Processor -- start 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 0x5651f7007af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.930 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7008030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f706ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101aab70 con 0x5651f6ff9810 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101abe90 con 0x5651f6ffb3a0 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c101ab1d0 con 0x5651f6ffa9e0 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.927+0000 7f2c19fe1740 10 client.0 _ll_register_callbacks cb 0x7f2c08005e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 0x5651f7007af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 0x5651f7007af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51254/0 (socket says 172.21.15.137:51254) 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 -- 172.21.15.137:0/192418445 learned_addr learned my addr 172.21.15.137:0/192418445 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c15200640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f706ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0f400640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7008030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 -- 172.21.15.137:0/192418445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 msgr2=0x5651f7008030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7008030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 -- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 msgr2=0x5651f706ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:25.931 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f706ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 -- 172.21.15.137:0/192418445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651f700b100 con 0x5651f6ff9810 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2c0fe00640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 0x5651f7007af0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2c00097f50 tx=0x7f2c0000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28481013050f0b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2bfa000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.928+0000 7f2bfa000640 1 -- 172.21.15.137:0/192418445 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0009b450 con 0x5651f6ff9810 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2bfa000640 1 -- 172.21.15.137:0/192418445 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f2c00099b90 con 0x5651f6ff9810 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bfc005050 con 0x5651f6ff9810 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2bfa000640 1 -- 172.21.15.137:0/192418445 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0009b630 con 0x5651f6ff9810 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 10 client.18609 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:25.932 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5651f706c2a0 con 0x5651f6ff9810 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2bfa000640 1 -- 172.21.15.137:0/192418445 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2bfc005050 con 0x5651f6ff9810 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2bfa000640 1 -- 172.21.15.137:0/192418445 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f2c000cf220 con 0x5651f6ff9810 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 10 client.18609 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 20 client.18609 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f706c750 0x5651f706eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:25.934 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 4 client.18609 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.929+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5651f6f65b50 con 0x5651f706c750 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.930+0000 7f2c0f400640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f706c750 0x5651f706eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c0f400640 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f706c750 0x5651f706eb70 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbe252749fa18954 server_cookie=1d09f6ccfe2c9db2 in_seq=0 out_seq=0 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2bfa000640 10 client.18609 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2bfa000640 1 -- 172.21.15.137:0/192418445 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5651f6f65b50 con 0x5651f706c750 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2bfa000640 10 client.18609 handle_command_reply: tid=0 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 client.18609 shutdown 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 msgr2=0x5651f7007af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 0x5651f7007af0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2c00097f50 tx=0x7f2c0000eec0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 shutdown_connections 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5651f706c750 0x5651f706eb70 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651f6ffb3a0 0x5651f706ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.935 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651f6ffa9e0 0x5651f7008030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 --2- 172.21.15.137:0/192418445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5651f6ff9810 0x5651f7007af0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 >> 172.21.15.137:0/192418445 conn(0x5651f6ff4c00 msgr2=0x5651f6ff3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.931+0000 7f2bfa000640 0 client.18609 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.932+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 shutdown_connections 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.932+0000 7f2c19fe1740 1 -- 172.21.15.137:0/192418445 wait complete. 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.932+0000 7f2c19fe1740 20 client.18609 trim_cache size 0 max 16384 2024-03-26T14:16:25.936 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.932+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2bf40a2800 msgr2=0x7f2bf40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.932+0000 7f2c17200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2bf40a2800 0x7f2bf40a4cc0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5651f6ffa8e0 tx=0x7f2c0800b450 comp rx=0 tx=0).stop 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 msgr2=0x7f2c10199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 0x7f2c10199b30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000ebc0 tx=0x7f2c0000c6a0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 shutdown_connections 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f2bf40a2800 0x7f2bf40a4cc0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c10107390 0x7f2c101a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.937 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c10104a60 0x7f2c1019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 --2- 172.21.15.137:0/4025107508 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c10102130 0x7f2c10199b30 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:25.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 >> 172.21.15.137:0/4025107508 conn(0x7f2c100fbb40 msgr2=0x7f2c10101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:25.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 shutdown_connections 2024-03-26T14:16:25.938 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:25.933+0000 7f2c17200640 1 -- 172.21.15.137:0/4025107508 wait complete. 2024-03-26T14:16:26.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:16:26.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:16:26.952 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:16:27.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 -- 172.21.15.137:0/1051079794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4102130 msgr2=0x7f6bb4104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.205 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1051079794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4102130 0x7f6bb4104520 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009900 tx=0x7f6ba802bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 -- 172.21.15.137:0/1051079794 shutdown_connections 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1051079794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb4104a60 0x7f6bb4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1051079794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bb4107390 0x7f6bb41097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1051079794 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4102130 0x7f6bb4104520 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 -- 172.21.15.137:0/1051079794 >> 172.21.15.137:0/1051079794 conn(0x7f6bb40fbb40 msgr2=0x7f6bb40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 -- 172.21.15.137:0/1051079794 shutdown_connections 2024-03-26T14:16:27.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.204+0000 7f6bbc400640 1 -- 172.21.15.137:0/1051079794 wait complete. 2024-03-26T14:16:27.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.205+0000 7f6bbc400640 1 Processor -- start 2024-03-26T14:16:27.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.205+0000 7f6bbc400640 1 -- start start 2024-03-26T14:16:27.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bbc400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb4102130 0x7f6bb4101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.209 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bbc400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bb4104a60 0x7f6bb41000e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bbc400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 0x7f6bb4100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bbc400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb410d250 con 0x7f6bb4107390 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bbc400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb410d0d0 con 0x7f6bb4102130 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bbc400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb410d3d0 con 0x7f6bb4104a60 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 0x7f6bb4100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 0x7f6bb4100620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51278/0 (socket says 172.21.15.137:51278) 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 -- 172.21.15.137:0/1764070946 learned_addr learned my addr 172.21.15.137:0/1764070946 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb4102130 0x7f6bb4101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bb4104a60 0x7f6bb41000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.210 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 -- 172.21.15.137:0/1764070946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bb4104a60 msgr2=0x7f6bb41000e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bb4104a60 0x7f6bb41000e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 -- 172.21.15.137:0/1764070946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb4102130 msgr2=0x7f6bb4101aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb4102130 0x7f6bb4101aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.206+0000 7f6bba400640 1 -- 172.21.15.137:0/1764070946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4100eb0 con 0x7f6bb4107390 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.207+0000 7f6bba400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 0x7f6bb4100620 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00eaa0 tx=0x7f6bac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.207+0000 7f6bb2000640 1 -- 172.21.15.137:0/1764070946 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac00cdb0 con 0x7f6bb4107390 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.207+0000 7f6bb2000640 1 -- 172.21.15.137:0/1764070946 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6bac005020 con 0x7f6bb4107390 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.207+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb41a1a70 con 0x7f6bb4107390 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.207+0000 7f6bb2000640 1 -- 172.21.15.137:0/1764070946 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0116d0 con 0x7f6bb4107390 2024-03-26T14:16:27.211 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.207+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb41a1fc0 con 0x7f6bb4107390 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 Processor -- start 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bb2000640 1 -- 172.21.15.137:0/1764070946 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6bac0051d0 con 0x7f6bb4107390 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 -- start start 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c11e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c123590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c126090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677c083eb0 con 0x55677c11e0a0 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677c02ca60 con 0x55677c123c00 2024-03-26T14:16:27.216 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.211+0000 7f6bbf056740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677c07ff50 con 0x55677c11ea90 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c11e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c11e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51284/0 (socket says 172.21.15.137:51284) 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 -- 172.21.15.137:0/3516195841 learned_addr learned my addr 172.21.15.137:0/3516195841 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c123590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c126090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 -- 172.21.15.137:0/3516195841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 msgr2=0x55677c123590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c123590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 -- 172.21.15.137:0/3516195841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 msgr2=0x55677c126090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c126090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.217 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.212+0000 7f6bb9000640 1 -- 172.21.15.137:0/3516195841 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55677be607e0 con 0x55677c11e0a0 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6bb2000640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6b9c0a2800 0x7f6b9c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6bb9000640 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c11e4c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00135d0 tx=0x7f6ba000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9566944ab012a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6bb2000640 1 -- 172.21.15.137:0/1764070946 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6bac0cefd0 con 0x7f6bb4107390 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6b9c0a2800 0x7f6b9c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6ba6a00640 1 -- 172.21.15.137:0/3516195841 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0019030 con 0x55677c11e0a0 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6ba6a00640 1 -- 172.21.15.137:0/3516195841 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ba0011970 con 0x55677c11e0a0 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.213+0000 7f6ba6a00640 1 -- 172.21.15.137:0/3516195841 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba000f880 con 0x55677c11e0a0 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 -- 172.21.15.137:0/3516195841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 msgr2=0x55677c11e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c11e4c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00135d0 tx=0x7f6ba000b6b0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6b9c0a2800 0x7f6b9c0a4cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8000c00 tx=0x7f6ba802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 -- 172.21.15.137:0/3516195841 shutdown_connections 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c126090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c123590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 --2- 172.21.15.137:0/3516195841 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c11e4c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 -- 172.21.15.137:0/3516195841 >> 172.21.15.137:0/3516195841 conn(0x55677c119610 msgr2=0x55677c119a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 -- 172.21.15.137:0/3516195841 shutdown_connections 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.214+0000 7f6bbf056740 1 -- 172.21.15.137:0/3516195841 wait complete. 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 -- start start 2024-03-26T14:16:27.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 Processor -- start 2024-03-26T14:16:27.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c128230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c128770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.224 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677c126bb0 con 0x55677c11e0a0 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677c03feb0 con 0x55677c123c00 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.215+0000 7f6bbf056740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55677c126d30 con 0x55677c11ea90 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bbf056740 10 client.0 _ll_register_callbacks cb 0x55677c034150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c128230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c128230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51290/0 (socket says 172.21.15.137:51290) 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 -- 172.21.15.137:0/1152518809 learned_addr learned my addr 172.21.15.137:0/1152518809 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bba400640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c128770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 -- 172.21.15.137:0/1152518809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 msgr2=0x55677c128770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c128770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 -- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 msgr2=0x55677c12f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.216+0000 7f6bb9a00640 1 -- 172.21.15.137:0/1152518809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55677bec0f70 con 0x55677c11e0a0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.217+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c128230 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005ab0 tx=0x7f6ba802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44884e818fbd8c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.217+0000 7f6b9b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.217+0000 7f6b9b400640 1 -- 172.21.15.137:0/1152518809 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba804f030 con 0x55677c11e0a0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.217+0000 7f6b9b400640 1 -- 172.21.15.137:0/1152518809 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6ba803a040 con 0x55677c11e0a0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.217+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba0011970 con 0x55677c11e0a0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6b9b400640 1 -- 172.21.15.137:0/1152518809 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba805c050 con 0x55677c11e0a0 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 10 client.18621 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:27.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55677bd58170 con 0x55677c11e0a0 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6b9b400640 1 -- 172.21.15.137:0/1152518809 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ba0011970 con 0x55677c11e0a0 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6b9b400640 1 -- 172.21.15.137:0/1152518809 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6ba80035f0 con 0x55677c11e0a0 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 10 client.18621 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 20 client.18621 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c1340e0 0x55677c136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 4 client.18621 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.218+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55677c0913f0 con 0x55677c1340e0 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.219+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c1340e0 0x55677c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.220+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c1340e0 0x55677c136530 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a774768b005bc27 server_cookie=221beac02532c1c9 in_seq=0 out_seq=0 2024-03-26T14:16:27.227 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.220+0000 7f6b9b400640 10 client.18621 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6b9b400640 1 -- 172.21.15.137:0/1152518809 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55677c0913f0 con 0x55677c1340e0 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6b9b400640 10 client.18621 handle_command_reply: tid=0 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 client.18621 shutdown 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 msgr2=0x55677c128230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c128230 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005ab0 tx=0x7f6ba802bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 shutdown_connections 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c1340e0 0x55677c136530 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11ea90 0x55677c128770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1152518809 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11e0a0 0x55677c128230 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6b9b400640 0 client.18621 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.228 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.221+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 >> 172.21.15.137:0/1152518809 conn(0x55677c119610 msgr2=0x55677c118830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.222+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 shutdown_connections 2024-03-26T14:16:27.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.222+0000 7f6bbf056740 1 -- 172.21.15.137:0/1152518809 wait complete. 2024-03-26T14:16:27.229 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.222+0000 7f6bbf056740 20 client.18621 trim_cache size 0 max 16384 2024-03-26T14:16:27.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.243+0000 7f6bbf056740 1 Processor -- start 2024-03-26T14:16:27.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.243+0000 7f6bbf056740 1 -- start start 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c1186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bbf056740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41ae970 con 0x55677c11ea90 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bbf056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41ac470 con 0x55677c123c00 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bbf056740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41aa4f0 con 0x55677c11e0a0 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c118bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51292/0 (socket says 172.21.15.137:51292) 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 -- 172.21.15.137:0/4017853940 learned_addr learned my addr 172.21.15.137:0/4017853940 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bba400640 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c1186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 -- 172.21.15.137:0/4017853940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 msgr2=0x55677c1186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c1186b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 -- 172.21.15.137:0/4017853940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 msgr2=0x55677c12f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9000640 1 -- 172.21.15.137:0/4017853940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55677c120510 con 0x55677c11ea90 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.244+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c1186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.245+0000 7f6bb9000640 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c118bf0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6ba001c000 tx=0x7f6ba000b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98328f99de1bb82d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.245+0000 7f6ba6000640 1 -- 172.21.15.137:0/4017853940 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0019040 con 0x55677c11ea90 2024-03-26T14:16:27.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.245+0000 7f6ba6000640 1 -- 172.21.15.137:0/4017853940 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ba0030020 con 0x55677c11ea90 2024-03-26T14:16:27.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.245+0000 7f6ba6000640 1 -- 172.21.15.137:0/4017853940 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00114c0 con 0x55677c11ea90 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 -- 172.21.15.137:0/4017853940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 msgr2=0x55677c118bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c118bf0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f6ba001c000 tx=0x7f6ba000b610 comp rx=0 tx=0).stop 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 -- 172.21.15.137:0/4017853940 shutdown_connections 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c118bf0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 --2- 172.21.15.137:0/4017853940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c1186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 -- 172.21.15.137:0/4017853940 >> 172.21.15.137:0/4017853940 conn(0x55677c119610 msgr2=0x55677c117fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 -- 172.21.15.137:0/4017853940 shutdown_connections 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.246+0000 7f6bbf056740 1 -- 172.21.15.137:0/4017853940 wait complete. 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.247+0000 7f6bbf056740 1 -- start start 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.247+0000 7f6bbf056740 1 Processor -- start 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c12f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c12dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41aaaf0 con 0x55677c11ea90 2024-03-26T14:16:27.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41abe10 con 0x55677c123c00 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb41ab150 con 0x55677c11e0a0 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bbf056740 10 client.0 _ll_register_callbacks cb 0x7f6ba8043050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c12dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c12dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51294/0 (socket says 172.21.15.137:51294) 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 -- 172.21.15.137:0/1287347166 learned_addr learned my addr 172.21.15.137:0/1287347166 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bba400640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c12f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 -- 172.21.15.137:0/1287347166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 msgr2=0x55677c12f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c12f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 -- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 msgr2=0x55677c12e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bb9000640 1 -- 172.21.15.137:0/1287347166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55677c12eb10 con 0x55677c11ea90 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.248+0000 7f6bba400640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6bb9000640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c12dd60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005d00 tx=0x7f6ba0011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171bafc958f5461a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6b9b400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6b9b400640 1 -- 172.21.15.137:0/1287347166 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba003b020 con 0x55677c11ea90 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ba0030020 con 0x55677c11ea90 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6b9b400640 1 -- 172.21.15.137:0/1287347166 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f6ba00114c0 con 0x55677c11ea90 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6b9b400640 1 -- 172.21.15.137:0/1287347166 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ba0046050 con 0x55677c11ea90 2024-03-26T14:16:27.256 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.249+0000 7f6bbf056740 10 client.18627 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55677c190a10 con 0x55677c11ea90 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6b9b400640 1 -- 172.21.15.137:0/1287347166 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ba0030020 con 0x55677c11ea90 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6b9b400640 1 -- 172.21.15.137:0/1287347166 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f6ba0037470 con 0x55677c11ea90 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 10 client.18627 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 20 client.18627 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c191160 0x55677c193550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 4 client.18627 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.250+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55677c08b740 con 0x55677c191160 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.251+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c191160 0x55677c193550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.251+0000 7f6bb9a00640 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c191160 0x55677c193550 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9d5d424236bd2c server_cookie=35208ba182dcbafa in_seq=0 out_seq=0 2024-03-26T14:16:27.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.251+0000 7f6b9b400640 10 client.18627 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6b9b400640 1 -- 172.21.15.137:0/1287347166 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55677c08b740 con 0x55677c191160 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6b9b400640 10 client.18627 handle_command_reply: tid=0 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 client.18627 shutdown 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 msgr2=0x55677c12dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c12dd60 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005d00 tx=0x7f6ba0011fa0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 shutdown_connections 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55677c191160 0x55677c193550 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55677c123c00 0x55677c12e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55677c11ea90 0x55677c12dd60 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 --2- 172.21.15.137:0/1287347166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55677c11e0a0 0x55677c12f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6b9b400640 0 client.18627 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 >> 172.21.15.137:0/1287347166 conn(0x55677c119610 msgr2=0x55677c11fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.254+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 shutdown_connections 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.255+0000 7f6bbf056740 1 -- 172.21.15.137:0/1287347166 wait complete. 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.255+0000 7f6bbf056740 20 client.18627 trim_cache size 0 max 16384 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "3e1dd5db-c4da-4c5c-86da-18ef44159418", 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:16:27.260 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:16:27.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.255+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6b9c0a2800 msgr2=0x7f6b9c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.255+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6b9c0a2800 0x7f6b9c0a4cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8000c00 tx=0x7f6ba802c7e0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.255+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 msgr2=0x7f6bb4100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.255+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 0x7f6bb4100620 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f6bac00eaa0 tx=0x7f6bac00ef60 comp rx=0 tx=0).stop 2024-03-26T14:16:27.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 shutdown_connections 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f6b9c0a2800 0x7f6b9c0a4cc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6bb4107390 0x7f6bb4100620 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bb4104a60 0x7f6bb41000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 --2- 172.21.15.137:0/1764070946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb4102130 0x7f6bb4101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 >> 172.21.15.137:0/1764070946 conn(0x7f6bb40fbb40 msgr2=0x7f6bb4102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 shutdown_connections 2024-03-26T14:16:27.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.256+0000 7f6bbc400640 1 -- 172.21.15.137:0/1764070946 wait complete. 2024-03-26T14:16:27.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:16:27.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:16:27.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 -- 172.21.15.137:0/168994161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c102770 msgr2=0x7f196c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 --2- 172.21.15.137:0/168994161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c102770 0x7f196c102bf0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1954009ca0 tx=0x7f195402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.495 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 -- 172.21.15.137:0/168994161 shutdown_connections 2024-03-26T14:16:27.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 --2- 172.21.15.137:0/168994161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f196c103130 0x7f196c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 --2- 172.21.15.137:0/168994161 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c102770 0x7f196c102bf0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 --2- 172.21.15.137:0/168994161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f196c101580 0x7f196c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 -- 172.21.15.137:0/168994161 >> 172.21.15.137:0/168994161 conn(0x7f196c0fcb10 msgr2=0x7f196c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 -- 172.21.15.137:0/168994161 shutdown_connections 2024-03-26T14:16:27.496 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.494+0000 7f1971600640 1 -- 172.21.15.137:0/168994161 wait complete. 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.495+0000 7f1971600640 1 Processor -- start 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.495+0000 7f1971600640 1 -- start start 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.495+0000 7f1971600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 0x7f196c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1971600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f196c102770 0x7f196c197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1971600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f196c103130 0x7f196c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1971600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10d0c0 con 0x7f196c101580 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1971600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10cf40 con 0x7f196c103130 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1971600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c10d240 con 0x7f196c102770 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 0x7f196c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.499 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 0x7f196c197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51316/0 (socket says 172.21.15.137:51316) 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 -- 172.21.15.137:0/4113356972 learned_addr learned my addr 172.21.15.137:0/4113356972 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1970c00640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f196c103130 0x7f196c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f1963400640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f196c102770 0x7f196c197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 -- 172.21.15.137:0/4113356972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f196c102770 msgr2=0x7f196c197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f196c102770 0x7f196c197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 -- 172.21.15.137:0/4113356972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f196c103130 msgr2=0x7f196c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f196c103130 0x7f196c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.496+0000 7f196be00640 1 -- 172.21.15.137:0/4113356972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c19f390 con 0x7f196c101580 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.497+0000 7f196be00640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 0x7f196c197770 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f195c00e9c0 tx=0x7f195c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.497+0000 7f1969600640 1 -- 172.21.15.137:0/4113356972 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00cd80 con 0x7f196c101580 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.497+0000 7f1969600640 1 -- 172.21.15.137:0/4113356972 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f195c0035f0 con 0x7f196c101580 2024-03-26T14:16:27.500 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.497+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c19f680 con 0x7f196c101580 2024-03-26T14:16:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.497+0000 7f1969600640 1 -- 172.21.15.137:0/4113356972 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c010690 con 0x7f196c101580 2024-03-26T14:16:27.501 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.497+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c19fbf0 con 0x7f196c101580 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 Processor -- start 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 -- start start 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1969600640 1 -- 172.21.15.137:0/4113356972 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f195c0037a0 con 0x7f196c101580 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a64540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a67040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4839c4250 con 0x55a483a5fa40 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48396ca30 con 0x55a483a64bb0 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.501+0000 7f1975a1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4839c02f0 con 0x55a483a5f050 2024-03-26T14:16:27.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f196be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a64540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1963400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a67040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f196be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a64540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51332/0 (socket says 172.21.15.137:51332) 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f196be00640 1 -- 172.21.15.137:0/1902912454 learned_addr learned my addr 172.21.15.137:0/1902912454 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1970c00640 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1963400640 1 -- 172.21.15.137:0/1902912454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 msgr2=0x55a483a5f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1963400640 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a5f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1963400640 1 -- 172.21.15.137:0/1902912454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 msgr2=0x55a483a64540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1963400640 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a64540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f1963400640 1 -- 172.21.15.137:0/1902912454 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a483957690 con 0x55a483a64bb0 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.502+0000 7f196be00640 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:27.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.503+0000 7f1963400640 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a67040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1954032420 tx=0x7f19540093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dde6ea67ceb66e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.503+0000 7f1961200640 1 -- 172.21.15.137:0/1902912454 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954043020 con 0x55a483a64bb0 2024-03-26T14:16:27.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.503+0000 7f1969600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19500a2880 0x7f19500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.503+0000 7f1961200640 1 -- 172.21.15.137:0/1902912454 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1954032db0 con 0x55a483a64bb0 2024-03-26T14:16:27.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.503+0000 7f1961200640 1 -- 172.21.15.137:0/1902912454 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954039040 con 0x55a483a64bb0 2024-03-26T14:16:27.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.504+0000 7f1969600640 1 -- 172.21.15.137:0/4113356972 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f195c0cf1f0 con 0x7f196c101580 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.504+0000 7f1970c00640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19500a2880 0x7f19500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.504+0000 7f1975a1b740 1 -- 172.21.15.137:0/1902912454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 msgr2=0x55a483a67040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.504+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a67040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1954032420 tx=0x7f19540093e0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.504+0000 7f1970c00640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19500a2880 0x7f19500a4d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55a483a602a0 tx=0x7f196400b2e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 -- 172.21.15.137:0/1902912454 shutdown_connections 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a67040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a64540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1902912454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a5f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 -- 172.21.15.137:0/1902912454 >> 172.21.15.137:0/1902912454 conn(0x55a483a27200 msgr2=0x55a483a5a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 -- 172.21.15.137:0/1902912454 shutdown_connections 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 -- 172.21.15.137:0/1902912454 wait complete. 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 -- start start 2024-03-26T14:16:27.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.505+0000 7f1975a1b740 1 Processor -- start 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5f050 0x55a483a6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a6d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a64bb0 0x55a483a748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a483a67b00 con 0x55a483a5fa40 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48397fe80 con 0x55a483a5f050 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a483a67c80 con 0x55a483a64bb0 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1975a1b740 10 client.0 _ll_register_callbacks cb 0x55a483992130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1963400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a6d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1963400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a6d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51344/0 (socket says 172.21.15.137:51344) 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f1963400640 1 -- 172.21.15.137:0/1388982504 learned_addr learned my addr 172.21.15.137:0/1388982504 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.506+0000 7f196be00640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5f050 0x55a483a6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1970c00640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a64bb0 0x55a483a748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1963400640 1 -- 172.21.15.137:0/1388982504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a64bb0 msgr2=0x55a483a748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1963400640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a64bb0 0x55a483a748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1963400640 1 -- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5f050 msgr2=0x55a483a6d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1963400640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5f050 0x55a483a6d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1963400640 1 -- 172.21.15.137:0/1388982504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4837a27d0 con 0x55a483a5fa40 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1970c00640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a64bb0 0x55a483a748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1963400640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a6d890 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f195402c630 tx=0x7f1954032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2c5c8a8e2dfd0f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1959600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1959600640 1 -- 172.21.15.137:0/1388982504 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195404e020 con 0x55a483a5fa40 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1959600640 1 -- 172.21.15.137:0/1388982504 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1954003f20 con 0x55a483a5fa40 2024-03-26T14:16:27.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.507+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1954032db0 con 0x55a483a5fa40 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1959600640 1 -- 172.21.15.137:0/1388982504 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1954059050 con 0x55a483a5fa40 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1975a1b740 10 client.18639 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a4836d83f0 con 0x55a483a5fa40 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1959600640 1 -- 172.21.15.137:0/1388982504 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1954032db0 con 0x55a483a5fa40 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1959600640 1 -- 172.21.15.137:0/1388982504 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f195403f7f0 con 0x55a483a5fa40 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1975a1b740 10 client.18639 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1975a1b740 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.508+0000 7f1975a1b740 20 client.18639 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.509+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483a78dc0 0x55a483a7b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.509+0000 7f1975a1b740 4 client.18639 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.509+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4839d1790 con 0x55a483a78dc0 2024-03-26T14:16:27.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.509+0000 7f196be00640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483a78dc0 0x55a483a7b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.510+0000 7f1959600640 10 client.18639 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.510+0000 7f196be00640 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483a78dc0 0x55a483a7b1b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c1bc5df96848ee server_cookie=fb5afbd8373170b in_seq=0 out_seq=0 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1959600640 1 -- 172.21.15.137:0/1388982504 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a4839d1790 con 0x55a483a78dc0 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1959600640 10 client.18639 handle_command_reply: tid=0 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 client.18639 shutdown 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 msgr2=0x55a483a6d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a6d890 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f195402c630 tx=0x7f1954032f80 comp rx=0 tx=0).stop 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 shutdown_connections 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483a78dc0 0x55a483a7b1b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a64bb0 0x55a483a748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.516 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a6d890 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1388982504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5f050 0x55a483a6d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1959600640 0 client.18639 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 >> 172.21.15.137:0/1388982504 conn(0x55a483a27200 msgr2=0x55a483a59bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 shutdown_connections 2024-03-26T14:16:27.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 1 -- 172.21.15.137:0/1388982504 wait complete. 2024-03-26T14:16:27.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.512+0000 7f1975a1b740 20 client.18639 trim_cache size 0 max 16384 2024-03-26T14:16:27.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.531+0000 7f1975a1b740 1 Processor -- start 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 -- start start 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a59a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a59f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1aa300 con 0x55a483a5fa40 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1a7e00 con 0x55a483a64bb0 2024-03-26T14:16:27.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1975a1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1a5e80 con 0x55a483a5f050 2024-03-26T14:16:27.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1963400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a59f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1963400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a59f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51350/0 (socket says 172.21.15.137:51350) 2024-03-26T14:16:27.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1963400640 1 -- 172.21.15.137:0/1985460515 learned_addr learned my addr 172.21.15.137:0/1985460515 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f1970c00640 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.532+0000 7f196be00640 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a59a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f1963400640 1 -- 172.21.15.137:0/1985460515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 msgr2=0x55a483a59a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f1963400640 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a59a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f1963400640 1 -- 172.21.15.137:0/1985460515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 msgr2=0x55a483a74890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f1963400640 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a74890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f1963400640 1 -- 172.21.15.137:0/1985460515 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4836980f0 con 0x55a483a5fa40 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f1963400640 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a59f80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1954005e30 tx=0x7f195402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920e228f73c7cd57 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f195be00640 1 -- 172.21.15.137:0/1985460515 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954043070 con 0x55a483a5fa40 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f195be00640 1 -- 172.21.15.137:0/1985460515 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1954032d70 con 0x55a483a5fa40 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.533+0000 7f195be00640 1 -- 172.21.15.137:0/1985460515 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195404e040 con 0x55a483a5fa40 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 -- 172.21.15.137:0/1985460515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 msgr2=0x55a483a59f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a59f80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f1954005e30 tx=0x7f195402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 -- 172.21.15.137:0/1985460515 shutdown_connections 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a64bb0 0x55a483a74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a5fa40 0x55a483a59f80 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 --2- 172.21.15.137:0/1985460515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a59a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 -- 172.21.15.137:0/1985460515 >> 172.21.15.137:0/1985460515 conn(0x55a483a27200 msgr2=0x55a483a59330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 -- 172.21.15.137:0/1985460515 shutdown_connections 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.534+0000 7f1975a1b740 1 -- 172.21.15.137:0/1985460515 wait complete. 2024-03-26T14:16:27.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 -- start start 2024-03-26T14:16:27.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 Processor -- start 2024-03-26T14:16:27.541 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a72780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5fa40 0x55a483a72cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 0x55a483a73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1a6480 con 0x55a483a64bb0 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1a77a0 con 0x55a483a5fa40 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.535+0000 7f1975a1b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1a6ae0 con 0x55a483a5f050 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1975a1b740 10 client.0 _ll_register_callbacks cb 0x7f195c0d2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 0x55a483a73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 0x55a483a73cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:51352/0 (socket says 172.21.15.137:51352) 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 -- 172.21.15.137:0/3405991492 learned_addr learned my addr 172.21.15.137:0/3405991492 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1963400640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5fa40 0x55a483a72cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f196be00640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a72780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 -- 172.21.15.137:0/3405991492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 msgr2=0x55a483a72780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a72780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 -- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5fa40 msgr2=0x55a483a72cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5fa40 0x55a483a72cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.536+0000 7f1970c00640 1 -- 172.21.15.137:0/3405991492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a483a742c0 con 0x55a483a64bb0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1970c00640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 0x55a483a73cd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1964003e70 tx=0x7f1964015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1a87d3892d2d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1959600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1959600640 1 -- 172.21.15.137:0/3405991492 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964028630 con 0x55a483a64bb0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1959600640 1 -- 172.21.15.137:0/3405991492 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f196401b430 con 0x55a483a64bb0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1954032d70 con 0x55a483a64bb0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1959600640 1 -- 172.21.15.137:0/3405991492 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19640287d0 con 0x55a483a64bb0 2024-03-26T14:16:27.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1975a1b740 10 client.18645 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.537+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a483ad4d00 con 0x55a483a64bb0 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1959600640 1 -- 172.21.15.137:0/3405991492 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1954032d70 con 0x55a483a64bb0 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1959600640 1 -- 172.21.15.137:0/3405991492 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f1964019040 con 0x55a483a64bb0 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1975a1b740 10 client.18645 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1975a1b740 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1975a1b740 20 client.18645 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1975a1b740 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483ad5760 0x55a483ad7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1975a1b740 4 client.18645 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a4839cbae0 con 0x55a483ad5760 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.538+0000 7f196be00640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483ad5760 0x55a483ad7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.539+0000 7f196be00640 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483ad5760 0x55a483ad7b80 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1042a282cc959ea server_cookie=fea0afe454789cdf in_seq=0 out_seq=0 2024-03-26T14:16:27.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.539+0000 7f1959600640 10 client.18645 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.539+0000 7f1959600640 1 -- 172.21.15.137:0/3405991492 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a4839cbae0 con 0x55a483ad5760 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.539+0000 7f1959600640 10 client.18645 handle_command_reply: tid=0 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.539+0000 7f1975a1b740 1 client.18645 shutdown 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 msgr2=0x55a483a73cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 0x55a483a73cd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1964003e70 tx=0x7f1964015000 comp rx=0 tx=0).stop 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 shutdown_connections 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a483ad5760 0x55a483ad7b80 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a483a5fa40 0x55a483a72cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a483a64bb0 0x55a483a73cd0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 --2- 172.21.15.137:0/3405991492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a483a5f050 0x55a483a72780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 >> 172.21.15.137:0/3405991492 conn(0x55a483a27200 msgr2=0x55a483a59bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1959600640 0 client.18645 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:27.545 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 shutdown_connections 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 1 -- 172.21.15.137:0/3405991492 wait complete. 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.540+0000 7f1975a1b740 20 client.18645 trim_cache size 0 max 16384 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: "3e1dd5db-c4da-4c5c-86da-18ef44159418": { 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "3e1dd5db-c4da-4c5c-86da-18ef44159418", 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:16:27.546 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:16:27.547 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:16:27.547 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:16:27.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19500a2880 msgr2=0x7f19500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19500a2880 0x7f19500a4d40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55a483a602a0 tx=0x7f196400b2e0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 msgr2=0x7f196c197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 0x7f196c197770 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f195c00e9c0 tx=0x7f195c00ee80 comp rx=0 tx=0).stop 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 shutdown_connections 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f19500a2880 0x7f19500a4d40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f196c103130 0x7f196c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f196c102770 0x7f196c197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 --2- 172.21.15.137:0/4113356972 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f196c101580 0x7f196c197770 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 >> 172.21.15.137:0/4113356972 conn(0x7f196c0fcb10 msgr2=0x7f196c105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 shutdown_connections 2024-03-26T14:16:27.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:27.541+0000 7f1971600640 1 -- 172.21.15.137:0/4113356972 wait complete. 2024-03-26T14:16:27.560 INFO:tasks.cephfs.filesystem:scrub status for tag:3e1dd5db-c4da-4c5c-86da-18ef44159418 - {'path': '/', 'tag': '3e1dd5db-c4da-4c5c-86da-18ef44159418', 'options': 'recursive,force'} 2024-03-26T14:16:27.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:27 smithi179 ceph-mon[35472]: pgmap v880: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:27.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:27 smithi159 ceph-mon[35501]: pgmap v880: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:27.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:27 smithi137 ceph-mon[31539]: pgmap v880: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:28.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:28 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:16:28.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:28 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:16:28.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:28 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:16:28.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:28 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:16:28.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:28 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:16:28.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:28 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:16:28.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:28 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:16:28.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:28 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:16:28.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:28 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:16:28.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:28 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:16:28.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:28 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:16:28.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:28 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:16:29.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:29 smithi159 ceph-mon[35501]: pgmap v881: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:29.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:29 smithi137 ceph-mon[31539]: pgmap v881: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:29.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:29 smithi179 ceph-mon[35472]: pgmap v881: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:31.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:31 smithi159 ceph-mon[35501]: pgmap v882: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:31.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:31 smithi137 ceph-mon[31539]: pgmap v882: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:31.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:31 smithi179 ceph-mon[35472]: pgmap v882: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:32.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:32 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:16:32.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:32 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:16:32.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:32 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:16:33.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:33 smithi159 ceph-mon[35501]: pgmap v883: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:33.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:33 smithi137 ceph-mon[31539]: pgmap v883: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:33.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:33 smithi179 ceph-mon[35472]: pgmap v883: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:34.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:34 smithi137 ceph-mon[31539]: pgmap v884: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:16:34.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:34 smithi179 ceph-mon[35472]: pgmap v884: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:16:34.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:34 smithi159 ceph-mon[35501]: pgmap v884: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-03-26T14:16:36.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:36 smithi159 ceph-mon[35501]: pgmap v885: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:37.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:36 smithi137 ceph-mon[31539]: pgmap v885: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:37.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:36 smithi179 ceph-mon[35472]: pgmap v885: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:38.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:38 smithi159 ceph-mon[35501]: pgmap v886: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:39.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:38 smithi137 ceph-mon[31539]: pgmap v886: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:39.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:38 smithi179 ceph-mon[35472]: pgmap v886: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-03-26T14:16:41.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:40 smithi137 ceph-mon[31539]: pgmap v887: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:41.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:40 smithi179 ceph-mon[35472]: pgmap v887: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:41.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:40 smithi159 ceph-mon[35501]: pgmap v887: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-03-26T14:16:43.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:42 smithi137 ceph-mon[31539]: pgmap v888: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:43.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:42 smithi179 ceph-mon[35472]: pgmap v888: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:43.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:42 smithi159 ceph-mon[35501]: pgmap v888: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 6 op/s 2024-03-26T14:16:45.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:44 smithi159 ceph-mon[35501]: pgmap v889: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:45.006 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:44 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:16:45.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:44 smithi137 ceph-mon[31539]: pgmap v889: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:45.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:44 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:16:45.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:44 smithi179 ceph-mon[35472]: pgmap v889: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:16:45.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:44 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:16:46.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:16:46.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:16:46.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:45 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:16:46.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:16:46.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:16:46.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:45 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:16:46.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:16:46.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:16:46.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:45 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:16:47.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:46 smithi137 ceph-mon[31539]: pgmap v890: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:47.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:46 smithi179 ceph-mon[35472]: pgmap v890: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:47.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:46 smithi159 ceph-mon[35501]: pgmap v890: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:16:48.675 INFO:tasks.workunit.client.0.smithi137.stdout:writing pattern 2024-03-26T14:16:48.675 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-26T14:16:48.676 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:48.677 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:48.678 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:48.679 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:48.680 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:48.681 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:48.682 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:48.683 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:48.795 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:48.795 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:48.795 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:48.795 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:48.795 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:48.795 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:48.796 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:48.797 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:48.798 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:48.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:48.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:48.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:48.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:48.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:48.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:48.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:48.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:48.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:48.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:48.923 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:48.924 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:48.925 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:48.926 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:48.927 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:48.928 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:48.929 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:48.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:48.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:48.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:48.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:48.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:48.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:49.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:48 smithi137 ceph-mon[31539]: pgmap v891: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:49.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:49.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:49.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:49.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:49.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:49.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:49.047 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-03-26T14:16:49.048 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-26T14:16:49.049 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:49.050 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:49.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:49.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:49.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:49.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:49.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:49.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:49.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:49.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:49.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-03-26T14:16:49.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-26T14:16:49.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-26T14:16:49.158 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-03-26T14:16:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-26T14:16:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-03-26T14:16:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-26T14:16:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-03-26T14:16:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-26T14:16:49.159 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-03-26T14:16:49.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:48 smithi179 ceph-mon[35472]: pgmap v891: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:49.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:48 smithi159 ceph-mon[35501]: pgmap v891: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:16:49.268 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-26T14:16:49.268 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-26T14:16:49.269 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-26T14:16:49.270 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-03-26T14:16:49.271 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-26T14:16:49.272 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-03-26T14:16:49.273 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-03-26T14:16:49.274 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-26T14:16:49.275 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-26T14:16:49.276 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-26T14:16:49.379 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-03-26T14:16:49.380 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-03-26T14:16:49.381 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-26T14:16:49.382 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-26T14:16:49.383 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-26T14:16:49.384 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-26T14:16:49.385 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-03-26T14:16:49.386 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-26T14:16:49.387 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-03-26T14:16:49.489 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-26T14:16:49.489 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-03-26T14:16:49.489 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-03-26T14:16:49.490 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-26T14:16:49.491 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-03-26T14:16:49.492 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-03-26T14:16:49.493 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-03-26T14:16:49.494 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-26T14:16:49.495 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-26T14:16:49.496 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-03-26T14:16:49.497 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-03-26T14:16:49.592 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-03-26T14:16:49.593 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-03-26T14:16:49.594 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-26T14:16:49.595 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-03-26T14:16:49.596 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-26T14:16:49.597 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-03-26T14:16:49.598 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-26T14:16:49.599 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-03-26T14:16:49.600 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-26T14:16:49.600 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-03-26T14:16:49.702 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-26T14:16:49.702 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-03-26T14:16:49.702 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-26T14:16:49.702 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-03-26T14:16:49.702 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-03-26T14:16:49.703 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-26T14:16:49.704 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-26T14:16:49.705 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-03-26T14:16:49.706 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-26T14:16:49.707 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-03-26T14:16:49.708 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-26T14:16:49.709 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-03-26T14:16:49.820 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-26T14:16:49.820 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-26T14:16:49.821 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-26T14:16:49.822 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-03-26T14:16:49.823 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-26T14:16:49.824 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-26T14:16:49.825 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-26T14:16:49.826 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-03-26T14:16:49.827 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-26T14:16:49.828 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-03-26T14:16:49.828 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-26T14:16:49.828 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-03-26T14:16:49.828 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-26T14:16:49.828 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-03-26T14:16:49.828 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-26T14:16:49.930 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-26T14:16:49.931 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-03-26T14:16:49.932 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-03-26T14:16:49.933 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-26T14:16:49.934 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-03-26T14:16:49.935 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-26T14:16:49.936 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-26T14:16:49.937 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-03-26T14:16:49.938 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-26T14:16:49.938 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-03-26T14:16:49.938 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-26T14:16:49.938 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-03-26T14:16:49.938 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-26T14:16:50.038 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-03-26T14:16:50.038 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-03-26T14:16:50.039 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-03-26T14:16:50.040 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-26T14:16:50.041 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-03-26T14:16:50.042 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-26T14:16:50.043 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-26T14:16:50.044 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-03-26T14:16:50.045 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-03-26T14:16:50.046 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-26T14:16:50.148 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-03-26T14:16:50.148 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-26T14:16:50.148 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-03-26T14:16:50.148 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-26T14:16:50.148 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-03-26T14:16:50.149 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-26T14:16:50.149 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-03-26T14:16:50.149 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-26T14:16:50.149 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-03-26T14:16:50.149 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-26T14:16:50.149 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-26T14:16:50.150 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-26T14:16:50.151 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-26T14:16:50.152 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-26T14:16:50.153 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-26T14:16:50.154 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-26T14:16:50.155 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-26T14:16:50.156 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-03-26T14:16:50.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-26T14:16:50.157 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-03-26T14:16:50.256 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-26T14:16:50.257 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-26T14:16:50.258 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-03-26T14:16:50.259 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-03-26T14:16:50.260 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-26T14:16:50.261 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-26T14:16:50.262 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-26T14:16:50.263 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-03-26T14:16:50.365 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-26T14:16:50.365 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-03-26T14:16:50.365 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-26T14:16:50.366 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-03-26T14:16:50.367 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-03-26T14:16:50.368 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-26T14:16:50.369 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-03-26T14:16:50.370 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-26T14:16:50.371 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-03-26T14:16:50.372 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-03-26T14:16:50.373 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-03-26T14:16:50.472 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-03-26T14:16:50.473 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-26T14:16:50.474 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-26T14:16:50.475 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-26T14:16:50.476 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-03-26T14:16:50.477 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-26T14:16:50.478 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-03-26T14:16:50.479 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-26T14:16:50.580 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-03-26T14:16:50.581 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-26T14:16:50.582 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-26T14:16:50.583 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-03-26T14:16:50.584 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-26T14:16:50.585 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-26T14:16:50.586 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-03-26T14:16:50.587 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-26T14:16:50.692 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-03-26T14:16:50.692 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-26T14:16:50.692 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-03-26T14:16:50.692 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-03-26T14:16:50.693 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-26T14:16:50.694 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-03-26T14:16:50.695 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-26T14:16:50.696 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-03-26T14:16:50.697 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-26T14:16:50.698 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-03-26T14:16:50.699 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-26T14:16:50.700 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-03-26T14:16:50.799 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-03-26T14:16:50.800 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-26T14:16:50.801 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-26T14:16:50.802 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-26T14:16:50.803 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-03-26T14:16:50.804 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-03-26T14:16:50.805 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-26T14:16:50.806 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-03-26T14:16:50.904 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-26T14:16:50.904 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-03-26T14:16:50.904 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-26T14:16:50.904 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-03-26T14:16:50.904 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-26T14:16:50.904 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-26T14:16:50.905 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-03-26T14:16:50.906 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-03-26T14:16:50.907 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-03-26T14:16:50.908 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-03-26T14:16:50.909 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-26T14:16:50.910 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-03-26T14:16:50.911 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-26T14:16:51.010 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-03-26T14:16:51.010 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-26T14:16:51.010 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-03-26T14:16:51.010 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-26T14:16:51.011 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-03-26T14:16:51.012 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-26T14:16:51.013 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-03-26T14:16:51.014 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-26T14:16:51.015 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-03-26T14:16:51.016 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-26T14:16:51.017 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-03-26T14:16:51.018 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-26T14:16:51.018 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-03-26T14:16:51.018 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-26T14:16:51.110 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-26T14:16:51.111 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-03-26T14:16:51.112 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-26T14:16:51.113 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-26T14:16:51.114 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-26T14:16:51.115 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-03-26T14:16:51.116 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-03-26T14:16:51.117 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-26T14:16:51.218 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-03-26T14:16:51.218 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-26T14:16:51.218 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-03-26T14:16:51.218 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-26T14:16:51.218 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-03-26T14:16:51.218 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-26T14:16:51.219 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-26T14:16:51.220 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-03-26T14:16:51.221 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-03-26T14:16:51.222 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-03-26T14:16:51.223 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-03-26T14:16:51.224 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-26T14:16:51.225 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-03-26T14:16:51.323 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-26T14:16:51.323 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-03-26T14:16:51.323 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-03-26T14:16:51.324 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-26T14:16:51.325 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-03-26T14:16:51.326 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-26T14:16:51.327 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-26T14:16:51.328 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-26T14:16:51.329 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-03-26T14:16:51.442 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-26T14:16:51.442 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-03-26T14:16:51.442 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-26T14:16:51.443 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-26T14:16:51.444 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-03-26T14:16:51.445 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-03-26T14:16:51.446 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-03-26T14:16:51.447 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-03-26T14:16:51.448 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-26T14:16:51.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:51 smithi137 ceph-mon[31539]: pgmap v892: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8 op/s 2024-03-26T14:16:51.551 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-03-26T14:16:51.551 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-26T14:16:51.551 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-03-26T14:16:51.551 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-26T14:16:51.551 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-03-26T14:16:51.552 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-26T14:16:51.553 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-26T14:16:51.554 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-03-26T14:16:51.555 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-03-26T14:16:51.556 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-26T14:16:51.557 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-03-26T14:16:51.558 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-26T14:16:51.659 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-03-26T14:16:51.659 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-26T14:16:51.660 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-26T14:16:51.661 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-03-26T14:16:51.662 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-26T14:16:51.663 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-26T14:16:51.664 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-03-26T14:16:51.665 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-26T14:16:51.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-03-26T14:16:51.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-26T14:16:51.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-03-26T14:16:51.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-26T14:16:51.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-03-26T14:16:51.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:51 smithi179 ceph-mon[35472]: pgmap v892: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8 op/s 2024-03-26T14:16:51.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:51 smithi159 ceph-mon[35501]: pgmap v892: 129 pgs: 129 active+clean; 4.8 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8 op/s 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-03-26T14:16:51.775 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-03-26T14:16:51.776 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-03-26T14:16:51.777 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-03-26T14:16:51.778 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-03-26T14:16:51.779 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-03-26T14:16:51.780 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-26T14:16:51.781 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-03-26T14:16:51.883 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-26T14:16:51.884 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-03-26T14:16:51.885 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-26T14:16:51.886 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-03-26T14:16:51.887 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-26T14:16:51.888 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-26T14:16:51.889 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-03-26T14:16:52.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-26T14:16:52.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-03-26T14:16:52.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-26T14:16:52.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-03-26T14:16:52.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-26T14:16:52.666 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-03-26T14:16:52.667 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-26T14:16:52.668 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-26T14:16:52.669 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-03-26T14:16:52.670 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:52.671 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:52.672 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-26T14:16:52.673 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:52.673 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:52.673 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:52.673 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-26T14:16:52.673 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:53.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:53 smithi137 ceph-mon[31539]: pgmap v893: 129 pgs: 129 active+clean; 8.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 342 KiB/s wr, 44 op/s 2024-03-26T14:16:53.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:53 smithi179 ceph-mon[35472]: pgmap v893: 129 pgs: 129 active+clean; 8.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 342 KiB/s wr, 44 op/s 2024-03-26T14:16:53.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:53 smithi159 ceph-mon[35501]: pgmap v893: 129 pgs: 129 active+clean; 8.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 342 KiB/s wr, 44 op/s 2024-03-26T14:16:54.993 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:54.993 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:54.993 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-26T14:16:54.993 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:54.994 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:54.995 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-26T14:16:54.996 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:54.997 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-26T14:16:54.998 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:54.999 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:54.999 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:54.999 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-26T14:16:54.999 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:54.999 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:54.999 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:55.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:55 smithi137 ceph-mon[31539]: pgmap v894: 129 pgs: 129 active+clean; 33 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 343 KiB/s wr, 45 op/s 2024-03-26T14:16:55.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:55 smithi179 ceph-mon[35472]: pgmap v894: 129 pgs: 129 active+clean; 33 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 343 KiB/s wr, 45 op/s 2024-03-26T14:16:55.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:55 smithi159 ceph-mon[35501]: pgmap v894: 129 pgs: 129 active+clean; 33 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 343 KiB/s wr, 45 op/s 2024-03-26T14:16:56.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-26T14:16:56.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:56.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:56.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:56.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:56.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:56.703 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:56.704 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-26T14:16:56.705 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-26T14:16:56.706 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:56.707 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:56.707 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:56.707 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-26T14:16:57.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:57 smithi137 ceph-mon[31539]: pgmap v895: 129 pgs: 129 active+clean; 69 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 349 KiB/s wr, 70 op/s 2024-03-26T14:16:57.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:16:57.562 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:16:57.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:57 smithi179 ceph-mon[35472]: pgmap v895: 129 pgs: 129 active+clean; 69 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 349 KiB/s wr, 70 op/s 2024-03-26T14:16:57.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:57 smithi159 ceph-mon[35501]: pgmap v895: 129 pgs: 129 active+clean; 69 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 349 KiB/s wr, 70 op/s 2024-03-26T14:16:57.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 -- 172.21.15.137:0/3161749270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 msgr2=0x7f3700103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 --2- 172.21.15.137:0/3161749270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700103d00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f36f0009900 tx=0x7f36f002bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:57.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 -- 172.21.15.137:0/3161749270 shutdown_connections 2024-03-26T14:16:57.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 --2- 172.21.15.137:0/3161749270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 0x7f3700104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 --2- 172.21.15.137:0/3161749270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3700105490 0x7f370010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 --2- 172.21.15.137:0/3161749270 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700103d00 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 -- 172.21.15.137:0/3161749270 >> 172.21.15.137:0/3161749270 conn(0x7f37000fee50 msgr2=0x7f3700101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:57.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 -- 172.21.15.137:0/3161749270 shutdown_connections 2024-03-26T14:16:57.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.779+0000 7f3705c00640 1 -- 172.21.15.137:0/3161749270 wait complete. 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.780+0000 7f3705c00640 1 Processor -- start 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.780+0000 7f3705c00640 1 -- start start 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f3705c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f3705c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 0x7f370019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f3705c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3700105490 0x7f37001a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f3705c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370010f430 con 0x7f37001038e0 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f3705c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370010f2b0 con 0x7f3700104ad0 2024-03-26T14:16:57.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f3705c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370010f5b0 con 0x7f3700105490 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59432/0 (socket says 172.21.15.137:59432) 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 -- 172.21.15.137:0/1125670712 learned_addr learned my addr 172.21.15.137:0/1125670712 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fe000640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 0x7f370019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 -- 172.21.15.137:0/1125670712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3700105490 msgr2=0x7f37001a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36ff400640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3700105490 0x7f37001a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3700105490 0x7f37001a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 -- 172.21.15.137:0/1125670712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 msgr2=0x7f370019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 0x7f370019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fea00640 1 -- 172.21.15.137:0/1125670712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37001a1750 con 0x7f37001038e0 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.781+0000 7f36fe000640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 0x7f370019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.782+0000 7f36fea00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700199b30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005ab0 tx=0x7f36f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.782+0000 7f36f7400640 1 -- 172.21.15.137:0/1125670712 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0043020 con 0x7f37001038e0 2024-03-26T14:16:57.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.782+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37001a19e0 con 0x7f37001038e0 2024-03-26T14:16:57.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.782+0000 7f36f7400640 1 -- 172.21.15.137:0/1125670712 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36f00054c0 con 0x7f37001038e0 2024-03-26T14:16:57.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.782+0000 7f36f7400640 1 -- 172.21.15.137:0/1125670712 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0003cb0 con 0x7f37001038e0 2024-03-26T14:16:57.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.782+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37001a1f30 con 0x7f37001038e0 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.786+0000 7f37089c9740 1 Processor -- start 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f36f7400640 1 -- 172.21.15.137:0/1125670712 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f36f0046020 con 0x7f37001038e0 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 -- start start 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba5352a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba53ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ba49a250 con 0x55d7ba535870 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ba442a30 con 0x55d7ba53a9e0 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f37089c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ba4962f0 con 0x55d7ba534e80 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f36fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f36fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59438/0 (socket says 172.21.15.137:59438) 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.787+0000 7f36fea00640 1 -- 172.21.15.137:0/2032157284 learned_addr learned my addr 172.21.15.137:0/2032157284 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:57.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fe000640 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba53ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36ff400640 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba5352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fea00640 1 -- 172.21.15.137:0/2032157284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 msgr2=0x55d7ba5352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fea00640 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba5352a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fea00640 1 -- 172.21.15.137:0/2032157284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 msgr2=0x55d7ba53ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fea00640 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba53ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fea00640 1 -- 172.21.15.137:0/2032157284 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ba42d690 con 0x55d7ba535870 2024-03-26T14:16:57.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36fea00640 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53a370 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f36f00f91e0 tx=0x7f36f00f9210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbeb3f7853a6a6d server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:57.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.788+0000 7f36f7400640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f36e40a2800 0x7f36e40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.792 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.789+0000 7f36f7400640 1 -- 172.21.15.137:0/1125670712 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f36f00f11e0 con 0x7f37001038e0 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.789+0000 7f36ebe00640 1 -- 172.21.15.137:0/2032157284 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00b9b30 con 0x55d7ba535870 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.789+0000 7f36ff400640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f36e40a2800 0x7f36e40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.789+0000 7f36ebe00640 1 -- 172.21.15.137:0/2032157284 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36f00fe030 con 0x55d7ba535870 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.789+0000 7f36ebe00640 1 -- 172.21.15.137:0/2032157284 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00f4d10 con 0x55d7ba535870 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 -- 172.21.15.137:0/2032157284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 msgr2=0x55d7ba53a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f36ff400640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f36e40a2800 0x7f36e40a4cc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55d7ba5360d0 tx=0x7f36f800b320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53a370 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f36f00f91e0 tx=0x7f36f00f9210 comp rx=0 tx=0).stop 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 -- 172.21.15.137:0/2032157284 shutdown_connections 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba53ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53a370 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 --2- 172.21.15.137:0/2032157284 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba5352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 -- 172.21.15.137:0/2032157284 >> 172.21.15.137:0/2032157284 conn(0x55d7ba4fd200 msgr2=0x55d7ba5307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 -- 172.21.15.137:0/2032157284 shutdown_connections 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 -- 172.21.15.137:0/2032157284 wait complete. 2024-03-26T14:16:57.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.790+0000 7f37089c9740 1 -- start start 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 Processor -- start 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba53eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba546120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ba53d750 con 0x55d7ba535870 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ba455e80 con 0x55d7ba53a9e0 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7ba53d8d0 con 0x55d7ba534e80 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.791+0000 7f37089c9740 10 client.0 _ll_register_callbacks cb 0x55d7ba468130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36fe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba546120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba546120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55672/0 (socket says 172.21.15.137:55672) 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36fe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59448/0 (socket says 172.21.15.137:59448) 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 -- 172.21.15.137:0/4281489291 learned_addr learned my addr 172.21.15.137:0/4281489291 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:57.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36fea00640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba53eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 -- 172.21.15.137:0/4281489291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 msgr2=0x55d7ba53eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba53eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 -- 172.21.15.137:0/4281489291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 msgr2=0x55d7ba53f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 -- 172.21.15.137:0/4281489291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7ba2787d0 con 0x55d7ba53a9e0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36fea00640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba53eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36fe000640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36ff400640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba546120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f36f8011e40 tx=0x7f36f8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdece597a03096 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.792+0000 7f36e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.793+0000 7f36e8c00640 1 -- 172.21.15.137:0/4281489291 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f801e420 con 0x55d7ba53a9e0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.793+0000 7f36e8c00640 1 -- 172.21.15.137:0/4281489291 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f36f801e5c0 con 0x55d7ba53a9e0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.793+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36f00fe030 con 0x55d7ba53a9e0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.793+0000 7f36e8c00640 1 -- 172.21.15.137:0/4281489291 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f801e7c0 con 0x55d7ba53a9e0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.793+0000 7f37089c9740 10 client.38168 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.793+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d7ba1ae3f0 con 0x55d7ba53a9e0 2024-03-26T14:16:57.797 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f36e8c00640 1 -- 172.21.15.137:0/4281489291 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36f00fe030 con 0x55d7ba53a9e0 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f36e8c00640 1 -- 172.21.15.137:0/4281489291 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f36f8026a10 con 0x55d7ba53a9e0 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f37089c9740 10 client.38168 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f37089c9740 10 client.38168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f37089c9740 20 client.38168 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f37089c9740 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba54aa50 0x55d7ba54ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f37089c9740 4 client.38168 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.794+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7ba4a7790 con 0x55d7ba54aa50 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.795+0000 7f36fea00640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba54aa50 0x55d7ba54ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.795+0000 7f36fea00640 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba54aa50 0x55d7ba54ce70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e2f314fdff5575 server_cookie=cd123ad79b64aba2 in_seq=0 out_seq=0 2024-03-26T14:16:57.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.795+0000 7f36e8c00640 10 client.38168 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:57.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.796+0000 7f36e8c00640 1 -- 172.21.15.137:0/4281489291 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d7ba4a7790 con 0x55d7ba54aa50 2024-03-26T14:16:57.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.796+0000 7f36e8c00640 10 client.38168 handle_command_reply: tid=0 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.796+0000 7f37089c9740 1 client.38168 shutdown 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 msgr2=0x55d7ba546120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba546120 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f36f8011e40 tx=0x7f36f8004a40 comp rx=0 tx=0).stop 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 shutdown_connections 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba54aa50 0x55d7ba54ce70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba546120 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba535870 0x55d7ba53f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 --2- 172.21.15.137:0/4281489291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba534e80 0x55d7ba53eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 >> 172.21.15.137:0/4281489291 conn(0x55d7ba4fd200 msgr2=0x55d7ba52fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f36e8c00640 0 client.38168 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 shutdown_connections 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 1 -- 172.21.15.137:0/4281489291 wait complete. 2024-03-26T14:16:57.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.797+0000 7f37089c9740 20 client.38168 trim_cache size 0 max 16384 2024-03-26T14:16:57.816 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.815+0000 7f37089c9740 1 Processor -- start 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.815+0000 7f37089c9740 1 -- start start 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f37089c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 0x55d7ba533a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f37089c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f37089c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba53a9e0 0x55d7ba533f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f37089c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001aea00 con 0x55d7ba53a9e0 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f37089c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001ac500 con 0x55d7ba534e80 2024-03-26T14:16:57.817 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f37089c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001aa580 con 0x55d7ba535870 2024-03-26T14:16:57.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f36fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 0x55d7ba533a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f36fea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 0x55d7ba533a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:55678/0 (socket says 172.21.15.137:55678) 2024-03-26T14:16:57.818 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f36fea00640 1 -- 172.21.15.137:0/2385756036 learned_addr learned my addr 172.21.15.137:0/2385756036 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f36ff400640 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba53a9e0 0x55d7ba533f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.816+0000 7f36fe000640 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36fea00640 1 -- 172.21.15.137:0/2385756036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 msgr2=0x55d7ba5460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36fea00640 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36fea00640 1 -- 172.21.15.137:0/2385756036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba53a9e0 msgr2=0x55d7ba533f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36fea00640 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba53a9e0 0x55d7ba533f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36fea00640 1 -- 172.21.15.137:0/2385756036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7ba16e0f0 con 0x55d7ba534e80 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36ff400640 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba53a9e0 0x55d7ba533f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.817+0000 7f36fea00640 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 0x55d7ba533a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005790 tx=0x7f36f0004ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3071a12026503f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.818+0000 7f36eb400640 1 -- 172.21.15.137:0/2385756036 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00044e0 con 0x55d7ba534e80 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.818+0000 7f36eb400640 1 -- 172.21.15.137:0/2385756036 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36f00fe040 con 0x55d7ba534e80 2024-03-26T14:16:57.819 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.818+0000 7f36eb400640 1 -- 172.21.15.137:0/2385756036 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00bd1f0 con 0x55d7ba534e80 2024-03-26T14:16:57.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 -- 172.21.15.137:0/2385756036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 msgr2=0x55d7ba533a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 0x55d7ba533a30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005790 tx=0x7f36f0004ac0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 -- 172.21.15.137:0/2385756036 shutdown_connections 2024-03-26T14:16:57.820 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba53a9e0 0x55d7ba533f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 --2- 172.21.15.137:0/2385756036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba534e80 0x55d7ba533a30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 -- 172.21.15.137:0/2385756036 >> 172.21.15.137:0/2385756036 conn(0x55d7ba4fd200 msgr2=0x55d7ba530310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 -- 172.21.15.137:0/2385756036 shutdown_connections 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.819+0000 7f37089c9740 1 -- 172.21.15.137:0/2385756036 wait complete. 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 -- start start 2024-03-26T14:16:57.821 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 Processor -- start 2024-03-26T14:16:57.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 0x55d7ba5433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba5448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001aab80 con 0x55d7ba534e80 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001abea0 con 0x55d7ba53a9e0 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001ab1e0 con 0x55d7ba535870 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.820+0000 7f37089c9740 10 client.0 _ll_register_callbacks cb 0x7f36f800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 0x55d7ba5433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36ff400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba5448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 0x55d7ba5433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:59480/0 (socket says 172.21.15.137:59480) 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 -- 172.21.15.137:0/961254875 learned_addr learned my addr 172.21.15.137:0/961254875 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fe000640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 -- 172.21.15.137:0/961254875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 msgr2=0x55d7ba5438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 -- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 msgr2=0x55d7ba5448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba5448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 -- 172.21.15.137:0/961254875 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7ba544ee0 con 0x55d7ba534e80 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36ff400640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba5448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36fea00640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 0x55d7ba5433a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f36f00bd820 tx=0x7f36f004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8d6a9ac871853 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.821+0000 7f36e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f36e8c00640 1 -- 172.21.15.137:0/961254875 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00d6060 con 0x55d7ba534e80 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f36e8c00640 1 -- 172.21.15.137:0/961254875 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f36f00bb930 con 0x55d7ba534e80 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36f00fe040 con 0x55d7ba534e80 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f36e8c00640 1 -- 172.21.15.137:0/961254875 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f00bd3d0 con 0x55d7ba534e80 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f37089c9740 10 client.18663 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:57.825 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d7ba5461f0 con 0x55d7ba534e80 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f36e8c00640 1 -- 172.21.15.137:0/961254875 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f36f00fe040 con 0x55d7ba534e80 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f36e8c00640 1 -- 172.21.15.137:0/961254875 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f36f00ee950 con 0x55d7ba534e80 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f37089c9740 10 client.18663 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f37089c9740 10 client.18663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.822+0000 7f37089c9740 20 client.18663 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.823+0000 7f37089c9740 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba5a6480 0x55d7ba5a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.823+0000 7f37089c9740 4 client.18663 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.823+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7ba4a1ae0 con 0x55d7ba5a6480 2024-03-26T14:16:57.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.823+0000 7f36fe000640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba5a6480 0x55d7ba5a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:57.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f36fe000640 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba5a6480 0x55d7ba5a8870 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad9a405a1d9df86 server_cookie=32937c0a171bb011 in_seq=0 out_seq=0 2024-03-26T14:16:57.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f36e8c00640 10 client.18663 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:57.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f36e8c00640 1 -- 172.21.15.137:0/961254875 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d7ba4a1ae0 con 0x55d7ba5a6480 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f36e8c00640 10 client.18663 handle_command_reply: tid=0 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 client.18663 shutdown 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 msgr2=0x55d7ba5433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 0x55d7ba5433a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f36f00bd820 tx=0x7f36f004f000 comp rx=0 tx=0).stop 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 shutdown_connections 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d7ba5a6480 0x55d7ba5a8870 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7ba53a9e0 0x55d7ba5448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7ba535870 0x55d7ba5438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 --2- 172.21.15.137:0/961254875 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d7ba534e80 0x55d7ba5433a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 >> 172.21.15.137:0/961254875 conn(0x55d7ba4fd200 msgr2=0x55d7ba533a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f36e8c00640 0 client.18663 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.824+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 shutdown_connections 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.825+0000 7f37089c9740 1 -- 172.21.15.137:0/961254875 wait complete. 2024-03-26T14:16:57.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.825+0000 7f37089c9740 20 client.18663 trim_cache size 0 max 16384 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.825+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f36e40a2800 msgr2=0x7f36e40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.825+0000 7f3705c00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f36e40a2800 0x7f36e40a4cc0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55d7ba5360d0 tx=0x7f36f800b320 comp rx=0 tx=0).stop 2024-03-26T14:16:57.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.825+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 msgr2=0x7f3700199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.825+0000 7f3705c00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700199b30 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f36f0005ab0 tx=0x7f36f00042f0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 shutdown_connections 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f36e40a2800 0x7f36e40a4cc0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3700104ad0 0x7f370019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3700105490 0x7f37001a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 --2- 172.21.15.137:0/1125670712 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f37001038e0 0x7f3700199b30 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 >> 172.21.15.137:0/1125670712 conn(0x7f37000fee50 msgr2=0x7f3700107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 shutdown_connections 2024-03-26T14:16:57.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:57.826+0000 7f3705c00640 1 -- 172.21.15.137:0/1125670712 wait complete. 2024-03-26T14:16:57.844 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:16:57.845 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:57.921 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:57.922 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-26T14:16:57.923 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:57.924 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:57.925 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:57.926 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 -- 172.21.15.137:0/2946245901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 msgr2=0x7f86f4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 --2- 172.21.15.137:0/2946245901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f4101540 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009be0 tx=0x7f86ec02bef0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 -- 172.21.15.137:0/2946245901 shutdown_connections 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 --2- 172.21.15.137:0/2946245901 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f4101540 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 --2- 172.21.15.137:0/2946245901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86f4100720 0x7f86f4100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 --2- 172.21.15.137:0/2946245901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 -- 172.21.15.137:0/2946245901 >> 172.21.15.137:0/2946245901 conn(0x7f86f40fb980 msgr2=0x7f86f40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 -- 172.21.15.137:0/2946245901 shutdown_connections 2024-03-26T14:16:58.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.048+0000 7f86f3e00640 1 -- 172.21.15.137:0/2946245901 wait complete. 2024-03-26T14:16:58.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 Processor -- start 2024-03-26T14:16:58.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 -- start start 2024-03-26T14:16:58.051 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86f4100720 0x7f86f4199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f41a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f410f260 con 0x7f86f41010c0 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f410f0e0 con 0x7f86f40ffdc0 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.049+0000 7f86f3e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f410f3e0 con 0x7f86f4100720 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f41a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39632/0 (socket says 172.21.15.137:39632) 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f2a00640 1 -- 172.21.15.137:0/1529924169 learned_addr learned my addr 172.21.15.137:0/1529924169 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f2000640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86f4100720 0x7f86f4199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f3400640 1 -- 172.21.15.137:0/1529924169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86f4100720 msgr2=0x7f86f4199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f3400640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86f4100720 0x7f86f4199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f3400640 1 -- 172.21.15.137:0/1529924169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 msgr2=0x7f86f41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f3400640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f3400640 1 -- 172.21.15.137:0/1529924169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f41a14c0 con 0x7f86f41010c0 2024-03-26T14:16:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.050+0000 7f86f2a00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.051+0000 7f86f3400640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f41a0e50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f86ec036040 tx=0x7f86ec02cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.051+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec043020 con 0x7f86f41010c0 2024-03-26T14:16:58.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.051+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86ec003e20 con 0x7f86f41010c0 2024-03-26T14:16:58.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.051+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec034690 con 0x7f86f41010c0 2024-03-26T14:16:58.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.051+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f41a1750 con 0x7f86f41010c0 2024-03-26T14:16:58.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.051+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f41a1cc0 con 0x7f86f41010c0 2024-03-26T14:16:58.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.053+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f410ab10 con 0x7f86f41010c0 2024-03-26T14:16:58.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.056+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f86ec046020 con 0x7f86f41010c0 2024-03-26T14:16:58.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.057+0000 7f86eb400640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f86d80a2880 0x7f86d80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.057+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f86ec0f1550 con 0x7f86f41010c0 2024-03-26T14:16:58.058 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.057+0000 7f86f2a00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f86d80a2880 0x7f86d80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.059 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.058+0000 7f86f2a00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f86d80a2880 0x7f86d80a4d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f86e40058c0 tx=0x7f86e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.066+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86ec04d020 con 0x7f86f41010c0 2024-03-26T14:16:58.329 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.329+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86f40622a0 con 0x7f86f41010c0 2024-03-26T14:16:58.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.330+0000 7f86eb400640 1 -- 172.21.15.137:0/1529924169 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f86ec0be220 con 0x7f86f41010c0 2024-03-26T14:16:58.331 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:16:58.331 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:16:58.332 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.333+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f86d80a2880 msgr2=0x7f86d80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.333+0000 7f86f3e00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f86d80a2880 0x7f86d80a4d40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f86e40058c0 tx=0x7f86e4009260 comp rx=0 tx=0).stop 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.333+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 msgr2=0x7f86f41a0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.333+0000 7f86f3e00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f41a0e50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f86ec036040 tx=0x7f86ec02cbc0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.333+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 shutdown_connections 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.333+0000 7f86f3e00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f86d80a2880 0x7f86d80a4d40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.334+0000 7f86f3e00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86f41010c0 0x7f86f41a0e50 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.334+0000 7f86f3e00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86f4100720 0x7f86f4199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.334+0000 7f86f3e00640 1 --2- 172.21.15.137:0/1529924169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f86f40ffdc0 0x7f86f41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.334+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 >> 172.21.15.137:0/1529924169 conn(0x7f86f40fb980 msgr2=0x7f86f40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.334+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 shutdown_connections 2024-03-26T14:16:58.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.334+0000 7f86f3e00640 1 -- 172.21.15.137:0/1529924169 wait complete. 2024-03-26T14:16:58.359 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:16:58.359 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:16:58.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 -- 172.21.15.137:0/1422105753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 msgr2=0x7f3378106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 --2- 172.21.15.137:0/1422105753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378106d20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3364009900 tx=0x7f336402bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:58.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 -- 172.21.15.137:0/1422105753 shutdown_connections 2024-03-26T14:16:58.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 --2- 172.21.15.137:0/1422105753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378106d20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 --2- 172.21.15.137:0/1422105753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3378107260 0x7f33781096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 --2- 172.21.15.137:0/1422105753 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 0x7f33781043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 -- 172.21.15.137:0/1422105753 >> 172.21.15.137:0/1422105753 conn(0x7f33780fba10 msgr2=0x7f33780fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 -- 172.21.15.137:0/1422105753 shutdown_connections 2024-03-26T14:16:58.574 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.572+0000 7f3377200640 1 -- 172.21.15.137:0/1422105753 wait complete. 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.573+0000 7f3377200640 1 Processor -- start 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 -- start start 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 0x7f3378199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3378107260 0x7f33781a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337810d150 con 0x7f3378102000 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337810cfd0 con 0x7f3378104930 2024-03-26T14:16:58.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3377200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337810d2d0 con 0x7f3378107260 2024-03-26T14:16:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3375e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 0x7f3378199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3375e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 0x7f3378199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41456/0 (socket says 172.21.15.137:41456) 2024-03-26T14:16:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.574+0000 7f3375e00640 1 -- 172.21.15.137:0/814576623 learned_addr learned my addr 172.21.15.137:0/814576623 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.576 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.575+0000 7f3375400640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.575+0000 7f3376800640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3378107260 0x7f33781a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.575+0000 7f3375400640 1 -- 172.21.15.137:0/814576623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3378107260 msgr2=0x7f33781a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.575+0000 7f3375400640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3378107260 0x7f33781a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.576+0000 7f3375400640 1 -- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 msgr2=0x7f3378199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.576+0000 7f3375400640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 0x7f3378199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.576+0000 7f3375400640 1 -- 172.21.15.137:0/814576623 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33781a1660 con 0x7f3378104930 2024-03-26T14:16:58.577 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.576+0000 7f3375400640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378199f80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3364036040 tx=0x7f3364004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.577+0000 7f336ea00640 1 -- 172.21.15.137:0/814576623 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364043020 con 0x7f3378104930 2024-03-26T14:16:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.577+0000 7f336ea00640 1 -- 172.21.15.137:0/814576623 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33640052b0 con 0x7f3378104930 2024-03-26T14:16:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.577+0000 7f336ea00640 1 -- 172.21.15.137:0/814576623 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364003bb0 con 0x7f3378104930 2024-03-26T14:16:58.578 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.577+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33781a18f0 con 0x7f3378104930 2024-03-26T14:16:58.579 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.577+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33781a1ee0 con 0x7f3378104930 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.584+0000 7f337ffa8740 1 Processor -- start 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.584+0000 7f337ffa8740 1 -- start start 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043874100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x562043875360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x56204387a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f337ffa8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620437dd2c0 con 0x562043874f00 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f337ffa8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x562043785c90 con 0x562043873ce0 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f337ffa8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620437d9360 con 0x5620438758a0 2024-03-26T14:16:58.586 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.585+0000 7f336ea00640 1 -- 172.21.15.137:0/814576623 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3364046020 con 0x7f3378104930 2024-03-26T14:16:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.586+0000 7f3376800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043874100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.586+0000 7f3376800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043874100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39640/0 (socket says 172.21.15.137:39640) 2024-03-26T14:16:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.586+0000 7f3376800640 1 -- 172.21.15.137:0/2755228565 learned_addr learned my addr 172.21.15.137:0/2755228565 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.586+0000 7f3375e00640 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x562043875360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.587 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.586+0000 7f3375400640 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x56204387a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.587+0000 7f3376800640 1 -- 172.21.15.137:0/2755228565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 msgr2=0x56204387a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.587+0000 7f3376800640 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x56204387a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.588 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.587+0000 7f3376800640 1 -- 172.21.15.137:0/2755228565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 msgr2=0x562043875360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.588+0000 7f336ea00640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f335c0a28d0 0x7f335c0a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.588+0000 7f3376800640 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x562043875360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.588+0000 7f3376800640 1 -- 172.21.15.137:0/2755228565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620437aa080 con 0x562043873ce0 2024-03-26T14:16:58.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.588+0000 7f3375e00640 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x562043875360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:58.589 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.588+0000 7f336ea00640 1 -- 172.21.15.137:0/814576623 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f33640f10a0 con 0x7f3378104930 2024-03-26T14:16:58.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.589+0000 7f3376800640 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043874100 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3370009f50 tx=0x7f337000c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f380f5b261052902 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.590 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.589+0000 7f3375400640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f335c0a28d0 0x7f335c0a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.590+0000 7f3363400640 1 -- 172.21.15.137:0/2755228565 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337001a070 con 0x562043873ce0 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.590+0000 7f3363400640 1 -- 172.21.15.137:0/2755228565 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33700158c0 con 0x562043873ce0 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.590+0000 7f3363400640 1 -- 172.21.15.137:0/2755228565 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000a860 con 0x562043873ce0 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.590+0000 7f3375400640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f335c0a28d0 0x7f335c0a4d90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f336402bc70 tx=0x7f33640f4ae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.591+0000 7f337ffa8740 1 -- 172.21.15.137:0/2755228565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 msgr2=0x562043874100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.591+0000 7f337ffa8740 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043874100 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3370009f50 tx=0x7f337000c8b0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 -- 172.21.15.137:0/2755228565 shutdown_connections 2024-03-26T14:16:58.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x56204387a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x562043875360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 --2- 172.21.15.137:0/2755228565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043874100 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 -- 172.21.15.137:0/2755228565 >> 172.21.15.137:0/2755228565 conn(0x56204387c5f0 msgr2=0x56204387c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 -- 172.21.15.137:0/2755228565 shutdown_connections 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.592+0000 7f337ffa8740 1 -- 172.21.15.137:0/2755228565 wait complete. 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.593+0000 7f337ffa8740 1 -- start start 2024-03-26T14:16:58.594 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.593+0000 7f337ffa8740 1 Processor -- start 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x562043893cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x5620438941f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620438758a0 0x56204389b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x56204387a8e0 con 0x562043874f00 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620437990d0 con 0x5620438758a0 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x56204387aa60 con 0x562043873ce0 2024-03-26T14:16:58.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.594+0000 7f337ffa8740 10 client.0 _ll_register_callbacks cb 0x56204379de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:58.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.595+0000 7f3375e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x5620438941f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.595+0000 7f3376800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620438758a0 0x56204389b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.595+0000 7f3375e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x5620438941f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41466/0 (socket says 172.21.15.137:41466) 2024-03-26T14:16:58.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.595+0000 7f3375e00640 1 -- 172.21.15.137:0/619649522 learned_addr learned my addr 172.21.15.137:0/619649522 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.596+0000 7f3376800640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x562043893cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.598+0000 7f3376800640 1 -- 172.21.15.137:0/619649522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 msgr2=0x562043893cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.598+0000 7f3376800640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x562043893cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.598+0000 7f3376800640 1 -- 172.21.15.137:0/619649522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 msgr2=0x5620438941f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.598+0000 7f3376800640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x5620438941f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.598+0000 7f3376800640 1 -- 172.21.15.137:0/619649522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5620434f13f0 con 0x5620438758a0 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.598+0000 7f3375e00640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x5620438941f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:58.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.599+0000 7f3376800640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620438758a0 0x56204389b230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3370009f50 tx=0x7f3370009a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a97fc2a76119f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.599+0000 7f3357e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:16:58.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.599+0000 7f3357e00640 1 -- 172.21.15.137:0/619649522 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370029060 con 0x5620438758a0 2024-03-26T14:16:58.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.600+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33700158c0 con 0x5620438758a0 2024-03-26T14:16:58.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.600+0000 7f3357e00640 1 -- 172.21.15.137:0/619649522 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3370015c80 con 0x5620438758a0 2024-03-26T14:16:58.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.600+0000 7f3357e00640 1 -- 172.21.15.137:0/619649522 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f337001a030 con 0x5620438758a0 2024-03-26T14:16:58.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.600+0000 7f337ffa8740 10 client.38192 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:58.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.600+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5620435bb7d0 con 0x5620438758a0 2024-03-26T14:16:58.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.601+0000 7f3357e00640 1 -- 172.21.15.137:0/619649522 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f33700158c0 con 0x5620438758a0 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f3357e00640 1 -- 172.21.15.137:0/619649522 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3370024960 con 0x5620438758a0 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f337ffa8740 10 client.38192 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f337ffa8740 10 client.38192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f337ffa8740 20 client.38192 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f337ffa8740 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56204389f4a0 0x5620438a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f337ffa8740 4 client.38192 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:58.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.602+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620437ea800 con 0x56204389f4a0 2024-03-26T14:16:58.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.604+0000 7f3375e00640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56204389f4a0 0x5620438a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.606+0000 7f3375e00640 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56204389f4a0 0x5620438a1890 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=245f76125d03ffcf server_cookie=ab6f61740d134492 in_seq=0 out_seq=0 2024-03-26T14:16:58.607 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.606+0000 7f3357e00640 10 client.38192 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.612+0000 7f3357e00640 1 -- 172.21.15.137:0/619649522 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5620437ea800 con 0x56204389f4a0 2024-03-26T14:16:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.612+0000 7f3357e00640 10 client.38192 handle_command_reply: tid=0 2024-03-26T14:16:58.613 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.612+0000 7f337ffa8740 1 client.38192 shutdown 2024-03-26T14:16:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620438758a0 msgr2=0x56204389b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.614 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620438758a0 0x56204389b230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3370009f50 tx=0x7f3370009a50 comp rx=0 tx=0).stop 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 shutdown_connections 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x56204389f4a0 0x5620438a1890 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620438758a0 0x56204389b230 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x5620438941f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 --2- 172.21.15.137:0/619649522 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x562043893cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f3357e00640 0 client.38192 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 >> 172.21.15.137:0/619649522 conn(0x56204387c5f0 msgr2=0x562043873160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 shutdown_connections 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.613+0000 7f337ffa8740 1 -- 172.21.15.137:0/619649522 wait complete. 2024-03-26T14:16:58.615 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.614+0000 7f337ffa8740 20 client.38192 trim_cache size 0 max 16384 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 Processor -- start 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 -- start start 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x56204387ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043874f00 0x56204389b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620438758a0 0x5620438a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781aa310 con 0x5620438758a0 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a7e10 con 0x562043874f00 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.641+0000 7f337ffa8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a5e90 con 0x562043873ce0 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3375e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043874f00 0x56204389b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x56204387ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3375e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043874f00 0x56204389b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39664/0 (socket says 172.21.15.137:39664) 2024-03-26T14:16:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3375e00640 1 -- 172.21.15.137:0/445812645 learned_addr learned my addr 172.21.15.137:0/445812645 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620438758a0 0x5620438a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 -- 172.21.15.137:0/445812645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043874f00 msgr2=0x56204389b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043874f00 0x56204389b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 -- 172.21.15.137:0/445812645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620438758a0 msgr2=0x5620438a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620438758a0 0x5620438a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.642+0000 7f3376800640 1 -- 172.21.15.137:0/445812645 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562043874960 con 0x562043873ce0 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.643+0000 7f3376800640 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x56204387ff60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3370002230 tx=0x7f337000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d18be1eb29d9f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.643+0000 7f3362a00640 1 -- 172.21.15.137:0/445812645 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370029070 con 0x562043873ce0 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.643+0000 7f3362a00640 1 -- 172.21.15.137:0/445812645 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f337001b040 con 0x562043873ce0 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.643+0000 7f3362a00640 1 -- 172.21.15.137:0/445812645 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370024400 con 0x562043873ce0 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 -- 172.21.15.137:0/445812645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 msgr2=0x56204387ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x56204387ff60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3370002230 tx=0x7f337000c8b0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 -- 172.21.15.137:0/445812645 shutdown_connections 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043874f00 0x56204389b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5620438758a0 0x5620438a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 --2- 172.21.15.137:0/445812645 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562043873ce0 0x56204387ff60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 -- 172.21.15.137:0/445812645 >> 172.21.15.137:0/445812645 conn(0x56204387c5f0 msgr2=0x56204387f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 -- 172.21.15.137:0/445812645 shutdown_connections 2024-03-26T14:16:58.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.644+0000 7f337ffa8740 1 -- 172.21.15.137:0/445812645 wait complete. 2024-03-26T14:16:58.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 -- start start 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 Processor -- start 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043899ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x56204389a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x5620438facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a6490 con 0x562043874f00 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a77b0 con 0x562043873ce0 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33781a6af0 con 0x5620438758a0 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.645+0000 7f337ffa8740 10 client.0 _ll_register_callbacks cb 0x7f3370018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043899ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043899ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39678/0 (socket says 172.21.15.137:39678) 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 -- 172.21.15.137:0/3667454856 learned_addr learned my addr 172.21.15.137:0/3667454856 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x5620438facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3375e00640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x56204389a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 -- 172.21.15.137:0/3667454856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 msgr2=0x5620438facc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x5620438facc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 -- 172.21.15.137:0/3667454856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 msgr2=0x56204389a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x56204389a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 -- 172.21.15.137:0/3667454856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56204389aa70 con 0x562043873ce0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3375e00640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x56204389a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3376800640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043899ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f337000a500 tx=0x7f33700070c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9943bc4df72d7ad server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.646+0000 7f3357e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f3357e00640 1 -- 172.21.15.137:0/3667454856 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370029040 con 0x562043873ce0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f337001b040 con 0x562043873ce0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f3357e00640 1 -- 172.21.15.137:0/3667454856 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f337000aba0 con 0x562043873ce0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f3357e00640 1 -- 172.21.15.137:0/3667454856 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3370024400 con 0x562043873ce0 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f337ffa8740 10 client.38198 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:58.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5620438fc350 con 0x562043873ce0 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.647+0000 7f3357e00640 1 -- 172.21.15.137:0/3667454856 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f337001b040 con 0x562043873ce0 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f3357e00640 1 -- 172.21.15.137:0/3667454856 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f3370023aa0 con 0x562043873ce0 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f337ffa8740 10 client.38198 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f337ffa8740 10 client.38198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f337ffa8740 20 client.38198 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f337ffa8740 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5620438fcb70 0x5620438fef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f337ffa8740 4 client.38198 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620437e4b50 con 0x5620438fcb70 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.648+0000 7f3375e00640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5620438fcb70 0x5620438fef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f3357e00640 10 client.38198 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f3375e00640 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5620438fcb70 0x5620438fef90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d589414a76196a8f server_cookie=d537c889adc20117 in_seq=0 out_seq=0 2024-03-26T14:16:58.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f3357e00640 1 -- 172.21.15.137:0/3667454856 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620437e4b50 con 0x5620438fcb70 2024-03-26T14:16:58.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f3357e00640 10 client.38198 handle_command_reply: tid=0 2024-03-26T14:16:58.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 client.38198 shutdown 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 msgr2=0x562043899ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043899ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f337000a500 tx=0x7f33700070c0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 shutdown_connections 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x5620438fcb70 0x5620438fef90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620438758a0 0x5620438facc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562043874f00 0x56204389a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f3357e00640 0 client.38198 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 --2- 172.21.15.137:0/3667454856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562043873ce0 0x562043899ca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.649+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 >> 172.21.15.137:0/3667454856 conn(0x56204387c5f0 msgr2=0x5620438800e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.650+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 shutdown_connections 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.650+0000 7f337ffa8740 1 -- 172.21.15.137:0/3667454856 wait complete. 2024-03-26T14:16:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.650+0000 7f337ffa8740 20 client.38198 trim_cache size 0 max 16384 2024-03-26T14:16:58.655 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:16:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f335c0a28d0 msgr2=0x7f335c0a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f335c0a28d0 0x7f335c0a4d90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f336402bc70 tx=0x7f33640f4ae0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 msgr2=0x7f3378199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378199f80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3364036040 tx=0x7f3364004270 comp rx=0 tx=0).stop 2024-03-26T14:16:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 shutdown_connections 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f335c0a28d0 0x7f335c0a4d90 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3378104930 0x7f3378199f80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3378107260 0x7f33781a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 --2- 172.21.15.137:0/814576623 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3378102000 0x7f3378199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 >> 172.21.15.137:0/814576623 conn(0x7f33780fba10 msgr2=0x7f3378101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 shutdown_connections 2024-03-26T14:16:58.656 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.651+0000 7f3377200640 1 -- 172.21.15.137:0/814576623 wait complete. 2024-03-26T14:16:58.670 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:16:58.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:16:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 -- 172.21.15.137:0/1075665257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 msgr2=0x7feee4103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 --2- 172.21.15.137:0/1075665257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 0x7feee4103d00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402bc30 comp rx=0 tx=0).stop 2024-03-26T14:16:58.873 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 -- 172.21.15.137:0/1075665257 shutdown_connections 2024-03-26T14:16:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 --2- 172.21.15.137:0/1075665257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4104ad0 0x7feee4104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 --2- 172.21.15.137:0/1075665257 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee4105490 0x7feee410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 --2- 172.21.15.137:0/1075665257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 0x7feee4103d00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 -- 172.21.15.137:0/1075665257 >> 172.21.15.137:0/1075665257 conn(0x7feee40fee50 msgr2=0x7feee4101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 -- 172.21.15.137:0/1075665257 shutdown_connections 2024-03-26T14:16:58.874 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.872+0000 7feee8a00640 1 -- 172.21.15.137:0/1075665257 wait complete. 2024-03-26T14:16:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.873+0000 7feee8a00640 1 Processor -- start 2024-03-26T14:16:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 -- start start 2024-03-26T14:16:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 0x7feee4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee4104ad0 0x7feee419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 0x7feee41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.875 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee410f530 con 0x7feee41038e0 2024-03-26T14:16:58.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee410f3b0 con 0x7feee4105490 2024-03-26T14:16:58.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee8a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee410f6b0 con 0x7feee4104ad0 2024-03-26T14:16:58.876 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 0x7feee4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 0x7feee41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.874+0000 7feee2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 0x7feee41a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39706/0 (socket says 172.21.15.137:39706) 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee2800640 1 -- 172.21.15.137:0/4098588273 learned_addr learned my addr 172.21.15.137:0/4098588273 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee1400640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee4104ad0 0x7feee419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee2800640 1 -- 172.21.15.137:0/4098588273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee4104ad0 msgr2=0x7feee419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee2800640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee4104ad0 0x7feee419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee2800640 1 -- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 msgr2=0x7feee4199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee2800640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 0x7feee4199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.875+0000 7feee2800640 1 -- 172.21.15.137:0/4098588273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee41a18c0 con 0x7feee4105490 2024-03-26T14:16:58.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.876+0000 7feee2800640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 0x7feee41a1220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b970 tx=0x7feedc00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.876+0000 7feedaa00640 1 -- 172.21.15.137:0/4098588273 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc019070 con 0x7feee4105490 2024-03-26T14:16:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.876+0000 7feedaa00640 1 -- 172.21.15.137:0/4098588273 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feedc014440 con 0x7feee4105490 2024-03-26T14:16:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.876+0000 7feedaa00640 1 -- 172.21.15.137:0/4098588273 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc0075d0 con 0x7feee4105490 2024-03-26T14:16:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.877+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee41a1bb0 con 0x7feee4105490 2024-03-26T14:16:58.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.877+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee41a2100 con 0x7feee4105490 2024-03-26T14:16:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.880+0000 7feeeb7d4740 1 Processor -- start 2024-03-26T14:16:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.880+0000 7feedaa00640 1 -- 172.21.15.137:0/4098588273 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7feedc01c070 con 0x7feee4105490 2024-03-26T14:16:58.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 -- start start 2024-03-26T14:16:58.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a93c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7a94e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7a9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a79f82c0 con 0x55b9a7a949e0 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a79a0c90 con 0x55b9a7a953a0 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a79f4360 con 0x55b9a7a93810 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feee1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7a94e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feee1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7a94e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41502/0 (socket says 172.21.15.137:41502) 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feee1e00640 1 -- 172.21.15.137:0/1526531478 learned_addr learned my addr 172.21.15.137:0/1526531478 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.881+0000 7feee1400640 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7a9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1400640 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a93c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1e00640 1 -- 172.21.15.137:0/1526531478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 msgr2=0x55b9a7a93c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1e00640 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a93c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1e00640 1 -- 172.21.15.137:0/1526531478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 msgr2=0x55b9a7a9bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1e00640 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7a9bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1e00640 1 -- 172.21.15.137:0/1526531478 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a79c5080 con 0x55b9a7a949e0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feee1400640 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7a9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feedaa00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feec80a2800 0x7feec80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.882+0000 7feedaa00640 1 -- 172.21.15.137:0/4098588273 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7feedc0ce400 con 0x7feee4105490 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.883+0000 7feee1400640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feec80a2800 0x7feec80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.883+0000 7feee1e00640 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7a94e60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7feed4005ab0 tx=0x7feed402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea1efe558948929 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.883+0000 7feecf000640 1 -- 172.21.15.137:0/1526531478 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4045020 con 0x55b9a7a949e0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.883+0000 7feecf000640 1 -- 172.21.15.137:0/1526531478 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feed4005040 con 0x55b9a7a949e0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.883+0000 7feecf000640 1 -- 172.21.15.137:0/1526531478 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4004920 con 0x55b9a7a949e0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 -- 172.21.15.137:0/1526531478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 msgr2=0x55b9a7a94e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7a94e60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7feed4005ab0 tx=0x7feed402ca10 comp rx=0 tx=0).stop 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feee1400640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feec80a2800 0x7feec80a4cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feed0006fd0 tx=0x7feed000a230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 -- 172.21.15.137:0/1526531478 shutdown_connections 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7a9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7a94e60 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/1526531478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a93c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 -- 172.21.15.137:0/1526531478 >> 172.21.15.137:0/1526531478 conn(0x55b9a7a8ec00 msgr2=0x55b9a7a8f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 -- 172.21.15.137:0/1526531478 shutdown_connections 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.884+0000 7feeeb7d4740 1 -- 172.21.15.137:0/1526531478 wait complete. 2024-03-26T14:16:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 -- start start 2024-03-26T14:16:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 Processor -- start 2024-03-26T14:16:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7aa6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7aad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a7aa4b60 con 0x55b9a7a949e0 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a79b40d0 con 0x55b9a7a953a0 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a7aa4ce0 con 0x55b9a7a93810 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feeeb7d4740 10 client.0 _ll_register_callbacks cb 0x55b9a79b8e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feee1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feee1400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41504/0 (socket says 172.21.15.137:41504) 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.885+0000 7feee1400640 1 -- 172.21.15.137:0/3691518068 learned_addr learned my addr 172.21.15.137:0/3691518068 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee2800640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7aad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1e00640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7aa6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1400640 1 -- 172.21.15.137:0/3691518068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 msgr2=0x55b9a7aa6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1400640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7aa6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1400640 1 -- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 msgr2=0x55b9a7aad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1400640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7aad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1400640 1 -- 172.21.15.137:0/3691518068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9a770c3f0 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feee1400640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa6660 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7feed0016fd0 tx=0x7feed00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7185214bf0bf69b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feec3400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feec3400640 1 -- 172.21.15.137:0/3691518068 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0028060 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feec3400640 1 -- 172.21.15.137:0/3691518068 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7feed000f050 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.886+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feed4005040 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feec3400640 1 -- 172.21.15.137:0/3691518068 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feed002f050 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feeeb7d4740 10 client.18696 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b9a77d67d0 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feec3400640 1 -- 172.21.15.137:0/3691518068 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feed4005040 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feec3400640 1 -- 172.21.15.137:0/3691518068 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7feed0015040 con 0x55b9a7a949e0 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feeeb7d4740 10 client.18696 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.887+0000 7feeeb7d4740 10 client.18696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.888+0000 7feeeb7d4740 20 client.18696 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.888+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7ab1bd0 0x55b9a7ab4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.888+0000 7feeeb7d4740 4 client.18696 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.888+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9a7a05800 con 0x55b9a7ab1bd0 2024-03-26T14:16:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.888+0000 7feee1e00640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7ab1bd0 0x55b9a7ab4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.889+0000 7feee1e00640 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7ab1bd0 0x55b9a7ab4070 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6853ca3108b1386c server_cookie=843701a85dc1c5b9 in_seq=0 out_seq=0 2024-03-26T14:16:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.889+0000 7feec3400640 10 client.18696 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feec3400640 1 -- 172.21.15.137:0/3691518068 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55b9a7a05800 con 0x55b9a7ab1bd0 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feec3400640 10 client.18696 handle_command_reply: tid=0 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 client.18696 shutdown 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 msgr2=0x55b9a7aa6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa6660 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7feed0016fd0 tx=0x7feed00047d0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 shutdown_connections 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7ab1bd0 0x55b9a7ab4070 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a953a0 0x55b9a7aad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa6660 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feec3400640 0 client.18696 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/3691518068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7aa6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 >> 172.21.15.137:0/3691518068 conn(0x55b9a7a8ec00 msgr2=0x55b9a7a8deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 shutdown_connections 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 1 -- 172.21.15.137:0/3691518068 wait complete. 2024-03-26T14:16:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.890+0000 7feeeb7d4740 20 client.18696 trim_cache size 0 max 16384 2024-03-26T14:16:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.906+0000 7feeeb7d4740 1 Processor -- start 2024-03-26T14:16:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.906+0000 7feeeb7d4740 1 -- start start 2024-03-26T14:16:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.906+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a8d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.908 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.906+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a949e0 0x55b9a7aad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.906+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7a8db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.906+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41aa590 con 0x55b9a7a953a0 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41a8090 con 0x55b9a7a949e0 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41a6110 con 0x55b9a7a93810 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feee1e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a8d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feee1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a949e0 0x55b9a7aad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feee2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7a8db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feee1e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a8d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.137:52126/0 (socket says 172.21.15.137:52126) 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feee1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a949e0 0x55b9a7aad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39730/0 (socket says 172.21.15.137:39730) 2024-03-26T14:16:58.909 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.907+0000 7feee1e00640 1 -- 172.21.15.137:0/295290361 learned_addr learned my addr 172.21.15.137:0/295290361 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee1e00640 1 -- 172.21.15.137:0/295290361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a949e0 msgr2=0x55b9a7aad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee1e00640 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a949e0 0x55b9a7aad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee1e00640 1 -- 172.21.15.137:0/295290361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 msgr2=0x55b9a7a8db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee1e00640 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7a8db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee1e00640 1 -- 172.21.15.137:0/295290361 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a7a94460 con 0x55b9a7a93810 2024-03-26T14:16:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee2800640 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7a8db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.908+0000 7feee1e00640 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a8d620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feed402c210 tx=0x7feed4009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e922e879c11120e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.909+0000 7feece600640 1 -- 172.21.15.137:0/295290361 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4050070 con 0x55b9a7a93810 2024-03-26T14:16:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.909+0000 7feece600640 1 -- 172.21.15.137:0/295290361 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feed40035f0 con 0x55b9a7a93810 2024-03-26T14:16:58.911 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.909+0000 7feece600640 1 -- 172.21.15.137:0/295290361 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed4045040 con 0x55b9a7a93810 2024-03-26T14:16:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 -- 172.21.15.137:0/295290361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 msgr2=0x55b9a7a8d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a8d620 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feed402c210 tx=0x7feed4009650 comp rx=0 tx=0).stop 2024-03-26T14:16:58.914 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 -- 172.21.15.137:0/295290361 shutdown_connections 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a949e0 0x55b9a7aad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7a8db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/295290361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a93810 0x55b9a7a8d620 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 -- 172.21.15.137:0/295290361 >> 172.21.15.137:0/295290361 conn(0x55b9a7a8ec00 msgr2=0x55b9a7a926f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 -- 172.21.15.137:0/295290361 shutdown_connections 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 -- 172.21.15.137:0/295290361 wait complete. 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.910+0000 7feeeb7d4740 1 -- start start 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 Processor -- start 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a93810 0x55b9a7aa94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7aaca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41a6710 con 0x55b9a7a953a0 2024-03-26T14:16:58.915 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41a7a30 con 0x55b9a7a93810 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee41a6d70 con 0x55b9a7a949e0 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feeeb7d4740 10 client.0 _ll_register_callbacks cb 0x7feed001a860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.911+0000 7feee2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7aaca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7aaca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41520/0 (socket says 172.21.15.137:41520) 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 -- 172.21.15.137:0/712202039 learned_addr learned my addr 172.21.15.137:0/712202039 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee1e00640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a93810 0x55b9a7aa94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee1400640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 -- 172.21.15.137:0/712202039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a949e0 msgr2=0x55b9a7aa9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 -- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a93810 msgr2=0x55b9a7aa94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a93810 0x55b9a7aa94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 -- 172.21.15.137:0/712202039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9a7aad010 con 0x55b9a7a953a0 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee1e00640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a93810 0x55b9a7aa94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:58.916 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feee2800640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7aaca20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7feedc0cba90 tx=0x7feedc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7504698718b391 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feec3400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feec3400640 1 -- 172.21.15.137:0/712202039 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc098e00 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feec3400640 1 -- 172.21.15.137:0/712202039 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7feedc0a2ca0 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.912+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feed40035f0 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feec3400640 1 -- 172.21.15.137:0/712202039 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7feedc0a2ea0 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 10 client.18702 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b9a7b1ed20 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feec3400640 1 -- 172.21.15.137:0/712202039 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feed40035f0 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feec3400640 1 -- 172.21.15.137:0/712202039 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7feedc096790 con 0x55b9a7a953a0 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 10 client.18702 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 10 client.18702 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 20 client.18702 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7b1f680 0x55b9a7b21aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:58.917 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 4 client.18702 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:16:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.913+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b9a79ffb50 con 0x55b9a7b1f680 2024-03-26T14:16:58.918 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.914+0000 7feee1e00640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7b1f680 0x55b9a7b21aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.914+0000 7feee1e00640 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7b1f680 0x55b9a7b21aa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15be20c193b8d463 server_cookie=ad3f1ec9e103034a in_seq=0 out_seq=0 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.914+0000 7feec3400640 10 client.18702 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feec3400640 1 -- 172.21.15.137:0/712202039 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b9a79ffb50 con 0x55b9a7b1f680 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feec3400640 10 client.18702 handle_command_reply: tid=0 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 client.18702 shutdown 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 msgr2=0x55b9a7aaca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7aaca20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7feedc0cba90 tx=0x7feedc004a40 comp rx=0 tx=0).stop 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 shutdown_connections 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55b9a7b1f680 0x55b9a7b21aa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55b9a7a953a0 0x55b9a7aaca20 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b9a7a949e0 0x55b9a7aa9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 --2- 172.21.15.137:0/712202039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9a7a93810 0x55b9a7aa94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 >> 172.21.15.137:0/712202039 conn(0x55b9a7a8ec00 msgr2=0x55b9a7a97c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feec3400640 0 client.18702 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 shutdown_connections 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 1 -- 172.21.15.137:0/712202039 wait complete. 2024-03-26T14:16:58.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.915+0000 7feeeb7d4740 20 client.18702 trim_cache size 0 max 16384 2024-03-26T14:16:58.921 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:16:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feec80a2800 msgr2=0x7feec80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feec80a2800 0x7feec80a4cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7feed0006fd0 tx=0x7feed000a230 comp rx=0 tx=0).stop 2024-03-26T14:16:58.921 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 msgr2=0x7feee41a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 0x7feee41a1220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feedc00b970 tx=0x7feedc00be30 comp rx=0 tx=0).stop 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 shutdown_connections 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7feec80a2800 0x7feec80a4cc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4105490 0x7feee41a1220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee4104ad0 0x7feee419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 --2- 172.21.15.137:0/4098588273 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7feee41038e0 0x7feee4199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 >> 172.21.15.137:0/4098588273 conn(0x7feee40fee50 msgr2=0x7feee4107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 shutdown_connections 2024-03-26T14:16:58.922 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:58.916+0000 7feee8a00640 1 -- 172.21.15.137:0/4098588273 wait complete. 2024-03-26T14:16:58.936 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:16:58.936 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 -- 172.21.15.137:0/1743183602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 msgr2=0x7ff5341089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 --2- 172.21.15.137:0/1743183602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 0x7ff5341089f0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009ca0 tx=0x7ff52c02c100 comp rx=0 tx=0).stop 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 -- 172.21.15.137:0/1743183602 shutdown_connections 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 --2- 172.21.15.137:0/1743183602 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 0x7ff5341089f0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 --2- 172.21.15.137:0/1743183602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5341017a0 0x7ff534101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 --2- 172.21.15.137:0/1743183602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341005b0 0x7ff5341009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.137+0000 7ff53c400640 1 -- 172.21.15.137:0/1743183602 >> 172.21.15.137:0/1743183602 conn(0x7ff5340fbb40 msgr2=0x7ff5340fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:59.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.138+0000 7ff53c400640 1 -- 172.21.15.137:0/1743183602 shutdown_connections 2024-03-26T14:16:59.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.138+0000 7ff53c400640 1 -- 172.21.15.137:0/1743183602 wait complete. 2024-03-26T14:16:59.142 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.139+0000 7ff53c400640 1 Processor -- start 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.139+0000 7ff53c400640 1 -- start start 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.139+0000 7ff53c400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5341005b0 0x7ff534199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff53c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 0x7ff534199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff53c400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 0x7ff5341a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff53c400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53410c0e0 con 0x7ff534102160 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff53c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53410bf60 con 0x7ff5341017a0 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff53c400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53410c260 con 0x7ff5341005b0 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff53a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 0x7ff5341a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 0x7ff534199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 0x7ff534199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39768/0 (socket says 172.21.15.137:39768) 2024-03-26T14:16:59.143 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 -- 172.21.15.137:0/1456542265 learned_addr learned my addr 172.21.15.137:0/1456542265 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539a00640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5341005b0 0x7ff534199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 -- 172.21.15.137:0/1456542265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5341005b0 msgr2=0x7ff534199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5341005b0 0x7ff534199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 -- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 msgr2=0x7ff5341a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 0x7ff5341a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.140+0000 7ff539000640 1 -- 172.21.15.137:0/1456542265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5341a16d0 con 0x7ff5341017a0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.141+0000 7ff539000640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 0x7ff534199ff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff52000d770 tx=0x7ff52000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.141+0000 7ff532000640 1 -- 172.21.15.137:0/1456542265 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520017070 con 0x7ff5341017a0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.141+0000 7ff532000640 1 -- 172.21.15.137:0/1456542265 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff5200054d0 con 0x7ff5341017a0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.141+0000 7ff532000640 1 -- 172.21.15.137:0/1456542265 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520004cf0 con 0x7ff5341017a0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.141+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5341a19c0 con 0x7ff5341017a0 2024-03-26T14:16:59.144 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.141+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5341a1ec0 con 0x7ff5341017a0 2024-03-26T14:16:59.147 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.145+0000 7ff53f22a740 1 Processor -- start 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.147+0000 7ff532000640 1 -- 172.21.15.137:0/1456542265 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff520004e90 con 0x7ff5341017a0 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.147+0000 7ff53f22a740 1 -- start start 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.147+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d45bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.147+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45c9e0 0x55725d45ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d463c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53f22a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725d3c02c0 con 0x55725d45b810 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53f22a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725d368c90 con 0x55725d45d3a0 2024-03-26T14:16:59.149 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53f22a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725d3bc360 con 0x55725d45c9e0 2024-03-26T14:16:59.150 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d45bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d45bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41530/0 (socket says 172.21.15.137:41530) 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff53a400640 1 -- 172.21.15.137:0/1721769507 learned_addr learned my addr 172.21.15.137:0/1721769507 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.148+0000 7ff539a00640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45c9e0 0x55725d45ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff53a400640 1 -- 172.21.15.137:0/1721769507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45c9e0 msgr2=0x55725d45ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff539000640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d463c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff53a400640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45c9e0 0x55725d45ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff53a400640 1 -- 172.21.15.137:0/1721769507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 msgr2=0x55725d463c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff53a400640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d463c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff53a400640 1 -- 172.21.15.137:0/1721769507 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725d38d080 con 0x55725d45b810 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff539000640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d463c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:59.152 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff539a00640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45c9e0 0x55725d45ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff532000640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff51c0a2800 0x7ff51c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff532000640 1 -- 172.21.15.137:0/1456542265 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff5200ce7e0 con 0x7ff5341017a0 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.149+0000 7ff53a400640 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d45bc30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02c710 tx=0x7ff52c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65b5503b2eb0b86 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.150+0000 7ff539a00640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff51c0a2800 0x7ff51c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.150+0000 7ff526a00640 1 -- 172.21.15.137:0/1721769507 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c043070 con 0x55725d45b810 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.150+0000 7ff526a00640 1 -- 172.21.15.137:0/1721769507 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff52c032dc0 con 0x55725d45b810 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.150+0000 7ff526a00640 1 -- 172.21.15.137:0/1721769507 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c039040 con 0x55725d45b810 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 -- 172.21.15.137:0/1721769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 msgr2=0x55725d45bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff539a00640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff51c0a2800 0x7ff51c0a4cc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff534101600 tx=0x7ff528005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:59.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d45bc30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02c710 tx=0x7ff52c009800 comp rx=0 tx=0).stop 2024-03-26T14:16:59.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 -- 172.21.15.137:0/1721769507 shutdown_connections 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d463c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45c9e0 0x55725d45ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1721769507 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d45bc30 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 -- 172.21.15.137:0/1721769507 >> 172.21.15.137:0/1721769507 conn(0x55725d456c00 msgr2=0x55725d457000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 -- 172.21.15.137:0/1721769507 shutdown_connections 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.151+0000 7ff53f22a740 1 -- 172.21.15.137:0/1721769507 wait complete. 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.152+0000 7ff53f22a740 1 -- start start 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.152+0000 7ff53f22a740 1 Processor -- start 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.152+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d465a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45c9e0 0x55725d465f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45d3a0 0x55725d46cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53f22a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725d464350 con 0x55725d45b810 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53f22a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725d37c0d0 con 0x55725d45c9e0 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53f22a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55725d4644d0 con 0x55725d45d3a0 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53f22a740 10 client.0 _ll_register_callbacks cb 0x55725d380e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff539a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d465a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff539a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d465a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41540/0 (socket says 172.21.15.137:41540) 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff539a00640 1 -- 172.21.15.137:0/1751278407 learned_addr learned my addr 172.21.15.137:0/1751278407 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff539000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45c9e0 0x55725d465f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.153+0000 7ff53a400640 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45d3a0 0x55725d46cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff539000640 1 -- 172.21.15.137:0/1751278407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45d3a0 msgr2=0x55725d46cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff539000640 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45d3a0 0x55725d46cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff539000640 1 -- 172.21.15.137:0/1751278407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 msgr2=0x55725d465a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff539000640 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d465a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff539000640 1 -- 172.21.15.137:0/1751278407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55725d0d43f0 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff539000640 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45c9e0 0x55725d465f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5200cbf10 tx=0x7ff52009ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef98b5e3673bc62a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff51b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.154+0000 7ff51b400640 1 -- 172.21.15.137:0/1751278407 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520020070 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52c032dc0 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff51b400640 1 -- 172.21.15.137:0/1751278407 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff52009a2b0 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff51b400640 1 -- 172.21.15.137:0/1751278407 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff520009c30 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff53f22a740 10 client.38219 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55725d19e7d0 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff51b400640 1 -- 172.21.15.137:0/1751278407 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff52c032dc0 con 0x55725d45c9e0 2024-03-26T14:16:59.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.155+0000 7ff51b400640 1 -- 172.21.15.137:0/1751278407 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff5200d0ce0 con 0x55725d45c9e0 2024-03-26T14:16:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff53f22a740 10 client.38219 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff53f22a740 10 client.38219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff53f22a740 20 client.38219 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4713d0 0x55725d473830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff53f22a740 4 client.38219 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:16:59.160 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55725d3cd800 con 0x55725d4713d0 2024-03-26T14:16:59.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.156+0000 7ff539a00640 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4713d0 0x55725d473830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.157+0000 7ff539a00640 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4713d0 0x55725d473830 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f6ac99e9a220ad server_cookie=e86fb97bdc3990c1 in_seq=0 out_seq=0 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.157+0000 7ff51b400640 10 client.38219 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.158+0000 7ff51b400640 1 -- 172.21.15.137:0/1751278407 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55725d3cd800 con 0x55725d4713d0 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.158+0000 7ff51b400640 10 client.38219 handle_command_reply: tid=0 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.158+0000 7ff53f22a740 1 client.38219 shutdown 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45c9e0 msgr2=0x55725d465f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45c9e0 0x55725d465f70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff5200cbf10 tx=0x7ff52009ad20 comp rx=0 tx=0).stop 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 shutdown_connections 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4713d0 0x55725d473830 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45c9e0 0x55725d465f70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45d3a0 0x55725d46cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 --2- 172.21.15.137:0/1751278407 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45b810 0x55725d465a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.162 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff51b400640 0 client.38219 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 >> 172.21.15.137:0/1751278407 conn(0x55725d456c00 msgr2=0x55725d455eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 shutdown_connections 2024-03-26T14:16:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 1 -- 172.21.15.137:0/1751278407 wait complete. 2024-03-26T14:16:59.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.159+0000 7ff53f22a740 20 client.38219 trim_cache size 0 max 16384 2024-03-26T14:16:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.173+0000 7ff53f22a740 1 Processor -- start 2024-03-26T14:16:59.175 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 -- start start 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d45a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d45fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341ae810 con 0x55725d45c9e0 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341ac310 con 0x55725d45d3a0 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53f22a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341aa390 con 0x55725d45b810 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff539000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d45fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d45fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39796/0 (socket says 172.21.15.137:39796) 2024-03-26T14:16:59.176 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff53a400640 1 -- 172.21.15.137:0/3522645634 learned_addr learned my addr 172.21.15.137:0/3522645634 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.174+0000 7ff539a00640 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d45a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff53a400640 1 -- 172.21.15.137:0/3522645634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 msgr2=0x55725d45a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff53a400640 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d45a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff53a400640 1 -- 172.21.15.137:0/3522645634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 msgr2=0x55725d46cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff53a400640 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.177 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff53a400640 1 -- 172.21.15.137:0/3522645634 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55725d45c460 con 0x55725d45d3a0 2024-03-26T14:16:59.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff53a400640 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d45fc10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff52c005e30 tx=0x7ff52c032340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3651a2c279e34ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:59.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.175+0000 7ff526000640 1 -- 172.21.15.137:0/3522645634 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c050070 con 0x55725d45d3a0 2024-03-26T14:16:59.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.176+0000 7ff526000640 1 -- 172.21.15.137:0/3522645634 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff52c003970 con 0x55725d45d3a0 2024-03-26T14:16:59.179 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.176+0000 7ff526000640 1 -- 172.21.15.137:0/3522645634 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c039030 con 0x55725d45d3a0 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.176+0000 7ff53f22a740 1 -- 172.21.15.137:0/3522645634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 msgr2=0x55725d45fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.176+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d45fc10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff52c005e30 tx=0x7ff52c032340 comp rx=0 tx=0).stop 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 -- 172.21.15.137:0/3522645634 shutdown_connections 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d45fc10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3522645634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d45a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 -- 172.21.15.137:0/3522645634 >> 172.21.15.137:0/3522645634 conn(0x55725d456c00 msgr2=0x55725d459e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 -- 172.21.15.137:0/3522645634 shutdown_connections 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 -- 172.21.15.137:0/3522645634 wait complete. 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 -- start start 2024-03-26T14:16:59.180 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.177+0000 7ff53f22a740 1 Processor -- start 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d46ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d46c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341aa990 con 0x55725d45c9e0 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341abcb0 con 0x55725d45d3a0 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341aaff0 con 0x55725d45b810 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53f22a740 10 client.0 _ll_register_callbacks cb 0x7ff520009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff539000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff539000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41552/0 (socket says 172.21.15.137:41552) 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff53a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d46c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff539000640 1 -- 172.21.15.137:0/3750121601 learned_addr learned my addr 172.21.15.137:0/3750121601 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:16:59.181 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.178+0000 7ff539a00640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d46ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff53a400640 1 -- 172.21.15.137:0/3750121601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 msgr2=0x55725d46ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff53a400640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d46ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff53a400640 1 -- 172.21.15.137:0/3750121601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 msgr2=0x55725d46b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff53a400640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff53a400640 1 -- 172.21.15.137:0/3750121601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55725d46c700 con 0x55725d45d3a0 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff539000640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:16:59.184 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff539a00640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d46ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff53a400640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d46c110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02e040 tx=0x7ff52c005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2db67fcf9a54ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.179+0000 7ff51b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.180+0000 7ff51b400640 1 -- 172.21.15.137:0/3750121601 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c058060 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.180+0000 7ff51b400640 1 -- 172.21.15.137:0/3750121601 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff52c0327d0 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.181+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52c003970 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.181+0000 7ff51b400640 1 -- 172.21.15.137:0/3750121601 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff52c050070 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.181+0000 7ff53f22a740 10 client.38225 fetch_fsmap learned FSMap version 24 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.181+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55725d4cd260 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff51b400640 1 -- 172.21.15.137:0/3750121601 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff52c003970 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff51b400640 1 -- 172.21.15.137:0/3750121601 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7ff52c048740 con 0x55725d45d3a0 2024-03-26T14:16:59.185 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff53f22a740 10 client.38225 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:16:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff53f22a740 10 client.38225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:16:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff53f22a740 20 client.38225 populate_metadata read hostname 'smithi137' 2024-03-26T14:16:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4cd630 0x55725d4cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:16:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff53f22a740 4 client.38225 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:16:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.182+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55725d3c7b50 con 0x55725d4cd630 2024-03-26T14:16:59.186 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.183+0000 7ff539a00640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4cd630 0x55725d4cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:16:59.187 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff539a00640 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4cd630 0x55725d4cfa20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df02b2ded861a4 server_cookie=822f3f124ef0c248 in_seq=0 out_seq=0 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff51b400640 10 client.38225 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff51b400640 1 -- 172.21.15.137:0/3750121601 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55725d3c7b50 con 0x55725d4cd630 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff51b400640 10 client.38225 handle_command_reply: tid=0 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 client.38225 shutdown 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 msgr2=0x55725d46c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d46c110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff52c02e040 tx=0x7ff52c005d80 comp rx=0 tx=0).stop 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 shutdown_connections 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55725d4cd630 0x55725d4cfa20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55725d45d3a0 0x55725d46c110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.188 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55725d45c9e0 0x55725d46b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 --2- 172.21.15.137:0/3750121601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55725d45b810 0x55725d46ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff51b400640 0 client.38225 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 >> 172.21.15.137:0/3750121601 conn(0x55725d456c00 msgr2=0x55725d45a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 shutdown_connections 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.184+0000 7ff53f22a740 1 -- 172.21.15.137:0/3750121601 wait complete. 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53f22a740 20 client.38225 trim_cache size 0 max 16384 2024-03-26T14:16:59.189 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:16:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff51c0a2800 msgr2=0x7ff51c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff51c0a2800 0x7ff51c0a4cc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff534101600 tx=0x7ff528005f90 comp rx=0 tx=0).stop 2024-03-26T14:16:59.190 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 msgr2=0x7ff534199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 0x7ff534199ff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff52000d770 tx=0x7ff52000dc30 comp rx=0 tx=0).stop 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 shutdown_connections 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff51c0a2800 0x7ff51c0a4cc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5341017a0 0x7ff534199ff0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff534102160 0x7ff5341a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 --2- 172.21.15.137:0/1456542265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5341005b0 0x7ff534199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.185+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 >> 172.21.15.137:0/1456542265 conn(0x7ff5340fbb40 msgr2=0x7ff5340fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.186+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 shutdown_connections 2024-03-26T14:16:59.191 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:16:59.186+0000 7ff53c400640 1 -- 172.21.15.137:0/1456542265 wait complete. 2024-03-26T14:16:59.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:59 smithi137 ceph-mon[31539]: pgmap v896: 129 pgs: 129 active+clean; 134 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 456 KiB/s wr, 113 op/s 2024-03-26T14:16:59.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:16:59 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1529924169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:16:59.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:59 smithi179 ceph-mon[35472]: pgmap v896: 129 pgs: 129 active+clean; 134 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 456 KiB/s wr, 113 op/s 2024-03-26T14:16:59.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:16:59 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1529924169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:16:59.694 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:59.695 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:59.696 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:59.697 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:59.698 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:59.699 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:59.700 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-26T14:16:59.701 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-03-26T14:16:59.702 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-26T14:16:59.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:59 smithi159 ceph-mon[35501]: pgmap v896: 129 pgs: 129 active+clean; 134 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 456 KiB/s wr, 113 op/s 2024-03-26T14:16:59.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:16:59 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1529924169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:17:00.205 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:17:00.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:17:00.205 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:17:00.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 -- 172.21.15.137:0/1567050562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc104930 msgr2=0x7f4efc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1567050562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc104930 0x7f4efc106d20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009860 tx=0x7f4ee402bb70 comp rx=0 tx=0).stop 2024-03-26T14:17:00.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 -- 172.21.15.137:0/1567050562 shutdown_connections 2024-03-26T14:17:00.430 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1567050562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4efc107260 0x7f4efc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1567050562 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc104930 0x7f4efc106d20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1567050562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4efc102000 0x7f4efc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 -- 172.21.15.137:0/1567050562 >> 172.21.15.137:0/1567050562 conn(0x7f4efc0fba10 msgr2=0x7f4efc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 -- 172.21.15.137:0/1567050562 shutdown_connections 2024-03-26T14:17:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.429+0000 7f4f00a00640 1 -- 172.21.15.137:0/1567050562 wait complete. 2024-03-26T14:17:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.430+0000 7f4f00a00640 1 Processor -- start 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 -- start start 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4efc102000 0x7f4efc1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4efc104930 0x7f4efc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 0x7f4efc192c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc10d0c0 con 0x7f4efc107260 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc10cf40 con 0x7f4efc104930 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4f00a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc10d240 con 0x7f4efc102000 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4efb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 0x7f4efc192c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4efb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 0x7f4efc192c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41572/0 (socket says 172.21.15.137:41572) 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4efb200640 1 -- 172.21.15.137:0/1136237743 learned_addr learned my addr 172.21.15.137:0/1136237743 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.431+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4efc104930 0x7f4efc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efa800640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4efc102000 0x7f4efc1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efb200640 1 -- 172.21.15.137:0/1136237743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4efc102000 msgr2=0x7f4efc1960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efb200640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4efc102000 0x7f4efc1960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efb200640 1 -- 172.21.15.137:0/1136237743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4efc104930 msgr2=0x7f4efc196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efb200640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4efc104930 0x7f4efc196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efb200640 1 -- 172.21.15.137:0/1136237743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc193520 con 0x7f4efc107260 2024-03-26T14:17:00.434 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4efb200640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 0x7f4efc192c60 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400eaa0 tx=0x7f4ef400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4ef3e00640 1 -- 172.21.15.137:0/1136237743 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400cdb0 con 0x7f4efc107260 2024-03-26T14:17:00.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4ef3e00640 1 -- 172.21.15.137:0/1136237743 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ef4005020 con 0x7f4efc107260 2024-03-26T14:17:00.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4ef3e00640 1 -- 172.21.15.137:0/1136237743 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40116d0 con 0x7f4efc107260 2024-03-26T14:17:00.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.432+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4efc1a5e50 con 0x7f4efc107260 2024-03-26T14:17:00.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.433+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4efc1a6250 con 0x7f4efc107260 2024-03-26T14:17:00.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4ef3e00640 1 -- 172.21.15.137:0/1136237743 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4ef4011870 con 0x7f4efc107260 2024-03-26T14:17:00.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 Processor -- start 2024-03-26T14:17:00.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 -- start start 2024-03-26T14:17:00.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdcb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fdd07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdd0e10 0x55624fdd32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624fd30eb0 con 0x55624fdcb2b0 2024-03-26T14:17:00.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624fcd9a60 con 0x55624fdd0e10 2024-03-26T14:17:00.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.436+0000 7f4f041b5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624fd2cf50 con 0x55624fdcbca0 2024-03-26T14:17:00.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdcb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdcb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41586/0 (socket says 172.21.15.137:41586) 2024-03-26T14:17:00.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 -- 172.21.15.137:0/2803740716 learned_addr learned my addr 172.21.15.137:0/2803740716 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdd0e10 0x55624fdd32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4efa800640 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fdd07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 -- 172.21.15.137:0/2803740716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 msgr2=0x55624fdd07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fdd07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 -- 172.21.15.137:0/2803740716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdd0e10 msgr2=0x55624fdd32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdd0e10 0x55624fdd32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.437+0000 7f4ef1e00640 1 -- 172.21.15.137:0/2803740716 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55624fb0d7e0 con 0x55624fdcb2b0 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdcb6d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40057c0 tx=0x7f4ee4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deddd46defdabe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4ef3e00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ee00a2880 0x7f4ee00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4eeb400640 1 -- 172.21.15.137:0/2803740716 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4048070 con 0x55624fdcb2b0 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4ef3e00640 1 -- 172.21.15.137:0/1136237743 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4ef40cefa0 con 0x7f4efc107260 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4eeb400640 1 -- 172.21.15.137:0/2803740716 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ee4034c80 con 0x55624fdcb2b0 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4eeb400640 1 -- 172.21.15.137:0/2803740716 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4039040 con 0x55624fdcb2b0 2024-03-26T14:17:00.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.438+0000 7f4efa800640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ee00a2880 0x7f4ee00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 -- 172.21.15.137:0/2803740716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 msgr2=0x55624fdcb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4efa800640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ee00a2880 0x7f4ee00a4d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0096c0 tx=0x7f4eec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdcb6d0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40057c0 tx=0x7f4ee4000bc0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 -- 172.21.15.137:0/2803740716 shutdown_connections 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdd0e10 0x55624fdd32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fdd07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2803740716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdcb6d0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 -- 172.21.15.137:0/2803740716 >> 172.21.15.137:0/2803740716 conn(0x55624fdc6820 msgr2=0x55624fdc6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 -- 172.21.15.137:0/2803740716 shutdown_connections 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.439+0000 7f4f041b5740 1 -- 172.21.15.137:0/2803740716 wait complete. 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 -- start start 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 Processor -- start 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdd5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcbca0 0x55624fdd5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdd0e10 0x55624fddcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624fdd3d60 con 0x55624fdcb2b0 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624fceceb0 con 0x55624fdcbca0 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55624fdd3ee0 con 0x55624fdd0e10 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4f041b5740 10 client.0 _ll_register_callbacks cb 0x55624fce1150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4efa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdd5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.440+0000 7f4efa800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdd5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41600/0 (socket says 172.21.15.137:41600) 2024-03-26T14:17:00.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 -- 172.21.15.137:0/2911268308 learned_addr learned my addr 172.21.15.137:0/2911268308 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcbca0 0x55624fdd5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efb200640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdd0e10 0x55624fddcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 -- 172.21.15.137:0/2911268308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdd0e10 msgr2=0x55624fddcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdd0e10 0x55624fddcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 -- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcbca0 msgr2=0x55624fdd5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcbca0 0x55624fdd5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 -- 172.21.15.137:0/2911268308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55624fb6df70 con 0x55624fdcb2b0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efb200640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdd0e10 0x55624fddcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4efa800640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdd5550 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4eec01d800 tx=0x7f4eec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8391c4b3a003e9b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.441+0000 7f4edfe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4edfe00640 1 -- 172.21.15.137:0/2911268308 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec02a060 con 0x55624fdcb2b0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4edfe00640 1 -- 172.21.15.137:0/2911268308 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4eec010b10 con 0x55624fdcb2b0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee4034c80 con 0x55624fdcb2b0 2024-03-26T14:17:00.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4edfe00640 1 -- 172.21.15.137:0/2911268308 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eec01de40 con 0x55624fdcb2b0 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4f041b5740 10 client.18732 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55624fa05170 con 0x55624fdcb2b0 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4edfe00640 1 -- 172.21.15.137:0/2911268308 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4ee4034c80 con 0x55624fdcb2b0 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4edfe00640 1 -- 172.21.15.137:0/2911268308 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4eec019040 con 0x55624fdcb2b0 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.442+0000 7f4f041b5740 10 client.18732 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.443+0000 7f4f041b5740 10 client.18732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.443+0000 7f4f041b5740 20 client.18732 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.443+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fde13b0 0x55624fde3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.443+0000 7f4f041b5740 4 client.18732 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.443+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55624fd3e3f0 con 0x55624fde13b0 2024-03-26T14:17:00.447 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.443+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fde13b0 0x55624fde3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.444+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fde13b0 0x55624fde3830 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87e658c82830a8c server_cookie=e32ba9db828e271b in_seq=0 out_seq=0 2024-03-26T14:17:00.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.444+0000 7f4edfe00640 10 client.18732 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.445+0000 7f4edfe00640 1 -- 172.21.15.137:0/2911268308 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55624fd3e3f0 con 0x55624fde13b0 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.445+0000 7f4edfe00640 10 client.18732 handle_command_reply: tid=0 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.445+0000 7f4f041b5740 1 client.18732 shutdown 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.445+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 msgr2=0x55624fdd5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.445+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdd5550 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4eec01d800 tx=0x7f4eec0046c0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 shutdown_connections 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fde13b0 0x55624fde3830 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcbca0 0x55624fdd5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdd0e10 0x55624fddcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 --2- 172.21.15.137:0/2911268308 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdcb2b0 0x55624fdd5550 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4edfe00640 0 client.18732 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 >> 172.21.15.137:0/2911268308 conn(0x55624fdc6820 msgr2=0x55624fdc5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 shutdown_connections 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 1 -- 172.21.15.137:0/2911268308 wait complete. 2024-03-26T14:17:00.449 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.446+0000 7f4f041b5740 20 client.18732 trim_cache size 0 max 16384 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 Processor -- start 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 -- start start 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fdc6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fdc5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1b2b60 con 0x55624fdd0e10 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1b0660 con 0x55624fdcb2b0 2024-03-26T14:17:00.467 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.466+0000 7f4f041b5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1ae6e0 con 0x55624fdcbca0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fdc6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fdc6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:39850/0 (socket says 172.21.15.137:39850) 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 -- 172.21.15.137:0/3330469441 learned_addr learned my addr 172.21.15.137:0/3330469441 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efb200640 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fdc5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 -- 172.21.15.137:0/3330469441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 msgr2=0x55624fddca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 -- 172.21.15.137:0/3330469441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 msgr2=0x55624fdc5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fdc5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 -- 172.21.15.137:0/3330469441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55624fdd3ca0 con 0x55624fdcb2b0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.467+0000 7f4efa800640 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fdc6e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0058c0 tx=0x7f4eec0107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5f5be50d96f3e40 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.468+0000 7f4eeaa00640 1 -- 172.21.15.137:0/3330469441 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec03a070 con 0x55624fdcb2b0 2024-03-26T14:17:00.469 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.468+0000 7f4eeaa00640 1 -- 172.21.15.137:0/3330469441 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4eec0128d0 con 0x55624fdcb2b0 2024-03-26T14:17:00.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.468+0000 7f4eeaa00640 1 -- 172.21.15.137:0/3330469441 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec025a40 con 0x55624fdcb2b0 2024-03-26T14:17:00.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 -- 172.21.15.137:0/3330469441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 msgr2=0x55624fdc6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fdc6e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0058c0 tx=0x7f4eec0107d0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 -- 172.21.15.137:0/3330469441 shutdown_connections 2024-03-26T14:17:00.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fdc5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.473 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 --2- 172.21.15.137:0/3330469441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fdc6e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 -- 172.21.15.137:0/3330469441 >> 172.21.15.137:0/3330469441 conn(0x55624fdc6820 msgr2=0x55624fdc51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 -- 172.21.15.137:0/3330469441 shutdown_connections 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 -- 172.21.15.137:0/3330469441 wait complete. 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 -- start start 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.469+0000 7f4f041b5740 1 Processor -- start 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fddcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fddb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1aece0 con 0x55624fdd0e10 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1b0000 con 0x55624fdcb2b0 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4efc1af340 con 0x55624fdcbca0 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4f041b5740 10 client.0 _ll_register_callbacks cb 0x7f4eec009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:00.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4efb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fddb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4efb200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fddb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41624/0 (socket says 172.21.15.137:41624) 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4efb200640 1 -- 172.21.15.137:0/1060610551 learned_addr learned my addr 172.21.15.137:0/1060610551 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4efa800640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fddcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.470+0000 7f4efb200640 1 -- 172.21.15.137:0/1060610551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 msgr2=0x55624fddb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4efb200640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4efb200640 1 -- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 msgr2=0x55624fddcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4efb200640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fddcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4efb200640 1 -- 172.21.15.137:0/1060610551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55624fe3ccc0 con 0x55624fdd0e10 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4ef1e00640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4efb200640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fddb710 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4ef409b230 tx=0x7f4ef409b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1408bf6e44aef78 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4edfe00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4edfe00640 1 -- 172.21.15.137:0/1060610551 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef401f040 con 0x55624fdd0e10 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4edfe00640 1 -- 172.21.15.137:0/1060610551 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f4ef40051d0 con 0x55624fdd0e10 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4eec0128d0 con 0x55624fdd0e10 2024-03-26T14:17:00.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.471+0000 7f4edfe00640 1 -- 172.21.15.137:0/1060610551 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ef40b4220 con 0x55624fdd0e10 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 10 client.18738 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55624fe3dec0 con 0x55624fdd0e10 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4edfe00640 1 -- 172.21.15.137:0/1060610551 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4eec0128d0 con 0x55624fdd0e10 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4edfe00640 1 -- 172.21.15.137:0/1060610551 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f4ef400f040 con 0x55624fdd0e10 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 10 client.18738 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 10 client.18738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 20 client.18738 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fe3e820 0x55624fe40c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 4 client.18738 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.472+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55624fd38740 con 0x55624fe3e820 2024-03-26T14:17:00.476 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.473+0000 7f4efa800640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fe3e820 0x55624fe40c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.473+0000 7f4efa800640 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fe3e820 0x55624fe40c40 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7913890f7eca516b server_cookie=e1a3e64c983d7133 in_seq=0 out_seq=0 2024-03-26T14:17:00.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.473+0000 7f4edfe00640 10 client.18738 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.477 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4edfe00640 1 -- 172.21.15.137:0/1060610551 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55624fd38740 con 0x55624fe3e820 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4edfe00640 10 client.18738 handle_command_reply: tid=0 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 client.18738 shutdown 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 msgr2=0x55624fddb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fddb710 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4ef409b230 tx=0x7f4ef409b5f0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 shutdown_connections 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55624fe3e820 0x55624fe40c40 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55624fdd0e10 0x55624fddb710 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55624fdcbca0 0x55624fddb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 --2- 172.21.15.137:0/1060610551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55624fdcb2b0 0x55624fddcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4edfe00640 0 client.18738 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 >> 172.21.15.137:0/1060610551 conn(0x55624fdc6820 msgr2=0x55624fdc5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 shutdown_connections 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 1 -- 172.21.15.137:0/1060610551 wait complete. 2024-03-26T14:17:00.478 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.475+0000 7f4f041b5740 20 client.18738 trim_cache size 0 max 16384 2024-03-26T14:17:00.479 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:17:00.479 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:17:00.479 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "ab6fac71-12b6-44d8-a571-e1c8f56cd02c", 2024-03-26T14:17:00.479 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:17:00.479 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ee00a2880 msgr2=0x7f4ee00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ee00a2880 0x7f4ee00a4d40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4eec0096c0 tx=0x7f4eec009260 comp rx=0 tx=0).stop 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 msgr2=0x7f4efc192c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 0x7f4efc192c60 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400eaa0 tx=0x7f4ef400ef60 comp rx=0 tx=0).stop 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 shutdown_connections 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f4ee00a2880 0x7f4ee00a4d40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4efc104930 0x7f4efc196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4efc107260 0x7f4efc192c60 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 --2- 172.21.15.137:0/1136237743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4efc102000 0x7f4efc1960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 >> 172.21.15.137:0/1136237743 conn(0x7f4efc0fba10 msgr2=0x7f4efc101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.476+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 shutdown_connections 2024-03-26T14:17:00.480 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.477+0000 7f4f00a00640 1 -- 172.21.15.137:0/1136237743 wait complete. 2024-03-26T14:17:00.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:17:00.494 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.732+0000 7fe89ec00640 1 -- 172.21.15.137:0/2223312327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898109870 msgr2=0x7fe898110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.732+0000 7fe89ec00640 1 --2- 172.21.15.137:0/2223312327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898109870 0x7fe898110100 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe890009860 tx=0x7fe89002bb70 comp rx=0 tx=0).stop 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 -- 172.21.15.137:0/2223312327 shutdown_connections 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 --2- 172.21.15.137:0/2223312327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898108eb0 0x7fe898109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 --2- 172.21.15.137:0/2223312327 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898109870 0x7fe898110100 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 --2- 172.21.15.137:0/2223312327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 0x7fe8981080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 -- 172.21.15.137:0/2223312327 >> 172.21.15.137:0/2223312327 conn(0x7fe898076c60 msgr2=0x7fe8980790a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 -- 172.21.15.137:0/2223312327 shutdown_connections 2024-03-26T14:17:00.734 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.733+0000 7fe89ec00640 1 -- 172.21.15.137:0/2223312327 wait complete. 2024-03-26T14:17:00.736 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 Processor -- start 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 -- start start 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 0x7fe89819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 0x7fe89819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898109870 0x7fe8981a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981137f0 con 0x7fe898108eb0 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe898113670 con 0x7fe898109870 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89ec00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe898113970 con 0x7fe898107cc0 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe897400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 0x7fe89819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe897400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 0x7fe89819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41652/0 (socket says 172.21.15.137:41652) 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe897400640 1 -- 172.21.15.137:0/4115025574 learned_addr learned my addr 172.21.15.137:0/4115025574 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.734+0000 7fe89cc00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898109870 0x7fe8981a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897e00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 0x7fe89819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.737 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897400640 1 -- 172.21.15.137:0/4115025574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 msgr2=0x7fe89819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897400640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 0x7fe89819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897400640 1 -- 172.21.15.137:0/4115025574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898109870 msgr2=0x7fe8981a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897400640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898109870 0x7fe8981a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897400640 1 -- 172.21.15.137:0/4115025574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8981a5ab0 con 0x7fe898108eb0 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897e00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 0x7fe89819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe897400640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 0x7fe89819e3d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e9c0 tx=0x7fe88000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe894c00640 1 -- 172.21.15.137:0/4115025574 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000ccc0 con 0x7fe898108eb0 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8981a5da0 con 0x7fe898108eb0 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe894c00640 1 -- 172.21.15.137:0/4115025574 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8800054d0 con 0x7fe898108eb0 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.735+0000 7fe894c00640 1 -- 172.21.15.137:0/4115025574 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880010430 con 0x7fe898108eb0 2024-03-26T14:17:00.738 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.736+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8981a6270 con 0x7fe898108eb0 2024-03-26T14:17:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.739+0000 7fe8a19a5740 1 Processor -- start 2024-03-26T14:17:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.739+0000 7fe894c00640 1 -- 172.21.15.137:0/4115025574 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe8800046c0 con 0x7fe898108eb0 2024-03-26T14:17:00.740 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.739+0000 7fe8a19a5740 1 -- start start 2024-03-26T14:17:00.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.739+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceae80 0x5581a9ceb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.739+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cf0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cf2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9c50250 con 0x5581a9ceb870 2024-03-26T14:17:00.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9bf8a30 con 0x5581a9cf09e0 2024-03-26T14:17:00.741 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9c4c2f0 con 0x5581a9ceae80 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cf0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cf0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41668/0 (socket says 172.21.15.137:41668) 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 -- 172.21.15.137:0/1334807937 learned_addr learned my addr 172.21.15.137:0/1334807937 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897400640 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cf2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe89cc00640 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceae80 0x5581a9ceb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 -- 172.21.15.137:0/1334807937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceae80 msgr2=0x5581a9ceb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceae80 0x5581a9ceb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 -- 172.21.15.137:0/1334807937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 msgr2=0x5581a9cf2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cf2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.743 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.740+0000 7fe897e00640 1 -- 172.21.15.137:0/1334807937 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a9be3690 con 0x5581a9ceb870 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe897e00640 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cf0370 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe888009f10 tx=0x7fe88800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade8b6ac5ad8226 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe894c00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe88d200640 1 -- 172.21.15.137:0/1334807937 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88801a680 con 0x5581a9ceb870 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe88d200640 1 -- 172.21.15.137:0/1334807937 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe88801acd0 con 0x5581a9ceb870 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe88d200640 1 -- 172.21.15.137:0/1334807937 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888011d90 con 0x5581a9ceb870 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe894c00640 1 -- 172.21.15.137:0/4115025574 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe8800cf150 con 0x7fe898108eb0 2024-03-26T14:17:00.744 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.741+0000 7fe89cc00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.742+0000 7fe89cc00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe8900349e0 tx=0x7fe8900056d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.742+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1334807937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 msgr2=0x5581a9cf0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.742+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cf0370 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe888009f10 tx=0x7fe88800f730 comp rx=0 tx=0).stop 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1334807937 shutdown_connections 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cf2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cf0370 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.747 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1334807937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceae80 0x5581a9ceb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1334807937 >> 172.21.15.137:0/1334807937 conn(0x5581a9cb3200 msgr2=0x5581a9ce67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1334807937 shutdown_connections 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1334807937 wait complete. 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.743+0000 7fe8a19a5740 1 -- start start 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 Processor -- start 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9cf3990 con 0x5581a9ceae80 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9c0be80 con 0x5581a9cf09e0 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.744+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581a9cf3b10 con 0x5581a9ceb870 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe8a19a5740 10 client.0 _ll_register_callbacks cb 0x5581a9c1e130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.748 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41672/0 (socket says 172.21.15.137:41672) 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 -- 172.21.15.137:0/1609151150 learned_addr learned my addr 172.21.15.137:0/1609151150 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe89cc00640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897400640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 -- 172.21.15.137:0/1609151150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 msgr2=0x5581a9cf5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 -- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 msgr2=0x5581a9cfc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.745+0000 7fe897e00640 1 -- 172.21.15.137:0/1609151150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581a9a2e7d0 con 0x5581a9ceae80 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe897e00640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf4e20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe888019000 tx=0x7fe888009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac2eb493ac2731a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe885600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe885600640 1 -- 172.21.15.137:0/1609151150 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888029060 con 0x5581a9ceae80 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe885600640 1 -- 172.21.15.137:0/1609151150 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe888011d20 con 0x5581a9ceae80 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe88801acd0 con 0x5581a9ceae80 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe885600640 1 -- 172.21.15.137:0/1609151150 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe888030050 con 0x5581a9ceae80 2024-03-26T14:17:00.749 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe8a19a5740 10 client.18750 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:00.750 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.746+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5581a99643f0 con 0x5581a9ceae80 2024-03-26T14:17:00.751 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe885600640 1 -- 172.21.15.137:0/1609151150 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe88801acd0 con 0x5581a9ceae80 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe885600640 1 -- 172.21.15.137:0/1609151150 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe888024870 con 0x5581a9ceae80 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe8a19a5740 10 client.18750 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe8a19a5740 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe8a19a5740 20 client.18750 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d00950 0x5581a9d02da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe8a19a5740 4 client.18750 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581a9c5d790 con 0x5581a9d00950 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.747+0000 7fe897400640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d00950 0x5581a9d02da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.748+0000 7fe897400640 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d00950 0x5581a9d02da0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73a399b225233e server_cookie=71ca09434725f4fa in_seq=0 out_seq=0 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.748+0000 7fe885600640 10 client.18750 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.749+0000 7fe885600640 1 -- 172.21.15.137:0/1609151150 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5581a9c5d790 con 0x5581a9d00950 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.749+0000 7fe885600640 10 client.18750 handle_command_reply: tid=0 2024-03-26T14:17:00.752 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.749+0000 7fe8a19a5740 1 client.18750 shutdown 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 msgr2=0x5581a9cf4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf4e20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe888019000 tx=0x7fe888009c50 comp rx=0 tx=0).stop 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 shutdown_connections 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d00950 0x5581a9d02da0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe885600640 0 client.18750 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1609151150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf4e20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 >> 172.21.15.137:0/1609151150 conn(0x5581a9cb3200 msgr2=0x5581a9ce5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 shutdown_connections 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1609151150 wait complete. 2024-03-26T14:17:00.753 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.750+0000 7fe8a19a5740 20 client.18750 trim_cache size 0 max 16384 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 Processor -- start 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 -- start start 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9ceae80 0x5581a9ce5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cfc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9cf09e0 0x5581a9d07660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981b2d50 con 0x5581a9ceb870 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981b0850 con 0x5581a9ceae80 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981ae8d0 con 0x5581a9cf09e0 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe897400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cfc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe897400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cfc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41680/0 (socket says 172.21.15.137:41680) 2024-03-26T14:17:00.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe897e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9ceae80 0x5581a9ce5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.768+0000 7fe897400640 1 -- 172.21.15.137:0/1288930468 learned_addr learned my addr 172.21.15.137:0/1288930468 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe89cc00640 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9cf09e0 0x5581a9d07660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe897400640 1 -- 172.21.15.137:0/1288930468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9cf09e0 msgr2=0x5581a9d07660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe897400640 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9cf09e0 0x5581a9d07660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe897400640 1 -- 172.21.15.137:0/1288930468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9ceae80 msgr2=0x5581a9ce5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe897400640 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9ceae80 0x5581a9ce5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe897400640 1 -- 172.21.15.137:0/1288930468 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581a9cf0730 con 0x5581a9ceb870 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe89cc00640 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9cf09e0 0x5581a9d07660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.769+0000 7fe897400640 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cfc360 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe8800cc840 tx=0x7fe8800cc710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57226a5dccbf92 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe887e00640 1 -- 172.21.15.137:0/1288930468 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880005680 con 0x5581a9ceb870 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe887e00640 1 -- 172.21.15.137:0/1288930468 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe88009bde0 con 0x5581a9ceb870 2024-03-26T14:17:00.772 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe887e00640 1 -- 172.21.15.137:0/1288930468 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880097d50 con 0x5581a9ceb870 2024-03-26T14:17:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1288930468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 msgr2=0x5581a9cfc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cfc360 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe8800cc840 tx=0x7fe8800cc710 comp rx=0 tx=0).stop 2024-03-26T14:17:00.777 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1288930468 shutdown_connections 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9cf09e0 0x5581a9d07660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceb870 0x5581a9cfc360 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/1288930468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9ceae80 0x5581a9ce5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.770+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1288930468 >> 172.21.15.137:0/1288930468 conn(0x5581a9cb3200 msgr2=0x5581a9cec960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.771+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1288930468 shutdown_connections 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.771+0000 7fe8a19a5740 1 -- 172.21.15.137:0/1288930468 wait complete. 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.771+0000 7fe8a19a5740 1 -- start start 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.771+0000 7fe8a19a5740 1 Processor -- start 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981aeed0 con 0x5581a9ceae80 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981b01f0 con 0x5581a9cf09e0 2024-03-26T14:17:00.778 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8981af530 con 0x5581a9ceb870 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe8a19a5740 10 client.0 _ll_register_callbacks cb 0x7fe888009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41692/0 (socket says 172.21.15.137:41692) 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 -- 172.21.15.137:0/2777381402 learned_addr learned my addr 172.21.15.137:0/2777381402 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe89cc00640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897400640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 -- 172.21.15.137:0/2777381402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 msgr2=0x5581a9cf9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 -- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 msgr2=0x5581a9cfac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe897e00640 1 -- 172.21.15.137:0/2777381402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581a99240f0 con 0x5581a9ceae80 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.772+0000 7fe89cc00640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe897e00640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf9730 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x5581a9ce2320 tx=0x7fe8880115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e8eb56f9c4e9b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe885600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe885600640 1 -- 172.21.15.137:0/2777381402 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888039060 con 0x5581a9ceae80 2024-03-26T14:17:00.779 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe885600640 1 -- 172.21.15.137:0/2777381402 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe888020040 con 0x5581a9ceae80 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe88009bde0 con 0x5581a9ceae80 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe885600640 1 -- 172.21.15.137:0/2777381402 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe888040050 con 0x5581a9ceae80 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe8a19a5740 10 client.18756 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5581a9cfc350 con 0x5581a9ceae80 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.773+0000 7fe885600640 1 -- 172.21.15.137:0/2777381402 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe88009bde0 con 0x5581a9ceae80 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe885600640 1 -- 172.21.15.137:0/2777381402 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe888010630 con 0x5581a9ceae80 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe8a19a5740 10 client.18756 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe8a19a5740 10 client.18756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe8a19a5740 20 client.18756 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d5c570 0x5581a9d5e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe8a19a5740 4 client.18756 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5581a9c57ae0 con 0x5581a9d5c570 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.774+0000 7fe897400640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d5c570 0x5581a9d5e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:00.780 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.775+0000 7fe885600640 10 client.18756 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.775+0000 7fe897400640 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d5c570 0x5581a9d5e960 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0ab061695b9d52 server_cookie=6981b3c95e7fc937 in_seq=0 out_seq=0 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe885600640 1 -- 172.21.15.137:0/2777381402 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5581a9c57ae0 con 0x5581a9d5c570 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe885600640 10 client.18756 handle_command_reply: tid=0 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 client.18756 shutdown 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 msgr2=0x5581a9cf9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf9730 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x5581a9ce2320 tx=0x7fe8880115f0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 shutdown_connections 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x5581a9d5c570 0x5581a9d5e960 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581a9cf09e0 0x5581a9cfac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5581a9ceb870 0x5581a9cf9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 --2- 172.21.15.137:0/2777381402 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x5581a9ceae80 0x5581a9cf9730 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 >> 172.21.15.137:0/2777381402 conn(0x5581a9cb3200 msgr2=0x5581a9cec040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe885600640 0 client.18756 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 shutdown_connections 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 1 -- 172.21.15.137:0/2777381402 wait complete. 2024-03-26T14:17:00.781 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.776+0000 7fe8a19a5740 20 client.18756 trim_cache size 0 max 16384 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: "ab6fac71-12b6-44d8-a571-e1c8f56cd02c": { 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "ab6fac71-12b6-44d8-a571-e1c8f56cd02c", 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:17:00.782 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:17:00.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe87c0a2800 msgr2=0x7fe87c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe8900349e0 tx=0x7fe8900056d0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 msgr2=0x7fe89819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:00.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 0x7fe89819e3d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fe88000e9c0 tx=0x7fe88000ee80 comp rx=0 tx=0).stop 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 shutdown_connections 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe87c0a2800 0x7fe87c0a4cc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898109870 0x7fe8981a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe898108eb0 0x7fe89819e3d0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 --2- 172.21.15.137:0/4115025574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe898107cc0 0x7fe89819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 >> 172.21.15.137:0/4115025574 conn(0x7fe898076c60 msgr2=0x7fe89810c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 shutdown_connections 2024-03-26T14:17:00.784 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:00.777+0000 7fe89ec00640 1 -- 172.21.15.137:0/4115025574 wait complete. 2024-03-26T14:17:00.798 INFO:tasks.cephfs.filesystem:scrub status for tag:ab6fac71-12b6-44d8-a571-e1c8f56cd02c - {'path': '/', 'tag': 'ab6fac71-12b6-44d8-a571-e1c8f56cd02c', 'options': 'recursive,force'} 2024-03-26T14:17:01.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:01 smithi179 ceph-mon[35472]: pgmap v897: 129 pgs: 129 active+clean; 114 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 491 KiB/s wr, 129 op/s 2024-03-26T14:17:01.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:01 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:17:01.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:01 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:01.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:01 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:17:01.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:01 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:01 smithi159 ceph-mon[35501]: pgmap v897: 129 pgs: 129 active+clean; 114 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 491 KiB/s wr, 129 op/s 2024-03-26T14:17:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:01 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:17:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:01 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:01.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:01 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:17:01.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:01 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-26T14:17:01.748 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-03-26T14:17:01.749 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-03-26T14:17:01.750 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-26T14:17:01.751 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-03-26T14:17:01.752 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-03-26T14:17:01.753 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-26T14:17:01.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:01 smithi137 ceph-mon[31539]: pgmap v897: 129 pgs: 129 active+clean; 114 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 491 KiB/s wr, 129 op/s 2024-03-26T14:17:01.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:01 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:17:01.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:01 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:01.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:01 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:17:01.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:01 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:02.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:02 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:17:02.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:02 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:17:02.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:02 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-03-26T14:17:02.985 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-03-26T14:17:02.986 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-26T14:17:02.987 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-26T14:17:02.988 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-03-26T14:17:02.989 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-03-26T14:17:02.990 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-26T14:17:03.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:03 smithi179 ceph-mon[35472]: pgmap v898: 129 pgs: 129 active+clean; 147 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 630 KiB/s wr, 161 op/s 2024-03-26T14:17:03.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:03 smithi159 ceph-mon[35501]: pgmap v898: 129 pgs: 129 active+clean; 147 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 630 KiB/s wr, 161 op/s 2024-03-26T14:17:03.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:03 smithi137 ceph-mon[31539]: pgmap v898: 129 pgs: 129 active+clean; 147 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 630 KiB/s wr, 161 op/s 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-03-26T14:17:04.353 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-26T14:17:04.354 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-26T14:17:04.355 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-26T14:17:04.356 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-03-26T14:17:04.357 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-26T14:17:04.358 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-03-26T14:17:04.359 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-26T14:17:04.360 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-03-26T14:17:04.361 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-26T14:17:04.362 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-03-26T14:17:05.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:05 smithi179 ceph-mon[35472]: pgmap v899: 129 pgs: 129 active+clean; 139 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 311 KiB/s wr, 142 op/s 2024-03-26T14:17:05.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:05 smithi159 ceph-mon[35501]: pgmap v899: 129 pgs: 129 active+clean; 139 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 311 KiB/s wr, 142 op/s 2024-03-26T14:17:05.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:05 smithi137 ceph-mon[31539]: pgmap v899: 129 pgs: 129 active+clean; 139 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 370 KiB/s rd, 311 KiB/s wr, 142 op/s 2024-03-26T14:17:06.413 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-26T14:17:06.413 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-03-26T14:17:06.414 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-26T14:17:06.415 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-26T14:17:06.416 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-26T14:17:06.417 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-26T14:17:06.418 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-26T14:17:06.419 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-03-26T14:17:06.420 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-26T14:17:06.420 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-03-26T14:17:07.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:07 smithi179 ceph-mon[35472]: pgmap v900: 129 pgs: 129 active+clean; 115 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 337 KiB/s wr, 155 op/s 2024-03-26T14:17:07.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:07 smithi159 ceph-mon[35501]: pgmap v900: 129 pgs: 129 active+clean; 115 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 337 KiB/s wr, 155 op/s 2024-03-26T14:17:07.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:07 smithi137 ceph-mon[31539]: pgmap v900: 129 pgs: 129 active+clean; 115 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 337 KiB/s wr, 155 op/s 2024-03-26T14:17:08.196 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-26T14:17:08.196 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-03-26T14:17:08.196 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-26T14:17:08.196 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-03-26T14:17:08.196 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-26T14:17:08.196 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-03-26T14:17:08.197 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-03-26T14:17:08.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-03-26T14:17:08.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-03-26T14:17:08.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-03-26T14:17:08.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-26T14:17:09.497 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-03-26T14:17:09.497 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-26T14:17:09.497 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-26T14:17:09.498 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-03-26T14:17:09.499 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-26T14:17:09.500 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-26T14:17:09.501 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-26T14:17:09.502 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-03-26T14:17:09.503 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-26T14:17:09.503 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-03-26T14:17:09.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:09 smithi137 ceph-mon[31539]: pgmap v901: 129 pgs: 129 active+clean; 105 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 486 KiB/s wr, 178 op/s 2024-03-26T14:17:09.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:09 smithi179 ceph-mon[35472]: pgmap v901: 129 pgs: 129 active+clean; 105 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 486 KiB/s wr, 178 op/s 2024-03-26T14:17:09.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:09 smithi159 ceph-mon[35501]: pgmap v901: 129 pgs: 129 active+clean; 105 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 486 KiB/s wr, 178 op/s 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-03-26T14:17:11.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-26T14:17:11.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-03-26T14:17:11.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-26T14:17:11.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-03-26T14:17:11.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-26T14:17:11.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-26T14:17:11.204 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-03-26T14:17:11.205 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-26T14:17:11.205 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-03-26T14:17:11.205 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-26T14:17:11.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:11 smithi179 ceph-mon[35472]: pgmap v902: 129 pgs: 129 active+clean; 156 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 153 op/s 2024-03-26T14:17:11.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:11 smithi159 ceph-mon[35501]: pgmap v902: 129 pgs: 129 active+clean; 156 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 153 op/s 2024-03-26T14:17:11.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:11 smithi137 ceph-mon[31539]: pgmap v902: 129 pgs: 129 active+clean; 156 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 410 KiB/s wr, 153 op/s 2024-03-26T14:17:13.279 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-03-26T14:17:13.280 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-26T14:17:13.281 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-03-26T14:17:13.282 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-26T14:17:13.283 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-03-26T14:17:13.284 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-03-26T14:17:13.285 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-03-26T14:17:13.286 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-26T14:17:13.287 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-03-26T14:17:13.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:13 smithi179 ceph-mon[35472]: pgmap v903: 129 pgs: 129 active+clean; 137 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 438 KiB/s wr, 157 op/s 2024-03-26T14:17:13.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:13 smithi159 ceph-mon[35501]: pgmap v903: 129 pgs: 129 active+clean; 137 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 438 KiB/s wr, 157 op/s 2024-03-26T14:17:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:13 smithi137 ceph-mon[31539]: pgmap v903: 129 pgs: 129 active+clean; 137 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 438 KiB/s wr, 157 op/s 2024-03-26T14:17:14.941 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-26T14:17:14.941 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-03-26T14:17:14.941 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-26T14:17:14.941 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-03-26T14:17:14.941 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-26T14:17:14.941 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-26T14:17:14.942 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-26T14:17:14.943 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-03-26T14:17:14.944 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-26T14:17:14.945 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-03-26T14:17:14.946 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-26T14:17:14.947 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-03-26T14:17:14.948 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-26T14:17:14.949 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-03-26T14:17:14.949 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-26T14:17:14.949 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-03-26T14:17:14.949 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-26T14:17:14.949 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-03-26T14:17:15.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:15 smithi179 ceph-mon[35472]: pgmap v904: 129 pgs: 129 active+clean; 130 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 419 KiB/s wr, 157 op/s 2024-03-26T14:17:15.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:15 smithi159 ceph-mon[35501]: pgmap v904: 129 pgs: 129 active+clean; 130 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 419 KiB/s wr, 157 op/s 2024-03-26T14:17:15.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:15 smithi137 ceph-mon[31539]: pgmap v904: 129 pgs: 129 active+clean; 130 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 419 KiB/s wr, 157 op/s 2024-03-26T14:17:16.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-26T14:17:16.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-03-26T14:17:16.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-26T14:17:16.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-03-26T14:17:16.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-26T14:17:16.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-03-26T14:17:16.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-03-26T14:17:16.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-03-26T14:17:16.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-26T14:17:16.324 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-03-26T14:17:16.325 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-26T14:17:16.325 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-03-26T14:17:16.325 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-26T14:17:16.325 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-03-26T14:17:16.325 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-26T14:17:17.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:17 smithi179 ceph-mon[35472]: pgmap v905: 129 pgs: 129 active+clean; 142 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 419 KiB/s wr, 152 op/s 2024-03-26T14:17:17.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:17 smithi159 ceph-mon[35501]: pgmap v905: 129 pgs: 129 active+clean; 142 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 419 KiB/s wr, 152 op/s 2024-03-26T14:17:17.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:17 smithi137 ceph-mon[31539]: pgmap v905: 129 pgs: 129 active+clean; 142 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 419 KiB/s wr, 152 op/s 2024-03-26T14:17:18.186 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-03-26T14:17:18.186 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-26T14:17:18.186 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-03-26T14:17:18.187 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-26T14:17:18.188 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-03-26T14:17:18.189 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-26T14:17:18.190 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-03-26T14:17:18.191 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-26T14:17:18.192 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-26T14:17:18.193 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-26T14:17:18.194 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-03-26T14:17:18.195 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-26T14:17:18.195 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-03-26T14:17:18.195 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-26T14:17:18.195 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-03-26T14:17:18.195 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-26T14:17:18.195 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-03-26T14:17:19.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:19 smithi179 ceph-mon[35472]: pgmap v906: 129 pgs: 129 active+clean; 115 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 539 KiB/s wr, 184 op/s 2024-03-26T14:17:19.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:19 smithi159 ceph-mon[35501]: pgmap v906: 129 pgs: 129 active+clean; 115 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 539 KiB/s wr, 184 op/s 2024-03-26T14:17:19.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:19 smithi137 ceph-mon[31539]: pgmap v906: 129 pgs: 129 active+clean; 115 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 539 KiB/s wr, 184 op/s 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-03-26T14:17:20.465 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-03-26T14:17:20.466 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-26T14:17:20.467 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-26T14:17:20.468 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-03-26T14:17:20.469 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-26T14:17:20.470 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-03-26T14:17:20.471 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-03-26T14:17:20.472 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-26T14:17:21.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:21 smithi179 ceph-mon[35472]: pgmap v907: 129 pgs: 129 active+clean; 115 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 416 KiB/s wr, 153 op/s 2024-03-26T14:17:21.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:21 smithi159 ceph-mon[35501]: pgmap v907: 129 pgs: 129 active+clean; 115 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 416 KiB/s wr, 153 op/s 2024-03-26T14:17:21.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:21 smithi137 ceph-mon[31539]: pgmap v907: 129 pgs: 129 active+clean; 115 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 416 KiB/s wr, 153 op/s 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-03-26T14:17:21.819 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-26T14:17:21.820 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-26T14:17:21.821 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-26T14:17:21.822 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-26T14:17:21.823 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-03-26T14:17:21.824 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-26T14:17:21.825 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-03-26T14:17:21.826 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-26T14:17:23.153 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-03-26T14:17:23.153 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-26T14:17:23.153 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-03-26T14:17:23.153 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-26T14:17:23.153 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-03-26T14:17:23.154 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-03-26T14:17:23.155 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-03-26T14:17:23.156 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-26T14:17:23.157 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-26T14:17:23.158 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-03-26T14:17:23.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:23 smithi179 ceph-mon[35472]: pgmap v908: 129 pgs: 129 active+clean; 139 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 399 KiB/s wr, 145 op/s 2024-03-26T14:17:23.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:23 smithi159 ceph-mon[35501]: pgmap v908: 129 pgs: 129 active+clean; 139 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 399 KiB/s wr, 145 op/s 2024-03-26T14:17:23.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:23 smithi137 ceph-mon[31539]: pgmap v908: 129 pgs: 129 active+clean; 139 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 399 KiB/s wr, 145 op/s 2024-03-26T14:17:25.371 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-03-26T14:17:25.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-03-26T14:17:25.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-26T14:17:25.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-03-26T14:17:25.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-26T14:17:25.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-03-26T14:17:25.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-26T14:17:25.674 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:25 smithi179 ceph-mon[35472]: pgmap v909: 129 pgs: 129 active+clean; 132 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 474 KiB/s wr, 162 op/s 2024-03-26T14:17:25.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:25 smithi159 ceph-mon[35501]: pgmap v909: 129 pgs: 129 active+clean; 132 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 474 KiB/s wr, 162 op/s 2024-03-26T14:17:25.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:25 smithi137 ceph-mon[31539]: pgmap v909: 129 pgs: 129 active+clean; 132 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 474 KiB/s wr, 162 op/s 2024-03-26T14:17:27.553 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-03-26T14:17:27.554 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-03-26T14:17:27.555 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-26T14:17:27.556 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-03-26T14:17:27.557 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-26T14:17:27.558 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-03-26T14:17:27.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:27 smithi179 ceph-mon[35472]: pgmap v910: 129 pgs: 129 active+clean; 69 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 376 KiB/s wr, 140 op/s 2024-03-26T14:17:27.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:27 smithi159 ceph-mon[35501]: pgmap v910: 129 pgs: 129 active+clean; 69 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 376 KiB/s wr, 140 op/s 2024-03-26T14:17:27.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:27 smithi137 ceph-mon[31539]: pgmap v910: 129 pgs: 129 active+clean; 69 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 376 KiB/s wr, 140 op/s 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-03-26T14:17:28.887 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-03-26T14:17:28.888 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-26T14:17:28.889 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-26T14:17:28.890 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-03-26T14:17:28.891 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-26T14:17:28.892 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-03-26T14:17:28.892 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-26T14:17:28.892 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-03-26T14:17:28.892 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-26T14:17:29.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:29 smithi179 ceph-mon[35472]: pgmap v911: 129 pgs: 129 active+clean; 149 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 463 KiB/s wr, 164 op/s 2024-03-26T14:17:29.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:29 smithi159 ceph-mon[35501]: pgmap v911: 129 pgs: 129 active+clean; 149 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 463 KiB/s wr, 164 op/s 2024-03-26T14:17:29.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:29 smithi137 ceph-mon[31539]: pgmap v911: 129 pgs: 129 active+clean; 149 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 463 KiB/s wr, 164 op/s 2024-03-26T14:17:30.451 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-03-26T14:17:30.451 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-26T14:17:30.451 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-03-26T14:17:30.452 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-26T14:17:30.453 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-26T14:17:30.454 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-26T14:17:30.455 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-26T14:17:30.456 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-03-26T14:17:30.457 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-26T14:17:30.799 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:17:30.799 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:17:31.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 -- 172.21.15.137:0/4256473689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 msgr2=0x7fe6bc1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/4256473689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc1099e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009ca0 tx=0x7fe6b402c100 comp rx=0 tx=0).stop 2024-03-26T14:17:31.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 -- 172.21.15.137:0/4256473689 shutdown_connections 2024-03-26T14:17:31.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/4256473689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc1099e0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/4256473689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc102790 0x7fe6bc102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/4256473689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc1015a0 0x7fe6bc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 -- 172.21.15.137:0/4256473689 >> 172.21.15.137:0/4256473689 conn(0x7fe6bc0fcb10 msgr2=0x7fe6bc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 -- 172.21.15.137:0/4256473689 shutdown_connections 2024-03-26T14:17:31.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.013+0000 7fe6c0e00640 1 -- 172.21.15.137:0/4256473689 wait complete. 2024-03-26T14:17:31.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 Processor -- start 2024-03-26T14:17:31.017 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 -- start start 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc1015a0 0x7fe6bc197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc102790 0x7fe6bc197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc10d0e0 con 0x7fe6bc103150 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc10cf60 con 0x7fe6bc102790 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.014+0000 7fe6c0e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc10d260 con 0x7fe6bc1015a0 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58290/0 (socket says 172.21.15.137:58290) 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 -- 172.21.15.137:0/315001669 learned_addr learned my addr 172.21.15.137:0/315001669 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc102790 0x7fe6bc197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bb400640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc1015a0 0x7fe6bc197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.018 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 -- 172.21.15.137:0/315001669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc1015a0 msgr2=0x7fe6bc197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc1015a0 0x7fe6bc197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 -- 172.21.15.137:0/315001669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc102790 msgr2=0x7fe6bc197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc102790 0x7fe6bc197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 -- 172.21.15.137:0/315001669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc19f320 con 0x7fe6bc103150 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.015+0000 7fe6bbe00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc19ecb0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c710 tx=0x7fe6b4035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.016+0000 7fe6b8c00640 1 -- 172.21.15.137:0/315001669 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4039040 con 0x7fe6bc103150 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.016+0000 7fe6b8c00640 1 -- 172.21.15.137:0/315001669 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6b4044070 con 0x7fe6bc103150 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.016+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc19f5b0 con 0x7fe6bc103150 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.016+0000 7fe6b8c00640 1 -- 172.21.15.137:0/315001669 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4004530 con 0x7fe6bc103150 2024-03-26T14:17:31.019 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.016+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc19faf0 con 0x7fe6bc103150 2024-03-26T14:17:31.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.019+0000 7fe6b8c00640 1 -- 172.21.15.137:0/315001669 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe6b40327c0 con 0x7fe6bc103150 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.019+0000 7fe6c51d1740 1 Processor -- start 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 -- start start 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 0x55a323eb7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7a40 0x55a323ebc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323ebf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a323e1c250 con 0x55a323ebcbb0 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a323dc4a30 con 0x55a323eb7050 2024-03-26T14:17:31.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a323e182f0 con 0x55a323eb7a40 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 0x55a323eb7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 0x55a323eb7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:48074/0 (socket says 172.21.15.137:48074) 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6b3e00640 1 -- 172.21.15.137:0/3127893925 learned_addr learned my addr 172.21.15.137:0/3127893925 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.020+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323ebf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7a40 0x55a323ebc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b8c00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6a00a6c90 0x7fe6a00a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b3e00640 1 -- 172.21.15.137:0/3127893925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7a40 msgr2=0x55a323ebc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7a40 0x55a323ebc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b3e00640 1 -- 172.21.15.137:0/3127893925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 msgr2=0x55a323ebf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323ebf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b3e00640 1 -- 172.21.15.137:0/3127893925 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a323daf690 con 0x55a323eb7050 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b8c00640 1 -- 172.21.15.137:0/315001669 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe6b40f1540 con 0x7fe6bc103150 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6bbe00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6a00a6c90 0x7fe6a00a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 0x55a323eb7470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40127a0 tx=0x7fe6a400b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1e0769887ae994 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.021+0000 7fe6b0c00640 1 -- 172.21.15.137:0/3127893925 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4012de0 con 0x55a323eb7050 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6b0c00640 1 -- 172.21.15.137:0/3127893925 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6a4005020 con 0x55a323eb7050 2024-03-26T14:17:31.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6b0c00640 1 -- 172.21.15.137:0/3127893925 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004a00 con 0x55a323eb7050 2024-03-26T14:17:31.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3127893925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 msgr2=0x55a323eb7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.029 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6bbe00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6a00a6c90 0x7fe6a00a9150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40ba4a0 tx=0x7fe6b4034850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 0x55a323eb7470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40127a0 tx=0x7fe6a400b7c0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3127893925 shutdown_connections 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.022+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323ebf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7a40 0x55a323ebc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3127893925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7050 0x55a323eb7470 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3127893925 >> 172.21.15.137:0/3127893925 conn(0x55a323e7f200 msgr2=0x55a323eb29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3127893925 shutdown_connections 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3127893925 wait complete. 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 -- start start 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.023+0000 7fe6c51d1740 1 Processor -- start 2024-03-26T14:17:31.030 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ec10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 0x55a323ec1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323ebcbb0 0x55a323ec8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a323ebfb60 con 0x55a323eb7a40 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a323dd7e80 con 0x55a323ebcbb0 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a323ebfce0 con 0x55a323eb7050 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6c51d1740 10 client.0 _ll_register_callbacks cb 0x55a323dea130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 0x55a323ec1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 0x55a323ec1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58314/0 (socket says 172.21.15.137:58314) 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6bb400640 1 -- 172.21.15.137:0/3559308689 learned_addr learned my addr 172.21.15.137:0/3559308689 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323ebcbb0 0x55a323ec8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ec10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6bb400640 1 -- 172.21.15.137:0/3559308689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 msgr2=0x55a323ec10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.024+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ec10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6bb400640 1 -- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323ebcbb0 msgr2=0x55a323ec8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323ebcbb0 0x55a323ec8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.031 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6bb400640 1 -- 172.21.15.137:0/3559308689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a323bfa7d0 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323ebcbb0 0x55a323ec8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 0x55a323ec1630 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac010760 tx=0x7fe6ac010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa949fbd7cb7f872 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6a9600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6a9600640 1 -- 172.21.15.137:0/3559308689 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004250 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6a4005020 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6a9600640 1 -- 172.21.15.137:0/3559308689 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe6ac0043f0 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.025+0000 7fe6a9600640 1 -- 172.21.15.137:0/3559308689 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6ac010e70 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 10 client.18768 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a323b303f0 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6a9600640 1 -- 172.21.15.137:0/3559308689 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6a4005020 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6a9600640 1 -- 172.21.15.137:0/3559308689 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe6ac0047a0 con 0x55a323eb7a40 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 10 client.18768 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 10 client.18768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:31.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 20 client.18768 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:31.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323ecd0b0 0x55a323ecf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 4 client.18768 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:31.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.026+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a323e29790 con 0x55a323ecd0b0 2024-03-26T14:17:31.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.027+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323ecd0b0 0x55a323ecf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.027+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323ecd0b0 0x55a323ecf4d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b5093db211df8b server_cookie=6a4b24aaad7d8e0f in_seq=0 out_seq=0 2024-03-26T14:17:31.033 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.027+0000 7fe6a9600640 10 client.18768 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6a9600640 1 -- 172.21.15.137:0/3559308689 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a323e29790 con 0x55a323ecd0b0 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6a9600640 10 client.18768 handle_command_reply: tid=0 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 client.18768 shutdown 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 msgr2=0x55a323ec1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 0x55a323ec1630 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac010760 tx=0x7fe6ac010c20 comp rx=0 tx=0).stop 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 shutdown_connections 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323ecd0b0 0x55a323ecf4d0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323ebcbb0 0x55a323ec8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7a40 0x55a323ec1630 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3559308689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ec10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6a9600640 0 client.18768 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.029+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 >> 172.21.15.137:0/3559308689 conn(0x55a323e7f200 msgr2=0x55a323eb1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.030+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 shutdown_connections 2024-03-26T14:17:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.030+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3559308689 wait complete. 2024-03-26T14:17:31.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.030+0000 7fe6c51d1740 20 client.18768 trim_cache size 0 max 16384 2024-03-26T14:17:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.050+0000 7fe6c51d1740 1 Processor -- start 2024-03-26T14:17:31.052 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.050+0000 7fe6c51d1740 1 -- start start 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.050+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ead920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323eb6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1aa220 con 0x55a323ebcbb0 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1a7d20 con 0x55a323eb7a40 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1a5da0 con 0x55a323eb7050 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:48102/0 (socket says 172.21.15.137:48102) 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 -- 172.21.15.137:0/3122847248 learned_addr learned my addr 172.21.15.137:0/3122847248 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323eb6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 -- 172.21.15.137:0/3122847248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 msgr2=0x55a323ead920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ead920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.053 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ead920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 -- 172.21.15.137:0/3122847248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 msgr2=0x55a323eb6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323eb6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6bb400640 1 -- 172.21.15.137:0/3122847248 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a323ec97a0 con 0x55a323eb7a40 2024-03-26T14:17:31.054 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.051+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ead920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:31.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.052+0000 7fe6bb400640 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec8630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac00ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d98cc8a3ed86cbc server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.053+0000 7fe6abe00640 1 -- 172.21.15.137:0/3122847248 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00ee50 con 0x55a323eb7a40 2024-03-26T14:17:31.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.053+0000 7fe6abe00640 1 -- 172.21.15.137:0/3122847248 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6ac02c070 con 0x55a323eb7a40 2024-03-26T14:17:31.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.053+0000 7fe6abe00640 1 -- 172.21.15.137:0/3122847248 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac003960 con 0x55a323eb7a40 2024-03-26T14:17:31.055 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3122847248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 msgr2=0x55a323ec8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec8630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac00ea60 comp rx=0 tx=0).stop 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3122847248 shutdown_connections 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec8630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323ebcbb0 0x55a323eb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/3122847248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323eb7050 0x55a323ead920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3122847248 >> 172.21.15.137:0/3122847248 conn(0x55a323e7f200 msgr2=0x55a323eb5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3122847248 shutdown_connections 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 -- 172.21.15.137:0/3122847248 wait complete. 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 -- start start 2024-03-26T14:17:31.056 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.054+0000 7fe6c51d1740 1 Processor -- start 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 0x55a323ec5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323ebcbb0 0x55a323ec7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1a63a0 con 0x55a323eb7050 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1a76c0 con 0x55a323eb7a40 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1a6a00 con 0x55a323ebcbb0 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6c51d1740 10 client.0 _ll_register_callbacks cb 0x7fe6ac00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 0x55a323ec5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.060 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6b3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 0x55a323ec5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58340/0 (socket says 172.21.15.137:58340) 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6b3e00640 1 -- 172.21.15.137:0/4082855353 learned_addr learned my addr 172.21.15.137:0/4082855353 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6bb400640 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6b3e00640 1 -- 172.21.15.137:0/4082855353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323ebcbb0 msgr2=0x55a323ec7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323ebcbb0 0x55a323ec7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.055+0000 7fe6b3e00640 1 -- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 msgr2=0x55a323ec6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6b3e00640 1 -- 172.21.15.137:0/4082855353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a323af00f0 con 0x55a323eb7050 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6b3e00640 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 0x55a323ec5ee0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40127a0 tx=0x7fe6a400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8602b2bc1e403359 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6a9600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6a9600640 1 -- 172.21.15.137:0/4082855353 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4033060 con 0x55a323eb7050 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6ac02c070 con 0x55a323eb7050 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6a9600640 1 -- 172.21.15.137:0/4082855353 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe6a402e4b0 con 0x55a323eb7050 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6a9600640 1 -- 172.21.15.137:0/4082855353 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6a400b9b0 con 0x55a323eb7050 2024-03-26T14:17:31.061 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6c51d1740 10 client.18774 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.056+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55a323f289b0 con 0x55a323eb7050 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6a9600640 1 -- 172.21.15.137:0/4082855353 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe6ac02c070 con 0x55a323eb7050 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6a9600640 1 -- 172.21.15.137:0/4082855353 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fe6a402e6b0 con 0x55a323eb7050 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6c51d1740 10 client.18774 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6c51d1740 10 client.18774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6c51d1740 20 client.18774 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323f29130 0x55a323f2b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6c51d1740 4 client.18774 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a323e23ae0 con 0x55a323f29130 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.057+0000 7fe6bb400640 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323f29130 0x55a323f2b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6bb400640 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323f29130 0x55a323f2b550 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ce9bfcd18cedc6 server_cookie=be52351914f23649 in_seq=0 out_seq=0 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6a9600640 10 client.18774 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:31.062 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6a9600640 1 -- 172.21.15.137:0/4082855353 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a323e23ae0 con 0x55a323f29130 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6a9600640 10 client.18774 handle_command_reply: tid=0 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6c51d1740 1 client.18774 shutdown 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 msgr2=0x55a323ec5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.058+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 0x55a323ec5ee0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe6a40127a0 tx=0x7fe6a400b670 comp rx=0 tx=0).stop 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 shutdown_connections 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55a323f29130 0x55a323f2b550 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a323eb7a40 0x55a323ec6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a323ebcbb0 0x55a323ec7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 --2- 172.21.15.137:0/4082855353 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55a323eb7050 0x55a323ec5ee0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 >> 172.21.15.137:0/4082855353 conn(0x55a323e7f200 msgr2=0x55a323eb6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6a9600640 0 client.18774 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 shutdown_connections 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 1 -- 172.21.15.137:0/4082855353 wait complete. 2024-03-26T14:17:31.063 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.059+0000 7fe6c51d1740 20 client.18774 trim_cache size 0 max 16384 2024-03-26T14:17:31.064 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:17:31.064 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:17:31.064 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:17:31.064 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6a00a6c90 msgr2=0x7fe6a00a9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6a00a6c90 0x7fe6a00a9150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40ba4a0 tx=0x7fe6b4034850 comp rx=0 tx=0).stop 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 msgr2=0x7fe6bc19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc19ecb0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c710 tx=0x7fe6b4035d60 comp rx=0 tx=0).stop 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 shutdown_connections 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe6a00a6c90 0x7fe6a00a9150 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6bc102790 0x7fe6bc197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6bc103150 0x7fe6bc19ecb0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 --2- 172.21.15.137:0/315001669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6bc1015a0 0x7fe6bc197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 >> 172.21.15.137:0/315001669 conn(0x7fe6bc0fcb10 msgr2=0x7fe6bc105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.065 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 shutdown_connections 2024-03-26T14:17:31.066 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.060+0000 7fe6c0e00640 1 -- 172.21.15.137:0/315001669 wait complete. 2024-03-26T14:17:31.078 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:17:31.079 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 -- 172.21.15.137:0/647284783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac40104ad0 msgr2=0x7fac40104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 --2- 172.21.15.137:0/647284783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac40104ad0 0x7fac40104f50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fac28009900 tx=0x7fac2802bc30 comp rx=0 tx=0).stop 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 -- 172.21.15.137:0/647284783 shutdown_connections 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 --2- 172.21.15.137:0/647284783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac40105490 0x7fac4010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 --2- 172.21.15.137:0/647284783 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac40104ad0 0x7fac40104f50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 --2- 172.21.15.137:0/647284783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac401038e0 0x7fac40103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 -- 172.21.15.137:0/647284783 >> 172.21.15.137:0/647284783 conn(0x7fac400fee50 msgr2=0x7fac40101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 -- 172.21.15.137:0/647284783 shutdown_connections 2024-03-26T14:17:31.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.282+0000 7fac45a00640 1 -- 172.21.15.137:0/647284783 wait complete. 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.283+0000 7fac45a00640 1 Processor -- start 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 -- start start 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 0x7fac40199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40104ad0 0x7fac4019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac40105490 0x7fac401a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4010f460 con 0x7fac401038e0 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4010f2e0 con 0x7fac40104ad0 2024-03-26T14:17:31.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac45a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4010f5e0 con 0x7fac40105490 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac3ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 0x7fac40199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac3ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 0x7fac40199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58350/0 (socket says 172.21.15.137:58350) 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac3ea00640 1 -- 172.21.15.137:0/928105188 learned_addr learned my addr 172.21.15.137:0/928105188 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.284+0000 7fac36000640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40104ad0 0x7fac4019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3f400640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac40105490 0x7fac401a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3ea00640 1 -- 172.21.15.137:0/928105188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac40105490 msgr2=0x7fac401a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3ea00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac40105490 0x7fac401a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3ea00640 1 -- 172.21.15.137:0/928105188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40104ad0 msgr2=0x7fac4019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3ea00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40104ad0 0x7fac4019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3ea00640 1 -- 172.21.15.137:0/928105188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac401a1750 con 0x7fac401038e0 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac36000640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40104ad0 0x7fac4019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:17:31.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.285+0000 7fac3ea00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 0x7fac40199b30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e9c0 tx=0x7fac3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.286+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000cd80 con 0x7fac401038e0 2024-03-26T14:17:31.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.286+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fac300035f0 con 0x7fac401038e0 2024-03-26T14:17:31.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.286+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac401a1a40 con 0x7fac401038e0 2024-03-26T14:17:31.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.286+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30010690 con 0x7fac401038e0 2024-03-26T14:17:31.288 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.286+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac401a1fb0 con 0x7fac401038e0 2024-03-26T14:17:31.289 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.288+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac40068490 con 0x7fac401038e0 2024-03-26T14:17:31.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.290+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fac300037a0 con 0x7fac401038e0 2024-03-26T14:17:31.292 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.291+0000 7fac37e00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fac240a2880 0x7fac240a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.292+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fac300cf290 con 0x7fac401038e0 2024-03-26T14:17:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.292+0000 7fac36000640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fac240a2880 0x7fac240a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.293 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.293+0000 7fac36000640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fac240a2880 0x7fac240a4d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac28006010 tx=0x7fac2802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.297 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.297+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fac3009bf60 con 0x7fac401038e0 2024-03-26T14:17:31.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:31 smithi137 ceph-mon[31539]: pgmap v912: 129 pgs: 129 active+clean; 157 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 338 KiB/s wr, 133 op/s 2024-03-26T14:17:31.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.567+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac4010aca0 con 0x7fac401038e0 2024-03-26T14:17:31.569 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.568+0000 7fac37e00640 1 -- 172.21.15.137:0/928105188 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fac4010aca0 con 0x7fac401038e0 2024-03-26T14:17:31.569 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:17:31.569 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:17:31.570 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fac240a2880 msgr2=0x7fac240a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fac240a2880 0x7fac240a4d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac28006010 tx=0x7fac2802c7e0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 msgr2=0x7fac40199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 0x7fac40199b30 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fac3000e9c0 tx=0x7fac3000ee80 comp rx=0 tx=0).stop 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 shutdown_connections 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fac240a2880 0x7fac240a4d40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac40104ad0 0x7fac4019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac40105490 0x7fac401a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.572 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 --2- 172.21.15.137:0/928105188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fac401038e0 0x7fac40199b30 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 >> 172.21.15.137:0/928105188 conn(0x7fac400fee50 msgr2=0x7fac40107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 shutdown_connections 2024-03-26T14:17:31.573 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.571+0000 7fac45a00640 1 -- 172.21.15.137:0/928105188 wait complete. 2024-03-26T14:17:31.591 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:17:31.591 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:17:31.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:31 smithi179 ceph-mon[35472]: pgmap v912: 129 pgs: 129 active+clean; 157 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 338 KiB/s wr, 133 op/s 2024-03-26T14:17:31.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:31 smithi159 ceph-mon[35501]: pgmap v912: 129 pgs: 129 active+clean; 157 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 338 KiB/s wr, 133 op/s 2024-03-26T14:17:31.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 -- 172.21.15.137:0/2077854422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 msgr2=0x7fb084104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.823 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 --2- 172.21.15.137:0/2077854422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084104e30 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb074009900 tx=0x7fb07402bc30 comp rx=0 tx=0).stop 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 -- 172.21.15.137:0/2077854422 shutdown_connections 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 --2- 172.21.15.137:0/2077854422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb084105370 0x7fb08410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 --2- 172.21.15.137:0/2077854422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084104e30 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 --2- 172.21.15.137:0/2077854422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0841037c0 0x7fb084103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 -- 172.21.15.137:0/2077854422 >> 172.21.15.137:0/2077854422 conn(0x7fb0840fed50 msgr2=0x7fb084101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 -- 172.21.15.137:0/2077854422 shutdown_connections 2024-03-26T14:17:31.824 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.822+0000 7fb08ae00640 1 -- 172.21.15.137:0/2077854422 wait complete. 2024-03-26T14:17:31.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.823+0000 7fb08ae00640 1 Processor -- start 2024-03-26T14:17:31.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.823+0000 7fb08ae00640 1 -- start start 2024-03-26T14:17:31.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08ae00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0841037c0 0x7fb084199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.826 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08ae00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08ae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb084105370 0x7fb0841a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08ae00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08410f340 con 0x7fb0841049b0 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08ae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08410f1c0 con 0x7fb084105370 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08ae00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08410f4c0 con 0x7fb0841037c0 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58372/0 (socket says 172.21.15.137:58372) 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 -- 172.21.15.137:0/3217948192 learned_addr learned my addr 172.21.15.137:0/3217948192 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089a00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0841037c0 0x7fb084199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb08a400640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb084105370 0x7fb0841a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 -- 172.21.15.137:0/3217948192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0841037c0 msgr2=0x7fb084199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0841037c0 0x7fb084199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 -- 172.21.15.137:0/3217948192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb084105370 msgr2=0x7fb0841a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb084105370 0x7fb0841a0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.824+0000 7fb089000640 1 -- 172.21.15.137:0/3217948192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0841a1630 con 0x7fb0841049b0 2024-03-26T14:17:31.827 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.825+0000 7fb089000640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084199f50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb074000c00 tx=0x7fb0740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.825+0000 7fb082000640 1 -- 172.21.15.137:0/3217948192 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074043020 con 0x7fb0841049b0 2024-03-26T14:17:31.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.825+0000 7fb082000640 1 -- 172.21.15.137:0/3217948192 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb074004aa0 con 0x7fb0841049b0 2024-03-26T14:17:31.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.825+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0841a18c0 con 0x7fb0841049b0 2024-03-26T14:17:31.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.825+0000 7fb082000640 1 -- 172.21.15.137:0/3217948192 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb074003d70 con 0x7fb0841049b0 2024-03-26T14:17:31.828 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.825+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0841a1e10 con 0x7fb0841049b0 2024-03-26T14:17:31.829 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.828+0000 7fb082000640 1 -- 172.21.15.137:0/3217948192 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb074046020 con 0x7fb0841049b0 2024-03-26T14:17:31.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.829+0000 7fb08f05c740 1 Processor -- start 2024-03-26T14:17:31.830 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.829+0000 7fb08f05c740 1 -- start start 2024-03-26T14:17:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbe9f0 0x555cf8bbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bc6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08f05c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8b232c0 con 0x555cf8bc0580 2024-03-26T14:17:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08f05c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8acbc90 con 0x555cf8bbe9f0 2024-03-26T14:17:31.831 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08f05c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8b1f360 con 0x555cf8bbfbc0 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb089000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bc6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb08a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbe9f0 0x555cf8bbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb089000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bc6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58374/0 (socket says 172.21.15.137:58374) 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb089000640 1 -- 172.21.15.137:0/777228836 learned_addr learned my addr 172.21.15.137:0/777228836 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.830+0000 7fb089a00640 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb082000640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb06c0a2800 0x7fb06c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 -- 172.21.15.137:0/777228836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbfbc0 msgr2=0x555cf8bc0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 -- 172.21.15.137:0/777228836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 msgr2=0x555cf8bc6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bc6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 -- 172.21.15.137:0/777228836 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf8af0080 con 0x555cf8bbe9f0 2024-03-26T14:17:31.833 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb089000640 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bc6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:17:31.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb082000640 1 -- 172.21.15.137:0/3217948192 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb0740f1290 con 0x7fb0841049b0 2024-03-26T14:17:31.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb06c0a2800 0x7fb06c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.831+0000 7fb08a400640 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbe9f0 0x555cf8bbee10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009ed0 tx=0x7fb07c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42355ad07120728 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.832+0000 7fb072a00640 1 -- 172.21.15.137:0/777228836 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c01d7e0 con 0x555cf8bbe9f0 2024-03-26T14:17:31.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.832+0000 7fb072a00640 1 -- 172.21.15.137:0/777228836 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb07c015070 con 0x555cf8bbe9f0 2024-03-26T14:17:31.834 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.832+0000 7fb072a00640 1 -- 172.21.15.137:0/777228836 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c01c5e0 con 0x555cf8bbe9f0 2024-03-26T14:17:31.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 -- 172.21.15.137:0/777228836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbe9f0 msgr2=0x555cf8bbee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08a400640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb06c0a2800 0x7fb06c0a4cc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb07c011010 tx=0x7fb07c01cc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbe9f0 0x555cf8bbee10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009ed0 tx=0x7fb07c0107a0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.837 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 -- 172.21.15.137:0/777228836 shutdown_connections 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bc6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 --2- 172.21.15.137:0/777228836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbe9f0 0x555cf8bbee10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 -- 172.21.15.137:0/777228836 >> 172.21.15.137:0/777228836 conn(0x555cf8bb9de0 msgr2=0x555cf8bba1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 -- 172.21.15.137:0/777228836 shutdown_connections 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 -- 172.21.15.137:0/777228836 wait complete. 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.833+0000 7fb08f05c740 1 -- start start 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 Processor -- start 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bc0580 0x555cf8bd0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.838 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8bc7590 con 0x555cf8bbfbc0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8adf0d0 con 0x555cf8bc0580 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf8bc7710 con 0x555cf8bbe9f0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08f05c740 10 client.0 _ll_register_callbacks cb 0x555cf8ae3e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb089000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb08a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bc0580 0x555cf8bd0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.834+0000 7fb089000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58376/0 (socket says 172.21.15.137:58376) 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 -- 172.21.15.137:0/1146412671 learned_addr learned my addr 172.21.15.137:0/1146412671 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089a00640 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 -- 172.21.15.137:0/1146412671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 msgr2=0x555cf8bc8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 -- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bc0580 msgr2=0x555cf8bd0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bc0580 0x555cf8bd0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 -- 172.21.15.137:0/1146412671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cf88373f0 con 0x555cf8bbfbc0 2024-03-26T14:17:31.839 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb089000640 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc9150 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb07402bc70 tx=0x7fb074005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ec9e44816627c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb067400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb067400640 1 -- 172.21.15.137:0/1146412671 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740bd2a0 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.835+0000 7fb067400640 1 -- 172.21.15.137:0/1146412671 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb0740bdba0 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb07c015070 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb067400640 1 -- 172.21.15.137:0/1146412671 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0740044e0 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 10 client.18792 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555cf89017d0 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb067400640 1 -- 172.21.15.137:0/1146412671 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb07c015070 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb067400640 1 -- 172.21.15.137:0/1146412671 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb0740c5c50 con 0x555cf8bbfbc0 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 10 client.18792 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 10 client.18792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 20 client.18792 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8bd4680 0x555cf8bd6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.840 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 4 client.18792 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:31.841 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.836+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cf8b30800 con 0x555cf8bd4680 2024-03-26T14:17:31.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.837+0000 7fb089a00640 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8bd4680 0x555cf8bd6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.838+0000 7fb089a00640 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8bd4680 0x555cf8bd6ac0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aea355410fd9a81d server_cookie=565de14e016f69bc in_seq=0 out_seq=0 2024-03-26T14:17:31.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.838+0000 7fb067400640 10 client.18792 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:17:31.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb067400640 1 -- 172.21.15.137:0/1146412671 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x555cf8b30800 con 0x555cf8bd4680 2024-03-26T14:17:31.842 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb067400640 10 client.18792 handle_command_reply: tid=0 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 client.18792 shutdown 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 msgr2=0x555cf8bc9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc9150 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fb07402bc70 tx=0x7fb074005780 comp rx=0 tx=0).stop 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 shutdown_connections 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8bd4680 0x555cf8bd6ac0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bc0580 0x555cf8bd0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbfbc0 0x555cf8bc9150 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb067400640 0 client.18792 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 --2- 172.21.15.137:0/1146412671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.839+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 >> 172.21.15.137:0/1146412671 conn(0x555cf8bb9de0 msgr2=0x555cf8bb9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.840+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 shutdown_connections 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.840+0000 7fb08f05c740 1 -- 172.21.15.137:0/1146412671 wait complete. 2024-03-26T14:17:31.843 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.840+0000 7fb08f05c740 20 client.18792 trim_cache size 0 max 16384 2024-03-26T14:17:31.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.853+0000 7fb08f05c740 1 Processor -- start 2024-03-26T14:17:31.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 -- start start 2024-03-26T14:17:31.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bd0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bc0580 0x555cf8bc3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.855 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841ae6c0 con 0x555cf8bbe9f0 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841ac1c0 con 0x555cf8bbfbc0 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb08f05c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841aa240 con 0x555cf8bc0580 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58384/0 (socket says 172.21.15.137:58384) 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 -- 172.21.15.137:0/2326520238 learned_addr learned my addr 172.21.15.137:0/2326520238 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089000640 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bd0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 -- 172.21.15.137:0/2326520238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bc0580 msgr2=0x555cf8bc3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bc0580 0x555cf8bc3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 -- 172.21.15.137:0/2326520238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 msgr2=0x555cf8bd0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bd0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.856 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.854+0000 7fb089a00640 1 -- 172.21.15.137:0/2326520238 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf8bd7140 con 0x555cf8bbe9f0 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.855+0000 7fb089a00640 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc2df0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x555cf8bd7340 tx=0x7fb07801da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5542984fb9fd6f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.855+0000 7fb072000640 1 -- 172.21.15.137:0/2326520238 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07801d020 con 0x555cf8bbe9f0 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.855+0000 7fb072000640 1 -- 172.21.15.137:0/2326520238 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb07801d1c0 con 0x555cf8bbe9f0 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.855+0000 7fb072000640 1 -- 172.21.15.137:0/2326520238 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb078013520 con 0x555cf8bbe9f0 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 -- 172.21.15.137:0/2326520238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 msgr2=0x555cf8bc2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc2df0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x555cf8bd7340 tx=0x7fb07801da50 comp rx=0 tx=0).stop 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 -- 172.21.15.137:0/2326520238 shutdown_connections 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bd0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bc0580 0x555cf8bc3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 --2- 172.21.15.137:0/2326520238 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bbe9f0 0x555cf8bc2df0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 -- 172.21.15.137:0/2326520238 >> 172.21.15.137:0/2326520238 conn(0x555cf8bb9de0 msgr2=0x555cf8bbd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 -- 172.21.15.137:0/2326520238 shutdown_connections 2024-03-26T14:17:31.859 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 -- 172.21.15.137:0/2326520238 wait complete. 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 -- start start 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.856+0000 7fb08f05c740 1 Processor -- start 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bcff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bcea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841aa840 con 0x555cf8bc0580 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841abb60 con 0x555cf8bbfbc0 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0841aaea0 con 0x555cf8bbe9f0 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08f05c740 10 client.0 _ll_register_callbacks cb 0x7fb0740ee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bcea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08a400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bcea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58394/0 (socket says 172.21.15.137:58394) 2024-03-26T14:17:31.860 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb08a400640 1 -- 172.21.15.137:0/963749593 learned_addr learned my addr 172.21.15.137:0/963749593 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb089000640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.857+0000 7fb089a00640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bcff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08a400640 1 -- 172.21.15.137:0/963749593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 msgr2=0x555cf8bcff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08a400640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bcff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08a400640 1 -- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 msgr2=0x555cf8bce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08a400640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08a400640 1 -- 172.21.15.137:0/963749593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cf8bcf160 con 0x555cf8bc0580 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08a400640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bcea70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fb07c010ca0 tx=0x7fb07c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c63e01ad08dfe2c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:31.861 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb067400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb067400640 1 -- 172.21.15.137:0/963749593 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c035020 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb067400640 1 -- 172.21.15.137:0/963749593 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fb07c011cb0 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.858+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb07801d1c0 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb067400640 1 -- 172.21.15.137:0/963749593 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb07c040050 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 10 client.18798 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555cf8c30f20 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb067400640 1 -- 172.21.15.137:0/963749593 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb07801d1c0 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb067400640 1 -- 172.21.15.137:0/963749593 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fb07c00e960 con 0x555cf8bc0580 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 10 client.18798 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:31.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 10 client.18798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 20 client.18798 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8c316a0 0x555cf8c33ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 4 client.18798 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.859+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555cf8b2ab50 con 0x555cf8c316a0 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.860+0000 7fb089a00640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8c316a0 0x555cf8c33ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.860+0000 7fb089a00640 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8c316a0 0x555cf8c33ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f782f3d98ea69964 server_cookie=c610060e73be8758 in_seq=0 out_seq=0 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.860+0000 7fb067400640 10 client.18798 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb067400640 1 -- 172.21.15.137:0/963749593 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555cf8b2ab50 con 0x555cf8c316a0 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb067400640 10 client.18798 handle_command_reply: tid=0 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 client.18798 shutdown 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 msgr2=0x555cf8bcea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bcea70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fb07c010ca0 tx=0x7fb07c016000 comp rx=0 tx=0).stop 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 shutdown_connections 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x555cf8c316a0 0x555cf8c33ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.865 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555cf8bbfbc0 0x555cf8bce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x555cf8bc0580 0x555cf8bcea70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 --2- 172.21.15.137:0/963749593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cf8bbe9f0 0x555cf8bcff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb067400640 0 client.18798 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 >> 172.21.15.137:0/963749593 conn(0x555cf8bb9de0 msgr2=0x555cf8bb88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.861+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 shutdown_connections 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.862+0000 7fb08f05c740 1 -- 172.21.15.137:0/963749593 wait complete. 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.862+0000 7fb08f05c740 20 client.18798 trim_cache size 0 max 16384 2024-03-26T14:17:31.866 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.862+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb06c0a2800 msgr2=0x7fb06c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.862+0000 7fb08ae00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb06c0a2800 0x7fb06c0a4cc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb07c011010 tx=0x7fb07c01cc30 comp rx=0 tx=0).stop 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.862+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 msgr2=0x7fb084199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.862+0000 7fb08ae00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084199f50 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb074000c00 tx=0x7fb0740042f0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 shutdown_connections 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fb06c0a2800 0x7fb06c0a4cc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb084105370 0x7fb0841a0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0841049b0 0x7fb084199f50 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 --2- 172.21.15.137:0/3217948192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0841037c0 0x7fb084199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 >> 172.21.15.137:0/3217948192 conn(0x7fb0840fed50 msgr2=0x7fb084107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 shutdown_connections 2024-03-26T14:17:31.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:31.863+0000 7fb08ae00640 1 -- 172.21.15.137:0/3217948192 wait complete. 2024-03-26T14:17:31.881 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:17:31.881 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:17:32.088 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 -- 172.21.15.137:0/3716779137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea80ff560 msgr2=0x7f8ea80ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 --2- 172.21.15.137:0/3716779137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea80ff560 0x7f8ea80ff980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009900 tx=0x7f8e9802bc50 comp rx=0 tx=0).stop 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 -- 172.21.15.137:0/3716779137 shutdown_connections 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 --2- 172.21.15.137:0/3716779137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea8101030 0x7f8ea81058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 --2- 172.21.15.137:0/3716779137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea8100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 --2- 172.21.15.137:0/3716779137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea80ff560 0x7f8ea80ff980 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 -- 172.21.15.137:0/3716779137 >> 172.21.15.137:0/3716779137 conn(0x7f8ea80fae60 msgr2=0x7f8ea80fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 -- 172.21.15.137:0/3716779137 shutdown_connections 2024-03-26T14:17:32.089 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.087+0000 7f8eada00640 1 -- 172.21.15.137:0/3716779137 wait complete. 2024-03-26T14:17:32.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.088+0000 7f8eada00640 1 Processor -- start 2024-03-26T14:17:32.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 -- start start 2024-03-26T14:17:32.091 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea80ff560 0x7f8ea819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea8101030 0x7f8ea81a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8113810 con 0x7f8ea8100690 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8113690 con 0x7f8ea8101030 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8eada00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8113990 con 0x7f8ea80ff560 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8ea6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8ea6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58398/0 (socket says 172.21.15.137:58398) 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8ea6000640 1 -- 172.21.15.137:0/2257918615 learned_addr learned my addr 172.21.15.137:0/2257918615 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8ea7400640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea8101030 0x7f8ea81a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.089+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea80ff560 0x7f8ea819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8ea6000640 1 -- 172.21.15.137:0/2257918615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea80ff560 msgr2=0x7f8ea819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8ea6000640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea80ff560 0x7f8ea819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.092 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8ea6000640 1 -- 172.21.15.137:0/2257918615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea8101030 msgr2=0x7f8ea81a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8ea6000640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea8101030 0x7f8ea81a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8ea6000640 1 -- 172.21.15.137:0/2257918615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea81a5b50 con 0x7f8ea8100690 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8ea6000640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea819e470 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000ea60 tx=0x7f8e9000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8e9f400640 1 -- 172.21.15.137:0/2257918615 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9000cde0 con 0x7f8ea8100690 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8e9f400640 1 -- 172.21.15.137:0/2257918615 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e900035f0 con 0x7f8ea8100690 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea81a5e40 con 0x7f8ea8100690 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8e9f400640 1 -- 172.21.15.137:0/2257918615 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90010690 con 0x7f8ea8100690 2024-03-26T14:17:32.093 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.090+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea81a6300 con 0x7f8ea8100690 2024-03-26T14:17:32.094 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8e9f400640 1 -- 172.21.15.137:0/2257918615 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8e900037a0 con 0x7f8ea8100690 2024-03-26T14:17:32.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 Processor -- start 2024-03-26T14:17:32.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 -- start start 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a809c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a80ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80b3a0 0x55fb6a811c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6a76e2c0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6a716c90 con 0x55fb6a809810 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.094+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6a76a360 con 0x55fb6a80b3a0 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a809c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a80ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a809c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:48186/0 (socket says 172.21.15.137:48186) 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 -- 172.21.15.137:0/1602362293 learned_addr learned my addr 172.21.15.137:0/1602362293 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea6000640 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80b3a0 0x55fb6a811c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 -- 172.21.15.137:0/1602362293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80b3a0 msgr2=0x55fb6a811c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.097 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80b3a0 0x55fb6a811c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 -- 172.21.15.137:0/1602362293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 msgr2=0x55fb6a80ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a80ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 -- 172.21.15.137:0/1602362293 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb6a73b080 con 0x55fb6a809810 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea6000640 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80b3a0 0x55fb6a811c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8e9f400640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e8c0a2800 0x7f8e8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8e9f400640 1 -- 172.21.15.137:0/2257918615 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8e900cf130 con 0x7f8ea8100690 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.095+0000 7f8ea7400640 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a809c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0010370 tx=0x7f8ea0010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fd77544354bbeae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.096+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e8c0a2800 0x7f8e8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.096+0000 7f8e97e00640 1 -- 172.21.15.137:0/1602362293 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0016070 con 0x55fb6a809810 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.096+0000 7f8e97e00640 1 -- 172.21.15.137:0/1602362293 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ea0004d20 con 0x55fb6a809810 2024-03-26T14:17:32.098 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.096+0000 7f8e97e00640 1 -- 172.21.15.137:0/1602362293 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0005310 con 0x55fb6a809810 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 -- 172.21.15.137:0/1602362293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 msgr2=0x55fb6a809c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a809c30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0010370 tx=0x7f8ea0010830 comp rx=0 tx=0).stop 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e8c0a2800 0x7f8e8c0a4cc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e98005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 -- 172.21.15.137:0/1602362293 shutdown_connections 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80b3a0 0x55fb6a811c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a80ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/1602362293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a809c30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 -- 172.21.15.137:0/1602362293 >> 172.21.15.137:0/1602362293 conn(0x55fb6a804c00 msgr2=0x55fb6a805000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 -- 172.21.15.137:0/1602362293 shutdown_connections 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 -- 172.21.15.137:0/1602362293 wait complete. 2024-03-26T14:17:32.102 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 -- start start 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.097+0000 7f8eb03fd740 1 Processor -- start 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a813ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a814000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6a8123b0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6a72a0d0 con 0x55fb6a80b3a0 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6a812530 con 0x55fb6a809810 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8eb03fd740 10 client.0 _ll_register_callbacks cb 0x55fb6a72ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8ea6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a814000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8ea6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a814000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58416/0 (socket says 172.21.15.137:58416) 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8ea6000640 1 -- 172.21.15.137:0/3024991142 learned_addr learned my addr 172.21.15.137:0/3024991142 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8ea7400640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.098+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a813ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8ea6000640 1 -- 172.21.15.137:0/3024991142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 msgr2=0x55fb6a813ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.103 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8ea6000640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a813ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8ea6000640 1 -- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80b3a0 msgr2=0x55fb6a81b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8ea6000640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8ea6000640 1 -- 172.21.15.137:0/3024991142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb6a4823f0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8ea6000640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a814000 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8e900a3930 tx=0x7f8e9009b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305d9814dfe169fb server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8e94c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8e94c00640 1 -- 172.21.15.137:0/3024991142 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9009bdc0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8e94c00640 1 -- 172.21.15.137:0/3024991142 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8e900d10f0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ea0004d20 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8e94c00640 1 -- 172.21.15.137:0/3024991142 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e900860d0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8eb03fd740 10 client.18810 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.099+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fb6a54c7d0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8e94c00640 1 -- 172.21.15.137:0/3024991142 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ea0004d20 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8e94c00640 1 -- 172.21.15.137:0/3024991142 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8e900ccce0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8eb03fd740 10 client.18810 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8eb03fd740 10 client.18810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8eb03fd740 20 client.18810 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a81f4a0 0x55fb6a8218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8eb03fd740 4 client.18810 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb6a77b800 con 0x55fb6a81f4a0 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.100+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a81f4a0 0x55fb6a8218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.101+0000 7f8e94c00640 10 client.18810 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:17:32.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.101+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a81f4a0 0x55fb6a8218b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbf3635560324532 server_cookie=8b5d75e59c5adf56 in_seq=0 out_seq=0 2024-03-26T14:17:32.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8e94c00640 1 -- 172.21.15.137:0/3024991142 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fb6a77b800 con 0x55fb6a81f4a0 2024-03-26T14:17:32.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8e94c00640 10 client.18810 handle_command_reply: tid=0 2024-03-26T14:17:32.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 client.18810 shutdown 2024-03-26T14:17:32.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 msgr2=0x55fb6a814000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a814000 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f8e900a3930 tx=0x7f8e9009b200 comp rx=0 tx=0).stop 2024-03-26T14:17:32.106 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 shutdown_connections 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a81f4a0 0x55fb6a8218b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80a9e0 0x55fb6a814000 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8e94c00640 0 client.18810 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/3024991142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a813ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 >> 172.21.15.137:0/3024991142 conn(0x55fb6a804c00 msgr2=0x55fb6a803eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 shutdown_connections 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 1 -- 172.21.15.137:0/3024991142 wait complete. 2024-03-26T14:17:32.107 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.102+0000 7f8eb03fd740 20 client.18810 trim_cache size 0 max 16384 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 Processor -- start 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 -- start start 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a808930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80a9e0 0x55fb6a81b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a803620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81b2e30 con 0x55fb6a80b3a0 2024-03-26T14:17:32.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81b0930 con 0x55fb6a809810 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81ae9b0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a803620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8ea6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a808930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8ea6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a808930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:48204/0 (socket says 172.21.15.137:48204) 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8ea6a00640 1 -- 172.21.15.137:0/475056764 learned_addr learned my addr 172.21.15.137:0/475056764 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.117+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a803620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58426/0 (socket says 172.21.15.137:58426) 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6000640 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80a9e0 0x55fb6a81b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6a00640 1 -- 172.21.15.137:0/475056764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80a9e0 msgr2=0x55fb6a81b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80a9e0 0x55fb6a81b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6a00640 1 -- 172.21.15.137:0/475056764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 msgr2=0x55fb6a803620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a803620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6a00640 1 -- 172.21.15.137:0/475056764 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb6a8004e0 con 0x55fb6a809810 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a808930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e980096c0 tx=0x7f8e98044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a000234bd21c87a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8e97400640 1 -- 172.21.15.137:0/475056764 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98004110 con 0x55fb6a809810 2024-03-26T14:17:32.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8e97400640 1 -- 172.21.15.137:0/475056764 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e98058070 con 0x55fb6a809810 2024-03-26T14:17:32.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.118+0000 7f8e97400640 1 -- 172.21.15.137:0/475056764 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98004920 con 0x55fb6a809810 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 -- 172.21.15.137:0/475056764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 msgr2=0x55fb6a808930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a808930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e980096c0 tx=0x7f8e98044000 comp rx=0 tx=0).stop 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 -- 172.21.15.137:0/475056764 shutdown_connections 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a803620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a80a9e0 0x55fb6a81b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/475056764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a809810 0x55fb6a808930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 -- 172.21.15.137:0/475056764 >> 172.21.15.137:0/475056764 conn(0x55fb6a804c00 msgr2=0x55fb6a80e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 -- 172.21.15.137:0/475056764 shutdown_connections 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.119+0000 7f8eb03fd740 1 -- 172.21.15.137:0/475056764 wait complete. 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 -- start start 2024-03-26T14:17:32.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 Processor -- start 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a8172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80a9e0 0x55fb6a8177e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81aefb0 con 0x55fb6a80b3a0 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81b02d0 con 0x55fb6a80a9e0 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea81af610 con 0x55fb6a809810 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.120+0000 7f8eb03fd740 10 client.0 _ll_register_callbacks cb 0x7f8e90003d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58428/0 (socket says 172.21.15.137:58428) 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 -- 172.21.15.137:0/955894904 learned_addr learned my addr 172.21.15.137:0/955894904 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea6000640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80a9e0 0x55fb6a8177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a8172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 -- 172.21.15.137:0/955894904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 msgr2=0x55fb6a8172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a8172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 -- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80a9e0 msgr2=0x55fb6a8177e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.125 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80a9e0 0x55fb6a8177e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 -- 172.21.15.137:0/955894904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb6a81ada0 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a8172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8ea7400640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81a7b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009f40 tx=0x7f8ea0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186855bc720a22a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.121+0000 7f8e94c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8e94c00640 1 -- 172.21.15.137:0/955894904 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0016030 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8e94c00640 1 -- 172.21.15.137:0/955894904 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f8ea00052b0 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e98058070 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8e94c00640 1 -- 172.21.15.137:0/955894904 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ea0034050 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 10 client.18816 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55fb6a87b7e0 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8e94c00640 1 -- 172.21.15.137:0/955894904 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e98058070 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8e94c00640 1 -- 172.21.15.137:0/955894904 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f8ea002f030 con 0x55fb6a80b3a0 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 10 client.18816 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 10 client.18816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:17:32.126 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 20 client.18816 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:32.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a87bec0 0x55fb6a87e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 4 client.18816 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:17:32.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.122+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fb6a775b50 con 0x55fb6a87bec0 2024-03-26T14:17:32.127 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.123+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a87bec0 0x55fb6a87e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.123+0000 7f8ea6a00640 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a87bec0 0x55fb6a87e2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec57b11530fd85d server_cookie=404e92becbcf2f20 in_seq=0 out_seq=0 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.123+0000 7f8e94c00640 10 client.18816 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8e94c00640 1 -- 172.21.15.137:0/955894904 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fb6a775b50 con 0x55fb6a87bec0 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8e94c00640 10 client.18816 handle_command_reply: tid=0 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 client.18816 shutdown 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 msgr2=0x55fb6a81a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81a7b0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009f40 tx=0x7f8ea0003ba0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 shutdown_connections 2024-03-26T14:17:32.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x55fb6a87bec0 0x55fb6a87e2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fb6a80a9e0 0x55fb6a8177e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55fb6a80b3a0 0x55fb6a81a7b0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 --2- 172.21.15.137:0/955894904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb6a809810 0x55fb6a8172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8e94c00640 0 client.18816 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 >> 172.21.15.137:0/955894904 conn(0x55fb6a804c00 msgr2=0x55fb6a803620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 shutdown_connections 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 1 -- 172.21.15.137:0/955894904 wait complete. 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.124+0000 7f8eb03fd740 20 client.18816 trim_cache size 0 max 16384 2024-03-26T14:17:32.129 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e8c0a2800 msgr2=0x7f8e8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e8c0a2800 0x7f8e8c0a4cc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e98005a20 comp rx=0 tx=0).stop 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 msgr2=0x7f8ea819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea819e470 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000ea60 tx=0x7f8e9000ef20 comp rx=0 tx=0).stop 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 shutdown_connections 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8e8c0a2800 0x7f8e8c0a4cc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ea8101030 0x7f8ea81a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ea8100690 0x7f8ea819e470 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 --2- 172.21.15.137:0/2257918615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ea80ff560 0x7f8ea819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.130 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 >> 172.21.15.137:0/2257918615 conn(0x7f8ea80fae60 msgr2=0x7f8ea81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 shutdown_connections 2024-03-26T14:17:32.131 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.125+0000 7f8eada00640 1 -- 172.21.15.137:0/2257918615 wait complete. 2024-03-26T14:17:32.144 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:17:32.144 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 -- 172.21.15.137:0/3386549768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 msgr2=0x7f5744104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 --2- 172.21.15.137:0/3386549768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744104e10 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5730009900 tx=0x7f573002bc30 comp rx=0 tx=0).stop 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 -- 172.21.15.137:0/3386549768 shutdown_connections 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 --2- 172.21.15.137:0/3386549768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5744105350 0x7f574410bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 --2- 172.21.15.137:0/3386549768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744104e10 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 --2- 172.21.15.137:0/3386549768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57441037a0 0x7f5744103bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.353+0000 7f574a800640 1 -- 172.21.15.137:0/3386549768 >> 172.21.15.137:0/3386549768 conn(0x7f57440fed10 msgr2=0x7f5744101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.354+0000 7f574a800640 1 -- 172.21.15.137:0/3386549768 shutdown_connections 2024-03-26T14:17:32.355 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.354+0000 7f574a800640 1 -- 172.21.15.137:0/3386549768 wait complete. 2024-03-26T14:17:32.356 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 Processor -- start 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 -- start start 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57441037a0 0x7f57441999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5744105350 0x7f57441a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574410f320 con 0x7f5744104990 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574410f1a0 con 0x7f57441037a0 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f574a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574410f4a0 con 0x7f5744105350 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f5748a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f5748a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58458/0 (socket says 172.21.15.137:58458) 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.355+0000 7f5748a00640 1 -- 172.21.15.137:0/568383159 learned_addr learned my addr 172.21.15.137:0/568383159 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5749e00640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5744105350 0x7f57441a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5748a00640 1 -- 172.21.15.137:0/568383159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5744105350 msgr2=0x7f57441a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5748a00640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5744105350 0x7f57441a0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5748a00640 1 -- 172.21.15.137:0/568383159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57441037a0 msgr2=0x7f57441999e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5748a00640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57441037a0 0x7f57441999e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.357 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5748a00640 1 -- 172.21.15.137:0/568383159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57441a1600 con 0x7f5744104990 2024-03-26T14:17:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.356+0000 7f5748a00640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744199f20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5730000c00 tx=0x7f57300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.357+0000 7f5742000640 1 -- 172.21.15.137:0/568383159 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730043020 con 0x7f5744104990 2024-03-26T14:17:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.357+0000 7f5742000640 1 -- 172.21.15.137:0/568383159 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f57300044c0 con 0x7f5744104990 2024-03-26T14:17:32.358 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.357+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57441a1890 con 0x7f5744104990 2024-03-26T14:17:32.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.357+0000 7f5742000640 1 -- 172.21.15.137:0/568383159 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730003c60 con 0x7f5744104990 2024-03-26T14:17:32.359 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.357+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57441a1de0 con 0x7f5744104990 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 Processor -- start 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 -- start start 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f5742000640 1 -- 172.21.15.137:0/568383159 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5730046020 con 0x7f5744104990 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492af5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492af7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492afde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492a5a2c0 con 0x563492af6bc0 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492a02c90 con 0x563492af7580 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.361+0000 7f574e9fa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492a56360 con 0x563492af59f0 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5749400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492af7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5748a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492afde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5749400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492af7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58468/0 (socket says 172.21.15.137:58468) 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5749400640 1 -- 172.21.15.137:0/3865018817 learned_addr learned my addr 172.21.15.137:0/3865018817 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5749e00640 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492af5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5749400640 1 -- 172.21.15.137:0/3865018817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 msgr2=0x563492af5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.362+0000 7f5749400640 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492af5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5749400640 1 -- 172.21.15.137:0/3865018817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 msgr2=0x563492afde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5749400640 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492afde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5749400640 1 -- 172.21.15.137:0/3865018817 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563492a27080 con 0x563492af6bc0 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5749e00640 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492af5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5749400640 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492af7040 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5738009e30 tx=0x7f573800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37800fc613179d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5736600640 1 -- 172.21.15.137:0/3865018817 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738010630 con 0x563492af6bc0 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5736600640 1 -- 172.21.15.137:0/3865018817 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5738010c80 con 0x563492af6bc0 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5742000640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f572c0a2800 0x7f572c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.363+0000 7f5736600640 1 -- 172.21.15.137:0/3865018817 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738018c50 con 0x563492af6bc0 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f5742000640 1 -- 172.21.15.137:0/568383159 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f57300f1210 con 0x7f5744104990 2024-03-26T14:17:32.365 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f5749e00640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f572c0a2800 0x7f572c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 -- 172.21.15.137:0/3865018817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 msgr2=0x563492af7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492af7040 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5738009e30 tx=0x7f573800e720 comp rx=0 tx=0).stop 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 -- 172.21.15.137:0/3865018817 shutdown_connections 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492afde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492af7040 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3865018817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492af5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.364+0000 7f574e9fa740 1 -- 172.21.15.137:0/3865018817 >> 172.21.15.137:0/3865018817 conn(0x563492af0de0 msgr2=0x563492af11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.365+0000 7f574e9fa740 1 -- 172.21.15.137:0/3865018817 shutdown_connections 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.365+0000 7f574e9fa740 1 -- 172.21.15.137:0/3865018817 wait complete. 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.365+0000 7f5749e00640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f572c0a2800 0x7f572c0a4cc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x563492af6ac0 tx=0x7f573c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.365+0000 7f574e9fa740 1 -- start start 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.365+0000 7f574e9fa740 1 Processor -- start 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af59f0 0x563492affcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492b00200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af7580 0x563492b071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492afe590 con 0x563492af6bc0 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492a160d0 con 0x563492af59f0 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492afe710 con 0x563492af7580 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f574e9fa740 10 client.0 _ll_register_callbacks cb 0x563492a1ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492b00200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5749400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af59f0 0x563492affcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492b00200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58482/0 (socket says 172.21.15.137:58482) 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 -- 172.21.15.137:0/2667894529 learned_addr learned my addr 172.21.15.137:0/2667894529 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5749e00640 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af7580 0x563492b071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 -- 172.21.15.137:0/2667894529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af7580 msgr2=0x563492b071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af7580 0x563492b071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 -- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af59f0 msgr2=0x563492affcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af59f0 0x563492affcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.366+0000 7f5748a00640 1 -- 172.21.15.137:0/2667894529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56349276e3f0 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f5748a00640 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492b00200 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f57300c5b30 tx=0x7f57300bd750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fde266be40987b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f572aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f572aa00640 1 -- 172.21.15.137:0/2667894529 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573003c350 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f572aa00640 1 -- 172.21.15.137:0/2667894529 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f57300f4dc0 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5738010c80 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f572aa00640 1 -- 172.21.15.137:0/2667894529 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57300bdfc0 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f574e9fa740 10 client.18828 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5634928387d0 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.367+0000 7f572aa00640 1 -- 172.21.15.137:0/2667894529 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5738010c80 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f572aa00640 1 -- 172.21.15.137:0/2667894529 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f57300c5bd0 con 0x563492af6bc0 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f574e9fa740 10 client.18828 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f574e9fa740 10 client.18828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:32.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f574e9fa740 20 client.18828 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:32.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f574e9fa740 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b0b640 0x563492b0da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f574e9fa740 4 client.18828 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:32.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563492a67800 con 0x563492b0b640 2024-03-26T14:17:32.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-03-26T14:17:32.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-26T14:17:32.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-03-26T14:17:32.372 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-03-26T14:17:32.373 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-26T14:17:32.374 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-26T14:17:32.375 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-03-26T14:17:32.376 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-26T14:17:32.377 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-03-26T14:17:32.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.368+0000 7f5749400640 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b0b640 0x563492b0da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.369+0000 7f5749400640 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b0b640 0x563492b0da90 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4e0079cccc4aa8 server_cookie=a7441a5cb32fad00 in_seq=0 out_seq=0 2024-03-26T14:17:32.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.369+0000 7f572aa00640 10 client.18828 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:32.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f572aa00640 1 -- 172.21.15.137:0/2667894529 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563492a67800 con 0x563492b0b640 2024-03-26T14:17:32.378 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f572aa00640 10 client.18828 handle_command_reply: tid=0 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 client.18828 shutdown 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 msgr2=0x563492b00200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492b00200 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f57300c5b30 tx=0x7f57300bd750 comp rx=0 tx=0).stop 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 shutdown_connections 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b0b640 0x563492b0da90 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af7580 0x563492b071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af6bc0 0x563492b00200 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 --2- 172.21.15.137:0/2667894529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af59f0 0x563492affcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f572aa00640 0 client.18828 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 >> 172.21.15.137:0/2667894529 conn(0x563492af0de0 msgr2=0x563492af0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 shutdown_connections 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 1 -- 172.21.15.137:0/2667894529 wait complete. 2024-03-26T14:17:32.379 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.371+0000 7f574e9fa740 20 client.18828 trim_cache size 0 max 16384 2024-03-26T14:17:32.385 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.384+0000 7f574e9fa740 1 Processor -- start 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 -- start start 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 0x563492af11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af6bc0 0x563492b07190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492af9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441ae730 con 0x563492af59f0 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441ac230 con 0x563492af7580 2024-03-26T14:17:32.386 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f574e9fa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441aa2b0 con 0x563492af6bc0 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 0x563492af11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 0x563492af11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58496/0 (socket says 172.21.15.137:58496) 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 -- 172.21.15.137:0/1409180439 learned_addr learned my addr 172.21.15.137:0/1409180439 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 -- 172.21.15.137:0/1409180439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af6bc0 msgr2=0x563492b07190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af6bc0 0x563492b07190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 -- 172.21.15.137:0/1409180439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 msgr2=0x563492af9c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.385+0000 7f5749400640 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492af9c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.386+0000 7f5749400640 1 -- 172.21.15.137:0/1409180439 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563492af1b50 con 0x563492af59f0 2024-03-26T14:17:32.387 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.386+0000 7f5749400640 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 0x563492af11c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5738006010 tx=0x7f5738009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26c5e89c7b75a39 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.386+0000 7f5735c00640 1 -- 172.21.15.137:0/1409180439 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738031070 con 0x563492af59f0 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.386+0000 7f5735c00640 1 -- 172.21.15.137:0/1409180439 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5738004c00 con 0x563492af59f0 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.386+0000 7f5735c00640 1 -- 172.21.15.137:0/1409180439 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738029070 con 0x563492af59f0 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 -- 172.21.15.137:0/1409180439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 msgr2=0x563492af11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 0x563492af11c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f5738006010 tx=0x7f5738009270 comp rx=0 tx=0).stop 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 -- 172.21.15.137:0/1409180439 shutdown_connections 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af7580 0x563492af9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af6bc0 0x563492b07190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 --2- 172.21.15.137:0/1409180439 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af59f0 0x563492af11c0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.388 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 -- 172.21.15.137:0/1409180439 >> 172.21.15.137:0/1409180439 conn(0x563492af0de0 msgr2=0x563492af48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 -- 172.21.15.137:0/1409180439 shutdown_connections 2024-03-26T14:17:32.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.387+0000 7f574e9fa740 1 -- 172.21.15.137:0/1409180439 wait complete. 2024-03-26T14:17:32.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.388+0000 7f574e9fa740 1 -- start start 2024-03-26T14:17:32.389 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.388+0000 7f574e9fa740 1 Processor -- start 2024-03-26T14:17:32.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492b069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af6bc0 0x563492b06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 0x563492b034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441aa8b0 con 0x563492af7580 2024-03-26T14:17:32.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441abbd0 con 0x563492af6bc0 2024-03-26T14:17:32.390 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57441aaf10 con 0x563492af59f0 2024-03-26T14:17:32.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f574e9fa740 10 client.0 _ll_register_callbacks cb 0x7f57300095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:32.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f5749e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 0x563492b034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f5749e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 0x563492b034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58506/0 (socket says 172.21.15.137:58506) 2024-03-26T14:17:32.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f5749e00640 1 -- 172.21.15.137:0/3953700940 learned_addr learned my addr 172.21.15.137:0/3953700940 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:32.391 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.389+0000 7f5749400640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492b069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.392 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:32 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/928105188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:17:32.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5748a00640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af6bc0 0x563492b06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5749e00640 1 -- 172.21.15.137:0/3953700940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 msgr2=0x563492b069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5749e00640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492b069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.392 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5749e00640 1 -- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af6bc0 msgr2=0x563492b06f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:32.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5749e00640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af6bc0 0x563492b06f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5749e00640 1 -- 172.21.15.137:0/3953700940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563492b03b30 con 0x563492af7580 2024-03-26T14:17:32.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5748a00640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af6bc0 0x563492b06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:17:32.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f5749e00640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 0x563492b034f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f573c005e00 tx=0x7f573c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c5c714160e221c server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:32.393 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.390+0000 7f572aa00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.395+0000 7f572aa00640 1 -- 172.21.15.137:0/3953700940 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c010ad0 con 0x563492af7580 2024-03-26T14:17:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.395+0000 7f572aa00640 1 -- 172.21.15.137:0/3953700940 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f573c010c70 con 0x563492af7580 2024-03-26T14:17:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.395+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5738004c00 con 0x563492af7580 2024-03-26T14:17:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f572aa00640 1 -- 172.21.15.137:0/3953700940 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f573c030050 con 0x563492af7580 2024-03-26T14:17:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f574e9fa740 10 client.18834 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:32.397 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563492b67bb0 con 0x563492af7580 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f572aa00640 1 -- 172.21.15.137:0/3953700940 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5738004c00 con 0x563492af7580 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f572aa00640 1 -- 172.21.15.137:0/3953700940 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f573c019040 con 0x563492af7580 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f574e9fa740 10 client.18834 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f574e9fa740 10 client.18834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f574e9fa740 20 client.18834 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.396+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b68570 0x563492b6a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.397+0000 7f574e9fa740 4 client.18834 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.397+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563492a61b50 con 0x563492b68570 2024-03-26T14:17:32.398 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.397+0000 7f5749400640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b68570 0x563492b6a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:32.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f572aa00640 10 client.18834 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:32.399 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f5749400640 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b68570 0x563492b6a990 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12669762d11fc025 server_cookie=53d1cad0156e6cb3 in_seq=0 out_seq=0 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f572aa00640 1 -- 172.21.15.137:0/3953700940 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563492a61b50 con 0x563492b68570 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f572aa00640 10 client.18834 handle_command_reply: tid=0 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 client.18834 shutdown 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 msgr2=0x563492b034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 0x563492b034f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f573c005e00 tx=0x7f573c0046c0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 shutdown_connections 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x563492b68570 0x563492b6a990 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563492af6bc0 0x563492b06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x563492af7580 0x563492b034f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 --2- 172.21.15.137:0/3953700940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563492af59f0 0x563492b069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f572aa00640 0 client.18834 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.398+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 >> 172.21.15.137:0/3953700940 conn(0x563492af0de0 msgr2=0x563492af48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 shutdown_connections 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574e9fa740 1 -- 172.21.15.137:0/3953700940 wait complete. 2024-03-26T14:17:32.400 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574e9fa740 20 client.18834 trim_cache size 0 max 16384 2024-03-26T14:17:32.401 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:17:32.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f572c0a2800 msgr2=0x7f572c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574a800640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f572c0a2800 0x7f572c0a4cc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x563492af6ac0 tx=0x7f573c005f90 comp rx=0 tx=0).stop 2024-03-26T14:17:32.401 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 msgr2=0x7f5744199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.399+0000 7f574a800640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744199f20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5730000c00 tx=0x7f57300040c0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 shutdown_connections 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f572c0a2800 0x7f572c0a4cc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5744105350 0x7f57441a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5744104990 0x7f5744199f20 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 --2- 172.21.15.137:0/568383159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f57441037a0 0x7f57441999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 >> 172.21.15.137:0/568383159 conn(0x7f57440fed10 msgr2=0x7f5744107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 shutdown_connections 2024-03-26T14:17:32.402 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:32.400+0000 7f574a800640 1 -- 172.21.15.137:0/568383159 wait complete. 2024-03-26T14:17:32.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:32 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/928105188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:17:32.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:32 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/928105188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:17:33.420 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-03-26T14:17:33.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:17:33.420 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-03-26T14:17:33.670 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 -- 172.21.15.137:0/3688442029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c10d9c0 msgr2=0x7fbb1c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 --2- 172.21.15.137:0/3688442029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c10d9c0 0x7fbb1c10de40 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009ca0 tx=0x7fbb1802c100 comp rx=0 tx=0).stop 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 -- 172.21.15.137:0/3688442029 shutdown_connections 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 --2- 172.21.15.137:0/3688442029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c10b010 0x7fbb1c10d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 --2- 172.21.15.137:0/3688442029 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c10d9c0 0x7fbb1c10de40 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 --2- 172.21.15.137:0/3688442029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c10aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 -- 172.21.15.137:0/3688442029 >> 172.21.15.137:0/3688442029 conn(0x7fbb1c06e500 msgr2=0x7fbb1c070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 -- 172.21.15.137:0/3688442029 shutdown_connections 2024-03-26T14:17:33.671 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.669+0000 7fbb24800640 1 -- 172.21.15.137:0/3688442029 wait complete. 2024-03-26T14:17:33.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.671+0000 7fbb24800640 1 Processor -- start 2024-03-26T14:17:33.672 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.671+0000 7fbb24800640 1 -- start start 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.672+0000 7fbb24800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c1a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.672+0000 7fbb24800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c10b010 0x7fbb1c1a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.672+0000 7fbb24800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb1c10d9c0 0x7fbb1c1adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.672+0000 7fbb24800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c11c000 con 0x7fbb1c1086a0 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.672+0000 7fbb24800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c11be80 con 0x7fbb1c10b010 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.672+0000 7fbb24800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c11c180 con 0x7fbb1c10d9c0 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.673+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c1a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.673+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c1a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58530/0 (socket says 172.21.15.137:58530) 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.673+0000 7fbb21e00640 1 -- 172.21.15.137:0/2976771532 learned_addr learned my addr 172.21.15.137:0/2976771532 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.674 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.673+0000 7fbb21400640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c10b010 0x7fbb1c1a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:33 smithi179 ceph-mon[35472]: pgmap v913: 129 pgs: 129 active+clean; 94 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 436 KiB/s wr, 149 op/s 2024-03-26T14:17:33.675 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.674+0000 7fbb22800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb1c10d9c0 0x7fbb1c1adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.674+0000 7fbb21e00640 1 -- 172.21.15.137:0/2976771532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb1c10d9c0 msgr2=0x7fbb1c1adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.674+0000 7fbb21e00640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb1c10d9c0 0x7fbb1c1adc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.674+0000 7fbb21e00640 1 -- 172.21.15.137:0/2976771532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c10b010 msgr2=0x7fbb1c1a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.674+0000 7fbb21e00640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c10b010 0x7fbb1c1a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.674+0000 7fbb21e00640 1 -- 172.21.15.137:0/2976771532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c1ae280 con 0x7fbb1c1086a0 2024-03-26T14:17:33.676 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.675+0000 7fbb21e00640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c1a6660 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d770 tx=0x7fbb1000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.676+0000 7fbb16a00640 1 -- 172.21.15.137:0/2976771532 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10017070 con 0x7fbb1c1086a0 2024-03-26T14:17:33.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.676+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c1ae570 con 0x7fbb1c1086a0 2024-03-26T14:17:33.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.676+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1aeac0 con 0x7fbb1c1086a0 2024-03-26T14:17:33.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.676+0000 7fbb16a00640 1 -- 172.21.15.137:0/2976771532 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb10009820 con 0x7fbb1c1086a0 2024-03-26T14:17:33.677 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.676+0000 7fbb16a00640 1 -- 172.21.15.137:0/2976771532 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10004ba0 con 0x7fbb1c1086a0 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.681+0000 7fbb2759c740 1 Processor -- start 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.681+0000 7fbb16a00640 1 -- 172.21.15.137:0/2976771532 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbb100099d0 con 0x7fbb1c1086a0 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 -- start start 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301b0a0 0x55f91301b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301ba90 0x55f913020590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913023090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f912f80eb0 con 0x55f91301ba90 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f912f29a60 con 0x55f913020c00 2024-03-26T14:17:33.683 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.682+0000 7fbb2759c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f912f7cf50 con 0x55f91301b0a0 2024-03-26T14:17:33.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.683+0000 7fbb22800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301b0a0 0x55f91301b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.683+0000 7fbb21400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913023090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.684 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.683+0000 7fbb21400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913023090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:48300/0 (socket says 172.21.15.137:48300) 2024-03-26T14:17:33.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.683+0000 7fbb21400640 1 -- 172.21.15.137:0/2735811212 learned_addr learned my addr 172.21.15.137:0/2735811212 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.685 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.683+0000 7fbb21e00640 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301ba90 0x55f913020590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.684+0000 7fbb22800640 1 -- 172.21.15.137:0/2735811212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 msgr2=0x55f913023090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.684+0000 7fbb22800640 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913023090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.684+0000 7fbb22800640 1 -- 172.21.15.137:0/2735811212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301ba90 msgr2=0x55f913020590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.684+0000 7fbb22800640 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301ba90 0x55f913020590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.686 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.684+0000 7fbb22800640 1 -- 172.21.15.137:0/2735811212 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f912d5d7e0 con 0x55f91301b0a0 2024-03-26T14:17:33.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.685+0000 7fbb22800640 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301b0a0 0x55f91301b4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb18037040 tx=0x7fbb18009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91825cc04527b1f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.687 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.686+0000 7fbb0b000640 1 -- 172.21.15.137:0/2735811212 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18043020 con 0x55f91301b0a0 2024-03-26T14:17:33.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.687+0000 7fbb0b000640 1 -- 172.21.15.137:0/2735811212 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb18003970 con 0x55f91301b0a0 2024-03-26T14:17:33.688 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.687+0000 7fbb0b000640 1 -- 172.21.15.137:0/2735811212 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18039040 con 0x55f91301b0a0 2024-03-26T14:17:33.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.688+0000 7fbb2759c740 1 -- 172.21.15.137:0/2735811212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301b0a0 msgr2=0x55f91301b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.689 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.688+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301b0a0 0x55f91301b4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb18037040 tx=0x7fbb18009890 comp rx=0 tx=0).stop 2024-03-26T14:17:33.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.689+0000 7fbb2759c740 1 -- 172.21.15.137:0/2735811212 shutdown_connections 2024-03-26T14:17:33.690 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.689+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913023090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.689+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301ba90 0x55f913020590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.690+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2735811212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301b0a0 0x55f91301b4c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.690+0000 7fbb2759c740 1 -- 172.21.15.137:0/2735811212 >> 172.21.15.137:0/2735811212 conn(0x55f913016610 msgr2=0x55f913016a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.690+0000 7fbb2759c740 1 -- 172.21.15.137:0/2735811212 shutdown_connections 2024-03-26T14:17:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.690+0000 7fbb16a00640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a28b0 0x7fbb040a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.691 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.690+0000 7fbb2759c740 1 -- 172.21.15.137:0/2735811212 wait complete. 2024-03-26T14:17:33.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.691+0000 7fbb16a00640 1 -- 172.21.15.137:0/2976771532 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb100ce820 con 0x7fbb1c1086a0 2024-03-26T14:17:33.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.691+0000 7fbb21400640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a28b0 0x7fbb040a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.692 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.691+0000 7fbb2759c740 1 -- start start 2024-03-26T14:17:33.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.692+0000 7fbb2759c740 1 Processor -- start 2024-03-26T14:17:33.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.692+0000 7fbb21400640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a28b0 0x7fbb040a4d70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005890 tx=0x7fbb0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.692+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913025250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f913025790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f91302c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb2759c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f913023b50 con 0x55f91301b0a0 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb2759c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f912f3ceb0 con 0x55f913020c00 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb2759c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f913023cd0 con 0x55f91301ba90 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb2759c740 10 client.0 _ll_register_callbacks cb 0x55f912f31150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913025250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913025250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58534/0 (socket says 172.21.15.137:58534) 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.693+0000 7fbb21e00640 1 -- 172.21.15.137:0/2187871585 learned_addr learned my addr 172.21.15.137:0/2187871585 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb22800640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f91302c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb21400640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f913025790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb22800640 1 -- 172.21.15.137:0/2187871585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 msgr2=0x55f913025790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb22800640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f913025790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb22800640 1 -- 172.21.15.137:0/2187871585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 msgr2=0x55f913025250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb22800640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913025250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.694+0000 7fbb22800640 1 -- 172.21.15.137:0/2187871585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f912dbdf70 con 0x55f913020c00 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.695+0000 7fbaff400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.695+0000 7fbb22800640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f91302c7d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb18033000 tx=0x7fbb180093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ab6e360ef390c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.696 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.695+0000 7fbaff400640 1 -- 172.21.15.137:0/2187871585 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18043030 con 0x55f913020c00 2024-03-26T14:17:33.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.695+0000 7fbaff400640 1 -- 172.21.15.137:0/2187871585 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb180327d0 con 0x55f913020c00 2024-03-26T14:17:33.697 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.696+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb18003970 con 0x55f913020c00 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.696+0000 7fbaff400640 1 -- 172.21.15.137:0/2187871585 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb18043030 con 0x55f913020c00 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.696+0000 7fbb2759c740 10 client.38336 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.696+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f912c55170 con 0x55f913020c00 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.696+0000 7fbaff400640 1 -- 172.21.15.137:0/2187871585 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb18003970 con 0x55f913020c00 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbaff400640 1 -- 172.21.15.137:0/2187871585 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbb1803f470 con 0x55f913020c00 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb2759c740 10 client.38336 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb2759c740 10 client.38336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb2759c740 20 client.38336 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f913030cc0 0x55f9130330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb2759c740 4 client.38336 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:33.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f912f8e3f0 con 0x55f913030cc0 2024-03-26T14:17:33.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.697+0000 7fbb21e00640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f913030cc0 0x55f9130330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.698+0000 7fbb21e00640 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f913030cc0 0x55f9130330b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628d71555de2eee7 server_cookie=fc8da3387b7aed06 in_seq=0 out_seq=0 2024-03-26T14:17:33.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.698+0000 7fbaff400640 10 client.38336 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbaff400640 1 -- 172.21.15.137:0/2187871585 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f912f8e3f0 con 0x55f913030cc0 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbaff400640 10 client.38336 handle_command_reply: tid=0 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbb2759c740 1 client.38336 shutdown 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 msgr2=0x55f91302c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f91302c7d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbb18033000 tx=0x7fbb180093d0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 shutdown_connections 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.699+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f913030cc0 0x55f9130330b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f91302c7d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f913025790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 1 --2- 172.21.15.137:0/2187871585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913025250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbaff400640 0 client.38336 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 >> 172.21.15.137:0/2187871585 conn(0x55f913016610 msgr2=0x55f913015830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 shutdown_connections 2024-03-26T14:17:33.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 1 -- 172.21.15.137:0/2187871585 wait complete. 2024-03-26T14:17:33.702 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.700+0000 7fbb2759c740 20 client.38336 trim_cache size 0 max 16384 2024-03-26T14:17:33.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:33 smithi159 ceph-mon[35501]: pgmap v913: 129 pgs: 129 active+clean; 94 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 436 KiB/s wr, 149 op/s 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.719+0000 7fbb2759c740 1 Processor -- start 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.719+0000 7fbb2759c740 1 -- start start 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913019ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f91302c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913037910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb2759c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000123b0 con 0x55f91301b0a0 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb2759c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000feb0 con 0x55f913020c00 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb2759c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000ded0 con 0x55f91301ba90 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913019ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913019ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58542/0 (socket says 172.21.15.137:58542) 2024-03-26T14:17:33.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 -- 172.21.15.137:0/846264005 learned_addr learned my addr 172.21.15.137:0/846264005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb22800640 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21400640 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f91302c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.722 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 -- 172.21.15.137:0/846264005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 msgr2=0x55f91302c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f91302c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 -- 172.21.15.137:0/846264005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 msgr2=0x55f913037910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913037910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21e00640 1 -- 172.21.15.137:0/846264005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f913017080 con 0x55f91301b0a0 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.720+0000 7fbb21400640 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f91302c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.721+0000 7fbb21e00640 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913019ee0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbb10097780 tx=0x7fbb100d0bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6473b46840a5f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.721+0000 7fbb0a600640 1 -- 172.21.15.137:0/846264005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10020040 con 0x55f91301b0a0 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.721+0000 7fbb0a600640 1 -- 172.21.15.137:0/846264005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb1009adb0 con 0x55f91301b0a0 2024-03-26T14:17:33.723 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.721+0000 7fbb0a600640 1 -- 172.21.15.137:0/846264005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10004d40 con 0x55f91301b0a0 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 -- 172.21.15.137:0/846264005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 msgr2=0x55f913019ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913019ee0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbb10097780 tx=0x7fbb100d0bf0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 -- 172.21.15.137:0/846264005 shutdown_connections 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f913020c00 0x55f913037910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f91301ba90 0x55f91302c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 --2- 172.21.15.137:0/846264005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f913019ee0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 -- 172.21.15.137:0/846264005 >> 172.21.15.137:0/846264005 conn(0x55f913016610 msgr2=0x55f913015830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.726 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 -- 172.21.15.137:0/846264005 shutdown_connections 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.722+0000 7fbb2759c740 1 -- 172.21.15.137:0/846264005 wait complete. 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 -- start start 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 Processor -- start 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f91302c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f91301ba90 0x55f91302ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f913020c00 0x55f91302b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000e530 con 0x55f91301b0a0 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000f850 con 0x55f91301ba90 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000eb90 con 0x55f913020c00 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb2759c740 10 client.0 _ll_register_callbacks cb 0x7fbb100d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f91302c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f91301ba90 0x55f91302ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.723+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f91302c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58552/0 (socket says 172.21.15.137:58552) 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 -- 172.21.15.137:0/1297146683 learned_addr learned my addr 172.21.15.137:0/1297146683 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb22800640 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f913020c00 0x55f91302b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.727 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 -- 172.21.15.137:0/1297146683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f913020c00 msgr2=0x55f91302b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f913020c00 0x55f91302b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 -- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f91301ba90 msgr2=0x55f91302ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f91301ba90 0x55f91302ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 -- 172.21.15.137:0/1297146683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91302b9c0 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb21e00640 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f91302c670 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbb1009aab0 tx=0x7fbb100a3690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8819b9f40199f3ae server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbaff400640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbaff400640 1 -- 172.21.15.137:0/1297146683 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb100b7030 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbaff400640 1 -- 172.21.15.137:0/1297146683 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb10009d60 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.724+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb1009adb0 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbaff400640 1 -- 172.21.15.137:0/1297146683 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb10020070 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 10 client.18852 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f91308d550 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbaff400640 1 -- 172.21.15.137:0/1297146683 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb1009adb0 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbaff400640 1 -- 172.21.15.137:0/1297146683 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fbb100a30c0 con 0x55f91301b0a0 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 10 client.18852 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:33.728 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 10 client.18852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:33.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 20 client.18852 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:33.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f91308de80 0x55f9130902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 4 client.18852 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-03-26T14:17:33.729 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.725+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f912f88740 con 0x55f91308de80 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.726+0000 7fbb21400640 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f91308de80 0x55f9130902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.726+0000 7fbb21400640 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f91308de80 0x55f9130902a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6174839ecba748 server_cookie=6de3a984c38f86c3 in_seq=0 out_seq=0 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.726+0000 7fbaff400640 10 client.18852 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbaff400640 1 -- 172.21.15.137:0/1297146683 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f912f88740 con 0x55f91308de80 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbaff400640 10 client.18852 handle_command_reply: tid=0 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 client.18852 shutdown 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 msgr2=0x55f91302c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.730 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f91302c670 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbb1009aab0 tx=0x7fbb100a3690 comp rx=0 tx=0).stop 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 shutdown_connections 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55f91308de80 0x55f9130902a0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f91301ba90 0x55f91302ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f913020c00 0x55f91302b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 --2- 172.21.15.137:0/1297146683 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f91301b0a0 0x55f91302c670 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 >> 172.21.15.137:0/1297146683 conn(0x55f913016610 msgr2=0x55f91301c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.727+0000 7fbaff400640 0 client.18852 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 shutdown_connections 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb2759c740 1 -- 172.21.15.137:0/1297146683 wait complete. 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb2759c740 20 client.18852 trim_cache size 0 max 16384 2024-03-26T14:17:33.731 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stdout: "return_code": 0, 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_tag": "8df14942-7d91-4475-95c0-2132818bf46b", 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": "asynchronous" 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a28b0 msgr2=0x7fbb040a4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb24800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a28b0 0x7fbb040a4d70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c005890 tx=0x7fbb0c00a2d0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.732 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 msgr2=0x7fbb1c1a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb24800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c1a6660 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d770 tx=0x7fbb1000db70 comp rx=0 tx=0).stop 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.728+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 shutdown_connections 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb040a28b0 0x7fbb040a4d70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb1c10b010 0x7fbb1c1a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb1c10d9c0 0x7fbb1c1adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 --2- 172.21.15.137:0/2976771532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb1c1086a0 0x7fbb1c1a6660 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 >> 172.21.15.137:0/2976771532 conn(0x7fbb1c06e500 msgr2=0x7fbb1c109c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 shutdown_connections 2024-03-26T14:17:33.733 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.729+0000 7fbb24800640 1 -- 172.21.15.137:0/2976771532 wait complete. 2024-03-26T14:17:33.745 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:33 smithi137 ceph-mon[31539]: pgmap v913: 129 pgs: 129 active+clean; 94 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 436 KiB/s wr, 149 op/s 2024-03-26T14:17:33.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:17:33.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:17:33.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.945+0000 7f16b1000640 1 -- 172.21.15.137:0/443708659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac109770 msgr2=0x7f16ac110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 --2- 172.21.15.137:0/443708659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac109770 0x7f16ac110000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009900 tx=0x7f16a402bc40 comp rx=0 tx=0).stop 2024-03-26T14:17:33.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 -- 172.21.15.137:0/443708659 shutdown_connections 2024-03-26T14:17:33.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 --2- 172.21.15.137:0/443708659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac109770 0x7f16ac110000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 --2- 172.21.15.137:0/443708659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ac108db0 0x7f16ac109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.947 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 --2- 172.21.15.137:0/443708659 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac107bc0 0x7f16ac107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 -- 172.21.15.137:0/443708659 >> 172.21.15.137:0/443708659 conn(0x7f16ac076b70 msgr2=0x7f16ac078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 -- 172.21.15.137:0/443708659 shutdown_connections 2024-03-26T14:17:33.948 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.946+0000 7f16b1000640 1 -- 172.21.15.137:0/443708659 wait complete. 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 Processor -- start 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 -- start start 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac107bc0 0x7f16ac19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ac108db0 0x7f16ac19e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 0x7f16ac1a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1136d0 con 0x7f16ac109770 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac113550 con 0x7f16ac107bc0 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.947+0000 7f16b1000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac113850 con 0x7f16ac108db0 2024-03-26T14:17:33.950 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 0x7f16ac1a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 0x7f16ac1a52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58558/0 (socket says 172.21.15.137:58558) 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 -- 172.21.15.137:0/1049723963 learned_addr learned my addr 172.21.15.137:0/1049723963 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16a2a00640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ac108db0 0x7f16ac19e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16ab400640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac107bc0 0x7f16ac19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 -- 172.21.15.137:0/1049723963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ac108db0 msgr2=0x7f16ac19e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ac108db0 0x7f16ac19e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 -- 172.21.15.137:0/1049723963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac107bc0 msgr2=0x7f16ac19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac107bc0 0x7f16ac19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 -- 172.21.15.137:0/1049723963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac1a5960 con 0x7f16ac109770 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.948+0000 7f16abe00640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 0x7f16ac1a52f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009230 tx=0x7f16a40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.949+0000 7f16a8c00640 1 -- 172.21.15.137:0/1049723963 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4043020 con 0x7f16ac109770 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.949+0000 7f16a8c00640 1 -- 172.21.15.137:0/1049723963 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f16a40044c0 con 0x7f16ac109770 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.949+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac1a5bf0 con 0x7f16ac109770 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.949+0000 7f16a8c00640 1 -- 172.21.15.137:0/1049723963 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4003bc0 con 0x7f16ac109770 2024-03-26T14:17:33.951 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.949+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac1a6140 con 0x7f16ac109770 2024-03-26T14:17:33.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.953+0000 7f16b5214740 1 Processor -- start 2024-03-26T14:17:33.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.953+0000 7f16a8c00640 1 -- 172.21.15.137:0/1049723963 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f16a4046020 con 0x7f16ac109770 2024-03-26T14:17:33.954 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 -- start start 2024-03-26T14:17:33.956 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7db6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc7d13250 con 0x55efc7db3bb0 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc7cbba30 con 0x55efc7dae050 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16b5214740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc7d0f2f0 con 0x55efc7daea40 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16a2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16a2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.137:48372/0 (socket says 172.21.15.137:48372) 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16a2a00640 1 -- 172.21.15.137:0/3536770584 learned_addr learned my addr 172.21.15.137:0/3536770584 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.954+0000 7f16a2a00640 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7db6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16ab400640 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a2a00640 1 -- 172.21.15.137:0/3536770584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 msgr2=0x55efc7db3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.957 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a2a00640 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a2a00640 1 -- 172.21.15.137:0/3536770584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 msgr2=0x55efc7db6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a2a00640 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7db6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a2a00640 1 -- 172.21.15.137:0/3536770584 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc7ca6690 con 0x55efc7dae050 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a8c00640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f16900a2800 0x7f16900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a2a00640 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dae470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1694012a60 tx=0x7f1694009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca6521e841c169a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.955+0000 7f16a8c00640 1 -- 172.21.15.137:0/1049723963 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f16a40f11f0 con 0x7f16ac109770 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.956+0000 7f16ab400640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f16900a2800 0x7f16900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.956+0000 7f16a0c00640 1 -- 172.21.15.137:0/3536770584 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694010770 con 0x55efc7dae050 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.956+0000 7f16a0c00640 1 -- 172.21.15.137:0/3536770584 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1694010dc0 con 0x55efc7dae050 2024-03-26T14:17:33.958 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.956+0000 7f16a0c00640 1 -- 172.21.15.137:0/3536770584 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169401ee20 con 0x55efc7dae050 2024-03-26T14:17:33.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 -- 172.21.15.137:0/3536770584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 msgr2=0x55efc7dae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dae470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1694012a60 tx=0x7f1694009780 comp rx=0 tx=0).stop 2024-03-26T14:17:33.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16ab400640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f16900a2800 0x7f16900a4cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55efc7daee40 tx=0x7f169c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.962 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 -- 172.21.15.137:0/3536770584 shutdown_connections 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7db6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 --2- 172.21.15.137:0/3536770584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dae470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 -- 172.21.15.137:0/3536770584 >> 172.21.15.137:0/3536770584 conn(0x55efc7d76200 msgr2=0x55efc7da99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 -- 172.21.15.137:0/3536770584 shutdown_connections 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 -- 172.21.15.137:0/3536770584 wait complete. 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 -- start start 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.957+0000 7f16b5214740 1 Processor -- start 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7db80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc7db6b60 con 0x55efc7db3bb0 2024-03-26T14:17:33.963 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc7ccee80 con 0x55efc7dae050 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc7db6ce0 con 0x55efc7daea40 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16b5214740 10 client.0 _ll_register_callbacks cb 0x55efc7ce1130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16ab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7db80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58582/0 (socket says 172.21.15.137:58582) 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.958+0000 7f16abe00640 1 -- 172.21.15.137:0/3357699505 learned_addr learned my addr 172.21.15.137:0/3357699505 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16a2a00640 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16abe00640 1 -- 172.21.15.137:0/3357699505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 msgr2=0x55efc7db8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16abe00640 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16abe00640 1 -- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 msgr2=0x55efc7db80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16abe00640 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7db80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16abe00640 1 -- 172.21.15.137:0/3357699505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efc7af17d0 con 0x55efc7db3bb0 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16abe00640 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbf640 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f16a40b9c20 tx=0x7f16a40bd120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1486756abf57a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.964 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f1699600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f1699600640 1 -- 172.21.15.137:0/3357699505 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40bd7c0 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f1699600640 1 -- 172.21.15.137:0/3357699505 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f16a4003d60 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1694010dc0 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f1699600640 1 -- 172.21.15.137:0/3357699505 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a40c1910 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16b5214740 10 client.18864 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.959+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55efc7a273f0 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f1699600640 1 -- 172.21.15.137:0/3357699505 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1694010dc0 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f1699600640 1 -- 172.21.15.137:0/3357699505 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f16a40beaf0 con 0x55efc7db3bb0 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16b5214740 10 client.18864 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16b5214740 10 client.18864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16b5214740 20 client.18864 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16b5214740 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7dc3fc0 0x55efc7dc6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16b5214740 4 client.18864 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:17:33.965 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efc7d20790 con 0x55efc7dc3fc0 2024-03-26T14:17:33.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.960+0000 7f16ab400640 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7dc3fc0 0x55efc7dc6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.961+0000 7f16ab400640 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7dc3fc0 0x55efc7dc6420 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43babe6b236d1800 server_cookie=552e384d462d4ce0 in_seq=0 out_seq=0 2024-03-26T14:17:33.966 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.961+0000 7f1699600640 10 client.18864 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.964+0000 7f1699600640 1 -- 172.21.15.137:0/3357699505 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55efc7d20790 con 0x55efc7dc3fc0 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.964+0000 7f1699600640 10 client.18864 handle_command_reply: tid=0 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.964+0000 7f16b5214740 1 client.18864 shutdown 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.964+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 msgr2=0x55efc7dbf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.964+0000 7f16b5214740 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbf640 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f16a40b9c20 tx=0x7f16a40bd120 comp rx=0 tx=0).stop 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 shutdown_connections 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7dc3fc0 0x55efc7dc6420 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbf640 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7db8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 --2- 172.21.15.137:0/3357699505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7db80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.967 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f1699600640 0 client.18864 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 >> 172.21.15.137:0/3357699505 conn(0x55efc7d76200 msgr2=0x55efc7da8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 shutdown_connections 2024-03-26T14:17:33.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 1 -- 172.21.15.137:0/3357699505 wait complete. 2024-03-26T14:17:33.968 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.965+0000 7f16b5214740 20 client.18864 trim_cache size 0 max 16384 2024-03-26T14:17:33.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 Processor -- start 2024-03-26T14:17:33.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 -- start start 2024-03-26T14:17:33.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 0x55efc7db3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7daea40 0x55efc7dbf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7db3bb0 0x55efc7da8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1b29c0 con 0x55efc7dae050 2024-03-26T14:17:33.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1b04c0 con 0x55efc7daea40 2024-03-26T14:17:33.986 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.984+0000 7f16b5214740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1ae540 con 0x55efc7db3bb0 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 0x55efc7db3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 0x55efc7db3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58590/0 (socket says 172.21.15.137:58590) 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 -- 172.21.15.137:0/1714589475 learned_addr learned my addr 172.21.15.137:0/1714589475 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16a2a00640 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7daea40 0x55efc7dbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16abe00640 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7db3bb0 0x55efc7da8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 -- 172.21.15.137:0/1714589475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7db3bb0 msgr2=0x55efc7da8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7db3bb0 0x55efc7da8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 -- 172.21.15.137:0/1714589475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7daea40 msgr2=0x55efc7dbf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7daea40 0x55efc7dbf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16ab400640 1 -- 172.21.15.137:0/1714589475 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc79e70f0 con 0x55efc7dae050 2024-03-26T14:17:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.985+0000 7f16a2a00640 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7daea40 0x55efc7dbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:17:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.986+0000 7f16ab400640 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 0x55efc7db3090 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f169c009460 tx=0x7f169c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e170114d83436248 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.986+0000 7f169be00640 1 -- 172.21.15.137:0/1714589475 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c02e070 con 0x55efc7dae050 2024-03-26T14:17:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.986+0000 7f169be00640 1 -- 172.21.15.137:0/1714589475 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f169c004230 con 0x55efc7dae050 2024-03-26T14:17:33.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.986+0000 7f169be00640 1 -- 172.21.15.137:0/1714589475 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c029780 con 0x55efc7dae050 2024-03-26T14:17:33.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 -- 172.21.15.137:0/1714589475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 msgr2=0x55efc7db3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 0x55efc7db3090 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f169c009460 tx=0x7f169c019000 comp rx=0 tx=0).stop 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 -- 172.21.15.137:0/1714589475 shutdown_connections 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7daea40 0x55efc7dbf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7db3bb0 0x55efc7da8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 --2- 172.21.15.137:0/1714589475 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7dae050 0x55efc7db3090 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 -- 172.21.15.137:0/1714589475 >> 172.21.15.137:0/1714589475 conn(0x55efc7d76200 msgr2=0x55efc7dad490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 -- 172.21.15.137:0/1714589475 shutdown_connections 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 -- 172.21.15.137:0/1714589475 wait complete. 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 -- start start 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.987+0000 7f16b5214740 1 Processor -- start 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dbeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7dbf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1aeb40 con 0x55efc7db3bb0 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1afe60 con 0x55efc7dae050 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1af1a0 con 0x55efc7daea40 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16b5214740 10 client.0 _ll_register_callbacks cb 0x7f16a4009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16abe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:58598/0 (socket says 172.21.15.137:58598) 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16abe00640 1 -- 172.21.15.137:0/2386097768 learned_addr learned my addr 172.21.15.137:0/2386097768 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.988+0000 7f16ab400640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dbeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16abe00640 1 -- 172.21.15.137:0/2386097768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 msgr2=0x55efc7dbf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16abe00640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7dbf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16abe00640 1 -- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 msgr2=0x55efc7dbeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16abe00640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dbeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16abe00640 1 -- 172.21.15.137:0/2386097768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efc7dbdc10 con 0x55efc7db3bb0 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16ab400640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dbeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16abe00640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbd4d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f16a40c1ad0 tx=0x7f16a4005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87118ae38526e65a server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:17:33.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f1699600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f1699600640 1 -- 172.21.15.137:0/2386097768 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40bd590 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f1699600640 1 -- 172.21.15.137:0/2386097768 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f16a40c5ad0 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f169c004230 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f1699600640 1 -- 172.21.15.137:0/2386097768 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16a403c350 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16b5214740 10 client.18870 fetch_fsmap learned FSMap version 24 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.989+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55efc7dbdc10 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f1699600640 1 -- 172.21.15.137:0/2386097768 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f169c004230 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f1699600640 1 -- 172.21.15.137:0/2386097768 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f16a40b9510 con 0x55efc7db3bb0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16b5214740 10 client.18870 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16b5214740 10 client.18870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16b5214740 20 client.18870 populate_metadata read hostname 'smithi137' 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16b5214740 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7e207f0 0x55efc7e22c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16b5214740 4 client.18870 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55efc7d1aae0 con 0x55efc7e207f0 2024-03-26T14:17:33.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.990+0000 7f16ab400640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7e207f0 0x55efc7e22c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:17:33.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.991+0000 7f16ab400640 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7e207f0 0x55efc7e22c10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04efef9a352c6c8 server_cookie=9e52909503dc7803 in_seq=0 out_seq=0 2024-03-26T14:17:33.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.991+0000 7f1699600640 10 client.18870 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f1699600640 1 -- 172.21.15.137:0/2386097768 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55efc7d1aae0 con 0x55efc7e207f0 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f1699600640 10 client.18870 handle_command_reply: tid=0 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 client.18870 shutdown 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 msgr2=0x55efc7dbd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbd4d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f16a40c1ad0 tx=0x7f16a4005890 comp rx=0 tx=0).stop 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 shutdown_connections 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55efc7e207f0 0x55efc7e22c10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55efc7db3bb0 0x55efc7dbd4d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55efc7daea40 0x55efc7dbf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 --2- 172.21.15.137:0/2386097768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efc7dae050 0x55efc7dbeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 >> 172.21.15.137:0/2386097768 conn(0x55efc7d76200 msgr2=0x55efc7dafd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f1699600640 0 client.18870 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 shutdown_connections 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 1 -- 172.21.15.137:0/2386097768 wait complete. 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.994+0000 7f16b5214740 20 client.18870 trim_cache size 0 max 16384 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": { 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: "8df14942-7d91-4475-95c0-2132818bf46b": { 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: "path": "/", 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: "tag": "8df14942-7d91-4475-95c0-2132818bf46b", 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: "options": "recursive,force" 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:17:33.997 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f16900a2800 msgr2=0x7f16900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f16900a2800 0x7f16900a4cc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x55efc7daee40 tx=0x7f169c005f90 comp rx=0 tx=0).stop 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 msgr2=0x7f16ac1a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 0x7f16ac1a52f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009230 tx=0x7f16a40040c0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 shutdown_connections 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f16900a2800 0x7f16900a4cc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16ac109770 0x7f16ac1a52f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f16ac108db0 0x7f16ac19e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 --2- 172.21.15.137:0/1049723963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16ac107bc0 0x7f16ac19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:17:33.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 >> 172.21.15.137:0/1049723963 conn(0x7f16ac076b70 msgr2=0x7f16ac10c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:17:33.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.995+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 shutdown_connections 2024-03-26T14:17:33.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:17:33.996+0000 7f16b1000640 1 -- 172.21.15.137:0/1049723963 wait complete. 2024-03-26T14:17:34.014 INFO:tasks.cephfs.filesystem:scrub status for tag:8df14942-7d91-4475-95c0-2132818bf46b - {'path': '/', 'tag': '8df14942-7d91-4475-95c0-2132818bf46b', 'options': 'recursive,force'} 2024-03-26T14:17:34.034 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-26T14:17:34.034 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-26T14:17:34.035 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-03-26T14:17:34.036 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-26T14:17:34.037 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-26T14:17:34.038 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-26T14:17:34.039 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-03-26T14:17:34.040 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-26T14:17:34.040 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-03-26T14:17:34.040 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-26T14:17:34.040 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-03-26T14:17:34.040 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-26T14:17:34.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:34 smithi179 ceph-mon[35472]: scrub queued for path: / 2024-03-26T14:17:34.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:34 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:34.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:34 smithi179 ceph-mon[35472]: scrub summary: active paths [/] 2024-03-26T14:17:34.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:34 smithi179 ceph-mon[35472]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:34.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:34 smithi159 ceph-mon[35501]: scrub queued for path: / 2024-03-26T14:17:34.703 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:34 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:34.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:34 smithi159 ceph-mon[35501]: scrub summary: active paths [/] 2024-03-26T14:17:34.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:34 smithi159 ceph-mon[35501]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:34.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:34 smithi137 ceph-mon[31539]: scrub queued for path: / 2024-03-26T14:17:34.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:34 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:34.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:34 smithi137 ceph-mon[31539]: scrub summary: active paths [/] 2024-03-26T14:17:34.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:34 smithi137 ceph-mon[31539]: scrub summary: idle+waiting paths [/] 2024-03-26T14:17:35.378 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-03-26T14:17:35.378 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-26T14:17:35.378 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-03-26T14:17:35.378 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-26T14:17:35.378 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-03-26T14:17:35.378 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-26T14:17:35.379 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-26T14:17:35.380 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-26T14:17:35.381 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-26T14:17:35.382 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-26T14:17:35.383 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-03-26T14:17:35.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:35 smithi179 ceph-mon[35472]: pgmap v914: 129 pgs: 129 active+clean; 102 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 457 KiB/s wr, 155 op/s 2024-03-26T14:17:35.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:35 smithi159 ceph-mon[35501]: pgmap v914: 129 pgs: 129 active+clean; 102 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 457 KiB/s wr, 155 op/s 2024-03-26T14:17:35.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:35 smithi137 ceph-mon[31539]: pgmap v914: 129 pgs: 129 active+clean; 102 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 457 KiB/s wr, 155 op/s 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-03-26T14:17:37.198 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-26T14:17:37.199 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-03-26T14:17:37.200 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-03-26T14:17:37.201 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-26T14:17:37.202 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-03-26T14:17:37.203 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-26T14:17:37.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:37 smithi159 ceph-mon[35501]: pgmap v915: 129 pgs: 129 active+clean; 170 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 339 KiB/s wr, 132 op/s 2024-03-26T14:17:37.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:37 smithi159 ceph-mon[35501]: scrub summary: idle 2024-03-26T14:17:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:37 smithi137 ceph-mon[31539]: pgmap v915: 129 pgs: 129 active+clean; 170 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 339 KiB/s wr, 132 op/s 2024-03-26T14:17:37.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:37 smithi137 ceph-mon[31539]: scrub summary: idle 2024-03-26T14:17:37.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:37 smithi179 ceph-mon[35472]: pgmap v915: 129 pgs: 129 active+clean; 170 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 339 KiB/s wr, 132 op/s 2024-03-26T14:17:37.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:37 smithi179 ceph-mon[35472]: scrub summary: idle 2024-03-26T14:17:39.339 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-03-26T14:17:39.339 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-26T14:17:39.339 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-03-26T14:17:39.339 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-26T14:17:39.339 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-03-26T14:17:39.339 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-03-26T14:17:39.340 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-03-26T14:17:39.341 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-26T14:17:39.342 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-03-26T14:17:39.343 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-03-26T14:17:39.344 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-03-26T14:17:39.345 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-03-26T14:17:39.346 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-26T14:17:39.347 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-03-26T14:17:39.347 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-26T14:17:39.347 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-03-26T14:17:39.347 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-26T14:17:39.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:39 smithi159 ceph-mon[35501]: pgmap v916: 129 pgs: 129 active+clean; 152 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 470 KiB/s wr, 167 op/s 2024-03-26T14:17:39.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:39 smithi137 ceph-mon[31539]: pgmap v916: 129 pgs: 129 active+clean; 152 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 470 KiB/s wr, 167 op/s 2024-03-26T14:17:39.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:39 smithi179 ceph-mon[35472]: pgmap v916: 129 pgs: 129 active+clean; 152 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 470 KiB/s wr, 167 op/s 2024-03-26T14:17:40.896 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-26T14:17:40.897 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-03-26T14:17:40.898 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-03-26T14:17:40.899 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-26T14:17:40.900 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-26T14:17:40.901 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-03-26T14:17:41.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:41 smithi159 ceph-mon[35501]: pgmap v917: 129 pgs: 129 active+clean; 116 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 395 KiB/s wr, 148 op/s 2024-03-26T14:17:41.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:41 smithi137 ceph-mon[31539]: pgmap v917: 129 pgs: 129 active+clean; 116 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 395 KiB/s wr, 148 op/s 2024-03-26T14:17:41.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:41 smithi179 ceph-mon[35472]: pgmap v917: 129 pgs: 129 active+clean; 116 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 395 KiB/s wr, 148 op/s 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-03-26T14:17:42.318 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-03-26T14:17:42.319 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-03-26T14:17:42.320 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-26T14:17:42.321 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-26T14:17:42.322 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-03-26T14:17:42.323 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-26T14:17:42.559 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-03-26T14:17:42.559 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-03-26T14:17:42.560 INFO:tasks.workunit.client.0.smithi137.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-26T14:17:42.561 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-26T14:17:42.561 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-26T14:17:42.732 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-03-26T14:17:42.732 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-26T14:17:43.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:43 smithi159 ceph-mon[35501]: pgmap v918: 129 pgs: 129 active+clean; 129 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 490 KiB/s wr, 163 op/s 2024-03-26T14:17:43.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:43 smithi137 ceph-mon[31539]: pgmap v918: 129 pgs: 129 active+clean; 129 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 490 KiB/s wr, 163 op/s 2024-03-26T14:17:43.832 DEBUG:teuthology.parallel:result is None 2024-03-26T14:17:43.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T14:17:43.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:43 smithi179 ceph-mon[35472]: pgmap v918: 129 pgs: 129 active+clean; 129 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 490 KiB/s wr, 163 op/s 2024-03-26T14:17:43.926 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-26T14:17:43.926 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-03-26T14:17:43.937 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-03-26T14:17:45.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:45 smithi159 ceph-mon[35501]: pgmap v919: 129 pgs: 129 active+clean; 85 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 390 KiB/s wr, 148 op/s 2024-03-26T14:17:45.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:45 smithi137 ceph-mon[31539]: pgmap v919: 129 pgs: 129 active+clean; 85 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 390 KiB/s wr, 148 op/s 2024-03-26T14:17:45.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:45 smithi179 ceph-mon[35472]: pgmap v919: 129 pgs: 129 active+clean; 85 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 390 KiB/s wr, 148 op/s 2024-03-26T14:17:46.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:46 smithi137 ceph-mon[31539]: pgmap v920: 129 pgs: 129 active+clean; 33 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 370 KiB/s wr, 140 op/s 2024-03-26T14:17:46.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:46 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:17:46.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:46 smithi179 ceph-mon[35472]: pgmap v920: 129 pgs: 129 active+clean; 33 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 370 KiB/s wr, 140 op/s 2024-03-26T14:17:46.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:46 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:17:46.953 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:46 smithi159 ceph-mon[35501]: pgmap v920: 129 pgs: 129 active+clean; 33 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 370 KiB/s wr, 140 op/s 2024-03-26T14:17:46.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:46 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:17:47.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi137 to 176.7M 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: Unable to set osd_memory_target on smithi137 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:17:47.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:17:47.827 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.827 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.827 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.828 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.828 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi137 to 176.7M 2024-03-26T14:17:47.828 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: Unable to set osd_memory_target on smithi137 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:17:47.828 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:17:47.828 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:17:47.828 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:17:47.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:17:47.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi137 to 176.7M 2024-03-26T14:17:47.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: Unable to set osd_memory_target on smithi137 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:17:47.955 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:17:47.955 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:17:47.955 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:17:48.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:48 smithi137 ceph-mon[31539]: pgmap v921: 129 pgs: 129 active+clean; 13 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 384 KiB/s wr, 141 op/s 2024-03-26T14:17:48.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:48 smithi179 ceph-mon[35472]: pgmap v921: 129 pgs: 129 active+clean; 13 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 384 KiB/s wr, 141 op/s 2024-03-26T14:17:48.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:48 smithi159 ceph-mon[35501]: pgmap v921: 129 pgs: 129 active+clean; 13 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 384 KiB/s wr, 141 op/s 2024-03-26T14:17:51.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:50 smithi137 ceph-mon[31539]: pgmap v922: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 232 KiB/s wr, 96 op/s 2024-03-26T14:17:51.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:50 smithi179 ceph-mon[35472]: pgmap v922: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 232 KiB/s wr, 96 op/s 2024-03-26T14:17:51.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:50 smithi159 ceph-mon[35501]: pgmap v922: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 232 KiB/s wr, 96 op/s 2024-03-26T14:17:53.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:52 smithi137 ceph-mon[31539]: pgmap v923: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 199 KiB/s wr, 80 op/s 2024-03-26T14:17:53.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:52 smithi179 ceph-mon[35472]: pgmap v923: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 199 KiB/s wr, 80 op/s 2024-03-26T14:17:53.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:52 smithi159 ceph-mon[35501]: pgmap v923: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 199 KiB/s wr, 80 op/s 2024-03-26T14:17:55.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:54 smithi137 ceph-mon[31539]: pgmap v924: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-03-26T14:17:55.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:54 smithi179 ceph-mon[35472]: pgmap v924: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-03-26T14:17:55.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:54 smithi159 ceph-mon[35501]: pgmap v924: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 83 KiB/s wr, 51 op/s 2024-03-26T14:17:57.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:56 smithi137 ceph-mon[31539]: pgmap v925: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-03-26T14:17:57.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:56 smithi179 ceph-mon[35472]: pgmap v925: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-03-26T14:17:57.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:56 smithi159 ceph-mon[35501]: pgmap v925: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 52 KiB/s wr, 32 op/s 2024-03-26T14:17:59.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:17:58 smithi137 ceph-mon[31539]: pgmap v926: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 39 KiB/s wr, 25 op/s 2024-03-26T14:17:59.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:17:58 smithi179 ceph-mon[35472]: pgmap v926: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 39 KiB/s wr, 25 op/s 2024-03-26T14:17:59.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:17:58 smithi159 ceph-mon[35501]: pgmap v926: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 39 KiB/s wr, 25 op/s 2024-03-26T14:18:01.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:00 smithi137 ceph-mon[31539]: pgmap v927: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-03-26T14:18:01.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:00 smithi179 ceph-mon[35472]: pgmap v927: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-03-26T14:18:01.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:00 smithi159 ceph-mon[35501]: pgmap v927: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-03-26T14:18:03.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:02 smithi137 ceph-mon[31539]: pgmap v928: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-03-26T14:18:03.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:02 smithi179 ceph-mon[35472]: pgmap v928: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-03-26T14:18:03.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:02 smithi159 ceph-mon[35501]: pgmap v928: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-03-26T14:18:04.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-03-26T14:18:04.015 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-03-26T14:18:04.232 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 -- 172.21.15.137:0/511965503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4881038e0 msgr2=0x7fc488103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 --2- 172.21.15.137:0/511965503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4881038e0 0x7fc488103d00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fc478009900 tx=0x7fc47802bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 -- 172.21.15.137:0/511965503 shutdown_connections 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 --2- 172.21.15.137:0/511965503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488104ad0 0x7fc488104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 --2- 172.21.15.137:0/511965503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc488105490 0x7fc48810bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 --2- 172.21.15.137:0/511965503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4881038e0 0x7fc488103d00 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 -- 172.21.15.137:0/511965503 >> 172.21.15.137:0/511965503 conn(0x7fc4880fee50 msgr2=0x7fc488101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 -- 172.21.15.137:0/511965503 shutdown_connections 2024-03-26T14:18:04.233 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.231+0000 7fc48de00640 1 -- 172.21.15.137:0/511965503 wait complete. 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.232+0000 7fc48de00640 1 Processor -- start 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 -- start start 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4881038e0 0x7fc488199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 0x7fc48819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488105490 0x7fc4881a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48810f410 con 0x7fc488104ad0 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48810f290 con 0x7fc488105490 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc48de00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48810f590 con 0x7fc4881038e0 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc47e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 0x7fc48819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc47e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 0x7fc48819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37868/0 (socket says 172.21.15.137:37868) 2024-03-26T14:18:04.236 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.233+0000 7fc47e000640 1 -- 172.21.15.137:0/2295725823 learned_addr learned my addr 172.21.15.137:0/2295725823 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc486a00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4881038e0 0x7fc488199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc487400640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488105490 0x7fc4881a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc47e000640 1 -- 172.21.15.137:0/2295725823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4881038e0 msgr2=0x7fc488199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc47e000640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4881038e0 0x7fc488199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc47e000640 1 -- 172.21.15.137:0/2295725823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488105490 msgr2=0x7fc4881a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc47e000640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488105490 0x7fc4881a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc47e000640 1 -- 172.21.15.137:0/2295725823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4881a1760 con 0x7fc488104ad0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc487400640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488105490 0x7fc4881a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc47e000640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 0x7fc48819a080 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc47000e9f0 tx=0x7fc47000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc48ca00640 1 -- 172.21.15.137:0/2295725823 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000cd80 con 0x7fc488104ad0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.234+0000 7fc48ca00640 1 -- 172.21.15.137:0/2295725823 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc4700108e0 con 0x7fc488104ad0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.235+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4881a1a50 con 0x7fc488104ad0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.235+0000 7fc48ca00640 1 -- 172.21.15.137:0/2295725823 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470018920 con 0x7fc488104ad0 2024-03-26T14:18:04.237 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.235+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4881a1ff0 con 0x7fc488104ad0 2024-03-26T14:18:04.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 Processor -- start 2024-03-26T14:18:04.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 -- start start 2024-03-26T14:18:04.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc48ca00640 1 -- 172.21.15.137:0/2295725823 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc470010a90 con 0x7fc488104ad0 2024-03-26T14:18:04.240 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9c9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa929250 con 0x55d3aa9c4870 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa8d1a30 con 0x55d3aa9c3e80 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.238+0000 7fc490b89740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9252f0 con 0x55d3aa9c99e0 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37876/0 (socket says 172.21.15.137:37876) 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 -- 172.21.15.137:0/1815444567 learned_addr learned my addr 172.21.15.137:0/1815444567 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc487400640 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc47e000640 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 -- 172.21.15.137:0/1815444567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 msgr2=0x55d3aa9cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.241 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 -- 172.21.15.137:0/1815444567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c3e80 msgr2=0x55d3aa9c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 -- 172.21.15.137:0/1815444567 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3aa8bc690 con 0x55d3aa9c4870 2024-03-26T14:18:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc487400640 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-26T14:18:04.242 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.239+0000 7fc486a00640 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9c9370 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc478005ab0 tx=0x7fc4780095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32425e2f7d623f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.240+0000 7fc477e00640 1 -- 172.21.15.137:0/1815444567 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478044020 con 0x55d3aa9c4870 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.240+0000 7fc477e00640 1 -- 172.21.15.137:0/1815444567 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47802cba0 con 0x55d3aa9c4870 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.240+0000 7fc477e00640 1 -- 172.21.15.137:0/1815444567 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4780046c0 con 0x55d3aa9c4870 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.240+0000 7fc48ca00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc46c0a2880 0x7fc46c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.240+0000 7fc48ca00640 1 -- 172.21.15.137:0/2295725823 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc4700cf240 con 0x7fc488104ad0 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.240+0000 7fc487400640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc46c0a2880 0x7fc46c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 -- 172.21.15.137:0/1815444567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 msgr2=0x55d3aa9c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9c9370 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fc478005ab0 tx=0x7fc4780095e0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.246 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 -- 172.21.15.137:0/1815444567 shutdown_connections 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9c9370 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 --2- 172.21.15.137:0/1815444567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 -- 172.21.15.137:0/1815444567 >> 172.21.15.137:0/1815444567 conn(0x55d3aa98c200 msgr2=0x55d3aa9bf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 -- 172.21.15.137:0/1815444567 shutdown_connections 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 -- 172.21.15.137:0/1815444567 wait complete. 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc487400640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc46c0a2880 0x7fc46c0a4d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55d3aa9c50d0 tx=0x7fc480009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.241+0000 7fc490b89740 1 -- start start 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 Processor -- start 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9cc930 con 0x55d3aa9c4870 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa8e4e80 con 0x55d3aa9c99e0 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3aa9ccab0 con 0x55d3aa9c3e80 2024-03-26T14:18:04.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc490b89740 10 client.0 _ll_register_callbacks cb 0x55d3aa8f7130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc47e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc47e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37890/0 (socket says 172.21.15.137:37890) 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.242+0000 7fc47e000640 1 -- 172.21.15.137:0/900234523 learned_addr learned my addr 172.21.15.137:0/900234523 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc487400640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc486a00640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc47e000640 1 -- 172.21.15.137:0/900234523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c3e80 msgr2=0x55d3aa9cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc47e000640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc47e000640 1 -- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c99e0 msgr2=0x55d3aa9d5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc47e000640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc47e000640 1 -- 172.21.15.137:0/900234523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3aa7077d0 con 0x55d3aa9c4870 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc47e000640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9ce2d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc4700a3a40 tx=0x7fc47009b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3edf23565c0963 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.243+0000 7fc474c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc474c00640 1 -- 172.21.15.137:0/900234523 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47009b7d0 con 0x55d3aa9c4870 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc474c00640 1 -- 172.21.15.137:0/900234523 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc4700860e0 con 0x55d3aa9c4870 2024-03-26T14:18:04.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc47802cba0 con 0x55d3aa9c4870 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc474c00640 1 -- 172.21.15.137:0/900234523 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc47009f920 con 0x55d3aa9c4870 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 10 client.18882 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d3aa63d3f0 con 0x55d3aa9c4870 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc474c00640 1 -- 172.21.15.137:0/900234523 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47802cba0 con 0x55d3aa9c4870 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc474c00640 1 -- 172.21.15.137:0/900234523 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc47009ac20 con 0x55d3aa9c4870 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 10 client.18882 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 10 client.18882 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 20 client.18882 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aa9d9b20 0x55d3aa9dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 4 client.18882 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.244+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3aa936790 con 0x55d3aa9d9b20 2024-03-26T14:18:04.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.245+0000 7fc486a00640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aa9d9b20 0x55d3aa9dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.246+0000 7fc486a00640 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aa9d9b20 0x55d3aa9dbf70 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2fd27d1e8b7c88 server_cookie=eed44c51ac6f8b2c in_seq=0 out_seq=0 2024-03-26T14:18:04.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.246+0000 7fc474c00640 10 client.18882 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:04.250 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc474c00640 1 -- 172.21.15.137:0/900234523 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d3aa936790 con 0x55d3aa9d9b20 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc474c00640 10 client.18882 handle_command_reply: tid=0 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 client.18882 shutdown 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 msgr2=0x55d3aa9ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9ce2d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc4700a3a40 tx=0x7fc47009b130 comp rx=0 tx=0).stop 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 shutdown_connections 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aa9d9b20 0x55d3aa9dbf70 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c4870 0x55d3aa9ce2d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 --2- 172.21.15.137:0/900234523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc474c00640 0 client.18882 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.248+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 >> 172.21.15.137:0/900234523 conn(0x55d3aa98c200 msgr2=0x55d3aa9bea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.249+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 shutdown_connections 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.249+0000 7fc490b89740 1 -- 172.21.15.137:0/900234523 wait complete. 2024-03-26T14:18:04.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.249+0000 7fc490b89740 20 client.18882 trim_cache size 0 max 16384 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 Processor -- start 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 -- start start 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9be200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881aea20 con 0x55d3aa9c3e80 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881ac520 con 0x55d3aa9c4870 2024-03-26T14:18:04.267 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.266+0000 7fc490b89740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881aa5a0 con 0x55d3aa9c99e0 2024-03-26T14:18:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9bfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37906/0 (socket says 172.21.15.137:37906) 2024-03-26T14:18:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 -- 172.21.15.137:0/52571357 learned_addr learned my addr 172.21.15.137:0/52571357 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc47e000640 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.269 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc487400640 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 -- 172.21.15.137:0/52571357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 msgr2=0x55d3aa9be200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9be200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 -- 172.21.15.137:0/52571357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 msgr2=0x55d3aa9d52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 -- 172.21.15.137:0/52571357 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3aa5fd0f0 con 0x55d3aa9c3e80 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.267+0000 7fc486a00640 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9bfa20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc478005ec0 tx=0x7fc47802c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad3614d6be73342 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc477400640 1 -- 172.21.15.137:0/52571357 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478044040 con 0x55d3aa9c3e80 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc477400640 1 -- 172.21.15.137:0/52571357 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47804f070 con 0x55d3aa9c3e80 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc477400640 1 -- 172.21.15.137:0/52571357 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478003d60 con 0x55d3aa9c3e80 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 -- 172.21.15.137:0/52571357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 msgr2=0x55d3aa9bfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9bfa20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc478005ec0 tx=0x7fc47802c660 comp rx=0 tx=0).stop 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 -- 172.21.15.137:0/52571357 shutdown_connections 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9be200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 --2- 172.21.15.137:0/52571357 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9bfa20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.270 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 -- 172.21.15.137:0/52571357 >> 172.21.15.137:0/52571357 conn(0x55d3aa98c200 msgr2=0x55d3aa9bf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 -- 172.21.15.137:0/52571357 shutdown_connections 2024-03-26T14:18:04.271 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.268+0000 7fc490b89740 1 -- 172.21.15.137:0/52571357 wait complete. 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 -- start start 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 Processor -- start 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881aaba0 con 0x55d3aa9c3e80 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881abec0 con 0x55d3aa9c4870 2024-03-26T14:18:04.273 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4881ab200 con 0x55d3aa9c99e0 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.269+0000 7fc490b89740 10 client.0 _ll_register_callbacks cb 0x7fc47000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37918/0 (socket says 172.21.15.137:37918) 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 -- 172.21.15.137:0/1214918711 learned_addr learned my addr 172.21.15.137:0/1214918711 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc47e000640 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 -- 172.21.15.137:0/1214918711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 msgr2=0x55d3aa9d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 -- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 msgr2=0x55d3aa9d35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 -- 172.21.15.137:0/1214918711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3aa9d4bd0 con 0x55d3aa9c3e80 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc486a00640 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9d3090 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc478043040 tx=0x7fc478005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcf8914fd273380 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc474c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:04.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc474c00640 1 -- 172.21.15.137:0/1214918711 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478057070 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc474c00640 1 -- 172.21.15.137:0/1214918711 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc4780042c0 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.270+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc47804f070 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc474c00640 1 -- 172.21.15.137:0/1214918711 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc478044040 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 10 client.18888 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d3aaa35ad0 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc474c00640 1 -- 172.21.15.137:0/1214918711 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc47804f070 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc474c00640 1 -- 172.21.15.137:0/1214918711 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc478040960 con 0x55d3aa9c3e80 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 10 client.18888 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 10 client.18888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 20 client.18888 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aaa35fa0 0x55d3aaa38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 4 client.18888 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.271+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d3aa930ae0 con 0x55d3aaa35fa0 2024-03-26T14:18:04.275 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.272+0000 7fc47e000640 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aaa35fa0 0x55d3aaa38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc474c00640 10 client.18888 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:04.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc47e000640 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aaa35fa0 0x55d3aaa38390 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2a4025230bd503 server_cookie=7be619058f52d36c in_seq=0 out_seq=0 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc474c00640 1 -- 172.21.15.137:0/1214918711 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d3aa930ae0 con 0x55d3aaa35fa0 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc474c00640 10 client.18888 handle_command_reply: tid=0 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 client.18888 shutdown 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 msgr2=0x55d3aa9d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9d3090 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fc478043040 tx=0x7fc478005dd0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 shutdown_connections 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55d3aaa35fa0 0x55d3aaa38390 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3aa9c4870 0x55d3aa9d35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3aa9c99e0 0x55d3aa9d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 --2- 172.21.15.137:0/1214918711 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55d3aa9c3e80 0x55d3aa9d3090 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc474c00640 0 client.18888 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 >> 172.21.15.137:0/1214918711 conn(0x55d3aa98c200 msgr2=0x55d3aa9c32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 shutdown_connections 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.273+0000 7fc490b89740 1 -- 172.21.15.137:0/1214918711 wait complete. 2024-03-26T14:18:04.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc490b89740 20 client.18888 trim_cache size 0 max 16384 2024-03-26T14:18:04.278 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:18:04.278 INFO:teuthology.orchestra.run.smithi137.stdout: "status": "no active scrubs running", 2024-03-26T14:18:04.278 INFO:teuthology.orchestra.run.smithi137.stdout: "scrubs": {} 2024-03-26T14:18:04.278 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc46c0a2880 msgr2=0x7fc46c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc46c0a2880 0x7fc46c0a4d40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55d3aa9c50d0 tx=0x7fc480009800 comp rx=0 tx=0).stop 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 msgr2=0x7fc48819a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 0x7fc48819a080 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fc47000e9f0 tx=0x7fc47000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 shutdown_connections 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc46c0a2880 0x7fc46c0a4d40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc488105490 0x7fc4881a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc488104ad0 0x7fc48819a080 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 --2- 172.21.15.137:0/2295725823 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4881038e0 0x7fc488199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 >> 172.21.15.137:0/2295725823 conn(0x7fc4880fee50 msgr2=0x7fc488107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.274+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 shutdown_connections 2024-03-26T14:18:04.279 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.275+0000 7fc48de00640 1 -- 172.21.15.137:0/2295725823 wait complete. 2024-03-26T14:18:04.292 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-26T14:18:04.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 -- 172.21.15.137:0/2103034130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5488077c40 msgr2=0x7f54880784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 --2- 172.21.15.137:0/2103034130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5488077c40 0x7f54880784e0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5480009ca0 tx=0x7f548002c100 comp rx=0 tx=0).stop 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 -- 172.21.15.137:0/2103034130 shutdown_connections 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 --2- 172.21.15.137:0/2103034130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54880772a0 0x7f5488077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 --2- 172.21.15.137:0/2103034130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5488077c40 0x7f54880784e0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 --2- 172.21.15.137:0/2103034130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5488076060 0x7f5488076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.504 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 -- 172.21.15.137:0/2103034130 >> 172.21.15.137:0/2103034130 conn(0x7f54880fee50 msgr2=0x7f5488101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 -- 172.21.15.137:0/2103034130 shutdown_connections 2024-03-26T14:18:04.505 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.503+0000 7f548dc00640 1 -- 172.21.15.137:0/2103034130 wait complete. 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 Processor -- start 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 -- start start 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5488076060 0x7f548819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 0x7f548819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5488077c40 0x7f54881a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5488113820 con 0x7f54880772a0 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54881136a0 con 0x7f5488076060 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f548dc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54881139a0 con 0x7f5488077c40 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f5486000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 0x7f548819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f5486000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 0x7f548819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37926/0 (socket says 172.21.15.137:37926) 2024-03-26T14:18:04.507 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.504+0000 7f5486000640 1 -- 172.21.15.137:0/255519260 learned_addr learned my addr 172.21.15.137:0/255519260 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486a00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5488076060 0x7f548819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5487400640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5488077c40 0x7f54881a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486000640 1 -- 172.21.15.137:0/255519260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5488077c40 msgr2=0x7f54881a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486000640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5488077c40 0x7f54881a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486000640 1 -- 172.21.15.137:0/255519260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5488076060 msgr2=0x7f548819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486000640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5488076060 0x7f548819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486000640 1 -- 172.21.15.137:0/255519260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54881a5af0 con 0x7f54880772a0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f5486000640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 0x7f548819e410 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f547400e9f0 tx=0x7f547400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547400cd80 con 0x7f54880772a0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54740108e0 con 0x7f54880772a0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.505+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474018920 con 0x7f54880772a0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.506+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54881a5de0 con 0x7f54880772a0 2024-03-26T14:18:04.508 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.506+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54881a6250 con 0x7f54880772a0 2024-03-26T14:18:04.509 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.507+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5488110440 con 0x7f54880772a0 2024-03-26T14:18:04.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.509+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5474010a90 con 0x7f54880772a0 2024-03-26T14:18:04.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.511+0000 7f547f400640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f546c0a2880 0x7f546c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.512+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f54740cf190 con 0x7f54880772a0 2024-03-26T14:18:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.512+0000 7f5486a00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f546c0a2880 0x7f546c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.513+0000 7f5486a00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f546c0a2880 0x7f546c0a4d40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5478009bd0 tx=0x7f5478005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.516+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f547409be20 con 0x7f54880772a0 2024-03-26T14:18:04.756 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.755+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54880623a0 con 0x7f54880772a0 2024-03-26T14:18:04.758 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.757+0000 7f547f400640 1 -- 172.21.15.137:0/255519260 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f547409b720 con 0x7f54880772a0 2024-03-26T14:18:04.758 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:18:04.759 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:18:04.759 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:18:04.761 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f546c0a2880 msgr2=0x7f546c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f546c0a2880 0x7f546c0a4d40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5478009bd0 tx=0x7f5478005f90 comp rx=0 tx=0).stop 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 msgr2=0x7f548819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 0x7f548819e410 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f547400e9f0 tx=0x7f547400eeb0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 shutdown_connections 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f546c0a2880 0x7f546c0a4d40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5488077c40 0x7f54881a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f54880772a0 0x7f548819e410 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 --2- 172.21.15.137:0/255519260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5488076060 0x7f548819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 >> 172.21.15.137:0/255519260 conn(0x7f54880fee50 msgr2=0x7f5488100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 shutdown_connections 2024-03-26T14:18:04.762 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.760+0000 7f548dc00640 1 -- 172.21.15.137:0/255519260 wait complete. 2024-03-26T14:18:04.780 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:18:04.781 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 -- 172.21.15.137:0/2474174494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63100756a0 msgr2=0x7f6310073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 --2- 172.21.15.137:0/2474174494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63100756a0 0x7f6310073af0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f6300009900 tx=0x7f630002bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 -- 172.21.15.137:0/2474174494 shutdown_connections 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 --2- 172.21.15.137:0/2474174494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6310109560 0x7f631010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 --2- 172.21.15.137:0/2474174494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100740c0 0x7f6310074540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 --2- 172.21.15.137:0/2474174494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f63100756a0 0x7f6310073af0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 -- 172.21.15.137:0/2474174494 >> 172.21.15.137:0/2474174494 conn(0x7f63100fcc40 msgr2=0x7f63100ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 -- 172.21.15.137:0/2474174494 shutdown_connections 2024-03-26T14:18:04.985 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.984+0000 7f6315a00640 1 -- 172.21.15.137:0/2474174494 wait complete. 2024-03-26T14:18:04.987 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 Processor -- start 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 -- start start 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63100740c0 0x7f6310199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100756a0 0x7f631019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 0x7f63101a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631010f410 con 0x7f6310109560 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631010f290 con 0x7f63100740c0 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f6315a00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631010f590 con 0x7f63100756a0 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.985+0000 7f630f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 0x7f63101a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 0x7f63101a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37932/0 (socket says 172.21.15.137:37932) 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 -- 172.21.15.137:0/2945324229 learned_addr learned my addr 172.21.15.137:0/2945324229 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630ea00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63100740c0 0x7f6310199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630e000640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100756a0 0x7f631019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 -- 172.21.15.137:0/2945324229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100756a0 msgr2=0x7f631019a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100756a0 0x7f631019a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 -- 172.21.15.137:0/2945324229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63100740c0 msgr2=0x7f6310199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.988 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63100740c0 0x7f6310199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 -- 172.21.15.137:0/2945324229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63101a1760 con 0x7f6310109560 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630e000640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100756a0 0x7f631019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f630f400640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 0x7f63101a10f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f630800efd0 tx=0x7f630800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f6307400640 1 -- 172.21.15.137:0/2945324229 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308003b70 con 0x7f6310109560 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f6307400640 1 -- 172.21.15.137:0/2945324229 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6308003d10 con 0x7f6310109560 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.986+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63101a1a50 con 0x7f6310109560 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.987+0000 7f6307400640 1 -- 172.21.15.137:0/2945324229 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308010980 con 0x7f6310109560 2024-03-26T14:18:04.989 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.987+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63101a1f50 con 0x7f6310109560 2024-03-26T14:18:04.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.990+0000 7f63187b6740 1 Processor -- start 2024-03-26T14:18:04.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.990+0000 7f6307400640 1 -- 172.21.15.137:0/2945324229 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6308004020 con 0x7f6310109560 2024-03-26T14:18:04.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.990+0000 7f63187b6740 1 -- start start 2024-03-26T14:18:04.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f63187b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f63187b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 0x55db2fd09e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f63187b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd0a3a0 0x55db2fd10c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f63187b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2fc6d2c0 con 0x55db2fd099e0 2024-03-26T14:18:04.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f63187b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2fc15c90 con 0x55db2fd08810 2024-03-26T14:18:04.992 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f63187b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2fc69360 con 0x55db2fd0a3a0 2024-03-26T14:18:04.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 0x55db2fd09e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.993 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 0x55db2fd09e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37936/0 (socket says 172.21.15.137:37936) 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 -- 172.21.15.137:0/2346184458 learned_addr learned my addr 172.21.15.137:0/2346184458 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630e000640 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630e000640 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd0a3a0 0x55db2fd10c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 -- 172.21.15.137:0/2346184458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd0a3a0 msgr2=0x55db2fd10c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd0a3a0 0x55db2fd10c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 -- 172.21.15.137:0/2346184458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 msgr2=0x55db2fd08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.991+0000 7f630ea00640 1 -- 172.21.15.137:0/2346184458 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db2fc3a080 con 0x55db2fd099e0 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f6307400640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62f40a2800 0x7f62f40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f630ea00640 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 0x55db2fd09e60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f630002c840 tx=0x7f6300009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461707248b510c8b server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f6307400640 1 -- 172.21.15.137:0/2945324229 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f63080cf420 con 0x7f6310109560 2024-03-26T14:18:04.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f630e000640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62f40a2800 0x7f62f40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f62fbe00640 1 -- 172.21.15.137:0/2346184458 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300044020 con 0x55db2fd099e0 2024-03-26T14:18:04.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f62fbe00640 1 -- 172.21.15.137:0/2346184458 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f630002cae0 con 0x55db2fd099e0 2024-03-26T14:18:04.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.992+0000 7f62fbe00640 1 -- 172.21.15.137:0/2346184458 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300004920 con 0x55db2fd099e0 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f630e000640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62f40a2800 0x7f62f40a4cc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0058c0 tx=0x7f62fc0095d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 -- 172.21.15.137:0/2346184458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 msgr2=0x55db2fd09e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 0x55db2fd09e60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f630002c840 tx=0x7f6300009670 comp rx=0 tx=0).stop 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 -- 172.21.15.137:0/2346184458 shutdown_connections 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd0a3a0 0x55db2fd10c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd099e0 0x55db2fd09e60 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.998 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 --2- 172.21.15.137:0/2346184458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 -- 172.21.15.137:0/2346184458 >> 172.21.15.137:0/2346184458 conn(0x55db2fd03c00 msgr2=0x55db2fd04000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 -- 172.21.15.137:0/2346184458 shutdown_connections 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.993+0000 7f63187b6740 1 -- 172.21.15.137:0/2346184458 wait complete. 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 -- start start 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 Processor -- start 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd12a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd12f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 0x55db2fd19f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2fd11350 con 0x55db2fd0a3a0 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2fc290d0 con 0x55db2fd08810 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.994+0000 7f63187b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db2fd114d0 con 0x55db2fd099e0 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f63187b6740 10 client.0 _ll_register_callbacks cb 0x55db2fc2de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 0x55db2fd19f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 0x55db2fd19f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37946/0 (socket says 172.21.15.137:37946) 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 -- 172.21.15.137:0/2389579041 learned_addr learned my addr 172.21.15.137:0/2389579041 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:04.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630ea00640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd12a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630e000640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd12f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 -- 172.21.15.137:0/2389579041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 msgr2=0x55db2fd12f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd12f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 -- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 msgr2=0x55db2fd12a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd12a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 -- 172.21.15.137:0/2389579041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db2f9813f0 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f630f400640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 0x55db2fd19f10 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6308098380 tx=0x7f6308097640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b69a4bc1c04694 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.995+0000 7f62f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f62f8c00640 1 -- 172.21.15.137:0/2389579041 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080b4060 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f62f8c00640 1 -- 172.21.15.137:0/2389579041 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f63080cf160 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f630002cae0 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f62f8c00640 1 -- 172.21.15.137:0/2389579041 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f630809c0b0 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f63187b6740 10 client.18906 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55db2fa4b7d0 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f62f8c00640 1 -- 172.21.15.137:0/2389579041 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f630002cae0 con 0x55db2fd0a3a0 2024-03-26T14:18:05.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f62f8c00640 1 -- 172.21.15.137:0/2389579041 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f630809cce0 con 0x55db2fd0a3a0 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f63187b6740 10 client.18906 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.996+0000 7f63187b6740 10 client.18906 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.997+0000 7f63187b6740 20 client.18906 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.997+0000 7f63187b6740 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd1e440 0x55db2fd208e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.997+0000 7f63187b6740 4 client.18906 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.997+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db2fc7a800 con 0x55db2fd1e440 2024-03-26T14:18:05.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.997+0000 7f630ea00640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd1e440 0x55db2fd208e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.998+0000 7f630ea00640 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd1e440 0x55db2fd208e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3ce468ce7bcff53 server_cookie=49bbf7b600190ed7 in_seq=0 out_seq=0 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.998+0000 7f62f8c00640 10 client.18906 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.999+0000 7f62f8c00640 1 -- 172.21.15.137:0/2389579041 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55db2fc7a800 con 0x55db2fd1e440 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:04.999+0000 7f62f8c00640 10 client.18906 handle_command_reply: tid=0 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 client.18906 shutdown 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 msgr2=0x55db2fd19f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 0x55db2fd19f10 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6308098380 tx=0x7f6308097640 comp rx=0 tx=0).stop 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 shutdown_connections 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd1e440 0x55db2fd208e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd0a3a0 0x55db2fd19f10 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd12f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 --2- 172.21.15.137:0/2389579041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd12a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f62f8c00640 0 client.18906 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 >> 172.21.15.137:0/2389579041 conn(0x55db2fd03c00 msgr2=0x55db2fd02eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 shutdown_connections 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 1 -- 172.21.15.137:0/2389579041 wait complete. 2024-03-26T14:18:05.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.000+0000 7f63187b6740 20 client.18906 trim_cache size 0 max 16384 2024-03-26T14:18:05.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 Processor -- start 2024-03-26T14:18:05.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 -- start start 2024-03-26T14:18:05.015 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 0x55db2fd0ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd19ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd0cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63101ae930 con 0x55db2fd03c00 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63101ac430 con 0x55db2fd08810 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f63187b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63101aa4b0 con 0x55db2fd099e0 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f630ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 0x55db2fd0ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f630ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 0x55db2fd0ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37954/0 (socket says 172.21.15.137:37954) 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.014+0000 7f630ea00640 1 -- 172.21.15.137:0/3975472594 learned_addr learned my addr 172.21.15.137:0/3975472594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630f400640 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd0cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630e000640 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd19ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630ea00640 1 -- 172.21.15.137:0/3975472594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 msgr2=0x55db2fd0cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630ea00640 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd0cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630ea00640 1 -- 172.21.15.137:0/3975472594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 msgr2=0x55db2fd19ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630ea00640 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd19ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.016 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630ea00640 1 -- 172.21.15.137:0/3975472594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db2fcfe2c0 con 0x55db2fd03c00 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f630ea00640 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 0x55db2fd0ca80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6300009230 tx=0x7f630002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7f12f4ef71a64f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f62fb400640 1 -- 172.21.15.137:0/3975472594 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300044040 con 0x55db2fd03c00 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f62fb400640 1 -- 172.21.15.137:0/3975472594 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6300005040 con 0x55db2fd03c00 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.015+0000 7f62fb400640 1 -- 172.21.15.137:0/3975472594 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300004920 con 0x55db2fd03c00 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 -- 172.21.15.137:0/3975472594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 msgr2=0x55db2fd0ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 0x55db2fd0ca80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6300009230 tx=0x7f630002bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 -- 172.21.15.137:0/3975472594 shutdown_connections 2024-03-26T14:18:05.020 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd08810 0x55db2fd19ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd099e0 0x55db2fd0cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 --2- 172.21.15.137:0/3975472594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd03c00 0x55db2fd0ca80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 -- 172.21.15.137:0/3975472594 >> 172.21.15.137:0/3975472594 conn(0x55db2fd11c10 msgr2=0x55db2fd076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 -- 172.21.15.137:0/3975472594 shutdown_connections 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 -- 172.21.15.137:0/3975472594 wait complete. 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 -- start start 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.016+0000 7f63187b6740 1 Processor -- start 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd03c00 0x55db2fd197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 0x55db2fd19cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd099e0 0x55db2fd16390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63101aaab0 con 0x55db2fd08810 2024-03-26T14:18:05.021 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63101abdd0 con 0x55db2fd099e0 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63101ab110 con 0x55db2fd03c00 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f63187b6740 10 client.0 _ll_register_callbacks cb 0x7f63080047e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f630e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 0x55db2fd19cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f630e000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 0x55db2fd19cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37958/0 (socket says 172.21.15.137:37958) 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f630e000640 1 -- 172.21.15.137:0/428703565 learned_addr learned my addr 172.21.15.137:0/428703565 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f630ea00640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd03c00 0x55db2fd197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.017+0000 7f630f400640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd099e0 0x55db2fd16390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f630e000640 1 -- 172.21.15.137:0/428703565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd03c00 msgr2=0x55db2fd197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f630e000640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd03c00 0x55db2fd197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f630e000640 1 -- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd099e0 msgr2=0x55db2fd16390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f630e000640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd099e0 0x55db2fd16390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f630e000640 1 -- 172.21.15.137:0/428703565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db2fd16a00 con 0x55db2fd08810 2024-03-26T14:18:05.022 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f630e000640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 0x55db2fd19cf0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0038a0 tx=0x7f62fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ee7ea079bd7269 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f62f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f62f8c00640 1 -- 172.21.15.137:0/428703565 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0119c0 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f62f8c00640 1 -- 172.21.15.137:0/428703565 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f62fc011b60 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6300005040 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f62f8c00640 1 -- 172.21.15.137:0/428703565 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62fc034050 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f63187b6740 10 client.18912 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.018+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55db2fd7aa10 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f62f8c00640 1 -- 172.21.15.137:0/428703565 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6300005040 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f62f8c00640 1 -- 172.21.15.137:0/428703565 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7f62fc023ca0 con 0x55db2fd08810 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f63187b6740 10 client.18912 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f63187b6740 10 client.18912 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f63187b6740 20 client.18912 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f63187b6740 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd0a3a0 0x55db2fd7d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.023 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f63187b6740 4 client.18912 mds_command: new command op to 34424 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:18:05.024 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 --> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55db2fc74b50 con 0x55db2fd0a3a0 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.019+0000 7f630ea00640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd0a3a0 0x55db2fd7d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.020+0000 7f630ea00640 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd0a3a0 0x55db2fd7d410 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b3604ccba1251a82 server_cookie=9703630b62abea45 in_seq=0 out_seq=0 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.020+0000 7f62f8c00640 10 client.18912 ms_handle_connect on v2:172.21.15.159:6834/3125472496 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f62f8c00640 1 -- 172.21.15.137:0/428703565 <== mds.2 v2:172.21.15.159:6834/3125472496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55db2fc74b50 con 0x55db2fd0a3a0 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f62f8c00640 10 client.18912 handle_command_reply: tid=0 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 client.18912 shutdown 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 msgr2=0x55db2fd19cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 0x55db2fd19cf0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0038a0 tx=0x7f62fc004730 comp rx=0 tx=0).stop 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 shutdown_connections 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:6834/3125472496,v1:172.21.15.159:6835/3125472496] conn(0x55db2fd0a3a0 0x55db2fd7d410 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db2fd099e0 0x55db2fd16390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.025 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55db2fd08810 0x55db2fd19cf0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 --2- 172.21.15.137:0/428703565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55db2fd03c00 0x55db2fd197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 >> 172.21.15.137:0/428703565 conn(0x55db2fd11c10 msgr2=0x55db2fd02620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f62f8c00640 0 client.18912 ms_handle_reset on v2:172.21.15.159:6834/3125472496 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 shutdown_connections 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 1 -- 172.21.15.137:0/428703565 wait complete. 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.021+0000 7f63187b6740 20 client.18912 trim_cache size 0 max 16384 2024-03-26T14:18:05.026 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62f40a2800 msgr2=0x7f62f40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62f40a2800 0x7f62f40a4cc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0058c0 tx=0x7f62fc0095d0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 msgr2=0x7f63101a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 0x7f63101a10f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f630800efd0 tx=0x7f630800c6a0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 shutdown_connections 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f62f40a2800 0x7f62f40a4cc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6310109560 0x7f63101a10f0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f63100756a0 0x7f631019a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.027 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 --2- 172.21.15.137:0/2945324229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63100740c0 0x7f6310199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 >> 172.21.15.137:0/2945324229 conn(0x7f63100fcc40 msgr2=0x7f63101054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 shutdown_connections 2024-03-26T14:18:05.028 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.022+0000 7f6315a00640 1 -- 172.21.15.137:0/2945324229 wait complete. 2024-03-26T14:18:05.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:04 smithi137 ceph-mon[31539]: pgmap v929: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-26T14:18:05.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:04 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/255519260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:18:05.039 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:18:05.040 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-03-26T14:18:05.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:04 smithi179 ceph-mon[35472]: pgmap v929: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-26T14:18:05.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:04 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/255519260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:18:05.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:04 smithi159 ceph-mon[35501]: pgmap v929: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-03-26T14:18:05.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:04 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/255519260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.243+0000 7fd47a000640 1 -- 172.21.15.137:0/385514761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474076060 msgr2=0x7fd474076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.243+0000 7fd47a000640 1 --2- 172.21.15.137:0/385514761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474076060 0x7fd474076480 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd464009900 tx=0x7fd46402bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 -- 172.21.15.137:0/385514761 shutdown_connections 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 --2- 172.21.15.137:0/385514761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4740772a0 0x7fd474077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 --2- 172.21.15.137:0/385514761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd474077c40 0x7fd4740784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 --2- 172.21.15.137:0/385514761 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474076060 0x7fd474076480 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 -- 172.21.15.137:0/385514761 >> 172.21.15.137:0/385514761 conn(0x7fd4740fee50 msgr2=0x7fd474101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 -- 172.21.15.137:0/385514761 shutdown_connections 2024-03-26T14:18:05.245 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.244+0000 7fd47a000640 1 -- 172.21.15.137:0/385514761 wait complete. 2024-03-26T14:18:05.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 Processor -- start 2024-03-26T14:18:05.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 -- start start 2024-03-26T14:18:05.247 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd474076060 0x7fd47419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4740772a0 0x7fd47419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 0x7fd4741a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd474113860 con 0x7fd474077c40 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741136e0 con 0x7fd474076060 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd47a000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741139e0 con 0x7fd4740772a0 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd473e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 0x7fd4741a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd473e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 0x7fd4741a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37970/0 (socket says 172.21.15.137:37970) 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.245+0000 7fd473e00640 1 -- 172.21.15.137:0/898334543 learned_addr learned my addr 172.21.15.137:0/898334543 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473400640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd474076060 0x7fd47419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd46be00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4740772a0 0x7fd47419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473e00640 1 -- 172.21.15.137:0/898334543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4740772a0 msgr2=0x7fd47419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473e00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4740772a0 0x7fd47419e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473e00640 1 -- 172.21.15.137:0/898334543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd474076060 msgr2=0x7fd47419df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473e00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd474076060 0x7fd47419df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473e00640 1 -- 172.21.15.137:0/898334543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4741a5b90 con 0x7fd474077c40 2024-03-26T14:18:05.248 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd473e00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 0x7fd4741a5520 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00eaa0 tx=0x7fd46c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd470c00640 1 -- 172.21.15.137:0/898334543 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00cdb0 con 0x7fd474077c40 2024-03-26T14:18:05.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd470c00640 1 -- 172.21.15.137:0/898334543 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd46c005020 con 0x7fd474077c40 2024-03-26T14:18:05.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4741a5e80 con 0x7fd474077c40 2024-03-26T14:18:05.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.246+0000 7fd470c00640 1 -- 172.21.15.137:0/898334543 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0116d0 con 0x7fd474077c40 2024-03-26T14:18:05.249 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.247+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4741a6390 con 0x7fd474077c40 2024-03-26T14:18:05.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 Processor -- start 2024-03-26T14:18:05.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd470c00640 1 -- 172.21.15.137:0/898334543 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd46c0051d0 con 0x7fd474077c40 2024-03-26T14:18:05.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 -- start start 2024-03-26T14:18:05.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf01dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.252 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 0x560fbf01ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01f3a0 0x560fbf025c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbef822c0 con 0x560fbf01e9e0 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbef2ac90 con 0x560fbf01f3a0 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.250+0000 7fd47cde1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbef7e360 con 0x560fbf01d810 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 0x560fbf01ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 0x560fbf01ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37982/0 (socket says 172.21.15.137:37982) 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 -- 172.21.15.137:0/2516934910 learned_addr learned my addr 172.21.15.137:0/2516934910 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd46be00640 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf01dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd46be00640 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01f3a0 0x560fbf025c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd470c00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4580a2880 0x7fd4580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd470c00640 1 -- 172.21.15.137:0/898334543 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd46c0cefd0 con 0x7fd474077c40 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 -- 172.21.15.137:0/2516934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 msgr2=0x560fbf01dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf01dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 -- 172.21.15.137:0/2516934910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01f3a0 msgr2=0x560fbf025c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01f3a0 0x560fbf025c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473400640 1 -- 172.21.15.137:0/2516934910 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fbef4f080 con 0x560fbf01e9e0 2024-03-26T14:18:05.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.251+0000 7fd473e00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4580a2880 0x7fd4580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.252+0000 7fd473400640 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 0x560fbf01ee60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd46402c840 tx=0x7fd464005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4386654d4b2a89 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.252+0000 7fd473e00640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4580a2880 0x7fd4580a4d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd46c09b4f0 tx=0x7fd46c00cf50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.257 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.252+0000 7fd463e00640 1 -- 172.21.15.137:0/2516934910 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464044020 con 0x560fbf01e9e0 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.252+0000 7fd463e00640 1 -- 172.21.15.137:0/2516934910 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd46402cbe0 con 0x560fbf01e9e0 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.252+0000 7fd463e00640 1 -- 172.21.15.137:0/2516934910 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464004920 con 0x560fbf01e9e0 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 -- 172.21.15.137:0/2516934910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 msgr2=0x560fbf01ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 0x560fbf01ee60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd46402c840 tx=0x7fd464005dd0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 -- 172.21.15.137:0/2516934910 shutdown_connections 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01f3a0 0x560fbf025c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01e9e0 0x560fbf01ee60 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 --2- 172.21.15.137:0/2516934910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf01dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 -- 172.21.15.137:0/2516934910 >> 172.21.15.137:0/2516934910 conn(0x560fbf018c00 msgr2=0x560fbf019000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 -- 172.21.15.137:0/2516934910 shutdown_connections 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 -- 172.21.15.137:0/2516934910 wait complete. 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.253+0000 7fd47cde1740 1 -- start start 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 Processor -- start 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf0279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf027f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 0x560fbf02ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbf026350 con 0x560fbf01d810 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbef3e0d0 con 0x560fbf01e9e0 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fbf0264d0 con 0x560fbf01f3a0 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd47cde1740 10 client.0 _ll_register_callbacks cb 0x560fbef42e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd46be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf0279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd46be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf0279d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37992/0 (socket says 172.21.15.137:37992) 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.254+0000 7fd46be00640 1 -- 172.21.15.137:0/602148250 learned_addr learned my addr 172.21.15.137:0/602148250 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd473400640 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf027f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd46be00640 1 -- 172.21.15.137:0/602148250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 msgr2=0x560fbf02ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd46be00640 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 0x560fbf02ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd46be00640 1 -- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 msgr2=0x560fbf027f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd46be00640 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf027f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd46be00640 1 -- 172.21.15.137:0/602148250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fbec963f0 con 0x560fbf01d810 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd46be00640 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf0279d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd45c010840 tx=0x7fd45c010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66b0dd7830a0f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.259 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd460c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd460c00640 1 -- 172.21.15.137:0/602148250 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00fbf0 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.255+0000 7fd460c00640 1 -- 172.21.15.137:0/602148250 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd45c00fd90 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd46402cbe0 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd460c00640 1 -- 172.21.15.137:0/602148250 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd45c013020 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 10 client.18924 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560fbed607d0 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd460c00640 1 -- 172.21.15.137:0/602148250 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd46402cbe0 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd460c00640 1 -- 172.21.15.137:0/602148250 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd45c00f040 con 0x560fbf01d810 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 10 client.18924 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 10 client.18924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 20 client.18924 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:05.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf0333f0 0x560fbf0358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 4 client.18924 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:18:05.261 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.256+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fbef8f800 con 0x560fbf0333f0 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.257+0000 7fd473400640 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf0333f0 0x560fbf0358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.257+0000 7fd473400640 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf0333f0 0x560fbf0358b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f5186a9657a5d55 server_cookie=556670671b2290f8 in_seq=0 out_seq=0 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.257+0000 7fd460c00640 10 client.18924 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.259+0000 7fd460c00640 1 -- 172.21.15.137:0/602148250 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x560fbef8f800 con 0x560fbf0333f0 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.259+0000 7fd460c00640 10 client.18924 handle_command_reply: tid=0 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.259+0000 7fd47cde1740 1 client.18924 shutdown 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 msgr2=0x560fbf0279d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf0279d0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fd45c010840 tx=0x7fd45c010d00 comp rx=0 tx=0).stop 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 shutdown_connections 2024-03-26T14:18:05.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf0333f0 0x560fbf0358b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf027f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 0x560fbf02ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd460c00640 0 client.18924 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 --2- 172.21.15.137:0/602148250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf0279d0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 >> 172.21.15.137:0/602148250 conn(0x560fbf018c00 msgr2=0x560fbf017eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 shutdown_connections 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 1 -- 172.21.15.137:0/602148250 wait complete. 2024-03-26T14:18:05.263 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.260+0000 7fd47cde1740 20 client.18924 trim_cache size 0 max 16384 2024-03-26T14:18:05.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.273+0000 7fd47cde1740 1 Processor -- start 2024-03-26T14:18:05.274 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 -- start start 2024-03-26T14:18:05.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf021a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 0x560fbf021fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741b2e90 con 0x560fbf01f3a0 2024-03-26T14:18:05.276 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741b0990 con 0x560fbf01e9e0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd47cde1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741aea10 con 0x560fbf01d810 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd46be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 0x560fbf021fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd46be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 0x560fbf021fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:37998/0 (socket says 172.21.15.137:37998) 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd46be00640 1 -- 172.21.15.137:0/1395816102 learned_addr learned my addr 172.21.15.137:0/1395816102 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd473400640 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.274+0000 7fd46be00640 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf021a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd46be00640 1 -- 172.21.15.137:0/1395816102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 msgr2=0x560fbf021a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd46be00640 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf021a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd46be00640 1 -- 172.21.15.137:0/1395816102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 msgr2=0x560fbf02ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd46be00640 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd46be00640 1 -- 172.21.15.137:0/1395816102 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fbf01e460 con 0x560fbf01f3a0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd46be00640 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 0x560fbf021fc0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd45c005fd0 tx=0x7fd45c00e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e09df32bc11fe35 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd463400640 1 -- 172.21.15.137:0/1395816102 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00f1c0 con 0x560fbf01f3a0 2024-03-26T14:18:05.277 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd463400640 1 -- 172.21.15.137:0/1395816102 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd45c00fbf0 con 0x560fbf01f3a0 2024-03-26T14:18:05.278 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.275+0000 7fd463400640 1 -- 172.21.15.137:0/1395816102 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c02be60 con 0x560fbf01f3a0 2024-03-26T14:18:05.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 -- 172.21.15.137:0/1395816102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 msgr2=0x560fbf021fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.280 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 0x560fbf021fc0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd45c005fd0 tx=0x7fd45c00e350 comp rx=0 tx=0).stop 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 -- 172.21.15.137:0/1395816102 shutdown_connections 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01f3a0 0x560fbf021fc0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1395816102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01d810 0x560fbf021a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 -- 172.21.15.137:0/1395816102 >> 172.21.15.137:0/1395816102 conn(0x560fbf018c00 msgr2=0x560fbf01c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 -- 172.21.15.137:0/1395816102 shutdown_connections 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 -- 172.21.15.137:0/1395816102 wait complete. 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 -- start start 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.276+0000 7fd47cde1740 1 Processor -- start 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf02afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 0x560fbf02e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741af010 con 0x560fbf01d810 2024-03-26T14:18:05.281 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741b0330 con 0x560fbf01e9e0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741af670 con 0x560fbf01f3a0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd47cde1740 10 client.0 _ll_register_callbacks cb 0x7fd45c010480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd46be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf02afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd46be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf02afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38004/0 (socket says 172.21.15.137:38004) 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd46be00640 1 -- 172.21.15.137:0/1928979030 learned_addr learned my addr 172.21.15.137:0/1928979030 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd473400640 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd46be00640 1 -- 172.21.15.137:0/1928979030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 msgr2=0x560fbf02e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd46be00640 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 0x560fbf02e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.277+0000 7fd46be00640 1 -- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 msgr2=0x560fbf02b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd46be00640 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd46be00640 1 -- 172.21.15.137:0/1928979030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fbf02eab0 con 0x560fbf01d810 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd46be00640 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf02afb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd45c010d40 tx=0x7fd45c00f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390ffdd3ad85231 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd460c00640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd460c00640 1 -- 172.21.15.137:0/1928979030 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c004680 con 0x560fbf01d810 2024-03-26T14:18:05.282 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd460c00640 1 -- 172.21.15.137:0/1928979030 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd45c02b430 con 0x560fbf01d810 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd45c00fbf0 con 0x560fbf01d810 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd460c00640 1 -- 172.21.15.137:0/1928979030 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd45c02b630 con 0x560fbf01d810 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd47cde1740 10 client.18930 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.278+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560fbf08f710 con 0x560fbf01d810 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd460c00640 1 -- 172.21.15.137:0/1928979030 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd45c00fbf0 con 0x560fbf01d810 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd460c00640 1 -- 172.21.15.137:0/1928979030 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fd45c041a20 con 0x560fbf01d810 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd47cde1740 10 client.18930 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd47cde1740 10 client.18930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd47cde1740 20 client.18930 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf090030 0x560fbf092450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd47cde1740 4 client.18930 mds_command: new command op to 34451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 --> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560fbef89b50 con 0x560fbf090030 2024-03-26T14:18:05.283 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.279+0000 7fd473400640 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf090030 0x560fbf092450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.284 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.280+0000 7fd473400640 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf090030 0x560fbf092450 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a383ff91cffe9a4 server_cookie=f2f235102b2e51f6 in_seq=0 out_seq=0 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.280+0000 7fd460c00640 10 client.18930 ms_handle_connect on v2:172.21.15.159:6836/16915360 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.280+0000 7fd460c00640 1 -- 172.21.15.137:0/1928979030 <== mds.1 v2:172.21.15.159:6836/16915360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fbef89b50 con 0x560fbf090030 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.280+0000 7fd460c00640 10 client.18930 handle_command_reply: tid=0 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.280+0000 7fd47cde1740 1 client.18930 shutdown 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 msgr2=0x560fbf02afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf02afb0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd45c010d40 tx=0x7fd45c00f1c0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 shutdown_connections 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:6836/16915360,v1:172.21.15.159:6837/16915360] conn(0x560fbf090030 0x560fbf092450 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fbf01e9e0 0x560fbf02b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560fbf01f3a0 0x560fbf02e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd460c00640 0 client.18930 ms_handle_reset on v2:172.21.15.159:6836/16915360 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 --2- 172.21.15.137:0/1928979030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560fbf01d810 0x560fbf02afb0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 >> 172.21.15.137:0/1928979030 conn(0x560fbf018c00 msgr2=0x560fbf017620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 shutdown_connections 2024-03-26T14:18:05.285 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 1 -- 172.21.15.137:0/1928979030 wait complete. 2024-03-26T14:18:05.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47cde1740 20 client.18930 trim_cache size 0 max 16384 2024-03-26T14:18:05.286 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:18:05.286 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4580a2880 msgr2=0x7fd4580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47a000640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4580a2880 0x7fd4580a4d40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd46c09b4f0 tx=0x7fd46c00cf50 comp rx=0 tx=0).stop 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.281+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 msgr2=0x7fd4741a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 0x7fd4741a5520 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00eaa0 tx=0x7fd46c00ef60 comp rx=0 tx=0).stop 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 shutdown_connections 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4580a2880 0x7fd4580a4d40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd474077c40 0x7fd4741a5520 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd4740772a0 0x7fd47419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 --2- 172.21.15.137:0/898334543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd474076060 0x7fd47419df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 >> 172.21.15.137:0/898334543 conn(0x7fd4740fee50 msgr2=0x7fd474100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 shutdown_connections 2024-03-26T14:18:05.287 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.282+0000 7fd47a000640 1 -- 172.21.15.137:0/898334543 wait complete. 2024-03-26T14:18:05.301 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-26T14:18:05.301 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 -- 172.21.15.137:0/3712300241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c104ad0 msgr2=0x7fc77c104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 --2- 172.21.15.137:0/3712300241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c104ad0 0x7fc77c104f50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009ca0 tx=0x7fc76c02c0f0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 -- 172.21.15.137:0/3712300241 shutdown_connections 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 --2- 172.21.15.137:0/3712300241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc77c105490 0x7fc77c10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 --2- 172.21.15.137:0/3712300241 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c104ad0 0x7fc77c104f50 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 --2- 172.21.15.137:0/3712300241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c1038e0 0x7fc77c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 -- 172.21.15.137:0/3712300241 >> 172.21.15.137:0/3712300241 conn(0x7fc77c0fee50 msgr2=0x7fc77c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 -- 172.21.15.137:0/3712300241 shutdown_connections 2024-03-26T14:18:05.512 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.510+0000 7fc783000640 1 -- 172.21.15.137:0/3712300241 wait complete. 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 Processor -- start 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 -- start start 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc77c1038e0 0x7fc77c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c104ad0 0x7fc77c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 0x7fc77c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c10f410 con 0x7fc77c105490 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c10f290 con 0x7fc77c104ad0 2024-03-26T14:18:05.513 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.511+0000 7fc783000640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c10f590 con 0x7fc77c1038e0 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 0x7fc77c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 0x7fc77c1a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38030/0 (socket says 172.21.15.137:38030) 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 -- 172.21.15.137:0/3426995949 learned_addr learned my addr 172.21.15.137:0/3426995949 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc77be00640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc77c1038e0 0x7fc77c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc77b400640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c104ad0 0x7fc77c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 -- 172.21.15.137:0/3426995949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc77c1038e0 msgr2=0x7fc77c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc77c1038e0 0x7fc77c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 -- 172.21.15.137:0/3426995949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c104ad0 msgr2=0x7fc77c19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c104ad0 0x7fc77c19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 -- 172.21.15.137:0/3426995949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c1a1710 con 0x7fc77c105490 2024-03-26T14:18:05.514 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc77b400640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c104ad0 0x7fc77c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:05.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.512+0000 7fc781000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 0x7fc77c1a10a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc77400d750 tx=0x7fc77400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.513+0000 7fc778c00640 1 -- 172.21.15.137:0/3426995949 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774017070 con 0x7fc77c105490 2024-03-26T14:18:05.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.513+0000 7fc778c00640 1 -- 172.21.15.137:0/3426995949 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc7740054d0 con 0x7fc77c105490 2024-03-26T14:18:05.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.513+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c1a1a00 con 0x7fc77c105490 2024-03-26T14:18:05.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.513+0000 7fc778c00640 1 -- 172.21.15.137:0/3426995949 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774004c60 con 0x7fc77c105490 2024-03-26T14:18:05.515 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.513+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c1a1f70 con 0x7fc77c105490 2024-03-26T14:18:05.517 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.516+0000 7fc785d6b740 1 Processor -- start 2024-03-26T14:18:05.519 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.517+0000 7fc778c00640 1 -- 172.21.15.137:0/3426995949 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc774004e00 con 0x7fc77c105490 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.517+0000 7fc785d6b740 1 -- start start 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 0x55ed1bfbfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfc7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc785d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed1bf242c0 con 0x55ed1bfbf810 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc785d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed1beccc90 con 0x55ed1bfc09e0 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc785d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed1bf20360 con 0x55ed1bfc13a0 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 0x55ed1bfbfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 0x55ed1bfbfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38046/0 (socket says 172.21.15.137:38046) 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 -- 172.21.15.137:0/1785177904 learned_addr learned my addr 172.21.15.137:0/1785177904 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77be00640 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.520 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 -- 172.21.15.137:0/1785177904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfc13a0 msgr2=0x55ed1bfc7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfc7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 -- 172.21.15.137:0/1785177904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 msgr2=0x55ed1bfc0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc778c00640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7600a2880 0x7fc7600a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.518+0000 7fc77b400640 1 -- 172.21.15.137:0/1785177904 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed1bef1080 con 0x55ed1bfbf810 2024-03-26T14:18:05.521 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.519+0000 7fc778c00640 1 -- 172.21.15.137:0/3426995949 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc7740ce820 con 0x7fc77c105490 2024-03-26T14:18:05.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.519+0000 7fc77b400640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7600a2880 0x7fc7600a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.519+0000 7fc77b400640 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 0x55ed1bfbfc30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc76c032380 tx=0x7fc76c0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abb0b7590f02630 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.519+0000 7fc771600640 1 -- 172.21.15.137:0/1785177904 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c039040 con 0x55ed1bfbf810 2024-03-26T14:18:05.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.520+0000 7fc771600640 1 -- 172.21.15.137:0/1785177904 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc76c0328b0 con 0x55ed1bfbf810 2024-03-26T14:18:05.525 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.520+0000 7fc771600640 1 -- 172.21.15.137:0/1785177904 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c034680 con 0x55ed1bfbf810 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.520+0000 7fc785d6b740 1 -- 172.21.15.137:0/1785177904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 msgr2=0x55ed1bfbfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.520+0000 7fc77b400640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7600a2880 0x7fc7600a4d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc76c044000 tx=0x7fc76c049000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.520+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 0x55ed1bfbfc30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc76c032380 tx=0x7fc76c0098e0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 -- 172.21.15.137:0/1785177904 shutdown_connections 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfc7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1785177904 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbf810 0x55ed1bfbfc30 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 -- 172.21.15.137:0/1785177904 >> 172.21.15.137:0/1785177904 conn(0x55ed1bfbac00 msgr2=0x55ed1bfbb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 -- 172.21.15.137:0/1785177904 shutdown_connections 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 -- 172.21.15.137:0/1785177904 wait complete. 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 -- start start 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.521+0000 7fc785d6b740 1 Processor -- start 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfd0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.526 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed1bfc83b0 con 0x55ed1bfc09e0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed1bee00d0 con 0x55ed1bfc13a0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed1bfc8530 con 0x55ed1bfbf810 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc785d6b740 10 client.0 _ll_register_callbacks cb 0x55ed1bee4e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38054/0 (socket says 172.21.15.137:38054) 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 -- 172.21.15.137:0/1017812707 learned_addr learned my addr 172.21.15.137:0/1017812707 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc781000640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfd0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77be00640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 -- 172.21.15.137:0/1017812707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 msgr2=0x55ed1bfc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfc9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 -- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc13a0 msgr2=0x55ed1bfd0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfd0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc77b400640 1 -- 172.21.15.137:0/1017812707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed1bc383f0 con 0x55ed1bfc09e0 2024-03-26T14:18:05.527 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.522+0000 7fc781000640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfd0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc77b400640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc9fa0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005e30 tx=0x7fc76c02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c93fb7b3c9f653 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc76a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc76a000640 1 -- 172.21.15.137:0/1017812707 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c050040 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc76a000640 1 -- 172.21.15.137:0/1017812707 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc76c034980 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc76c0328b0 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc76a000640 1 -- 172.21.15.137:0/1017812707 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc76c060050 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc785d6b740 10 client.18942 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.523+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ed1bd027d0 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc76a000640 1 -- 172.21.15.137:0/1017812707 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc76c0328b0 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc76a000640 1 -- 172.21.15.137:0/1017812707 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc76c053070 con 0x55ed1bfc09e0 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc785d6b740 10 client.18942 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc785d6b740 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc785d6b740 20 client.18942 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfd5140 0x55ed1bfd7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc785d6b740 4 client.18942 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed1bf31800 con 0x55ed1bfd5140 2024-03-26T14:18:05.528 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.524+0000 7fc77be00640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfd5140 0x55ed1bfd7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.525+0000 7fc77be00640 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfd5140 0x55ed1bfd7530 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1961931044a0e1a server_cookie=2132f4d4ac042847 in_seq=0 out_seq=0 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.525+0000 7fc76a000640 10 client.18942 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc76a000640 1 -- 172.21.15.137:0/1017812707 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55ed1bf31800 con 0x55ed1bfd5140 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc76a000640 10 client.18942 handle_command_reply: tid=0 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 client.18942 shutdown 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 msgr2=0x55ed1bfc9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc9fa0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005e30 tx=0x7fc76c02c770 comp rx=0 tx=0).stop 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 shutdown_connections 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfd5140 0x55ed1bfd7530 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc13a0 0x55ed1bfd0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfc9fa0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 --2- 172.21.15.137:0/1017812707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc76a000640 0 client.18942 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.526+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 >> 172.21.15.137:0/1017812707 conn(0x55ed1bfbac00 msgr2=0x55ed1bfb9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.530 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.527+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 shutdown_connections 2024-03-26T14:18:05.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.527+0000 7fc785d6b740 1 -- 172.21.15.137:0/1017812707 wait complete. 2024-03-26T14:18:05.531 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.527+0000 7fc785d6b740 20 client.18942 trim_cache size 0 max 16384 2024-03-26T14:18:05.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 Processor -- start 2024-03-26T14:18:05.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 -- start start 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbac00 0x55ed1bfb9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfdbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1ae950 con 0x55ed1bfc09e0 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1ac450 con 0x55ed1bfbf810 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.540+0000 7fc785d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1aa4d0 con 0x55ed1bfbac00 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfdbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfdbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38058/0 (socket says 172.21.15.137:38058) 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 -- 172.21.15.137:0/3449902527 learned_addr learned my addr 172.21.15.137:0/3449902527 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc77b400640 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 -- 172.21.15.137:0/3449902527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbac00 msgr2=0x55ed1bfb9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbac00 0x55ed1bfb9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 -- 172.21.15.137:0/3449902527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfbf810 msgr2=0x55ed1bfd0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.543 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 -- 172.21.15.137:0/3449902527 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed1bfbb670 con 0x55ed1bfc09e0 2024-03-26T14:18:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc781000640 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfdbc40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc774004770 tx=0x7fc77409a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a908b2be50184af server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.541+0000 7fc770c00640 1 -- 172.21.15.137:0/3449902527 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774020040 con 0x55ed1bfc09e0 2024-03-26T14:18:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc770c00640 1 -- 172.21.15.137:0/3449902527 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc77409e030 con 0x55ed1bfc09e0 2024-03-26T14:18:05.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc770c00640 1 -- 172.21.15.137:0/3449902527 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774097420 con 0x55ed1bfc09e0 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 -- 172.21.15.137:0/3449902527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 msgr2=0x55ed1bfdbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfdbc40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc774004770 tx=0x7fc77409a820 comp rx=0 tx=0).stop 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 -- 172.21.15.137:0/3449902527 shutdown_connections 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfdbc40 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 --2- 172.21.15.137:0/3449902527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbac00 0x55ed1bfb9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 -- 172.21.15.137:0/3449902527 >> 172.21.15.137:0/3449902527 conn(0x55ed1bfc8c70 msgr2=0x55ed1bfb9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 -- 172.21.15.137:0/3449902527 shutdown_connections 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.542+0000 7fc785d6b740 1 -- 172.21.15.137:0/3449902527 wait complete. 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 -- start start 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 Processor -- start 2024-03-26T14:18:05.547 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 0x55ed1bfd0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfcd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1aaad0 con 0x55ed1bfbac00 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1abdf0 con 0x55ed1bfc09e0 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c1ab130 con 0x55ed1bfbf810 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc785d6b740 10 client.0 _ll_register_callbacks cb 0x7fc76400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.543+0000 7fc77be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 0x55ed1bfd0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 0x55ed1bfd0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38060/0 (socket says 172.21.15.137:38060) 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 -- 172.21.15.137:0/113279387 learned_addr learned my addr 172.21.15.137:0/113279387 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc781000640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfcd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 -- 172.21.15.137:0/113279387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 msgr2=0x55ed1bfd1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 -- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 msgr2=0x55ed1bfcd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfcd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 -- 172.21.15.137:0/113279387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed1bfcde40 con 0x55ed1bfbac00 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc781000640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfcd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:05.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc77be00640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 0x55ed1bfd0ac0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc764005f20 tx=0x7fc764012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555a5022631e2f9f server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc76a000640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc76a000640 1 -- 172.21.15.137:0/113279387 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76401f070 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc76a000640 1 -- 172.21.15.137:0/113279387 <== mon.0 v2:172.21.15.137:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc764004220 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.544+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc77409e030 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc76a000640 1 -- 172.21.15.137:0/113279387 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc76402a050 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 10 client.18948 fetch_fsmap learned FSMap version 24 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ed1c031700 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc76a000640 1 -- 172.21.15.137:0/113279387 <== mon.0 v2:172.21.15.137:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc77409e030 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc76a000640 1 -- 172.21.15.137:0/113279387 <== mon.0 v2:172.21.15.137:3300/0 5 ==== fsmap(e 24) v1 ==== 7926+0+0 (secure 0 0 0) 0x7fc764003680 con 0x55ed1bfbac00 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 10 client.18948 fetch_fsmap finished waiting for FSMap version 24 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 10 client.18948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 20 client.18948 populate_metadata read hostname 'smithi137' 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfc13a0 0x55ed1c0340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 4 client.18948 mds_command: new command op to 34496 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-03-26T14:18:05.549 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.545+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 --> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed1bfc9850 con 0x55ed1bfc13a0 2024-03-26T14:18:05.550 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.546+0000 7fc77b400640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfc13a0 0x55ed1c0340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.546+0000 7fc77b400640 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfc13a0 0x55ed1c0340d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c8106cc43ec6d server_cookie=4e6556460472193f in_seq=0 out_seq=0 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.546+0000 7fc76a000640 10 client.18948 ms_handle_connect on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.546+0000 7fc76a000640 1 -- 172.21.15.137:0/113279387 <== mds.0 v2:172.21.15.159:6838/2964052982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed1bfc9850 con 0x55ed1bfc13a0 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.546+0000 7fc76a000640 10 client.18948 handle_command_reply: tid=0 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 client.18948 shutdown 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 msgr2=0x55ed1bfd0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 0x55ed1bfd0ac0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc764005f20 tx=0x7fc764012820 comp rx=0 tx=0).stop 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 shutdown_connections 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:6838/2964052982,v1:172.21.15.159:6839/2964052982] conn(0x55ed1bfc13a0 0x55ed1c0340d0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1bfc09e0 0x55ed1bfcd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed1bfbf810 0x55ed1bfd1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc76a000640 0 client.18948 ms_handle_reset on v2:172.21.15.159:6838/2964052982 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 --2- 172.21.15.137:0/113279387 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55ed1bfbac00 0x55ed1bfd0ac0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 >> 172.21.15.137:0/113279387 conn(0x55ed1bfc8c70 msgr2=0x55ed1bfb9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 shutdown_connections 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 1 -- 172.21.15.137:0/113279387 wait complete. 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.547+0000 7fc785d6b740 20 client.18948 trim_cache size 0 max 16384 2024-03-26T14:18:05.552 INFO:teuthology.orchestra.run.smithi137.stdout:[] 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7600a2880 msgr2=0x7fc7600a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7600a2880 0x7fc7600a4d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc76c044000 tx=0x7fc76c049000 comp rx=0 tx=0).stop 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 msgr2=0x7fc77c1a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 0x7fc77c1a10a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc77400d750 tx=0x7fc77400dc10 comp rx=0 tx=0).stop 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 shutdown_connections 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc7600a2880 0x7fc7600a4d40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc77c104ad0 0x7fc77c19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc77c105490 0x7fc77c1a10a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 --2- 172.21.15.137:0/3426995949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc77c1038e0 0x7fc77c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:05.553 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 >> 172.21.15.137:0/3426995949 conn(0x7fc77c0fee50 msgr2=0x7fc77c107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:05.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 shutdown_connections 2024-03-26T14:18:05.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:05.548+0000 7fc783000640 1 -- 172.21.15.137:0/3426995949 wait complete. 2024-03-26T14:18:06.566 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-03-26T14:18:06.567 INFO:tasks.fwd_scrub:done joining 2024-03-26T14:18:06.567 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-26T14:18:06.578 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:18:06.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.793+0000 7f3741600640 1 -- 172.21.15.137:0/1858100199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c1048a0 msgr2=0x7f373c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:06.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.793+0000 7f3741600640 1 --2- 172.21.15.137:0/1858100199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c1048a0 0x7f373c106c90 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3724009900 tx=0x7f372402bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:06.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 -- 172.21.15.137:0/1858100199 shutdown_connections 2024-03-26T14:18:06.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 --2- 172.21.15.137:0/1858100199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1071d0 0x7f373c109620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:06.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 --2- 172.21.15.137:0/1858100199 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c1048a0 0x7f373c106c90 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:06.795 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 --2- 172.21.15.137:0/1858100199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c101f70 0x7f373c104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:06.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 -- 172.21.15.137:0/1858100199 >> 172.21.15.137:0/1858100199 conn(0x7f373c0fb980 msgr2=0x7f373c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:06.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 -- 172.21.15.137:0/1858100199 shutdown_connections 2024-03-26T14:18:06.796 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.794+0000 7f3741600640 1 -- 172.21.15.137:0/1858100199 wait complete. 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 Processor -- start 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 -- start start 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 0x7f373c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1048a0 0x7f373c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c1071d0 0x7f373c1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c10d080 con 0x7f373c101f70 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c10cf00 con 0x7f373c1048a0 2024-03-26T14:18:06.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f3741600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c10d200 con 0x7f373c1071d0 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f373be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 0x7f373c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f373be00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 0x7f373c199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38080/0 (socket says 172.21.15.137:38080) 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.795+0000 7f373be00640 1 -- 172.21.15.137:0/1105507871 learned_addr learned my addr 172.21.15.137:0/1105507871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3733400640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1048a0 0x7f373c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f373be00640 1 -- 172.21.15.137:0/1105507871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c1071d0 msgr2=0x7f373c1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f373be00640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c1071d0 0x7f373c1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f373be00640 1 -- 172.21.15.137:0/1105507871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1048a0 msgr2=0x7f373c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f373be00640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1048a0 0x7f373c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f373be00640 1 -- 172.21.15.137:0/1105507871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c1a15a0 con 0x7f373c101f70 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3733400640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1048a0 0x7f373c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f373be00640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 0x7f373c199980 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f372c00d770 tx=0x7f372c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c017070 con 0x7f373c101f70 2024-03-26T14:18:06.799 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f372c0054d0 con 0x7f373c101f70 2024-03-26T14:18:06.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c1a1890 con 0x7f373c101f70 2024-03-26T14:18:06.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c004cf0 con 0x7f373c101f70 2024-03-26T14:18:06.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.796+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c1a1d90 con 0x7f373c101f70 2024-03-26T14:18:06.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.797+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c108d20 con 0x7f373c101f70 2024-03-26T14:18:06.800 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.799+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f372c004e90 con 0x7f373c101f70 2024-03-26T14:18:06.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.801+0000 7f3739600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37200a2800 0x7f37200a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:06.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.801+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f372c0ce980 con 0x7f373c101f70 2024-03-26T14:18:06.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.801+0000 7f3733400640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37200a2800 0x7f37200a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:06.802 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.802+0000 7f3733400640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37200a2800 0x7f37200a4cc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3724036040 tx=0x7f3724005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:06.806 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:06.805+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f372c09b610 con 0x7f373c101f70 2024-03-26T14:18:07.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.006+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f373c062340 con 0x7f373c101f70 2024-03-26T14:18:07.010 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.009+0000 7f3739600640 1 -- 172.21.15.137:0/1105507871 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f372c09af10 con 0x7f373c101f70 2024-03-26T14:18:07.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:18:07.011 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:18:07.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37200a2800 msgr2=0x7f37200a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37200a2800 0x7f37200a4cc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3724036040 tx=0x7f3724005fd0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 msgr2=0x7f373c199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.013 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 0x7f373c199980 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f372c00d770 tx=0x7f372c00dc30 comp rx=0 tx=0).stop 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 shutdown_connections 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f37200a2800 0x7f37200a4cc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f373c1048a0 0x7f373c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f373c1071d0 0x7f373c1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 --2- 172.21.15.137:0/1105507871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f373c101f70 0x7f373c199980 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 >> 172.21.15.137:0/1105507871 conn(0x7f373c0fb980 msgr2=0x7f373c101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 shutdown_connections 2024-03-26T14:18:07.014 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.012+0000 7f3741600640 1 -- 172.21.15.137:0/1105507871 wait complete. 2024-03-26T14:18:07.029 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:06 smithi137 ceph-mon[31539]: pgmap v930: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:18:07.033 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:18:07.033 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:18:07.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:06 smithi179 ceph-mon[35472]: pgmap v930: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:18:07.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:06 smithi159 ceph-mon[35501]: pgmap v930: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 -- 172.21.15.137:0/2391848466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32640756e0 msgr2=0x7f3264073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 --2- 172.21.15.137:0/2391848466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32640756e0 0x7f3264073b30 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f3254009930 tx=0x7f325402bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 -- 172.21.15.137:0/2391848466 shutdown_connections 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 --2- 172.21.15.137:0/2391848466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f326410db70 0x7f326410ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 --2- 172.21.15.137:0/2391848466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3264074100 0x7f3264074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 --2- 172.21.15.137:0/2391848466 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f32640756e0 0x7f3264073b30 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 -- 172.21.15.137:0/2391848466 >> 172.21.15.137:0/2391848466 conn(0x7f32640fcb10 msgr2=0x7f32640fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.249+0000 7f3269800640 1 -- 172.21.15.137:0/2391848466 shutdown_connections 2024-03-26T14:18:07.251 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.250+0000 7f3269800640 1 -- 172.21.15.137:0/2391848466 wait complete. 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.250+0000 7f3269800640 1 Processor -- start 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 -- start start 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 0x7f326419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32640756e0 0x7f326419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326410db70 0x7f32641a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264113710 con 0x7f3264074100 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264113590 con 0x7f32640756e0 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3269800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264113890 con 0x7f326410db70 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3263e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 0x7f326419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3263e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 0x7f326419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38098/0 (socket says 172.21.15.137:38098) 2024-03-26T14:18:07.253 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.251+0000 7f3263e00640 1 -- 172.21.15.137:0/2657667264 learned_addr learned my addr 172.21.15.137:0/2657667264 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3263e00640 1 -- 172.21.15.137:0/2657667264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326410db70 msgr2=0x7f32641a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3263e00640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326410db70 0x7f32641a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3263e00640 1 -- 172.21.15.137:0/2657667264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32640756e0 msgr2=0x7f326419e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3263e00640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32640756e0 0x7f326419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3263e00640 1 -- 172.21.15.137:0/2657667264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32641a5a50 con 0x7f3264074100 2024-03-26T14:18:07.254 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3263e00640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 0x7f326419de30 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3254005ab0 tx=0x7f32540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:07.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254043020 con 0x7f3264074100 2024-03-26T14:18:07.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32540054c0 con 0x7f3264074100 2024-03-26T14:18:07.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3254003d10 con 0x7f3264074100 2024-03-26T14:18:07.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.252+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32641a5ce0 con 0x7f3264074100 2024-03-26T14:18:07.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.253+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32641a6220 con 0x7f3264074100 2024-03-26T14:18:07.255 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.254+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264103d40 con 0x7f3264074100 2024-03-26T14:18:07.258 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.257+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3254046020 con 0x7f3264074100 2024-03-26T14:18:07.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.258+0000 7f3260c00640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32480a2880 0x7f32480a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.259+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f32540f1580 con 0x7f3264074100 2024-03-26T14:18:07.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.259+0000 7f3263400640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32480a2880 0x7f32480a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:07.260 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.259+0000 7f3263400640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32480a2880 0x7f32480a4d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3250005fd0 tx=0x7f3250005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:07.262 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.261+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f325404d020 con 0x7f3264074100 2024-03-26T14:18:07.470 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.469+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f326402d290 con 0x7f3264074100 2024-03-26T14:18:07.471 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.470+0000 7f3260c00640 1 -- 172.21.15.137:0/2657667264 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32540be210 con 0x7f3264074100 2024-03-26T14:18:07.471 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:18:07.474 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32480a2880 msgr2=0x7f32480a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32480a2880 0x7f32480a4d40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3250005fd0 tx=0x7f3250005850 comp rx=0 tx=0).stop 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 msgr2=0x7f326419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 0x7f326419de30 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3254005ab0 tx=0x7f32540042f0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 shutdown_connections 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f32480a2880 0x7f32480a4d40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32640756e0 0x7f326419e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f326410db70 0x7f32641a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 --2- 172.21.15.137:0/2657667264 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3264074100 0x7f326419de30 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 >> 172.21.15.137:0/2657667264 conn(0x7f32640fcb10 msgr2=0x7f3264100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 shutdown_connections 2024-03-26T14:18:07.475 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.473+0000 7f3269800640 1 -- 172.21.15.137:0/2657667264 wait complete. 2024-03-26T14:18:07.493 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:18:07.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 -- 172.21.15.137:0/2960631555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e8102150 msgr2=0x7f29e81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 --2- 172.21.15.137:0/2960631555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e8102150 0x7f29e81089e0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009900 tx=0x7f29e002bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:07.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 -- 172.21.15.137:0/2960631555 shutdown_connections 2024-03-26T14:18:07.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 --2- 172.21.15.137:0/2960631555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29e8101790 0x7f29e8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 --2- 172.21.15.137:0/2960631555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e8102150 0x7f29e81089e0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.708 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 --2- 172.21.15.137:0/2960631555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e81005a0 0x7f29e81009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 -- 172.21.15.137:0/2960631555 >> 172.21.15.137:0/2960631555 conn(0x7f29e80fbb30 msgr2=0x7f29e80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:07.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 -- 172.21.15.137:0/2960631555 shutdown_connections 2024-03-26T14:18:07.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.707+0000 7f29f0600640 1 -- 172.21.15.137:0/2960631555 wait complete. 2024-03-26T14:18:07.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.708+0000 7f29f0600640 1 Processor -- start 2024-03-26T14:18:07.709 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.708+0000 7f29f0600640 1 -- start start 2024-03-26T14:18:07.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29f0600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 0x7f29e8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29f0600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29e8101790 0x7f29e819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.711 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29f0600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e8102150 0x7f29e81a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29f0600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e810c0b0 con 0x7f29e81005a0 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29f0600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e810bf30 con 0x7f29e8101790 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29f0600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e810c230 con 0x7f29e8102150 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29edc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 0x7f29e8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29edc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 0x7f29e8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:38116/0 (socket says 172.21.15.137:38116) 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29edc00640 1 -- 172.21.15.137:0/3088259153 learned_addr learned my addr 172.21.15.137:0/3088259153 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29ed200640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29e8101790 0x7f29e819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.709+0000 7f29ee600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e8102150 0x7f29e81a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29edc00640 1 -- 172.21.15.137:0/3088259153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e8102150 msgr2=0x7f29e81a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29edc00640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e8102150 0x7f29e81a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29edc00640 1 -- 172.21.15.137:0/3088259153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29e8101790 msgr2=0x7f29e819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29edc00640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29e8101790 0x7f29e819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.712 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29edc00640 1 -- 172.21.15.137:0/3088259153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e81a1700 con 0x7f29e81005a0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29edc00640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 0x7f29e8199ae0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00ccb0 tx=0x7f29dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc003990 con 0x7f29e81005a0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29dc004020 con 0x7f29e81005a0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81a19f0 con 0x7f29e81005a0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc00f6b0 con 0x7f29e81005a0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.710+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e81a1f40 con 0x7f29e81005a0 2024-03-26T14:18:07.713 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.712+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e8068490 con 0x7f29e81005a0 2024-03-26T14:18:07.714 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.714+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f29dc003b30 con 0x7f29e81005a0 2024-03-26T14:18:07.716 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.716+0000 7f29e6000640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29d00a2800 0x7f29d00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:07.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.716+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f29dc0cf320 con 0x7f29e81005a0 2024-03-26T14:18:07.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.716+0000 7f29ed200640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29d00a2800 0x7f29d00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:07.717 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.717+0000 7f29ed200640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29d00a2800 0x7f29d00a4cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005fd0 tx=0x7f29d8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:07.721 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.721+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f29dc014030 con 0x7f29e81005a0 2024-03-26T14:18:07.919 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.918+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29e81a29c0 con 0x7f29e81005a0 2024-03-26T14:18:07.920 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.919+0000 7f29e6000640 1 -- 172.21.15.137:0/3088259153 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29dc09bfb0 con 0x7f29e81005a0 2024-03-26T14:18:07.920 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:18:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29d00a2800 msgr2=0x7f29d00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29d00a2800 0x7f29d00a4cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005fd0 tx=0x7f29d8005850 comp rx=0 tx=0).stop 2024-03-26T14:18:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 msgr2=0x7f29e8199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 0x7f29e8199ae0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00ccb0 tx=0x7f29dc00a620 comp rx=0 tx=0).stop 2024-03-26T14:18:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 shutdown_connections 2024-03-26T14:18:07.923 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f29d00a2800 0x7f29d00a4cc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29e8101790 0x7f29e819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29e8102150 0x7f29e81a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 --2- 172.21.15.137:0/3088259153 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f29e81005a0 0x7f29e8199ae0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 >> 172.21.15.137:0/3088259153 conn(0x7f29e80fbb30 msgr2=0x7f29e8068800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 shutdown_connections 2024-03-26T14:18:07.924 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:07.922+0000 7f29f0600640 1 -- 172.21.15.137:0/3088259153 wait complete. 2024-03-26T14:18:07.942 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:18:08.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 -- 172.21.15.137:0/1524397468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 msgr2=0x7f7d9810bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 --2- 172.21.15.137:0/1524397468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d9810bd20 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009ca0 tx=0x7f7d9002c0f0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.153 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 -- 172.21.15.137:0/1524397468 shutdown_connections 2024-03-26T14:18:08.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 --2- 172.21.15.137:0/1524397468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d98104ad0 0x7f7d98104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 --2- 172.21.15.137:0/1524397468 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d9810bd20 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 --2- 172.21.15.137:0/1524397468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d981038e0 0x7f7d98103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 -- 172.21.15.137:0/1524397468 >> 172.21.15.137:0/1524397468 conn(0x7f7d980fee50 msgr2=0x7f7d98101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:08.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 -- 172.21.15.137:0/1524397468 shutdown_connections 2024-03-26T14:18:08.154 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.152+0000 7f7d9d200640 1 -- 172.21.15.137:0/1524397468 wait complete. 2024-03-26T14:18:08.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.153+0000 7f7d9d200640 1 Processor -- start 2024-03-26T14:18:08.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.153+0000 7f7d9d200640 1 -- start start 2024-03-26T14:18:08.156 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d9d200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d981038e0 0x7f7d98199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d9d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d98104ad0 0x7f7d98199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d9d200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d981a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d9d200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9810f3f0 con 0x7f7d98105490 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d9d200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9810f270 con 0x7f7d98104ad0 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d9d200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9810f570 con 0x7f7d981038e0 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d981a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d981a0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47474/0 (socket says 172.21.15.137:47474) 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 -- 172.21.15.137:0/3649134243 learned_addr learned my addr 172.21.15.137:0/3649134243 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d8d400640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d98104ad0 0x7f7d98199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d95e00640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d981038e0 0x7f7d98199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 -- 172.21.15.137:0/3649134243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d981038e0 msgr2=0x7f7d98199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d981038e0 0x7f7d98199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 -- 172.21.15.137:0/3649134243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d98104ad0 msgr2=0x7f7d98199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.157 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d98104ad0 0x7f7d98199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.154+0000 7f7d96800640 1 -- 172.21.15.137:0/3649134243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d981a1660 con 0x7f7d98105490 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.155+0000 7f7d96800640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d981a0ff0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7d9002c700 tx=0x7f7d900328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.155+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90041020 con 0x7f7d98105490 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.155+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d981a18f0 con 0x7f7d98105490 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.155+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7d90003e20 con 0x7f7d98105490 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.155+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90039040 con 0x7f7d98105490 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.155+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d981a1de0 con 0x7f7d98105490 2024-03-26T14:18:08.158 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.156+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9810ac00 con 0x7f7d98105490 2024-03-26T14:18:08.159 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.158+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7d90044020 con 0x7f7d98105490 2024-03-26T14:18:08.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.159+0000 7f7d8f400640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.160+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f7d900f1630 con 0x7f7d98105490 2024-03-26T14:18:08.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.160+0000 7f7d95e00640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.161 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.160+0000 7f7d95e00640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d880058c0 tx=0x7f7d88005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:08.163 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.162+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d9004b020 con 0x7f7d98105490 2024-03-26T14:18:08.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:07 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1105507871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:08.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:07 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2657667264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:18:08.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:07 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1105507871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:08.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:07 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2657667264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:18:08.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:07 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1105507871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:08.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:07 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2657667264' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:18:08.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.362+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d9802d200 con 0x7f7d98105490 2024-03-26T14:18:08.363 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.362+0000 7f7d8f400640 1 -- 172.21.15.137:0/3649134243 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d900333a0 con 0x7f7d98105490 2024-03-26T14:18:08.364 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:18:08.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d7c0a2800 msgr2=0x7f7d7c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.366 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7d880058c0 tx=0x7f7d88005850 comp rx=0 tx=0).stop 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 msgr2=0x7f7d981a0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d981a0ff0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7d9002c700 tx=0x7f7d900328a0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 shutdown_connections 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7d7c0a2800 0x7f7d7c0a4cc0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d98104ad0 0x7f7d98199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7d98105490 0x7f7d981a0ff0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 --2- 172.21.15.137:0/3649134243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d981038e0 0x7f7d98199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 >> 172.21.15.137:0/3649134243 conn(0x7f7d980fee50 msgr2=0x7f7d98107dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 shutdown_connections 2024-03-26T14:18:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.365+0000 7f7d9d200640 1 -- 172.21.15.137:0/3649134243 wait complete. 2024-03-26T14:18:08.386 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.596+0000 7fdaccc00640 1 -- 172.21.15.137:0/668089552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8076060 msgr2=0x7fdac8076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.596+0000 7fdaccc00640 1 --2- 172.21.15.137:0/668089552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8076060 0x7fdac8076480 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009900 tx=0x7fdab802bc50 comp rx=0 tx=0).stop 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.596+0000 7fdaccc00640 1 -- 172.21.15.137:0/668089552 shutdown_connections 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.596+0000 7fdaccc00640 1 --2- 172.21.15.137:0/668089552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac8077c40 0x7fdac80784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.596+0000 7fdaccc00640 1 --2- 172.21.15.137:0/668089552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdac80772a0 0x7fdac8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.596+0000 7fdaccc00640 1 --2- 172.21.15.137:0/668089552 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8076060 0x7fdac8076480 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.597+0000 7fdaccc00640 1 -- 172.21.15.137:0/668089552 >> 172.21.15.137:0/668089552 conn(0x7fdac80fee50 msgr2=0x7fdac8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.597+0000 7fdaccc00640 1 -- 172.21.15.137:0/668089552 shutdown_connections 2024-03-26T14:18:08.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.597+0000 7fdaccc00640 1 -- 172.21.15.137:0/668089552 wait complete. 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.597+0000 7fdaccc00640 1 Processor -- start 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 -- start start 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdac8076060 0x7fdac819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac80772a0 0x7fdac819e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 0x7fdac81a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8113860 con 0x7fdac8077c40 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81136e0 con 0x7fdac80772a0 2024-03-26T14:18:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdaccc00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81139e0 con 0x7fdac8076060 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdac6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 0x7fdac81a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdac6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 0x7fdac81a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47488/0 (socket says 172.21.15.137:47488) 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdac6800640 1 -- 172.21.15.137:0/1322035813 learned_addr learned my addr 172.21.15.137:0/1322035813 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.598+0000 7fdabd400640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac80772a0 0x7fdac819e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac5e00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdac8076060 0x7fdac819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac6800640 1 -- 172.21.15.137:0/1322035813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdac8076060 msgr2=0x7fdac819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac6800640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdac8076060 0x7fdac819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac6800640 1 -- 172.21.15.137:0/1322035813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac80772a0 msgr2=0x7fdac819e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac6800640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac80772a0 0x7fdac819e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac6800640 1 -- 172.21.15.137:0/1322035813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac81a5b60 con 0x7fdac8077c40 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdac6800640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 0x7fdac81a54f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fdac000eaa0 tx=0x7fdac000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:08.602 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac000cdb0 con 0x7fdac8077c40 2024-03-26T14:18:08.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdac0005020 con 0x7fdac8077c40 2024-03-26T14:18:08.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac81a5e50 con 0x7fdac8077c40 2024-03-26T14:18:08.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.599+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac00116d0 con 0x7fdac8077c40 2024-03-26T14:18:08.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.600+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac81a62c0 con 0x7fdac8077c40 2024-03-26T14:18:08.603 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.601+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8110250 con 0x7fdac8077c40 2024-03-26T14:18:08.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.603+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdac00051d0 con 0x7fdac8077c40 2024-03-26T14:18:08.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.604+0000 7fdabf400640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:08.605 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.604+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdac00cf050 con 0x7fdac8077c40 2024-03-26T14:18:08.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.604+0000 7fdac5e00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:08.606 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.605+0000 7fdac5e00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdab8006010 tx=0x7fdab8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:08.609 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.608+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdac0016030 con 0x7fdac8077c40 2024-03-26T14:18:08.810 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.809+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdac8062340 con 0x7fdac8077c40 2024-03-26T14:18:08.811 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.810+0000 7fdabf400640 1 -- 172.21.15.137:0/1322035813 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdac009bce0 con 0x7fdac8077c40 2024-03-26T14:18:08.811 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:18:08.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaac0a2800 msgr2=0x7fdaac0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdab8006010 tx=0x7fdab8000bc0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.814 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 msgr2=0x7fdac81a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 0x7fdac81a54f0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fdac000eaa0 tx=0x7fdac000ef60 comp rx=0 tx=0).stop 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 shutdown_connections 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdaac0a2800 0x7fdaac0a4cc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac80772a0 0x7fdac819e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdac8077c40 0x7fdac81a54f0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 --2- 172.21.15.137:0/1322035813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdac8076060 0x7fdac819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 >> 172.21.15.137:0/1322035813 conn(0x7fdac80fee50 msgr2=0x7fdac8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 shutdown_connections 2024-03-26T14:18:08.815 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:08.813+0000 7fdaccc00640 1 -- 172.21.15.137:0/1322035813 wait complete. 2024-03-26T14:18:08.833 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.034+0000 7fbb92e00640 1 -- 172.21.15.137:0/2324899132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 msgr2=0x7fbb8c110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.034+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2324899132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c110120 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009ca0 tx=0x7fbb8402c100 comp rx=0 tx=0).stop 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 -- 172.21.15.137:0/2324899132 shutdown_connections 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2324899132 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c110120 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2324899132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb8c108ed0 0x7fbb8c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2324899132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c107ce0 0x7fbb8c108100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 -- 172.21.15.137:0/2324899132 >> 172.21.15.137:0/2324899132 conn(0x7fbb8c076cb0 msgr2=0x7fbb8c0790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:09.036 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 -- 172.21.15.137:0/2324899132 shutdown_connections 2024-03-26T14:18:09.037 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.035+0000 7fbb92e00640 1 -- 172.21.15.137:0/2324899132 wait complete. 2024-03-26T14:18:09.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 Processor -- start 2024-03-26T14:18:09.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 -- start start 2024-03-26T14:18:09.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb8c107ce0 0x7fbb8c19de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:09.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c108ed0 0x7fbb8c19e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:09.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c1a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:09.039 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c113810 con 0x7fbb8c109890 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c113690 con 0x7fbb8c108ed0 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb92e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c113990 con 0x7fbb8c107ce0 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb90e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c1a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb90e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c1a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:47492/0 (socket says 172.21.15.137:47492) 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb90e00640 1 -- 172.21.15.137:0/2918105630 learned_addr learned my addr 172.21.15.137:0/2918105630 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.036+0000 7fbb8b400640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c108ed0 0x7fbb8c19e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb8be00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb8c107ce0 0x7fbb8c19de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb90e00640 1 -- 172.21.15.137:0/2918105630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb8c107ce0 msgr2=0x7fbb8c19de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb90e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb8c107ce0 0x7fbb8c19de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb90e00640 1 -- 172.21.15.137:0/2918105630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c108ed0 msgr2=0x7fbb8c19e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb90e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c108ed0 0x7fbb8c19e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb90e00640 1 -- 172.21.15.137:0/2918105630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c1a5ab0 con 0x7fbb8c109890 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb8b400640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c108ed0 0x7fbb8c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:09.040 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb90e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c1a5440 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fbb84032520 tx=0x7fbb84035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:09.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84039040 con 0x7fbb8c109890 2024-03-26T14:18:09.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb84044070 con 0x7fbb8c109890 2024-03-26T14:18:09.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c1a5d40 con 0x7fbb8c109890 2024-03-26T14:18:09.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb840043c0 con 0x7fbb8c109890 2024-03-26T14:18:09.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.037+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c1a6220 con 0x7fbb8c109890 2024-03-26T14:18:09.041 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.039+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c10f000 con 0x7fbb8c109890 2024-03-26T14:18:09.042 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.040+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbb84004560 con 0x7fbb8c109890 2024-03-26T14:18:09.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.042+0000 7fbb88c00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb700a2880 0x7fbb700a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:09.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.042+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fbb840f1590 con 0x7fbb8c109890 2024-03-26T14:18:09.043 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.042+0000 7fbb8be00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb700a2880 0x7fbb700a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:09.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.043+0000 7fbb8be00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb700a2880 0x7fbb700a4d40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c108d30 tx=0x7fbb7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.046+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbb840be2e0 con 0x7fbb8c109890 2024-03-26T14:18:09.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:08 smithi179 ceph-mon[35472]: pgmap v931: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-26T14:18:09.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3088259153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:18:09.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3649134243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:18:09.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:08 smithi159 ceph-mon[35501]: pgmap v931: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-26T14:18:09.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3088259153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:18:09.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3649134243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:18:09.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:08 smithi137 ceph-mon[31539]: pgmap v931: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-03-26T14:18:09.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3088259153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:18:09.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3649134243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:18:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.295+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fbb8c062340 con 0x7fbb8c109890 2024-03-26T14:18:09.324 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.323+0000 7fbb88c00640 1 -- 172.21.15.137:0/2918105630 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+739838 (secure 0 0 0) 0x7fbb840bdbe0 con 0x7fbb8c109890 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "epoch": 20, 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "active_gid": 14152, 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "active_name": "x", 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "active_addrs": { 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.338 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:6800", 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3167395503 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v1", 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:6801", 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 3167395503 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "active_addr": "172.21.15.137:6801/3167395503", 2024-03-26T14:18:09.339 INFO:teuthology.orchestra.run.smithi137.stdout: "active_change": "2024-03-26T13:49:52.910706+0000", 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "active_mgr_features": 4540701547738038271, 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "available": true, 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "standbys": [ 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "gid": 14223, 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "y", 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr_features": 4540701547738038271, 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: "available_modules": [ 2024-03-26T14:18:09.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "alerts", 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.341 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.342 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.343 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.344 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.345 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.346 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.347 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_destination": { 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_destination", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email address to send alerts to", 2024-03-26T14:18:09.348 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_from_name": { 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_from_name", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "Ceph", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email From: name", 2024-03-26T14:18:09.349 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_host": { 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_host", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP server", 2024-03-26T14:18:09.350 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_password": { 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_password", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Password to authenticate with", 2024-03-26T14:18:09.351 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_port": { 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_port", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "465", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP port", 2024-03-26T14:18:09.352 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_sender": { 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_sender", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP envelope sender", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.353 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_ssl": { 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_ssl", 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.354 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_user": { 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_user", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.355 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "User to authenticate as", 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "balancer", 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.356 INFO:teuthology.orchestra.run.smithi137.stdout: "active": { 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "active", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_time": { 2024-03-26T14:18:09.357 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_time", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0000", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.358 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_weekday": { 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_weekday", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.359 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_max_iterations": { 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_max_iterations", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "25", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "250", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_metrics": { 2024-03-26T14:18:09.360 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_metrics", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_step": { 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_step", 2024-03-26T14:18:09.361 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.5", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0.001", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "0.999", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "aggressiveness of optimization", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "end_time": { 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_time", 2024-03-26T14:18:09.362 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2359", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "end_weekday": { 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_weekday", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.363 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.364 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.365 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.366 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.367 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.368 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "min_score": { 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_score", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.369 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-03-26T14:18:09.370 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "upmap", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "crush-compat", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "none", 2024-03-26T14:18:09.371 INFO:teuthology.orchestra.run.smithi137.stdout: "read", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap-read" 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Balancer mode", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_ids": { 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_ids", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.372 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.373 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.374 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_deviation": { 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_deviation", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.375 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_optimizations": { 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_optimizations", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.376 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm", 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_down_multiplier": { 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_down_multiplier", 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.377 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_refresh_rate": { 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_refresh_rate", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.378 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "20", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_starting_port": { 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_starting_port", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.379 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4721", 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_ptrace": { 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_ptrace", 2024-03-26T14:18:09.380 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_interval": { 2024-03-26T14:18:09.381 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_interval", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_memory_target_ratio": { 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T14:18:09.382 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.7", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm_log_destination": { 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm_log_destination", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.383 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "file", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "file,syslog", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "syslog" 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.384 INFO:teuthology.orchestra.run.smithi137.stdout: "cgroups_split": { 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cgroups_split", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "config_checks_enabled": { 2024-03-26T14:18:09.385 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_checks_enabled", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "config_dashboard": { 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_dashboard", 2024-03-26T14:18:09.386 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_alertmanager": { 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_alertmanager", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.387 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_base": { 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_base", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.388 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Container image name, without the tag", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_elasticsearch": { 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_elasticsearch", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.389 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "elasticsearch container image", 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_grafana": { 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_grafana", 2024-03-26T14:18:09.390 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_haproxy": { 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_haproxy", 2024-03-26T14:18:09.391 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "HAproxy container image", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_agent": { 2024-03-26T14:18:09.392 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_agent", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger agent container image", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.393 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_collector": { 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_collector", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger collector container image", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.394 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_query": { 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_query", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger query container image", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.395 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_keepalived": { 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_keepalived", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Keepalived container image", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.396 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_loki": { 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_loki", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Loki container image", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.397 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_node_exporter": { 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_node_exporter", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.398 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_nvmeof": { 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_nvmeof", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Nvme-of container image", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_prometheus": { 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_prometheus", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.400 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_promtail": { 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_promtail", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Promtail container image", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.401 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_snmp_gateway": { 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_snmp_gateway", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SNMP Gateway container image", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.402 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "container_init": { 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_init", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.403 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "daemon_cache_timeout": { 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "daemon_cache_timeout", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.404 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "default_cephadm_command_timeout": { 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "900", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.405 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "default_registry": { 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_registry", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T14:18:09.406 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "device_cache_timeout": { 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_cache_timeout", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1800", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache device inventory", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.407 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "device_enhanced_scan": { 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_enhanced_scan", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.408 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "facts_cache_timeout": { 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "facts_cache_timeout", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache host facts data", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.409 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "host_check_interval": { 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "host_check_interval", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to perform a host check", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "hw_monitoring": { 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hw_monitoring", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.411 INFO:teuthology.orchestra.run.smithi137.stdout: "inventory_list_all": { 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "inventory_list_all", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.412 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.413 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "log_refresh_metadata": { 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_refresh_metadata", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.414 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.415 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.416 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.417 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T14:18:09.418 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.419 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "*", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.420 INFO:teuthology.orchestra.run.smithi137.stdout: "max_count_per_host": { 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_count_per_host", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of daemons per service per host", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "max_osd_draining_count": { 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_osd_draining_count", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.422 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "migration_current": { 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "migration_current", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "internal - do not modify", 2024-03-26T14:18:09.423 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "root", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm-package", 2024-03-26T14:18:09.424 INFO:teuthology.orchestra.run.smithi137.stdout: "root" 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "oob_default_addr": { 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "oob_default_addr", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "169.254.1.1", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.425 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "prometheus_alerts_path": { 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus_alerts_path", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.426 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_insecure": { 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_insecure", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T14:18:09.427 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_password": { 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_password", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.428 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_url": { 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_url", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.429 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_username": { 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_username", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.430 INFO:teuthology.orchestra.run.smithi137.stdout: "secure_monitoring_stack": { 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "secure_monitoring_stack", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "service_discovery_port": { 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "service_discovery_port", 2024-03-26T14:18:09.431 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8765", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "cephadm service discovery port", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "ssh_config_file": { 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssh_config_file", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.432 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "use_agent": { 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_agent", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.433 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "use_repo_digest": { 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_repo_digest", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.434 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_failed_host_check": { 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_failed_host_check", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.435 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_daemons": { 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_daemons", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T14:18:09.436 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_hosts": { 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_hosts", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.437 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crash", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.438 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.439 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.440 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "retain_interval": { 2024-03-26T14:18:09.442 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retain_interval", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "31536000", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_recent_interval": { 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_recent_interval", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.443 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1209600", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dashboard", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.444 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T14:18:09.445 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.446 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_ENABLED": { 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.447 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.448 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.450 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T14:18:09.451 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T14:18:09.452 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T14:18:09.453 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.454 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.455 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.456 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.457 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.458 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_URL": { 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_URL", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.459 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T14:18:09.460 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.461 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.462 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.463 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "{}", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.464 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.465 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.466 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T14:18:09.467 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.468 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.469 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.470 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.471 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.472 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T14:18:09.473 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T14:18:09.474 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.475 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "45", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.476 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.477 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.478 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T14:18:09.479 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.480 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.481 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.482 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.483 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "cross_origin_url": { 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cross_origin_url", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "crt_file": { 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crt_file", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "debug": { 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "debug", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.485 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable/disable debug options", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "jwt_token_ttl": { 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "jwt_token_ttl", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.486 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "28800", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.487 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.488 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.489 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.490 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.491 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "motd": { 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "motd", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.492 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "The message of the day", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect_resolve_ip_addr": { 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.493 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.494 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8080", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl_server_port": { 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl_server_port", 2024-03-26T14:18:09.496 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8443", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.497 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "redirect", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect" 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.498 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "url_prefix": { 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url_prefix", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.499 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_monitoring": { 2024-03-26T14:18:09.500 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_monitoring", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "monitor device health metrics", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.501 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.502 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.503 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.504 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.505 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "mark_out_threshold": { 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mark_out_threshold", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2419200", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.506 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_name": { 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_name", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "device_health_metrics", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "retention_period": { 2024-03-26T14:18:09.507 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retention_period", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15552000", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain device health metrics", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_frequency": { 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_frequency", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.508 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "self_heal": { 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "self_heal", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.509 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.510 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_threshold": { 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_threshold", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7257600", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.511 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "diskprediction_local", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.512 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.513 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.514 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.515 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.516 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "predict_interval": { 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predict_interval", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "predictor_model": { 2024-03-26T14:18:09.517 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predictor_model", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "prophetstor", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.518 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "influx", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": false, 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "influxdb python module not found", 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.519 INFO:teuthology.orchestra.run.smithi137.stdout: "batch_size": { 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "batch_size", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5000", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "database": { 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "database", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.520 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "ceph", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": { 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hostname", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.521 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server hostname", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "30", 2024-03-26T14:18:09.522 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "5", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.523 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.524 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.525 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.526 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "password": { 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "password", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.527 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "password of InfluxDB server user", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "port": { 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "port", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8086", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.528 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server port", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "false", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.529 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "threads": { 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threads", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "32", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "username": { 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "username", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "username of InfluxDB server user", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.531 INFO:teuthology.orchestra.run.smithi137.stdout: "verify_ssl": { 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "verify_ssl", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "true", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "insights", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.533 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.534 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.535 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.536 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "iostat", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.537 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.538 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.540 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.541 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "k8sevents", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_event_retention_days": { 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ceph_event_retention_days", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.542 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "config_check_secs": { 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_check_secs", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "10", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.543 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.544 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.545 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.546 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.547 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "localpool", 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.548 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "failure_domain": { 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "failure_domain", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "host", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "failure domain for any created local pool", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.549 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.550 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.551 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.552 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.553 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "min_size": { 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_size", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default min_size for any created local pool", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T14:18:09.554 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "num_rep": { 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "num_rep", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default replica count for any created local pool", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_num": { 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_num", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "128", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "prefix": { 2024-03-26T14:18:09.556 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prefix", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name prefix for any created local pool", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "subtree": { 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "subtree", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.557 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "rack", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mds_autoscaler", 2024-03-26T14:18:09.558 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.559 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.560 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.561 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.562 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirroring", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.563 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.564 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.565 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.566 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.567 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "nfs", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.568 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.569 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.570 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.571 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.572 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "fail_fs": { 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "fail_fs", 2024-03-26T14:18:09.573 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.574 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.576 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.577 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.578 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator": { 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "rook", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "test_orchestrator" 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Orchestrator backend", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.579 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_perf_query", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.580 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.581 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.582 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_support", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.585 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.586 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.587 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.588 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_autoscaler", 2024-03-26T14:18:09.589 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.590 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.591 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.592 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.593 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.594 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "threshold": { 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threshold", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1.0", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "scaling threshold", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T14:18:09.595 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "progress", 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_pg_recovery_event": { 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_pg_recovery_event", 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.596 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.597 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.598 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.599 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.600 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.601 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "max_completed_events": { 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_completed_events", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "50", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.602 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "number of past completed events to remember", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long the module is going to sleep", 2024-03-26T14:18:09.603 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus", 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "cache": { 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cache", 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.604 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "exclude_perf_counters": { 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "exclude_perf_counters", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.605 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.606 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.607 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.608 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.609 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools": { 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.610 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "300", 2024-03-26T14:18:09.611 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_interval": { 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_interval", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15.0", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.612 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.613 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "9283", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.614 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "stale_cache_strategy": { 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stale_cache_strategy", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "log", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.615 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "default", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "default", 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "error" 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.616 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.617 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.618 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.619 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.620 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.621 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.622 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_snap_create": { 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_snap_create", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.623 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "mirror_snapshot_schedule": { 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "trash_purge_schedule": { 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "trash_purge_schedule", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "restful", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_auth": { 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_auth", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.626 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.627 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.628 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.629 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.630 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.631 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.632 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.633 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rgw", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.634 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.635 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.636 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.637 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rook", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.639 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.640 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.641 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.642 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "storage_class": { 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "storage_class", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.643 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "local", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "selftest", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.644 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.645 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.646 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.647 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.648 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "roption1": { 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption1", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.649 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "roption2": { 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption2", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "xyz", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption1": { 2024-03-26T14:18:09.650 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption1", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption2": { 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption2", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.651 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption3": { 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption3", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.652 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption4": { 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption4", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.653 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption5": { 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption5", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.654 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption6": { 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption6", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.655 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption7": { 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption7", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "42", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.656 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "testkey": { 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testkey", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "testlkey": { 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testlkey", 2024-03-26T14:18:09.657 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "testnewline": { 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testnewline", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.658 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.659 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_m_granularity": { 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_m_granularity", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "dump_on_update": { 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dump_on_update", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.660 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "dump database to debug log on update", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.661 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.662 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.663 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.664 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stats", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.666 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.667 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.668 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.669 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "status", 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.670 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.671 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.672 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.673 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.674 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telegraf", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.675 INFO:teuthology.orchestra.run.smithi137.stdout: "address": { 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "address", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.676 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.677 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.678 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.679 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.680 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.681 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telemetry", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_basic": { 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_basic", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.682 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_crash": { 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_crash", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.683 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_device": { 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_device", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.684 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_ident": { 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_ident", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.685 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_perf": { 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_perf", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "contact": { 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "contact", 2024-03-26T14:18:09.686 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "description": { 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "description", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.687 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "device_url": { 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_url", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T14:18:09.688 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.689 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "24", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "8", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.690 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "last_opt_revision": { 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "last_opt_revision", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.691 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard": { 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard_description": { 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard_description", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.693 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.694 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.696 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "organization": { 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "organization", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "proxy": { 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "proxy", 2024-03-26T14:18:09.698 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "url": { 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.699 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.700 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "test_orchestrator", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.701 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.702 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.703 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.706 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.707 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.708 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.709 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_clones": { 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_clones", 2024-03-26T14:18:09.710 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "periodic_async_work": { 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "periodic_async_work", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.711 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Periodically check for async work", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_delay": { 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_delay", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.712 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_no_wait": { 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.713 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix", 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "discovery_interval": { 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "discovery_interval", 2024-03-26T14:18:09.714 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "100", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "identifier": { 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "identifier", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.715 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.716 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.717 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.718 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.719 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.720 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_host": { 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_host", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.721 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_port": { 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_port", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10051", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.722 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_sender": { 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_sender", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.723 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "gid": 24112, 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "z", 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "mgr_features": 4540701547738038271, 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "available_modules": [ 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "alerts", 2024-03-26T14:18:09.724 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.725 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.726 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.727 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.728 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.729 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_destination": { 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_destination", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email address to send alerts to", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.730 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_from_name": { 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_from_name", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "Ceph", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email From: name", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.731 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_host": { 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_host", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP server", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.732 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_password": { 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_password", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Password to authenticate with", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.733 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_port": { 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_port", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "465", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP port", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.734 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_sender": { 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_sender", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP envelope sender", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.735 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_ssl": { 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_ssl", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_user": { 2024-03-26T14:18:09.736 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_user", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "User to authenticate as", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.737 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "balancer", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "active": { 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "active", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.738 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_time": { 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_time", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0000", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.739 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_weekday": { 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_weekday", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_max_iterations": { 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_max_iterations", 2024-03-26T14:18:09.740 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "25", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "250", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_metrics": { 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_metrics", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.741 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_step": { 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_step", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.5", 2024-03-26T14:18:09.742 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0.001", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "0.999", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "aggressiveness of optimization", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "end_time": { 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_time", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2359", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.743 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "end_weekday": { 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_weekday", 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.744 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.745 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.746 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.747 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.748 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.749 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "min_score": { 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_score", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.750 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-03-26T14:18:09.751 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "upmap", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "crush-compat", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "none", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "read", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap-read" 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Balancer mode", 2024-03-26T14:18:09.752 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_ids": { 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_ids", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T14:18:09.753 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.754 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_deviation": { 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_deviation", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T14:18:09.755 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_optimizations": { 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_optimizations", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.756 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_down_multiplier": { 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_down_multiplier", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.757 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_refresh_rate": { 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_refresh_rate", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "20", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.758 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_starting_port": { 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_starting_port", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4721", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T14:18:09.759 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_ptrace": { 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_ptrace", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T14:18:09.760 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_interval": { 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_interval", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.761 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_memory_target_ratio": { 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.7", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.762 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm_log_destination": { 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm_log_destination", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "file", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "file,syslog", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "syslog" 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.763 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "cgroups_split": { 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cgroups_split", 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.764 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "config_checks_enabled": { 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_checks_enabled", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.765 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "config_dashboard": { 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_dashboard", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.766 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_alertmanager": { 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_alertmanager", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_base": { 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_base", 2024-03-26T14:18:09.767 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Container image name, without the tag", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_elasticsearch": { 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_elasticsearch", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.768 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "elasticsearch container image", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_grafana": { 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_grafana", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.769 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_haproxy": { 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_haproxy", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "HAproxy container image", 2024-03-26T14:18:09.770 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_agent": { 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_agent", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger agent container image", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.771 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_collector": { 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_collector", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger collector container image", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.772 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_query": { 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_query", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger query container image", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_keepalived": { 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_keepalived", 2024-03-26T14:18:09.773 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Keepalived container image", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_loki": { 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_loki", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.774 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Loki container image", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_node_exporter": { 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_node_exporter", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.775 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_nvmeof": { 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_nvmeof", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Nvme-of container image", 2024-03-26T14:18:09.776 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_prometheus": { 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_prometheus", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_promtail": { 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_promtail", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Promtail container image", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_snmp_gateway": { 2024-03-26T14:18:09.778 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_snmp_gateway", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SNMP Gateway container image", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "container_init": { 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_init", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.779 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "daemon_cache_timeout": { 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "daemon_cache_timeout", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.780 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "default_cephadm_command_timeout": { 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "900", 2024-03-26T14:18:09.781 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "default_registry": { 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_registry", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.782 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "device_cache_timeout": { 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_cache_timeout", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1800", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache device inventory", 2024-03-26T14:18:09.783 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "device_enhanced_scan": { 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_enhanced_scan", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.784 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "facts_cache_timeout": { 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "facts_cache_timeout", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache host facts data", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.785 INFO:teuthology.orchestra.run.smithi137.stdout: "host_check_interval": { 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "host_check_interval", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to perform a host check", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "hw_monitoring": { 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hw_monitoring", 2024-03-26T14:18:09.786 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "inventory_list_all": { 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "inventory_list_all", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.787 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.788 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "log_refresh_metadata": { 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_refresh_metadata", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.789 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.790 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.791 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.792 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T14:18:09.793 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.794 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "*", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "max_count_per_host": { 2024-03-26T14:18:09.795 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_count_per_host", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of daemons per service per host", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.796 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "max_osd_draining_count": { 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_osd_draining_count", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.797 INFO:teuthology.orchestra.run.smithi137.stdout: "migration_current": { 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "migration_current", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "internal - do not modify", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.798 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "root", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm-package", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "root" 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T14:18:09.799 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "oob_default_addr": { 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "oob_default_addr", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "169.254.1.1", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T14:18:09.800 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "prometheus_alerts_path": { 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus_alerts_path", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.801 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_insecure": { 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_insecure", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.802 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_password": { 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_password", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_url": { 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_url", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_username": { 2024-03-26T14:18:09.804 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_username", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "secure_monitoring_stack": { 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "secure_monitoring_stack", 2024-03-26T14:18:09.805 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "service_discovery_port": { 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "service_discovery_port", 2024-03-26T14:18:09.806 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8765", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "cephadm service discovery port", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "ssh_config_file": { 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssh_config_file", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.807 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "use_agent": { 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_agent", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.808 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "use_repo_digest": { 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_repo_digest", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.809 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_failed_host_check": { 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_failed_host_check", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T14:18:09.810 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_daemons": { 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_daemons", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_hosts": { 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_hosts", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crash", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.813 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.814 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.815 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.816 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "retain_interval": { 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retain_interval", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.817 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "31536000", 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_recent_interval": { 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_recent_interval", 2024-03-26T14:18:09.818 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1209600", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dashboard", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.820 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.821 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.822 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_ENABLED": { 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.823 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.824 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.825 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.826 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.827 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.828 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.829 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.830 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.831 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.832 INFO:teuthology.orchestra.run.smithi137.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T14:18:09.833 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.834 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_URL": { 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_URL", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.835 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.836 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.837 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.838 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.840 INFO:teuthology.orchestra.run.smithi137.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "{}", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T14:18:09.841 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.842 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.843 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.844 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.845 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.846 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.847 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.848 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.849 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.850 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.851 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.852 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.853 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "45", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.854 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.855 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.856 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.857 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T14:18:09.858 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.859 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.860 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.861 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "cross_origin_url": { 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cross_origin_url", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.862 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "crt_file": { 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crt_file", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.863 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "debug": { 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "debug", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable/disable debug options", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.864 INFO:teuthology.orchestra.run.smithi137.stdout: "jwt_token_ttl": { 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "jwt_token_ttl", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "28800", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-03-26T14:18:09.865 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.866 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.868 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.869 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "motd": { 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "motd", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "The message of the day", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect_resolve_ip_addr": { 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T14:18:09.871 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.872 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8080", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.873 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.874 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl_server_port": { 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl_server_port", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8443", 2024-03-26T14:18:09.875 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "redirect", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.876 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect" 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.877 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "url_prefix": { 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url_prefix", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.878 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_monitoring": { 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_monitoring", 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.879 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "monitor device health metrics", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.880 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.881 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.882 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "mark_out_threshold": { 2024-03-26T14:18:09.884 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mark_out_threshold", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2419200", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_name": { 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_name", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.885 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "device_health_metrics", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "retention_period": { 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retention_period", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15552000", 2024-03-26T14:18:09.886 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain device health metrics", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_frequency": { 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_frequency", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.887 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "self_heal": { 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "self_heal", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.888 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_threshold": { 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_threshold", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7257600", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.890 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "diskprediction_local", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.891 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.892 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.893 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.894 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "predict_interval": { 2024-03-26T14:18:09.895 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predict_interval", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.896 INFO:teuthology.orchestra.run.smithi137.stdout: "predictor_model": { 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predictor_model", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "prophetstor", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.897 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "influx", 2024-03-26T14:18:09.898 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": false, 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "influxdb python module not found", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "batch_size": { 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "batch_size", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5000", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.899 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "database": { 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "database", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "ceph", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.900 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": { 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hostname", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server hostname", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.901 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "30", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "5", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.902 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.903 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.904 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.905 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.906 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "password": { 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "password", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.907 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "password of InfluxDB server user", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "port": { 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "port", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8086", 2024-03-26T14:18:09.908 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server port", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "false", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T14:18:09.909 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "threads": { 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threads", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "32", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.910 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "username": { 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "username", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "username of InfluxDB server user", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "verify_ssl": { 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "verify_ssl", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "true", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "insights", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.913 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.914 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.915 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.916 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "iostat", 2024-03-26T14:18:09.917 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.918 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.919 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.920 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.921 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "k8sevents", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_event_retention_days": { 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ceph_event_retention_days", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.923 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "config_check_secs": { 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_check_secs", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "10", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.924 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.925 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.926 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.927 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.928 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "localpool", 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "failure_domain": { 2024-03-26T14:18:09.929 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "failure_domain", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "host", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "failure domain for any created local pool", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.930 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.931 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.932 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.933 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.934 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "min_size": { 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_size", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default min_size for any created local pool", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "num_rep": { 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "num_rep", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default replica count for any created local pool", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_num": { 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_num", 2024-03-26T14:18:09.936 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "128", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "prefix": { 2024-03-26T14:18:09.937 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prefix", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name prefix for any created local pool", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "subtree": { 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "subtree", 2024-03-26T14:18:09.938 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "rack", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.939 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mds_autoscaler", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.940 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.941 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.942 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.943 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.944 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirroring", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.945 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.946 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.947 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.948 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.949 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "nfs", 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.950 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.951 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.953 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.954 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "fail_fs": { 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "fail_fs", 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.955 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.956 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.957 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.958 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.959 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.960 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator": { 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "rook", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "test_orchestrator" 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Orchestrator backend", 2024-03-26T14:18:09.961 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_perf_query", 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.962 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.964 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.965 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.966 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_support", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.967 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.968 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.969 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.970 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.971 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_autoscaler", 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.972 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.973 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.974 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.975 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "threshold": { 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threshold", 2024-03-26T14:18:09.977 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1.0", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "scaling threshold", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.978 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "progress", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_pg_recovery_event": { 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_pg_recovery_event", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.980 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.981 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.982 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.983 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.984 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "max_completed_events": { 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_completed_events", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "50", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "number of past completed events to remember", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.985 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long the module is going to sleep", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.986 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus", 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "cache": { 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cache", 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.987 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "exclude_perf_counters": { 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "exclude_perf_counters", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.988 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.989 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.990 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:09.991 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:09.992 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools": { 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.993 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "300", 2024-03-26T14:18:09.994 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_interval": { 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_interval", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15.0", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.995 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.996 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "9283", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "stale_cache_strategy": { 2024-03-26T14:18:09.997 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stale_cache_strategy", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "log", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-03-26T14:18:09.998 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "default", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "default", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "error" 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-03-26T14:18:09.999 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.000 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.001 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.002 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.003 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_snap_create": { 2024-03-26T14:18:10.005 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_snap_create", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "mirror_snapshot_schedule": { 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.006 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "trash_purge_schedule": { 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "trash_purge_schedule", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.007 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "restful", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_auth": { 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_auth", 2024-03-26T14:18:10.008 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.009 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.010 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.011 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.012 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.013 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.014 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.015 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rgw", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.016 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.017 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.019 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.020 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rook", 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.021 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.022 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.023 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.024 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.025 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "storage_class": { 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "storage_class", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.026 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "local", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "selftest", 2024-03-26T14:18:10.027 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.028 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.029 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.030 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.031 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.032 INFO:teuthology.orchestra.run.smithi137.stdout: "roption1": { 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption1", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "roption2": { 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption2", 2024-03-26T14:18:10.033 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "xyz", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption1": { 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption1", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.034 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption2": { 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption2", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.035 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption3": { 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption3", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.036 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption4": { 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption4", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.037 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption5": { 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption5", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.038 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption6": { 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption6", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.039 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption7": { 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption7", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.040 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "42", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "testkey": { 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testkey", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.041 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "testlkey": { 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testlkey", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.042 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "testnewline": { 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testnewline", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.043 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_m_granularity": { 2024-03-26T14:18:10.044 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_m_granularity", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "dump_on_update": { 2024-03-26T14:18:10.045 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dump_on_update", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "dump database to debug log on update", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.046 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.047 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.048 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.049 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.050 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stats", 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.051 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.052 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.053 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.054 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.055 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "status", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.056 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.058 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.059 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.060 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telegraf", 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "address": { 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "address", 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.061 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.062 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.063 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.064 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.065 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.066 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.067 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telemetry", 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_basic": { 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_basic", 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.068 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_crash": { 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_crash", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.069 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_device": { 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_device", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.070 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_ident": { 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_ident", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.071 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_perf": { 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_perf", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.072 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "contact": { 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "contact", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.073 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "description": { 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "description", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.074 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "device_url": { 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_url", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.076 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "24", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "8", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "last_opt_revision": { 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "last_opt_revision", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.077 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard": { 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.078 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard_description": { 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard_description", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.079 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.080 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.081 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.082 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.083 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "organization": { 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "organization", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.084 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "proxy": { 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "proxy", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.085 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "url": { 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.086 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "test_orchestrator", 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.087 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.088 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.089 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.090 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.091 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.092 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.093 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.094 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.095 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.096 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_clones": { 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_clones", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4", 2024-03-26T14:18:10.097 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "periodic_async_work": { 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "periodic_async_work", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.098 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Periodically check for async work", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_delay": { 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_delay", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.099 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_no_wait": { 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "discovery_interval": { 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "discovery_interval", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "100", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.101 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "identifier": { 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "identifier", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.102 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.104 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.105 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.106 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.107 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_host": { 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_host", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.108 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_port": { 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_port", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10051", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.109 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_sender": { 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_sender", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.110 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "modules": [ 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "dashboard", 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "iostat", 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "nfs", 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "restful" 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "available_modules": [ 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "alerts", 2024-03-26T14:18:10.111 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How frequently to reexamine health status", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.112 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.113 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.114 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.115 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.116 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.117 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_destination": { 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_destination", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email address to send alerts to", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.118 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_from_name": { 2024-03-26T14:18:10.119 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_from_name", 2024-03-26T14:18:10.119 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.119 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "Ceph", 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Email From: name", 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_host": { 2024-03-26T14:18:10.121 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_host", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP server", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_password": { 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_password", 2024-03-26T14:18:10.122 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Password to authenticate with", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_port": { 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_port", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.123 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "465", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP port", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_sender": { 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_sender", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.124 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SMTP envelope sender", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_ssl": { 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_ssl", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-26T14:18:10.125 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "smtp_user": { 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "smtp_user", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "User to authenticate as", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.126 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "balancer", 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "active": { 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "active", 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.127 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically balance PGs across cluster", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_time": { 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_time", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0000", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.128 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "beginning time of day to automatically balance", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "begin_weekday": { 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "begin_weekday", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T14:18:10.129 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_max_iterations": { 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_max_iterations", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "25", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "250", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.130 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_metrics": { 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_metrics", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "pgs,objects,bytes", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.131 INFO:teuthology.orchestra.run.smithi137.stdout: "crush_compat_step": { 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crush_compat_step", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.5", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0.001", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "0.999", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "aggressiveness of optimization", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "end_time": { 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_time", 2024-03-26T14:18:10.132 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2359", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ending time of day to automatically balance", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "end_weekday": { 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "end_weekday", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:10.133 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "0", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "6", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.134 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.135 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.136 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.137 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.138 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.139 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "min_score": { 2024-03-26T14:18:10.140 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_score", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.141 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "upmap", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "crush-compat", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "none", 2024-03-26T14:18:10.142 INFO:teuthology.orchestra.run.smithi137.stdout: "read", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap-read" 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Balancer mode", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_ids": { 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_ids", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.143 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:10.144 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_deviation": { 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_deviation", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.145 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "upmap_max_optimizations": { 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "upmap_max_optimizations", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.146 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm", 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_down_multiplier": { 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_down_multiplier", 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-03-26T14:18:10.147 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_refresh_rate": { 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_refresh_rate", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "20", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.148 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "agent_starting_port": { 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "agent_starting_port", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4721", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.149 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_ptrace": { 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_ptrace", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.150 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_interval": { 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_interval", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.151 INFO:teuthology.orchestra.run.smithi137.stdout: "autotune_memory_target_ratio": { 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "autotune_memory_target_ratio", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0.7", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm_log_destination": { 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cephadm_log_destination", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.152 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "file", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "file,syslog", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "syslog" 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.153 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "cgroups_split": { 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cgroups_split", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.154 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "config_checks_enabled": { 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_checks_enabled", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "config_dashboard": { 2024-03-26T14:18:10.155 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_dashboard", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_alertmanager": { 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_alertmanager", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.156 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_base": { 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_base", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.157 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Container image name, without the tag", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_elasticsearch": { 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_elasticsearch", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.158 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "elasticsearch container image", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_grafana": { 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_grafana", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.159 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_haproxy": { 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_haproxy", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "HAproxy container image", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.160 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_agent": { 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_agent", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger agent container image", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.161 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_collector": { 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_collector", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger collector container image", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.162 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_jaeger_query": { 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_jaeger_query", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Jaeger query container image", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.163 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_keepalived": { 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_keepalived", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Keepalived container image", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.164 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_loki": { 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_loki", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Loki container image", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_node_exporter": { 2024-03-26T14:18:10.165 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_node_exporter", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.166 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_nvmeof": { 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_nvmeof", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Nvme-of container image", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_prometheus": { 2024-03-26T14:18:10.167 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_prometheus", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Prometheus container image", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_promtail": { 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_promtail", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.168 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Promtail container image", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "container_image_snmp_gateway": { 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_image_snmp_gateway", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.169 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "SNMP Gateway container image", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "container_init": { 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "container_init", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.170 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Run podman/docker with `--init`", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "daemon_cache_timeout": { 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "daemon_cache_timeout", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.171 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "default_cephadm_command_timeout": { 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_cephadm_command_timeout", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "900", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.172 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "default_registry": { 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "default_registry", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "docker.io", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.173 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "device_cache_timeout": { 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_cache_timeout", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1800", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache device inventory", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.174 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "device_enhanced_scan": { 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_enhanced_scan", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-26T14:18:10.175 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "facts_cache_timeout": { 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "facts_cache_timeout", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "seconds to cache host facts data", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.176 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "host_check_interval": { 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "host_check_interval", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to perform a host check", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.177 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "hw_monitoring": { 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hw_monitoring", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.178 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "inventory_list_all": { 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "inventory_list_all", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.179 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.180 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "log_refresh_metadata": { 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_refresh_metadata", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-26T14:18:10.181 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.182 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.183 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.184 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.185 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "*", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.186 INFO:teuthology.orchestra.run.smithi137.stdout: "max_count_per_host": { 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_count_per_host", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of daemons per service per host", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "max_osd_draining_count": { 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_osd_draining_count", 2024-03-26T14:18:10.187 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "migration_current": { 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "migration_current", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.188 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "internal - do not modify", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "mode": { 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mode", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "root", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.189 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm-package", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "root" 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "mode for remote execution of cephadm", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "oob_default_addr": { 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "oob_default_addr", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "169.254.1.1", 2024-03-26T14:18:10.190 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "prometheus_alerts_path": { 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus_alerts_path", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.191 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_insecure": { 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_insecure", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.192 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_password": { 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_password", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_url": { 2024-03-26T14:18:10.193 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_url", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "registry_username": { 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "registry_username", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.194 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "secure_monitoring_stack": { 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "secure_monitoring_stack", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.195 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "service_discovery_port": { 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "service_discovery_port", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8765", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.196 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "cephadm service discovery port", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "ssh_config_file": { 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssh_config_file", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-26T14:18:10.197 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "use_agent": { 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_agent", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.198 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "use_repo_digest": { 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "use_repo_digest", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_failed_host_check": { 2024-03-26T14:18:10.199 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_failed_host_check", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if the host check fails", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_daemons": { 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_daemons", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.200 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_on_stray_hosts": { 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_on_stray_hosts", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.201 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crash", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.202 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.203 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.204 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.205 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.206 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "retain_interval": { 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retain_interval", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.207 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "31536000", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_recent_interval": { 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_recent_interval", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.208 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1209600", 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dashboard", 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.209 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.210 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_HOST": { 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.211 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.212 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_ENABLED": { 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_ENABLED", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.213 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.214 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "ENABLE_BROWSABLE_API": { 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.215 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.216 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.217 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-26T14:18:10.218 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-26T14:18:10.219 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-26T14:18:10.220 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.221 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.222 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_PASSWORD": { 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:10.223 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.224 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_URL": { 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_URL", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.225 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_API_USERNAME": { 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.227 INFO:teuthology.orchestra.run.smithi137.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-26T14:18:10.228 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.229 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "MULTICLUSTER_CONFIG": { 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.230 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "{}", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_HOST": { 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.231 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.232 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.233 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.234 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.235 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.236 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.237 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-26T14:18:10.238 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_ENABLED": { 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.239 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.240 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.241 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.242 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "45", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.243 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ACCESS_KEY": { 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-26T14:18:10.244 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "admin", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SECRET_KEY": { 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.245 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "RGW_API_SSL_VERIFY": { 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.246 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "UNSAFE_TLS_v1_2": { 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.247 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.248 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-26T14:18:10.249 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "cross_origin_url": { 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cross_origin_url", 2024-03-26T14:18:10.250 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "crt_file": { 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "crt_file", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.251 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "debug": { 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "debug", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.252 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Enable/disable debug options", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "jwt_token_ttl": { 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "jwt_token_ttl", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "28800", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.253 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.254 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.255 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.256 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.257 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.258 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "motd": { 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "motd", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "The message of the day", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.259 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect_resolve_ip_addr": { 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "redirect_resolve_ip_addr", 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.260 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.261 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8080", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.262 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl_server_port": { 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl_server_port", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.263 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8443", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.264 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "redirect", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "redirect" 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.265 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "url_prefix": { 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url_prefix", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.266 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_monitoring": { 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_monitoring", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.267 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "monitor device health metrics", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.268 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.269 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.270 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.271 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.272 INFO:teuthology.orchestra.run.smithi137.stdout: "mark_out_threshold": { 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mark_out_threshold", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "2419200", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "pool_name": { 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pool_name", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.273 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "device_health_metrics", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "retention_period": { 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "retention_period", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.274 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15552000", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long to retain device health metrics", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_frequency": { 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_frequency", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.275 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "self_heal": { 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "self_heal", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.276 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.277 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how frequently to wake up and check device health", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "warn_threshold": { 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "warn_threshold", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7257600", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.278 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "diskprediction_local", 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.279 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.280 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.281 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.282 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.283 INFO:teuthology.orchestra.run.smithi137.stdout: "predict_interval": { 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predict_interval", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "86400", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "predictor_model": { 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "predictor_model", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.284 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "prophetstor", 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:10.285 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "600", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "influx", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": false, 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "influxdb python module not found", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "batch_size": { 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "batch_size", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5000", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.287 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "database": { 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "database", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "ceph", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.288 INFO:teuthology.orchestra.run.smithi137.stdout: "hostname": { 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "hostname", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server hostname", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.289 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "30", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "5", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.290 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.292 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.293 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.294 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "password": { 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "password", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "password of InfluxDB server user", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.295 INFO:teuthology.orchestra.run.smithi137.stdout: "port": { 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "port", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "8086", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "InfluxDB server port", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "ssl": { 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ssl", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.296 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "false", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "threads": { 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threads", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:10.297 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "32", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "username": { 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "username", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.298 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "username of InfluxDB server user", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "verify_ssl": { 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "verify_ssl", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "true", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.299 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "insights", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.300 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.301 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.302 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.303 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.304 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "iostat", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.305 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.306 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.307 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "k8sevents", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_event_retention_days": { 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "ceph_event_retention_days", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "7", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.310 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "config_check_secs": { 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "config_check_secs", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "10", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.311 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.312 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.313 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.314 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.315 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.316 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "localpool", 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.317 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "failure_domain": { 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "failure_domain", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "host", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "failure domain for any created local pool", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.318 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.319 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.320 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.321 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.322 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "min_size": { 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "min_size", 2024-03-26T14:18:10.325 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default min_size for any created local pool", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "num_rep": { 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "num_rep", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.326 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default replica count for any created local pool", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_num": { 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_num", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.327 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "128", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "default pg_num for any created local pool", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "prefix": { 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prefix", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.328 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "name prefix for any created local pool", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "subtree": { 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "subtree", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "rack", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.329 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mds_autoscaler", 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.330 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.331 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.332 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.333 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.334 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirroring", 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.335 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.337 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.338 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.339 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "nfs", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.340 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.341 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.342 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.343 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.344 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "fail_fs": { 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "fail_fs", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.346 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.347 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.348 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.349 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator": { 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "orchestrator", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.350 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "cephadm", 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "rook", 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "test_orchestrator" 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Orchestrator backend", 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_perf_query", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.352 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.353 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.354 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.355 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "osd_support", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.357 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.358 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.359 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.360 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "pg_autoscaler", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.361 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.362 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.363 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.364 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.365 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.366 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "threshold": { 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "threshold", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "3.0", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1.0", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "scaling threshold", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.367 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "progress", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_pg_recovery_event": { 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_pg_recovery_event", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.368 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.369 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.370 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.371 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.372 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.373 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "max_completed_events": { 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_completed_events", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "50", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "number of past completed events to remember", 2024-03-26T14:18:10.374 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "sleep_interval": { 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "sleep_interval", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "5", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "how long the module is going to sleep", 2024-03-26T14:18:10.375 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "prometheus", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "cache": { 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "cache", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.376 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "exclude_perf_counters": { 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "exclude_perf_counters", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.377 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.378 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.379 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.380 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.381 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.382 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools": { 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-26T14:18:10.383 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "300", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "scrape_interval": { 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "scrape_interval", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.384 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15.0", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "::", 2024-03-26T14:18:10.385 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "9283", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.386 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "stale_cache_strategy": { 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stale_cache_strategy", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "log", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.387 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_behaviour": { 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_behaviour", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "default", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "default", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "error" 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.388 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "standby_error_status_code": { 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "standby_error_status_code", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "500", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "400", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "599", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.389 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.390 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.391 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.392 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.393 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_snap_create": { 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_snap_create", 2024-03-26T14:18:10.394 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "mirror_snapshot_schedule": { 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "mirror_snapshot_schedule", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.395 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "trash_purge_schedule": { 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "trash_purge_schedule", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.396 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "restful", 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "enable_auth": { 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enable_auth", 2024-03-26T14:18:10.397 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "key_file": { 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "key_file", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.398 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.399 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.400 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.401 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.402 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.403 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "server_addr": { 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_addr", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.404 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "server_port": { 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "server_port", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.405 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rgw", 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.406 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.407 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.408 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.409 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.410 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rook", 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.411 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.412 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.413 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.414 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.415 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "storage_class": { 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "storage_class", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "local", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.416 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "selftest", 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.417 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.418 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.419 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "roption1": { 2024-03-26T14:18:10.421 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption1", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "roption2": { 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "roption2", 2024-03-26T14:18:10.422 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "xyz", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption1": { 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption1", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.423 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption2": { 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption2", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.424 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption3": { 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption3", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "float", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.425 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption4": { 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption4", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption5": { 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption5", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption6": { 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption6", 2024-03-26T14:18:10.427 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "rwoption7": { 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rwoption7", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.428 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "1", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "42", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "testkey": { 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testkey", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.429 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "testlkey": { 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testlkey", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.430 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "testnewline": { 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "testnewline", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.431 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snap_schedule", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "allow_m_granularity": { 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "allow_m_granularity", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.432 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "allow minute scheduled snapshots", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "dump_on_update": { 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "dump_on_update", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "dump database to debug log on update", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.433 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.434 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.435 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.436 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.437 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "stats", 2024-03-26T14:18:10.438 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.439 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.440 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.441 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.442 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "status", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.443 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.444 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.445 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.446 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.447 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telegraf", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "address": { 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "address", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.448 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "15", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.449 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.450 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.451 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.452 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.454 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "telemetry", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_basic": { 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_basic", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.455 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_crash": { 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_crash", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.456 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_device": { 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_device", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_ident": { 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_ident", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "channel_perf": { 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "channel_perf", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "contact": { 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "contact", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.460 INFO:teuthology.orchestra.run.smithi137.stdout: "description": { 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "description", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "device_url": { 2024-03-26T14:18:10.461 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "device_url", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "enabled": { 2024-03-26T14:18:10.462 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "enabled", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.463 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "24", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "8", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "last_opt_revision": { 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "last_opt_revision", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.464 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "1", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard": { 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.465 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "leaderboard_description": { 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "leaderboard_description", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.466 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.467 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.468 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.469 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.470 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "organization": { 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "organization", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.471 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "proxy": { 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "proxy", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.472 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "url": { 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "url", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.473 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "test_orchestrator", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.474 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.476 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.477 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.479 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.480 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.481 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.482 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.483 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "max_concurrent_clones": { 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "max_concurrent_clones", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "4", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "periodic_async_work": { 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "periodic_async_work", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Periodically check for async work", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_delay": { 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_delay", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "0", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.486 INFO:teuthology.orchestra.run.smithi137.stdout: "snapshot_clone_no_wait": { 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "snapshot_clone_no_wait", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "True", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.487 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "can_run": true, 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "error_string": "", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "module_options": { 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "discovery_interval": { 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "discovery_interval", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "uint", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "100", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.488 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "identifier": { 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "identifier", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.489 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "interval": { 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "interval", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "secs", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "60", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "log_level": { 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_level", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.491 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster": { 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.492 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_cluster_level": { 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_cluster_level", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "info", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [ 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "critical", 2024-03-26T14:18:10.493 INFO:teuthology.orchestra.run.smithi137.stdout: "debug", 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "error", 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "info", 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "warning" 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.494 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "log_to_file": { 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "log_to_file", 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "bool", 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 1, 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "False", 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.495 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_host": { 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_host", 2024-03-26T14:18:10.496 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "", 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.497 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_port": { 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_port", 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "int", 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.498 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "10051", 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "zabbix_sender": { 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "zabbix_sender", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "str", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "level": "advanced", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "flags": 0, 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "min": "", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "max": "", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "enum_allowed": [], 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "desc": "", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "long_desc": "", 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "tags": [], 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: "see_also": [] 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.500 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "services": { 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "dashboard": "https://172.21.15.137:8443/" 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "always_on_modules": { 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "octopus": [ 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-03-26T14:18:10.501 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "pacific": [ 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-03-26T14:18:10.502 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "quincy": [ 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "reef": [ 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-03-26T14:18:10.503 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "squid": [ 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "balancer", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "crash", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "devicehealth", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "orchestrator", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "pg_autoscaler", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "progress", 2024-03-26T14:18:10.504 INFO:teuthology.orchestra.run.smithi137.stdout: "rbd_support", 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "status", 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "telemetry", 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "volumes" 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "last_failure_osd_epoch": 3, 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "active_clients": [ 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "devicehealth", 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 1857852421 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.505 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "libcephsqlite", 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 1371432069 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "rbd_support", 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-03-26T14:18:10.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 406354822 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "name": "volumes", 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "addrvec": [ 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "type": "v2", 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "addr": "172.21.15.137:0", 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: "nonce": 1685729166 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.507 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:10.508 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-03-26T14:18:10.508 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:18:10.509 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1322035813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:18:10.509 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2918105630' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T14:18:10.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1322035813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:18:10.509 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2918105630' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb700a2880 msgr2=0x7fbb700a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb700a2880 0x7fbb700a4d40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c108d30 tx=0x7fbb7c005f90 comp rx=0 tx=0).stop 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 msgr2=0x7fbb8c1a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c1a5440 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fbb84032520 tx=0x7fbb84035ce0 comp rx=0 tx=0).stop 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 shutdown_connections 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fbb700a2880 0x7fbb700a4d40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb8c108ed0 0x7fbb8c19e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbb8c109890 0x7fbb8c1a5440 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 --2- 172.21.15.137:0/2918105630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb8c107ce0 0x7fbb8c19de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:10.510 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.333+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 >> 172.21.15.137:0/2918105630 conn(0x7fbb8c076cb0 msgr2=0x7fbb8c10c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:10.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.334+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 shutdown_connections 2024-03-26T14:18:10.511 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:09.334+0000 7fbb92e00640 1 -- 172.21.15.137:0/2918105630 wait complete. 2024-03-26T14:18:10.547 DEBUG:tasks.check_counter:Getting stats from a 2024-03-26T14:18:10.548 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-26T14:18:10.548 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1322035813' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:18:10.548 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2918105630' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-26T14:18:10.995 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:18:11.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:10 smithi137 ceph-mon[31539]: pgmap v932: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T14:18:11.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:10 smithi179 ceph-mon[35472]: pgmap v932: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T14:18:11.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:10 smithi159 ceph-mon[35501]: pgmap v932: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-03-26T14:18:13.073 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:12 smithi137 ceph-mon[31539]: pgmap v933: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:18:13.176 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:12 smithi179 ceph-mon[35472]: pgmap v933: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:18:13.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:12 smithi159 ceph-mon[35501]: pgmap v933: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-03-26T14:18:13.405 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:18:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 310, 2024-03-26T14:18:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 305, 2024-03-26T14:18:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 777136, 2024-03-26T14:18:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 183488, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.116844182, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.050078864, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.052146006, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.014850820, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 304, 2024-03-26T14:18:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.027341343, 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000089938 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 777136, 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 183488 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1, 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 232, 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 160, 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 118368, 2024-03-26T14:18:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 3, 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.046099905, 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.037121352, 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.003564513, 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 230, 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.021249461, 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000092388 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 118368 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4, 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 4, 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3040, 2024-03-26T14:18:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 448, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.001758297, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000083538, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001648479, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001630007, 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000543335 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-26T14:18:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148984, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-26T14:18:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 311, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 733328, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 311, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 311, 2024-03-26T14:18:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 733328, 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:13.424 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:18:14.312 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-03-26T14:18:14.312 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-03-26T14:18:14.312 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-03-26T14:18:14.312 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-03-26T14:18:14.312 DEBUG:tasks.check_counter:Getting stats from d 2024-03-26T14:18:14.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-26T14:18:14.747 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:18:15.086 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:14 smithi137 ceph-mon[31539]: pgmap v934: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:15.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:14 smithi179 ceph-mon[35472]: pgmap v934: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:15.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:14 smithi159 ceph-mon[35501]: pgmap v934: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:16.987 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:18:16.987 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:16.987 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.003611986, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000795212, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:16.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 322, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 315, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 782352, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 188928, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-03-26T14:18:16.989 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.118703524, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.051040899, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.054665111, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.016490181, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 315, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.029917146, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000094975 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:16.990 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 782352, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 188928 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 239, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 320, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 197488, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.048758754, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.039006464, 2024-03-26T14:18:16.991 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.004230438, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 237, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.023927074, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000100958 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T14:18:16.992 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 197488 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:16.993 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:16.994 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:16.995 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148722, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:16.996 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:16.997 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-26T14:18:16.998 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 320, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 734395, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 320, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 320, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 734395, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:16.999 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:18:17.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:16 smithi137 ceph-mon[31539]: pgmap v935: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:17.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:16 smithi179 ceph-mon[35472]: pgmap v935: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:17.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:16 smithi159 ceph-mon[35501]: pgmap v935: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:17.856 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-26T14:18:17.857 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-26T14:18:17.857 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-03-26T14:18:17.857 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-03-26T14:18:17.857 DEBUG:tasks.check_counter:Getting stats from g 2024-03-26T14:18:17.857 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-26T14:18:18.313 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:18:19.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:18 smithi137 ceph-mon[31539]: pgmap v936: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:19.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:18 smithi179 ceph-mon[35472]: pgmap v936: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:19.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:18 smithi159 ceph-mon[35501]: pgmap v936: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:20.546 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 312, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 307, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 777680, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 184736, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.114739242, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.048374582, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.051521923, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.014939970, 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:20.547 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 306, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.028643861, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000093607 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 777680, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 184736 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:20.548 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 6, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 237, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3360, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 197232, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.045592186, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.036584683, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.003821732, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 236, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.023995249, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000101674 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:20.549 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 197232 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.000616252, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T14:18:20.550 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000459967, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.551 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:20.552 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:20.553 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148704, 2024-03-26T14:18:20.554 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 29, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-03-26T14:18:20.555 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 314, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 733615, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 314, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-26T14:18:20.556 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 314, 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 733615, 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:20.557 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:18:21.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:20 smithi179 ceph-mon[35472]: pgmap v937: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:21.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:20 smithi159 ceph-mon[35501]: pgmap v937: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:21.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:20 smithi137 ceph-mon[31539]: pgmap v937: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:21.394 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-03-26T14:18:21.394 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-03-26T14:18:21.395 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-03-26T14:18:21.395 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-03-26T14:18:21.395 DEBUG:tasks.check_counter:Getting stats from j 2024-03-26T14:18:21.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-26T14:18:21.845 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.a/config 2024-03-26T14:18:23.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:22 smithi179 ceph-mon[35472]: pgmap v938: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:23.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:22 smithi159 ceph-mon[35501]: pgmap v938: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:23.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:22 smithi137 ceph-mon[31539]: pgmap v938: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 4, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3040, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 448, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.004468295, 2024-03-26T14:18:24.217 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000080807, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001151481, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001576163, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000525387 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.218 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 241, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 320, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 197840, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.046452956, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.038079503, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.002668891, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:24.219 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 239, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.024689931, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000103305 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 197840 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.220 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 320, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 315, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 779856, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 189728, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.117294973, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.049856732, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.053803039, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.016083788, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 314, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.027765074, 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000088423 2024-03-26T14:18:24.221 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 779856, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 189728 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.222 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:24.223 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:24.224 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148704, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 29, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:24.225 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-03-26T14:18:24.226 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 322, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 734655, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 322, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 322, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 734655, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-03-26T14:18:24.227 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-03-26T14:18:24.228 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:24.228 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-03-26T14:18:24.228 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-03-26T14:18:25.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:24 smithi137 ceph-mon[31539]: pgmap v939: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:25.122 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-26T14:18:25.122 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-26T14:18:25.122 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-26T14:18:25.122 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-26T14:18:25.122 DEBUG:tasks.check_counter:Getting stats from b 2024-03-26T14:18:25.122 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-26T14:18:25.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:24 smithi179 ceph-mon[35472]: pgmap v939: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:25.195 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:24 smithi159 ceph-mon[35501]: pgmap v939: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:25.605 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T14:18:27.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:26 smithi179 ceph-mon[35472]: pgmap v940: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:27.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:26 smithi159 ceph-mon[35501]: pgmap v940: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:27.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:26 smithi137 ceph-mon[31539]: pgmap v940: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:28.190 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 5279, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 6469, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 2001598, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 8513079, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 227, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.200356006, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.485753497, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.509157213, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.141434576, 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:28.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6472, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.172212330, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000181120 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5856, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001910391, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000326 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 864208, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 228000 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 4952, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 6458, 2024-03-26T14:18:28.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 1334297, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 6737318, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 20, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.896220504, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.420305519, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.324658039, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.139370321, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6459, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.321279695, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000204564 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5557, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002197175, 2024-03-26T14:18:28.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000395 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 467408 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 10890, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 15017, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3605778, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 10229261, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 18, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 8, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.650308339, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.682448062, 2024-03-26T14:18:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.771814030, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.218703776, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 15009, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.380162303, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000091955 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 17528, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.006108159, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000348 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 56080, 2024-03-26T14:18:28.195 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 36944 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5340, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 5.829363757, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001091641 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:28.196 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5253, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.344180177, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000065520 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 2514, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 2514, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2514, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 12.716451969, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005058254 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-03-26T14:18:28.197 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 12, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 24, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 24, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 23, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 2, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 2, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 5047, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 2565, 2024-03-26T14:18:28.198 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-26T14:18:28.199 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 400, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 400, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 258, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 29, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 229, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 142, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 1263, 2024-03-26T14:18:28.200 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 3631, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 3892, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 1776, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 57, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 3653 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 13, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 23, 2024-03-26T14:18:28.201 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 1212, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 1212, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 29, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-26T14:18:28.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 6376, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 6376, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 11, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 11, 2024-03-26T14:18:28.203 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 16907175, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2505, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 10.236760053, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004086530 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 23, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 1264, 2024-03-26T14:18:28.204 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 1241, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 18, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 24, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 6, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 24, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 1243, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 1219, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 2, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 1227, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 1225, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 57876, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 231804 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 5216, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-26T14:18:28.205 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 2514, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 27, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1210, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 8.336562843, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.006889721 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 40, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.022962050, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000574051 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 24, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.012151084, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000506295 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-26T14:18:28.207 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 12, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.058931998, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004910999 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002968486, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000989495 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.006984575, 2024-03-26T14:18:28.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003492287 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.065438719, 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.010906453 2024-03-26T14:18:28.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002235719, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001117859 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9, 2024-03-26T14:18:28.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.044109955, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004901106 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1206, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.164106540, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003452824 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-26T14:18:28.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 4, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 3, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 1797, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 1797, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 967, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-26T14:18:28.216 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:28.217 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-26T14:18:28.218 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 291644, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 117, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9088, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 175, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:28.219 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 94890, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 703, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 8539, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 12971508, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 37, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 8539, 2024-03-26T14:18:28.220 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8539, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 76.333522642, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.008939398 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8539, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 8693 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 8539, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 568, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 7971, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-26T14:18:28.221 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 11, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 139, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 2615, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 83, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 5173, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 429, 2024-03-26T14:18:28.222 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 22, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-26T14:18:28.223 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 94, 2024-03-26T14:18:28.224 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 9, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 160, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 61 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.225 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 19, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 6, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 4658, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 1212, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:28.226 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 21117, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 5007404, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 21117, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 21117, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 5007404, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.227 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 8539, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 13103568, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 8539, 2024-03-26T14:18:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 13103568, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:28.229 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 8539, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 8539, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 8539, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 8539, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1212, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 132060, 2024-03-26T14:18:28.230 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1212, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 110, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 132060, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x563e395185a0": { 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:28.231 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 6376, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 12712871, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 6376, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2505, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 12712871, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:28.232 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-26T14:18:28.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:28 smithi159 ceph-mon[35501]: pgmap v941: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:18:29.119 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:28 smithi137 ceph-mon[31539]: pgmap v941: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:18:29.152 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-26T14:18:29.152 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-26T14:18:29.152 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=13 2024-03-26T14:18:29.152 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-26T14:18:29.152 DEBUG:tasks.check_counter:Getting stats from e 2024-03-26T14:18:29.152 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-26T14:18:29.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:28 smithi179 ceph-mon[35472]: pgmap v941: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-03-26T14:18:29.702 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T14:18:31.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:30 smithi137 ceph-mon[31539]: pgmap v942: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:31.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:30 smithi179 ceph-mon[35472]: pgmap v942: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:31.203 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:30 smithi159 ceph-mon[35501]: pgmap v942: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:32.176 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 399, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 391, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 817498, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 251474, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 14, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.124769755, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.041703014, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.058863567, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.013997781, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 390, 2024-03-26T14:18:32.177 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.044690491, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000114591 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 39, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000011794, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000302 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 811424, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 198128 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 78, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 315, 2024-03-26T14:18:32.178 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 15531, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1157455, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 66, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.083101567, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.022976588, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.050957234, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000444025, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 314, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.075616281, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000240816 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 58, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000012655, 2024-03-26T14:18:32.179 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000218 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 511440 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 3448, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2175, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 913509, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 508845, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 7, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.573493982, 2024-03-26T14:18:32.180 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.309521873, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.325227083, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.008364248, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2173, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.181619184, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000083579 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5570, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001513464, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000271 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:32.181 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 49, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002359216, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000048147 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-26T14:18:32.182 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000008027, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000004013 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 2, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 1, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000267969, 2024-03-26T14:18:32.183 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000267969 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 1, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 17, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 1, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 16, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 8, 2024-03-26T14:18:32.184 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 3, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 13, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 8, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 4, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-26T14:18:32.185 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-26T14:18:32.186 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 1 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 16, 2024-03-26T14:18:32.187 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 10, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 11, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-26T14:18:32.188 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 5, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 5, 2024-03-26T14:18:32.189 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 2, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4195917, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.010820931, 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002705232 2024-03-26T14:18:32.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 21, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 27, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 6, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 20, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 24, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 4, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 17, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 25, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 8, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 1, 2024-03-26T14:18:32.191 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 1, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 50816, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223612 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 2, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 2, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 16, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-26T14:18:32.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000267969, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000267969 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-26T14:18:32.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.197 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.198 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.199 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.201 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-26T14:18:32.203 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 1, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 1, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 969, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:32.204 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:32.205 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 240295, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 83, 2024-03-26T14:18:32.206 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9088, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 175, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 83773, 2024-03-26T14:18:32.207 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 335, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 40, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 11915, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 40, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 40, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.554989256, 2024-03-26T14:18:32.208 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.013874731 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 40, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 62 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 40, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 0, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 40, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 0, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 4, 2024-03-26T14:18:32.209 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 7, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 6, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-26T14:18:32.210 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-26T14:18:32.211 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 94, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 8, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 11, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 3, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 14, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-26T14:18:32.212 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 1 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-26T14:18:32.213 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 3921, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1371813, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 3921, 2024-03-26T14:18:32.214 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3921, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1371813, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:32.215 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 40, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 11915, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 40, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 11915, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-26T14:18:32.216 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 40, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 40, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 40, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 40, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-26T14:18:32.217 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55723c6f85a0": { 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 5, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1613, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 5, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1613, 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:32.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:32.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:32.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:32.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.220 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:32.220 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-26T14:18:32.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:32 smithi159 ceph-mon[35501]: pgmap v943: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:33.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:32 smithi179 ceph-mon[35472]: pgmap v943: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:33.205 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-26T14:18:33.205 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-26T14:18:33.205 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-26T14:18:33.205 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=16 2024-03-26T14:18:33.205 DEBUG:tasks.check_counter:Getting stats from h 2024-03-26T14:18:33.206 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-26T14:18:33.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:32 smithi137 ceph-mon[31539]: pgmap v943: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:33.693 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T14:18:35.116 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:34 smithi159 ceph-mon[35501]: pgmap v944: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:35.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:34 smithi179 ceph-mon[35472]: pgmap v944: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:35.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:34 smithi137 ceph-mon[31539]: pgmap v944: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:36.102 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-26T14:18:36.102 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:36.102 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 3464, 2024-03-26T14:18:36.102 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2199, 2024-03-26T14:18:36.102 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 916815, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 537495, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 11, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.599932944, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.313442616, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.337577002, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.008634795, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2197, 2024-03-26T14:18:36.103 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.180821334, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000082303 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5591, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001613994, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000288 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.104 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 31, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 268, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 6446, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 509422, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.037859578, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.022020431, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.010761127, 2024-03-26T14:18:36.105 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.000261106, 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 266, 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.053121382, 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000199704 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 36, 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000011663, 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000323 2024-03-26T14:18:36.106 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 456800 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 440, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 428, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 828995, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 820426, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 63, 2024-03-26T14:18:36.107 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.156945072, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.045305820, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.104691325, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.013102457, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 428, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.079728399, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000186281 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:36.108 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 73, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000018638, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000255 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 818384, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 198736 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-26T14:18:36.109 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 52, 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002035880, 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000039151 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:36.110 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000008967, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000004483 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 9, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 6, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.002454867, 2024-03-26T14:18:36.111 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000409144 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 3, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 17, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 1, 2024-03-26T14:18:36.112 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 16, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 8, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 3, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 3, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 3, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 21, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 16, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 1, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 4, 2024-03-26T14:18:36.113 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-26T14:18:36.114 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T14:18:36.115 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-03-26T14:18:36.116 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 18, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 11, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 12, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-03-26T14:18:36.117 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-26T14:18:36.118 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 5, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 5, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 2, 2024-03-26T14:18:36.119 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 2, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 4195917, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.016608566, 2024-03-26T14:18:36.120 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004152141 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 21, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 27, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 6, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 20, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 26, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 6, 2024-03-26T14:18:36.121 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 17, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 25, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 8, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 3, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 6, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 3, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 51072, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223612 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 9, 2024-03-26T14:18:36.122 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 9, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 16, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-26T14:18:36.123 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4, 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.001477821, 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000369455 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.124 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2, 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000977046, 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000488523 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-26T14:18:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-26T14:18:36.126 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.127 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-26T14:18:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-26T14:18:36.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.133 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.134 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-26T14:18:36.135 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.136 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-26T14:18:36.137 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 5, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 2, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 976, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:36.138 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:36.139 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:36.140 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 256923, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 97, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 176, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 2, 2024-03-26T14:18:36.141 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9088, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 175, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:36.142 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 84941, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 349, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 41, 2024-03-26T14:18:36.143 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 12134, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 41, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 41, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.583988701, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.014243626 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 41, 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 64 2024-03-26T14:18:36.144 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 41, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 41, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 4, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 7, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-26T14:18:36.145 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 6, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-26T14:18:36.146 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-26T14:18:36.147 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 94, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 9, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-03-26T14:18:36.148 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 3, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 16, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 1 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-26T14:18:36.149 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:36.150 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 3931, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1376584, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 3931, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3931, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1376584, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.151 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 41, 2024-03-26T14:18:36.152 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 12134, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 41, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 12134, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:36.153 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 41, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 41, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 41, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 41, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-26T14:18:36.155 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x56412390a5a0": { 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 5, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1613, 2024-03-26T14:18:36.156 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 5, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 4, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1613, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:36.157 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-26T14:18:36.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:36 smithi159 ceph-mon[35501]: pgmap v945: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:37.164 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-26T14:18:37.164 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-26T14:18:37.165 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-26T14:18:37.165 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=18 2024-03-26T14:18:37.165 DEBUG:tasks.check_counter:Getting stats from k 2024-03-26T14:18:37.165 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-26T14:18:37.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:36 smithi179 ceph-mon[35472]: pgmap v945: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:37.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:36 smithi137 ceph-mon[31539]: pgmap v945: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:37.670 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.b/config 2024-03-26T14:18:39.076 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:38 smithi159 ceph-mon[35501]: pgmap v946: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:39.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:38 smithi179 ceph-mon[35472]: pgmap v946: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:39.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:38 smithi137 ceph-mon[31539]: pgmap v946: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:39.992 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2270, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2270, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 4873567, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 721046, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.422325599, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.182013447, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.099866575, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.101600939, 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:39.993 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2269, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.139512228, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000061486 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 461, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 455, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 5158341, 2024-03-26T14:18:39.994 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 234360, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.122889322, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.044523612, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.061841574, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.017870859, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 454, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.036615932, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000080651 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:39.995 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 801312, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 194544 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1081, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1324, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 4460615, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 834625, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 3, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.210157139, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.077700920, 2024-03-26T14:18:39.996 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.052149216, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.057862320, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1322, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.086447236, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000065391 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-26T14:18:39.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 491184 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5770, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.246494923, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000042720 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-03-26T14:18:39.998 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-03-26T14:18:39.999 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 1235, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 1235, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 1235, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-03-26T14:18:40.000 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-03-26T14:18:40.001 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T14:18:40.002 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-03-26T14:18:40.003 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-03-26T14:18:40.004 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 6376, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 11, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-03-26T14:18:40.005 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 16907175, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 16902874, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 6376 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 23, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 1235, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 1212, 2024-03-26T14:18:40.006 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 18, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 24, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 6, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 1235, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 1242, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 7, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 52576, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215420 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-03-26T14:18:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.011 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.012 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.013 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.014 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.015 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.016 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-03-26T14:18:40.017 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 4, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 3, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 3918, 2024-03-26T14:18:40.018 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:40.019 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:40.020 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 190184, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 53, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-03-26T14:18:40.021 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9088, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 175, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 942522, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 5233, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-03-26T14:18:40.022 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 3471, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 26, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 4, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 3471, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3471, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.288936354, 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000947547 2024-03-26T14:18:40.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3471, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3472 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 3471, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 3471, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 2254, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1214, 2024-03-26T14:18:40.024 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-03-26T14:18:40.025 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-03-26T14:18:40.026 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 94, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 4, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 5, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 7, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 2, 2024-03-26T14:18:40.027 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T14:18:40.028 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 3808, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 14129494, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 3808, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3808, 2024-03-26T14:18:40.029 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 14129494, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:40.030 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3471, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 12712881, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3471, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 12712881, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:40.031 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 3471, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 3471, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 3471, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 3471, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:40.032 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55bc5593e780": { 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-03-26T14:18:40.033 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-03-26T14:18:40.034 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-03-26T14:18:41.006 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-26T14:18:41.006 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-26T14:18:41.006 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-26T14:18:41.006 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-03-26T14:18:41.006 DEBUG:tasks.check_counter:Getting stats from c 2024-03-26T14:18:41.007 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-26T14:18:41.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:40 smithi137 ceph-mon[31539]: pgmap v947: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:41.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:40 smithi179 ceph-mon[35472]: pgmap v947: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:41.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:40 smithi159 ceph-mon[35501]: pgmap v947: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:41.476 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T14:18:43.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:42 smithi179 ceph-mon[35472]: pgmap v948: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:43.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:42 smithi159 ceph-mon[35501]: pgmap v948: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:43.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:42 smithi137 ceph-mon[31539]: pgmap v948: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:43.800 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-03-26T14:18:43.800 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:43.800 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1138, 2024-03-26T14:18:43.800 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1140, 2024-03-26T14:18:43.800 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 300344, 2024-03-26T14:18:43.800 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 361948, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.235856793, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.080748455, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.062517110, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.063462483, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1138, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.104892759, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000092172 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.801 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1401, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1395, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1063306, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 532137, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.344399090, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.117115972, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.121807799, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.090324678, 2024-03-26T14:18:43.802 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1394, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.124392638, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000089234 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 800064, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 191872 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1136, 2024-03-26T14:18:43.803 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1374, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 383216, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 820634, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.267686525, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.107681929, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.062442599, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.064236232, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1373, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.145441713, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000105929 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:43.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 460976 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5350, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.287943499, 2024-03-26T14:18:43.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000053821 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-03-26T14:18:43.806 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-03-26T14:18:43.807 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 1, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-03-26T14:18:43.808 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T14:18:43.809 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-03-26T14:18:43.810 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T14:18:43.811 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 5, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-03-26T14:18:43.812 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195917, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 4195424, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 5 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-03-26T14:18:43.813 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 3, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 3, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 1, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 1, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 48896, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215420 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-03-26T14:18:43.814 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.815 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.816 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.817 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.818 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-03-26T14:18:43.819 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.820 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.821 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.822 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.823 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-03-26T14:18:43.824 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 2452, 2024-03-26T14:18:43.825 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:43.826 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:43.827 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 190167, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 52, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:43.828 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9088, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 175, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 8320, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 13, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-03-26T14:18:43.829 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3339, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 21, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 3, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3339, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3339, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.612918868, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001082036 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3339, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3340 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.830 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3339, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3339, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 2199, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1138, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-03-26T14:18:43.831 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 1, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-03-26T14:18:43.832 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-03-26T14:18:43.833 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 5, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 6, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-03-26T14:18:43.834 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3671, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1396421, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:43.835 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3671, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3671, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1396421, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:43.836 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3339, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 1618, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3339, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1618, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:43.837 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3339, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3339, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3339, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3339, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-03-26T14:18:43.838 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:43.839 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55f791f78780": { 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:43.840 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.841 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:43.841 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-03-26T14:18:44.661 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-26T14:18:44.661 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-26T14:18:44.661 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-03-26T14:18:44.661 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-03-26T14:18:44.661 DEBUG:tasks.check_counter:Getting stats from f 2024-03-26T14:18:44.662 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-26T14:18:45.087 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T14:18:45.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:44 smithi179 ceph-mon[35472]: pgmap v949: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:45.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:44 smithi159 ceph-mon[35501]: pgmap v949: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:45.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:44 smithi137 ceph-mon[31539]: pgmap v949: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:47.056 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:46 smithi179 ceph-mon[35472]: pgmap v950: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:47.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:46 smithi159 ceph-mon[35501]: pgmap v950: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:47.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:46 smithi137 ceph-mon[31539]: pgmap v950: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 323, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 318, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 780672, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 191600, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 0, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.120528754, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.046006731, 2024-03-26T14:18:47.409 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.062774359, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.015566347, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 317, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.033284161, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000104997 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 780672, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 191600 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:47.410 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 243, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 320, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 134736, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.029237446, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.021063391, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.002406977, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 241, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.027743593, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000115118 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.411 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 134736 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 4, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 4, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3040, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 448, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 2, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.001381929, 2024-03-26T14:18:47.412 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.000087285, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.001249358, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001541676, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000513892 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 448 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.413 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:47.414 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:47.415 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 148704, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 29, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:47.416 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 928, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 2, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:47.417 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 325, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 735045, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 325, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 325, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 735045, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:47.418 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-03-26T14:18:48.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:47 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:18:48.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:47 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:18:48.286 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-03-26T14:18:48.286 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-03-26T14:18:48.286 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-03-26T14:18:48.286 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-03-26T14:18:48.286 DEBUG:tasks.check_counter:Getting stats from i 2024-03-26T14:18:48.286 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-26T14:18:48.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:47 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-26T14:18:48.698 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: pgmap v951: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.168 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:48 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: pgmap v951: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:48 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: pgmap v951: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:49.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:48 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi179 to 176.7M 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: Unable to set osd_memory_target on smithi179 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: Adjusting osd_memory_target on smithi159 to 176.8M 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: Unable to set osd_memory_target on smithi159 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:18:50.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:49 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi179 to 176.7M 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: Unable to set osd_memory_target on smithi179 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: Adjusting osd_memory_target on smithi159 to 176.8M 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: Unable to set osd_memory_target on smithi159 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:18:50.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:49 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:18:50.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi179 to 176.7M 2024-03-26T14:18:50.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: Unable to set osd_memory_target on smithi179 to 185372364: error parsing value: Value '185372364' is below minimum 939524096 2024-03-26T14:18:50.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: Adjusting osd_memory_target on smithi159 to 176.8M 2024-03-26T14:18:50.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: Unable to set osd_memory_target on smithi159 to 185389568: error parsing value: Value '185389568' is below minimum 939524096 2024-03-26T14:18:50.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-26T14:18:50.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-26T14:18:50.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:49 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' 2024-03-26T14:18:50.782 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 3271, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 3273, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 927530, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 1040185, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 6, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.625882197, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.226507742, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.157291263, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.189013253, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3271, 2024-03-26T14:18:50.783 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.265569649, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000081189 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 8, 2024-03-26T14:18:50.784 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 246, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 3916, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 518714, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.032880468, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.022105457, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.004818630, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000079092, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 245, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.030079656, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000122774 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.785 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 518016 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 332, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 326, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 800332, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 192206, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.121120656, 2024-03-26T14:18:50.786 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.046839589, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.064352526, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.015684530, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 325, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.033518113, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000103132 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 800064, 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 191888 2024-03-26T14:18:50.787 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5552, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.278781687, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000050212 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-03-26T14:18:50.788 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.789 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-03-26T14:18:50.790 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 1, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-03-26T14:18:50.791 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-03-26T14:18:50.792 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-03-26T14:18:50.793 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-03-26T14:18:50.794 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 5, 2024-03-26T14:18:50.795 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195917, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 4195776, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.796 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 5 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 3, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 3, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 0, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 1, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 1, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 0, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 0, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 0, 2024-03-26T14:18:50.797 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 49408, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215420 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.798 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.799 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.800 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-03-26T14:18:50.801 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.802 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.803 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-03-26T14:18:50.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.803 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.803 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.803 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.804 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.805 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.806 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.807 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.808 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.809 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.810 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 2, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 3935, 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-03-26T14:18:50.811 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:50.812 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:50.813 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:50.814 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 182087, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 48, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9088, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 175, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:50.815 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 8320, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 13, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3274, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 21, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 3, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3274, 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-03-26T14:18:50.816 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3274, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3.504921311, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001070531 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3274, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3275 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3274, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3274, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 2127, 2024-03-26T14:18:50.817 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1145, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-03-26T14:18:50.818 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 1, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-03-26T14:18:50.819 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-03-26T14:18:50.820 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 94, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-03-26T14:18:50.821 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 4, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 4, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 5, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-03-26T14:18:50.822 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:50.823 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3607, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1387050, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3607, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3607, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1387050, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-03-26T14:18:50.824 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3274, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 1618, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3274, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1618, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.825 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3274, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3274, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3274, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3274, 2024-03-26T14:18:50.826 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-03-26T14:18:50.827 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x558417f42780": { 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:50.828 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:50.829 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-03-26T14:18:51.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:50 smithi179 ceph-mon[35472]: pgmap v952: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:51.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:50 smithi137 ceph-mon[31539]: pgmap v952: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:51.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:50 smithi159 ceph-mon[35501]: pgmap v952: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:51.723 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-26T14:18:51.723 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-26T14:18:51.723 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-03-26T14:18:51.723 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-03-26T14:18:51.723 DEBUG:tasks.check_counter:Getting stats from l 2024-03-26T14:18:51.723 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033 shell --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-26T14:18:52.188 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/mon.c/config 2024-03-26T14:18:53.282 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:52 smithi179 ceph-mon[35472]: pgmap v953: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:53.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:52 smithi137 ceph-mon[31539]: pgmap v953: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:53.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:52 smithi159 ceph-mon[35501]: pgmap v953: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:18:54.520 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-03-26T14:18:54.520 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-03-26T14:18:54.520 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 0, 2024-03-26T14:18:54.520 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 2, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 0, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 0, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 0, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.003352594, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.000000000, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.000670290, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:54.521 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.522 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 334, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 327, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 785616, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 196416, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 2, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2024-03-26T14:18:54.523 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.122558627, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.047891125, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.066583010, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.015847496, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 327, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.035529359, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000108652 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:54.524 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 785616, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 196416 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 2, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 249, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 320, 2024-03-26T14:18:54.525 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 135792, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.030929831, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.022415693, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.002193210, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-03-26T14:18:54.526 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 247, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.028733302, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000116329 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-26T14:18:54.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 135792 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 0, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-03-26T14:18:54.528 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-03-26T14:18:54.529 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-03-26T14:18:54.530 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-03-26T14:18:54.531 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 148784, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 31, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-03-26T14:18:54.532 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 928, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 2, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-03-26T14:18:54.533 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 332, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 735955, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 332, 2024-03-26T14:18:54.534 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 332, 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 735955, 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-03-26T14:18:54.535 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-03-26T14:18:55.238 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:54 smithi179 ceph-mon[35472]: pgmap v954: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:55.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:54 smithi137 ceph-mon[31539]: pgmap v954: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:55.313 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-03-26T14:18:55.313 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-03-26T14:18:55.313 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-03-26T14:18:55.313 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-03-26T14:18:55.314 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-26T14:18:55.324 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-26T14:18:55.325 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T14:18:55.325 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T14:18:55.383 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T14:18:55.383 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T14:18:55.437 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:18:55.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:54 smithi159 ceph-mon[35501]: pgmap v954: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 -- 172.21.15.137:0/3138142643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5141038e0 msgr2=0x7fd514103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 --2- 172.21.15.137:0/3138142643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5141038e0 0x7fd514103d00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd504009ca0 tx=0x7fd50402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 -- 172.21.15.137:0/3138142643 shutdown_connections 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 --2- 172.21.15.137:0/3138142643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 0x7fd51410bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 --2- 172.21.15.137:0/3138142643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd514104ad0 0x7fd514104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 --2- 172.21.15.137:0/3138142643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd5141038e0 0x7fd514103d00 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 -- 172.21.15.137:0/3138142643 >> 172.21.15.137:0/3138142643 conn(0x7fd5140fee50 msgr2=0x7fd514101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 -- 172.21.15.137:0/3138142643 shutdown_connections 2024-03-26T14:18:55.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.646+0000 7fd519c00640 1 -- 172.21.15.137:0/3138142643 wait complete. 2024-03-26T14:18:55.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.647+0000 7fd519c00640 1 Processor -- start 2024-03-26T14:18:55.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.647+0000 7fd519c00640 1 -- start start 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd519c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5141038e0 0x7fd514199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd519c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 0x7fd51419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd519c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 0x7fd5141a1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd519c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51410f400 con 0x7fd514104ad0 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd519c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51410f280 con 0x7fd514105490 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd519c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51410f580 con 0x7fd5141038e0 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 0x7fd51419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 0x7fd51419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35828/0 (socket says 172.21.15.137:35828) 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 -- 172.21.15.137:0/2795883933 learned_addr learned my addr 172.21.15.137:0/2795883933 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd513400640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 0x7fd5141a1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 -- 172.21.15.137:0/2795883933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5141038e0 msgr2=0x7fd514199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:55.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512a00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5141038e0 0x7fd514199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5141038e0 0x7fd514199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 -- 172.21.15.137:0/2795883933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 msgr2=0x7fd5141a1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 0x7fd5141a1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd512000640 1 -- 172.21.15.137:0/2795883933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5141a1770 con 0x7fd514104ad0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.648+0000 7fd513400640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 0x7fd5141a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.649+0000 7fd512000640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 0x7fd51419a090 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00d770 tx=0x7fd4fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.649+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc017070 con 0x7fd514104ad0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.649+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd4fc0054d0 con 0x7fd514104ad0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.649+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5141a1a60 con 0x7fd514104ad0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.649+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc004cf0 con 0x7fd514104ad0 2024-03-26T14:18:55.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.649+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5141a1f60 con 0x7fd514104ad0 2024-03-26T14:18:55.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.650+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51410acd0 con 0x7fd514104ad0 2024-03-26T14:18:55.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.652+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd4fc004e90 con 0x7fd514104ad0 2024-03-26T14:18:55.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.653+0000 7fd50b400640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4f80a2800 0x7fd4f80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:55.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.654+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd4fc0ce940 con 0x7fd514104ad0 2024-03-26T14:18:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.654+0000 7fd512a00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4f80a2800 0x7fd4f80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:55.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.654+0000 7fd512a00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4f80a2800 0x7fd4f80a4cc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd504037040 tx=0x7fd504032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:55.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.660+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4fc09b610 con 0x7fd514104ad0 2024-03-26T14:18:55.864 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.863+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5140622a0 con 0x7fd514104ad0 2024-03-26T14:18:55.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.866+0000 7fd50b400640 1 -- 172.21.15.137:0/2795883933 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7fd4fc09af10 con 0x7fd514104ad0 2024-03-26T14:18:55.868 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:18:55.868 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4f80a2800 msgr2=0x7fd4f80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4f80a2800 0x7fd4f80a4cc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd504037040 tx=0x7fd504032340 comp rx=0 tx=0).stop 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 msgr2=0x7fd51419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 0x7fd51419a090 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00d770 tx=0x7fd4fc00dc30 comp rx=0 tx=0).stop 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 shutdown_connections 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4f80a2800 0x7fd4f80a4cc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd514105490 0x7fd5141a1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd514104ad0 0x7fd51419a090 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 --2- 172.21.15.137:0/2795883933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd5141038e0 0x7fd514199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:55.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.869+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 >> 172.21.15.137:0/2795883933 conn(0x7fd5140fee50 msgr2=0x7fd514107ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:55.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.870+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 shutdown_connections 2024-03-26T14:18:55.872 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:55.870+0000 7fd519c00640 1 -- 172.21.15.137:0/2795883933 wait complete. 2024-03-26T14:18:55.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:18:55.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 -- 172.21.15.137:0/748208907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c105470 msgr2=0x7f066c10bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 --2- 172.21.15.137:0/748208907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c105470 0x7f066c10bd00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f0664009900 tx=0x7f066402bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 -- 172.21.15.137:0/748208907 shutdown_connections 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 --2- 172.21.15.137:0/748208907 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c105470 0x7f066c10bd00 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 --2- 172.21.15.137:0/748208907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f066c104ab0 0x7f066c104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 --2- 172.21.15.137:0/748208907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c1038c0 0x7f066c103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 -- 172.21.15.137:0/748208907 >> 172.21.15.137:0/748208907 conn(0x7f066c0fee50 msgr2=0x7f066c101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 -- 172.21.15.137:0/748208907 shutdown_connections 2024-03-26T14:18:56.110 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.108+0000 7f0673e00640 1 -- 172.21.15.137:0/748208907 wait complete. 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 Processor -- start 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 -- start start 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 0x7f066c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f066c104ab0 0x7f066c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c105470 0x7f066c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c10f440 con 0x7f066c1038c0 2024-03-26T14:18:56.112 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c10f2c0 con 0x7f066c105470 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0673e00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c10f5c0 con 0x7f066c104ab0 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0671400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 0x7f066c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0671400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 0x7f066c199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35840/0 (socket says 172.21.15.137:35840) 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.110+0000 7f0671400640 1 -- 172.21.15.137:0/3574388064 learned_addr learned my addr 172.21.15.137:0/3574388064 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0670a00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f066c104ab0 0x7f066c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c105470 0x7f066c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671400640 1 -- 172.21.15.137:0/3574388064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f066c104ab0 msgr2=0x7f066c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671400640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f066c104ab0 0x7f066c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671400640 1 -- 172.21.15.137:0/3574388064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c105470 msgr2=0x7f066c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671400640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c105470 0x7f066c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671400640 1 -- 172.21.15.137:0/3574388064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c1a1620 con 0x7f066c1038c0 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c105470 0x7f066c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:18:56.113 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f0671400640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 0x7f066c199a00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c970 tx=0x7f065c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:56.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c007c10 con 0x7f066c1038c0 2024-03-26T14:18:56.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.111+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f065c007db0 con 0x7f066c1038c0 2024-03-26T14:18:56.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.112+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c00f870 con 0x7f066c1038c0 2024-03-26T14:18:56.114 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.112+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c1a1910 con 0x7f066c1038c0 2024-03-26T14:18:56.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.112+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c1a1e10 con 0x7f066c1038c0 2024-03-26T14:18:56.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.113+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066c10ab80 con 0x7f066c1038c0 2024-03-26T14:18:56.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.115+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f065c00fa10 con 0x7f066c1038c0 2024-03-26T14:18:56.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.117+0000 7f066a000640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f06500a2800 0x7f06500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.117+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f065c0cf2e0 con 0x7f066c1038c0 2024-03-26T14:18:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.118+0000 7f0670a00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f06500a2800 0x7f06500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.118+0000 7f0670a00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f06500a2800 0x7f06500a4cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f066c19af10 tx=0x7f0658009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:56.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.121+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f065c09c000 con 0x7f066c1038c0 2024-03-26T14:18:56.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:55 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2795883933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:56.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.320+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f066c02d2f0 con 0x7f066c1038c0 2024-03-26T14:18:56.322 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.321+0000 7f066a000640 1 -- 172.21.15.137:0/3574388064 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f065c09b900 con 0x7f066c1038c0 2024-03-26T14:18:56.322 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f06500a2800 msgr2=0x7f06500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f06500a2800 0x7f06500a4cc0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f066c19af10 tx=0x7f0658009870 comp rx=0 tx=0).stop 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 msgr2=0x7f066c199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 0x7f066c199a00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f065c00c970 tx=0x7f065c00ce30 comp rx=0 tx=0).stop 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 shutdown_connections 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f06500a2800 0x7f06500a4cc0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f066c105470 0x7f066c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f066c104ab0 0x7f066c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 --2- 172.21.15.137:0/3574388064 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f066c1038c0 0x7f066c199a00 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.323+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 >> 172.21.15.137:0/3574388064 conn(0x7f066c0fee50 msgr2=0x7f066c107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.324+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 shutdown_connections 2024-03-26T14:18:56.325 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.324+0000 7f0673e00640 1 -- 172.21.15.137:0/3574388064 wait complete. 2024-03-26T14:18:56.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:18:56.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:55 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2795883933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:56.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:55 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2795883933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.553+0000 7f1466800640 1 -- 172.21.15.137:0/2385817390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14601038e0 msgr2=0x7f1460103d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.553+0000 7f1466800640 1 --2- 172.21.15.137:0/2385817390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14601038e0 0x7f1460103d00 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f1450009900 tx=0x7f145002bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 -- 172.21.15.137:0/2385817390 shutdown_connections 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 --2- 172.21.15.137:0/2385817390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460104ad0 0x7f1460104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 --2- 172.21.15.137:0/2385817390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1460105490 0x7f146010bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 --2- 172.21.15.137:0/2385817390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f14601038e0 0x7f1460103d00 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 -- 172.21.15.137:0/2385817390 >> 172.21.15.137:0/2385817390 conn(0x7f14600fee50 msgr2=0x7f1460101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 -- 172.21.15.137:0/2385817390 shutdown_connections 2024-03-26T14:18:56.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.554+0000 7f1466800640 1 -- 172.21.15.137:0/2385817390 wait complete. 2024-03-26T14:18:56.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 Processor -- start 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 -- start start 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14601038e0 0x7f1460199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 0x7f146019a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460105490 0x7f14601a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146010f460 con 0x7f1460104ad0 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146010f2e0 con 0x7f1460105490 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1466800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146010f5e0 con 0x7f14601038e0 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1457e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 0x7f146019a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.555+0000 7f1457e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 0x7f146019a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35856/0 (socket says 172.21.15.137:35856) 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 -- 172.21.15.137:0/2487328521 learned_addr learned my addr 172.21.15.137:0/2487328521 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f145fe00640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460105490 0x7f14601a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f145f400640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14601038e0 0x7f1460199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.558 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 -- 172.21.15.137:0/2487328521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14601038e0 msgr2=0x7f1460199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14601038e0 0x7f1460199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 -- 172.21.15.137:0/2487328521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460105490 msgr2=0x7f14601a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460105490 0x7f14601a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 -- 172.21.15.137:0/2487328521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14601a1760 con 0x7f1460104ad0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f1457e00640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 0x7f146019a080 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f144800d950 tx=0x7f144800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448017070 con 0x7f1460104ad0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.556+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1448009c30 con 0x7f1460104ad0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.557+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144800f620 con 0x7f1460104ad0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.557+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14601a1a50 con 0x7f1460104ad0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.557+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14601a1ff0 con 0x7f1460104ad0 2024-03-26T14:18:56.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.558+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146010ac80 con 0x7f1460104ad0 2024-03-26T14:18:56.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.560+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f14480109a0 con 0x7f1460104ad0 2024-03-26T14:18:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.561+0000 7f145cc00640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f14440a2880 0x7f14440a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.561+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f14480ceae0 con 0x7f1460104ad0 2024-03-26T14:18:56.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.561+0000 7f145f400640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f14440a2880 0x7f14440a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.562+0000 7f145f400640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f14440a2880 0x7f14440a4d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f145002c240 tx=0x7f1450005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:56.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.565+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1448009de0 con 0x7f1460104ad0 2024-03-26T14:18:56.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.761+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14600622d0 con 0x7f1460104ad0 2024-03-26T14:18:56.763 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.762+0000 7f145cc00640 1 -- 172.21.15.137:0/2487328521 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f144809b770 con 0x7f1460104ad0 2024-03-26T14:18:56.763 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:18:56.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f14440a2880 msgr2=0x7f14440a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f14440a2880 0x7f14440a4d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f145002c240 tx=0x7f1450005fd0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.766 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 msgr2=0x7f146019a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 0x7f146019a080 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f144800d950 tx=0x7f144800de10 comp rx=0 tx=0).stop 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 shutdown_connections 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f14440a2880 0x7f14440a4d40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460105490 0x7f14601a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1460104ad0 0x7f146019a080 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 --2- 172.21.15.137:0/2487328521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14601038e0 0x7f1460199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 >> 172.21.15.137:0/2487328521 conn(0x7f14600fee50 msgr2=0x7f1460107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 shutdown_connections 2024-03-26T14:18:56.767 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.765+0000 7f1466800640 1 -- 172.21.15.137:0/2487328521 wait complete. 2024-03-26T14:18:56.785 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 -- 172.21.15.137:0/3496156612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc104ad0 msgr2=0x7fe7cc104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3496156612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc104ad0 0x7fe7cc104f50 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009900 tx=0x7fe7b402bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 -- 172.21.15.137:0/3496156612 shutdown_connections 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3496156612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7cc105490 0x7fe7cc10bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3496156612 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc104ad0 0x7fe7cc104f50 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3496156612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7cc1038e0 0x7fe7cc103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.994+0000 7fe7d3200640 1 -- 172.21.15.137:0/3496156612 >> 172.21.15.137:0/3496156612 conn(0x7fe7cc0fee50 msgr2=0x7fe7cc101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:56.996 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.995+0000 7fe7d3200640 1 -- 172.21.15.137:0/3496156612 shutdown_connections 2024-03-26T14:18:56.997 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.995+0000 7fe7d3200640 1 -- 172.21.15.137:0/3496156612 wait complete. 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.995+0000 7fe7d3200640 1 Processor -- start 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 -- start start 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7cc1038e0 0x7fe7cc199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7cc104ad0 0x7fe7cc19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 0x7fe7cc1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc10f4d0 con 0x7fe7cc105490 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc10f350 con 0x7fe7cc1038e0 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d3200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc10f650 con 0x7fe7cc104ad0 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d1200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 0x7fe7cc1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d1200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 0x7fe7cc1a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35880/0 (socket says 172.21.15.137:35880) 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d1200640 1 -- 172.21.15.137:0/3695655802 learned_addr learned my addr 172.21.15.137:0/3695655802 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7cbe00640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7cc1038e0 0x7fe7cc199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:56.999 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7c3400640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7cc104ad0 0x7fe7cc19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d1200640 1 -- 172.21.15.137:0/3695655802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7cc104ad0 msgr2=0x7fe7cc19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.996+0000 7fe7d1200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7cc104ad0 0x7fe7cc19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7d1200640 1 -- 172.21.15.137:0/3695655802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7cc1038e0 msgr2=0x7fe7cc199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7d1200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7cc1038e0 0x7fe7cc199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7d1200640 1 -- 172.21.15.137:0/3695655802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc1a17d0 con 0x7fe7cc105490 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7d1200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 0x7fe7cc1a1160 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400eaa0 tx=0x7fe7c400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4004d20 con 0x7fe7cc105490 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc1a1ac0 con 0x7fe7cc105490 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe7c4005020 con 0x7fe7cc105490 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40116b0 con 0x7fe7cc105490 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.997+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc1a1fe0 con 0x7fe7cc105490 2024-03-26T14:18:57.000 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:56.998+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc10ad20 con 0x7fe7cc105490 2024-03-26T14:18:57.001 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.000+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe7c40051d0 con 0x7fe7cc105490 2024-03-26T14:18:57.002 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.001+0000 7fe7c9600640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7b00a2800 0x7fe7b00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.002+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fe7c40cf0f0 con 0x7fe7cc105490 2024-03-26T14:18:57.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.002+0000 7fe7cbe00640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7b00a2800 0x7fe7b00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.003 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.002+0000 7fe7cbe00640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7b00a2800 0x7fe7b00a4cc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc104930 tx=0x7fe7bc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:57.007 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.006+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe7c4016030 con 0x7fe7cc105490 2024-03-26T14:18:57.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:56 smithi137 ceph-mon[31539]: pgmap v955: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:57.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3574388064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:18:57.036 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:56 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2487328521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:18:57.202 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.201+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7cc0622a0 con 0x7fe7cc105490 2024-03-26T14:18:57.203 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.202+0000 7fe7c9600640 1 -- 172.21.15.137:0/3695655802 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7c409bd80 con 0x7fe7cc105490 2024-03-26T14:18:57.203 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:18:57.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:56 smithi159 ceph-mon[35501]: pgmap v955: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:57.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3574388064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:18:57.204 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:56 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2487328521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:18:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.204+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7b00a2800 msgr2=0x7fe7b00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.204+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7b00a2800 0x7fe7b00a4cc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc104930 tx=0x7fe7bc009110 comp rx=0 tx=0).stop 2024-03-26T14:18:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.204+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 msgr2=0x7fe7cc1a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.204+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 0x7fe7cc1a1160 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400eaa0 tx=0x7fe7c400eea0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 shutdown_connections 2024-03-26T14:18:57.206 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fe7b00a2800 0x7fe7b00a4cc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe7cc105490 0x7fe7cc1a1160 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe7cc104ad0 0x7fe7cc19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 --2- 172.21.15.137:0/3695655802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7cc1038e0 0x7fe7cc199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 >> 172.21.15.137:0/3695655802 conn(0x7fe7cc0fee50 msgr2=0x7fe7cc107ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:57.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 shutdown_connections 2024-03-26T14:18:57.207 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.205+0000 7fe7d3200640 1 -- 172.21.15.137:0/3695655802 wait complete. 2024-03-26T14:18:57.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:18:57.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:56 smithi179 ceph-mon[35472]: pgmap v955: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:18:57.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3574388064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:18:57.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:56 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2487328521' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:18:57.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 -- 172.21.15.137:0/979667517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500077c40 msgr2=0x7fd5000784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 --2- 172.21.15.137:0/979667517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500077c40 0x7fd5000784e0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009ca0 tx=0x7fd4f802c100 comp rx=0 tx=0).stop 2024-03-26T14:18:57.435 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 -- 172.21.15.137:0/979667517 shutdown_connections 2024-03-26T14:18:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 --2- 172.21.15.137:0/979667517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5000772a0 0x7fd500077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 --2- 172.21.15.137:0/979667517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500077c40 0x7fd5000784e0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 --2- 172.21.15.137:0/979667517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500076060 0x7fd500076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 -- 172.21.15.137:0/979667517 >> 172.21.15.137:0/979667517 conn(0x7fd5000fee50 msgr2=0x7fd500101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 -- 172.21.15.137:0/979667517 shutdown_connections 2024-03-26T14:18:57.436 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.434+0000 7fd507800640 1 -- 172.21.15.137:0/979667517 wait complete. 2024-03-26T14:18:57.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 Processor -- start 2024-03-26T14:18:57.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 -- start start 2024-03-26T14:18:57.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 0x7fd50019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5000772a0 0x7fd50019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.438 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500077c40 0x7fd5001a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd500113820 con 0x7fd500076060 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5001136a0 con 0x7fd5000772a0 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.435+0000 7fd507800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5001139a0 con 0x7fd500077c40 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 0x7fd50019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 0x7fd50019deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35902/0 (socket says 172.21.15.137:35902) 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 -- 172.21.15.137:0/677708360 learned_addr learned my addr 172.21.15.137:0/677708360 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd4ffe00640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5000772a0 0x7fd50019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 -- 172.21.15.137:0/677708360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500077c40 msgr2=0x7fd5001a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd505800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500077c40 0x7fd5001a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500077c40 0x7fd5001a5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 -- 172.21.15.137:0/677708360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5000772a0 msgr2=0x7fd50019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5000772a0 0x7fd50019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 -- 172.21.15.137:0/677708360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5001a5ad0 con 0x7fd500076060 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.436+0000 7fd504e00640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 0x7fd50019deb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400d950 tx=0x7fd4f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:57.439 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.437+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4017070 con 0x7fd500076060 2024-03-26T14:18:57.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.437+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd4f4009c30 con 0x7fd500076060 2024-03-26T14:18:57.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.437+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5001a5dc0 con 0x7fd500076060 2024-03-26T14:18:57.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.437+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400f620 con 0x7fd500076060 2024-03-26T14:18:57.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.437+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5001a6230 con 0x7fd500076060 2024-03-26T14:18:57.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.438+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd500110250 con 0x7fd500076060 2024-03-26T14:18:57.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.440+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd4f40109a0 con 0x7fd500076060 2024-03-26T14:18:57.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.441+0000 7fd4fd600640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.441+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fd4f40ceb10 con 0x7fd500076060 2024-03-26T14:18:57.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.441+0000 7fd4ffe00640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.443 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.442+0000 7fd4ffe00640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec005fd0 tx=0x7fd4ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:57.446 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.445+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd4f4009de0 con 0x7fd500076060 2024-03-26T14:18:57.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.646+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd500061ae0 con 0x7fd500076060 2024-03-26T14:18:57.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.647+0000 7fd4fd600640 1 -- 172.21.15.137:0/677708360 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd4f409b7a0 con 0x7fd500076060 2024-03-26T14:18:57.648 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.649+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4e80a2800 msgr2=0x7fd4e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.649+0000 7fd507800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec005fd0 tx=0x7fd4ec005850 comp rx=0 tx=0).stop 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.649+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 msgr2=0x7fd50019deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.649+0000 7fd507800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 0x7fd50019deb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400d950 tx=0x7fd4f400de10 comp rx=0 tx=0).stop 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 shutdown_connections 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fd4e80a2800 0x7fd4e80a4cc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5000772a0 0x7fd50019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd500077c40 0x7fd5001a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 --2- 172.21.15.137:0/677708360 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd500076060 0x7fd50019deb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.651 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 >> 172.21.15.137:0/677708360 conn(0x7fd5000fee50 msgr2=0x7fd500100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:57.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 shutdown_connections 2024-03-26T14:18:57.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.650+0000 7fd507800640 1 -- 172.21.15.137:0/677708360 wait complete. 2024-03-26T14:18:57.671 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T14:18:57.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 -- 172.21.15.137:0/3321531092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00076060 msgr2=0x7ffa00076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 --2- 172.21.15.137:0/3321531092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00076060 0x7ffa00076480 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4009900 tx=0x7ff9f402bc30 comp rx=0 tx=0).stop 2024-03-26T14:18:57.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 -- 172.21.15.137:0/3321531092 shutdown_connections 2024-03-26T14:18:57.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 --2- 172.21.15.137:0/3321531092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00077c40 0x7ffa000784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.878 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 --2- 172.21.15.137:0/3321531092 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa000772a0 0x7ffa00077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 --2- 172.21.15.137:0/3321531092 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa00076060 0x7ffa00076480 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 -- 172.21.15.137:0/3321531092 >> 172.21.15.137:0/3321531092 conn(0x7ffa000fee50 msgr2=0x7ffa00101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:57.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 -- 172.21.15.137:0/3321531092 shutdown_connections 2024-03-26T14:18:57.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.877+0000 7ffa08400640 1 -- 172.21.15.137:0/3321531092 wait complete. 2024-03-26T14:18:57.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.878+0000 7ffa08400640 1 Processor -- start 2024-03-26T14:18:57.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.878+0000 7ffa08400640 1 -- start start 2024-03-26T14:18:57.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa08400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00076060 0x7ffa0019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa08400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 0x7ffa0019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa08400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa00077c40 0x7ffa001a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.880 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa08400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00113860 con 0x7ffa000772a0 2024-03-26T14:18:57.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa08400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001136e0 con 0x7ffa00076060 2024-03-26T14:18:57.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa08400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001139e0 con 0x7ffa00077c40 2024-03-26T14:18:57.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa05000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 0x7ffa0019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa05000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 0x7ffa0019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35932/0 (socket says 172.21.15.137:35932) 2024-03-26T14:18:57.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa05000640 1 -- 172.21.15.137:0/3044806731 learned_addr learned my addr 172.21.15.137:0/3044806731 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:57.881 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa05a00640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00076060 0x7ffa0019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa06400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa00077c40 0x7ffa001a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa05000640 1 -- 172.21.15.137:0/3044806731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa00077c40 msgr2=0x7ffa001a5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.879+0000 7ffa05000640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa00077c40 0x7ffa001a5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa05000640 1 -- 172.21.15.137:0/3044806731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00076060 msgr2=0x7ffa0019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:57.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa05000640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00076060 0x7ffa0019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:57.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa05000640 1 -- 172.21.15.137:0/3044806731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa001a5b90 con 0x7ffa000772a0 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa06400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa00077c40 0x7ffa001a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa05000640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 0x7ffa0019e4b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00d770 tx=0x7ff9ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec017070 con 0x7ffa000772a0 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff9ec0054d0 con 0x7ffa000772a0 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa001a5e80 con 0x7ffa000772a0 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec004cf0 con 0x7ffa000772a0 2024-03-26T14:18:57.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.880+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001a6340 con 0x7ffa000772a0 2024-03-26T14:18:57.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.881+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa00110250 con 0x7ffa000772a0 2024-03-26T14:18:57.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.884+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff9ec004e90 con 0x7ffa000772a0 2024-03-26T14:18:57.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.885+0000 7ff9fe000640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9e80a2880 0x7ff9e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:57.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.885+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7ff9ec0ce820 con 0x7ffa000772a0 2024-03-26T14:18:57.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.886+0000 7ffa05a00640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9e80a2880 0x7ff9e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:57.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.886+0000 7ffa05a00640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9e80a2880 0x7ff9e80a4d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4000c00 tx=0x7ff9f402c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:57.891 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:57.890+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff9ec09b4b0 con 0x7ffa000772a0 2024-03-26T14:18:58.104 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.103+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa00062340 con 0x7ffa000772a0 2024-03-26T14:18:58.105 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.104+0000 7ff9fe000640 1 -- 172.21.15.137:0/3044806731 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+600 (secure 0 0 0) 0x7ff9ec09adb0 con 0x7ffa000772a0 2024-03-26T14:18:58.105 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T14:18:58.105 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T14:18:58.105 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/460394311 2024-03-27T13:49:52.910355+0000 2024-03-26T14:18:58.105 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1426358112 2024-03-27T13:49:19.001493+0000 2024-03-26T14:18:58.105 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2012111099 2024-03-27T13:49:52.910355+0000 2024-03-26T14:18:58.106 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1602182932 2024-03-27T13:49:19.001493+0000 2024-03-26T14:18:58.106 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T14:18:58.106 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3592180121 2024-03-27T13:49:19.001493+0000 2024-03-26T14:18:58.106 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T14:18:58.106 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/487629871 2024-03-27T13:49:52.910355+0000 2024-03-26T14:18:58.106 INFO:teuthology.orchestra.run.smithi137.stderr:listed 10 entries 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9e80a2880 msgr2=0x7ff9e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9e80a2880 0x7ff9e80a4d40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4000c00 tx=0x7ff9f402c7e0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 msgr2=0x7ffa0019e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 0x7ffa0019e4b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00d770 tx=0x7ff9ec00dc30 comp rx=0 tx=0).stop 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 shutdown_connections 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7ff9e80a2880 0x7ff9e80a4d40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa00077c40 0x7ffa001a5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.108 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffa000772a0 0x7ffa0019e4b0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 --2- 172.21.15.137:0/3044806731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa00076060 0x7ffa0019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 >> 172.21.15.137:0/3044806731 conn(0x7ffa000fee50 msgr2=0x7ffa001021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:58.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 shutdown_connections 2024-03-26T14:18:58.109 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.107+0000 7ffa08400640 1 -- 172.21.15.137:0/3044806731 wait complete. 2024-03-26T14:18:58.127 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-03-26T14:18:58.127 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-26T14:18:58.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:18:58.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:57 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3695655802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:18:58.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:57 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/677708360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:18:58.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:57 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3695655802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:18:58.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:57 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/677708360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.435+0000 7f52b9600640 1 -- 172.21.15.137:0/562707205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b4108910 msgr2=0x7f52b410ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.435+0000 7f52b9600640 1 --2- 172.21.15.137:0/562707205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b4108910 0x7f52b410ad00 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f52a400aa10 tx=0x7f52a402cf90 comp rx=0 tx=0).stop 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 -- 172.21.15.137:0/562707205 shutdown_connections 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 --2- 172.21.15.137:0/562707205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b410dd70 0x7f52b4110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 --2- 172.21.15.137:0/562707205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52b410b2d0 0x7f52b410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 --2- 172.21.15.137:0/562707205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b4108910 0x7f52b410ad00 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 -- 172.21.15.137:0/562707205 >> 172.21.15.137:0/562707205 conn(0x7f52b406e8b0 msgr2=0x7f52b4070cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 -- 172.21.15.137:0/562707205 shutdown_connections 2024-03-26T14:18:58.437 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.436+0000 7f52b9600640 1 -- 172.21.15.137:0/562707205 wait complete. 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 Processor -- start 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 -- start start 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52b4108910 0x7f52b41a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b410b2d0 0x7f52b41a2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 0x7f52b41a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4115a60 con 0x7f52b410dd70 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b41158e0 con 0x7f52b410b2d0 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b9600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b4115be0 con 0x7f52b4108910 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 0x7f52b41a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 0x7f52b41a9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53916/0 (socket says 172.21.15.137:53916) 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.437+0000 7f52b3400640 1 -- 172.21.15.137:0/3058620099 learned_addr learned my addr 172.21.15.137:0/3058620099 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:58.440 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b2000640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b410b2d0 0x7f52b41a2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b2a00640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52b4108910 0x7f52b41a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b3400640 1 -- 172.21.15.137:0/3058620099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52b4108910 msgr2=0x7f52b41a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b3400640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52b4108910 0x7f52b41a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b3400640 1 -- 172.21.15.137:0/3058620099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b410b2d0 msgr2=0x7f52b41a2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b3400640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b410b2d0 0x7f52b41a2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b3400640 1 -- 172.21.15.137:0/3058620099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b41aa000 con 0x7f52b410dd70 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b3400640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 0x7f52b41a9990 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c970 tx=0x7f52ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac007bb0 con 0x7f52b410dd70 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.438+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b41aa2f0 con 0x7f52b410dd70 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.439+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f52ac007d50 con 0x7f52b410dd70 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.439+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b41aa8c0 con 0x7f52b410dd70 2024-03-26T14:18:58.441 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.439+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac01d060 con 0x7f52b410dd70 2024-03-26T14:18:58.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.440+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b410f910 con 0x7f52b410dd70 2024-03-26T14:18:58.442 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.442+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f52ac004e30 con 0x7f52b410dd70 2024-03-26T14:18:58.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.443+0000 7f52ab400640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f52980a2800 0x7f52980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.443+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f52ac0ceaa0 con 0x7f52b410dd70 2024-03-26T14:18:58.444 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.443+0000 7f52b2a00640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f52980a2800 0x7f52980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.445 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.444+0000 7f52b2a00640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f52980a2800 0x7f52980a4cc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f52a4006010 tx=0x7f52a4005b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:58.448 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.447+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f52ac014030 con 0x7f52b410dd70 2024-03-26T14:18:58.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:57 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3695655802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:18:58.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:57 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/677708360' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:18:58.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.648+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52b40622a0 con 0x7f52b410dd70 2024-03-26T14:18:58.652 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.650+0000 7f52ab400640 1 -- 172.21.15.137:0/3058620099 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f52ac09b730 con 0x7f52b410dd70 2024-03-26T14:18:58.652 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:18:58.652 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:18:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f52980a2800 msgr2=0x7f52980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f52980a2800 0x7f52980a4cc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f52a4006010 tx=0x7f52a4005b50 comp rx=0 tx=0).stop 2024-03-26T14:18:58.654 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 msgr2=0x7f52b41a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 0x7f52b41a9990 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00c970 tx=0x7f52ac00ce30 comp rx=0 tx=0).stop 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 shutdown_connections 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f52980a2800 0x7f52980a4cc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b410b2d0 0x7f52b41a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f52b410dd70 0x7f52b41a9990 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 --2- 172.21.15.137:0/3058620099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52b4108910 0x7f52b41a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 >> 172.21.15.137:0/3058620099 conn(0x7f52b406e8b0 msgr2=0x7f52b4114910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 shutdown_connections 2024-03-26T14:18:58.655 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.653+0000 7f52b9600640 1 -- 172.21.15.137:0/3058620099 wait complete. 2024-03-26T14:18:58.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-26T13:53:05.611202+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-26T13:59:20.941382+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-26T13:59:22.182445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-26T13:59:25.824350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-26T14:18:58.673 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 -- 172.21.15.137:0/72897709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c1016b0 msgr2=0x7f5d2c101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 --2- 172.21.15.137:0/72897709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c101ad0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009860 tx=0x7f5d1c02bb70 comp rx=0 tx=0).stop 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 -- 172.21.15.137:0/72897709 shutdown_connections 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 --2- 172.21.15.137:0/72897709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 0x7f5d2c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 --2- 172.21.15.137:0/72897709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c103260 0x7f5d2c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 --2- 172.21.15.137:0/72897709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c101ad0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 -- 172.21.15.137:0/72897709 >> 172.21.15.137:0/72897709 conn(0x7f5d2c0fcc40 msgr2=0x7f5d2c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 -- 172.21.15.137:0/72897709 shutdown_connections 2024-03-26T14:18:58.885 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.883+0000 7f5d31200640 1 -- 172.21.15.137:0/72897709 wait complete. 2024-03-26T14:18:58.886 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 Processor -- start 2024-03-26T14:18:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 -- start start 2024-03-26T14:18:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c1075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 0x7f5d2c107b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.887 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c103260 0x7f5d2c104130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c10d1b0 con 0x7f5d2c1016b0 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c10d030 con 0x7f5d2c1028a0 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.885+0000 7f5d31200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c10d330 con 0x7f5d2c103260 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c1075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c1075c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53926/0 (socket says 172.21.15.137:53926) 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 -- 172.21.15.137:0/2829847151 learned_addr learned my addr 172.21.15.137:0/2829847151 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d21400640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 0x7f5d2c107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d2a800640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c103260 0x7f5d2c104130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 -- 172.21.15.137:0/2829847151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c103260 msgr2=0x7f5d2c104130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c103260 0x7f5d2c104130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 -- 172.21.15.137:0/2829847151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 msgr2=0x7f5d2c107b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 0x7f5d2c107b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 -- 172.21.15.137:0/2829847151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c1049f0 con 0x7f5d2c1016b0 2024-03-26T14:18:58.888 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d21400640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 0x7f5d2c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.886+0000 7f5d29e00640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c1075c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0057c0 tx=0x7f5d1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:58.889 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.887+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c041030 con 0x7f5d2c1016b0 2024-03-26T14:18:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.887+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d1c02c9c0 con 0x7f5d2c1016b0 2024-03-26T14:18:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.887+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c1a5f70 con 0x7f5d2c1016b0 2024-03-26T14:18:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.887+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c039040 con 0x7f5d2c1016b0 2024-03-26T14:18:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.887+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c1a6320 con 0x7f5d2c1016b0 2024-03-26T14:18:58.890 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.888+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c1a64f0 con 0x7f5d2c1016b0 2024-03-26T14:18:58.892 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.891+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5d1c044070 con 0x7f5d2c1016b0 2024-03-26T14:18:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.892+0000 7f5d23400640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5d100a2880 0x7f5d100a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:58.893 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.892+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f5d1c0f0bd0 con 0x7f5d2c1016b0 2024-03-26T14:18:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.892+0000 7f5d21400640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5d100a2880 0x7f5d100a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:58.894 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.893+0000 7f5d21400640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5d100a2880 0x7f5d100a4d40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c105540 tx=0x7f5d14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:58.895 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:58.894+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5d1c0bd8e0 con 0x7f5d2c1016b0 2024-03-26T14:18:59.095 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.094+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d2c1a6880 con 0x7f5d2c1016b0 2024-03-26T14:18:59.096 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.095+0000 7f5d23400640 1 -- 172.21.15.137:0/2829847151 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d1c02cb90 con 0x7f5d2c1016b0 2024-03-26T14:18:59.096 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 1 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.097+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5d100a2880 msgr2=0x7f5d100a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.097+0000 7f5d31200640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5d100a2880 0x7f5d100a4d40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c105540 tx=0x7f5d14009260 comp rx=0 tx=0).stop 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.097+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 msgr2=0x7f5d2c1075c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.097+0000 7f5d31200640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c1075c0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0057c0 tx=0x7f5d1c0042f0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 shutdown_connections 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f5d100a2880 0x7f5d100a4d40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d2c1028a0 0x7f5d2c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d2c103260 0x7f5d2c104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.099 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 --2- 172.21.15.137:0/2829847151 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d2c1016b0 0x7f5d2c1075c0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 >> 172.21.15.137:0/2829847151 conn(0x7f5d2c0fcc40 msgr2=0x7f5d2c108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 shutdown_connections 2024-03-26T14:18:59.100 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.098+0000 7f5d31200640 1 -- 172.21.15.137:0/2829847151 wait complete. 2024-03-26T14:18:59.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-26T14:18:59.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:59 smithi137 ceph-mon[31539]: pgmap v956: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:59.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:59 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3044806731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T14:18:59.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:18:59 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3058620099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:59.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 -- 172.21.15.137:0/1067085793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8077c40 msgr2=0x7f34d80784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.331 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 --2- 172.21.15.137:0/1067085793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8077c40 0x7f34d80784e0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009ca0 tx=0x7f34d002c100 comp rx=0 tx=0).stop 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 -- 172.21.15.137:0/1067085793 shutdown_connections 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 --2- 172.21.15.137:0/1067085793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34d80772a0 0x7f34d8077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 --2- 172.21.15.137:0/1067085793 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8077c40 0x7f34d80784e0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 --2- 172.21.15.137:0/1067085793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d8076060 0x7f34d8076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 -- 172.21.15.137:0/1067085793 >> 172.21.15.137:0/1067085793 conn(0x7f34d80fee50 msgr2=0x7f34d8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 -- 172.21.15.137:0/1067085793 shutdown_connections 2024-03-26T14:18:59.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.330+0000 7f34dde00640 1 -- 172.21.15.137:0/1067085793 wait complete. 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.331+0000 7f34dde00640 1 Processor -- start 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.331+0000 7f34dde00640 1 -- start start 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34dde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 0x7f34d819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34dde00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d80772a0 0x7f34d819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34d8077c40 0x7f34d81a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34dde00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8113860 con 0x7f34d8076060 2024-03-26T14:18:59.334 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34dde00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81136e0 con 0x7f34d8077c40 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34dde00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d81139e0 con 0x7f34d80772a0 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 0x7f34d819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 0x7f34d819dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53942/0 (socket says 172.21.15.137:53942) 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 -- 172.21.15.137:0/2496861137 learned_addr learned my addr 172.21.15.137:0/2496861137 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d7400640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34d8077c40 0x7f34d81a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6000640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d80772a0 0x7f34d819e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 -- 172.21.15.137:0/2496861137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d80772a0 msgr2=0x7f34d819e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d80772a0 0x7f34d819e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 -- 172.21.15.137:0/2496861137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34d8077c40 msgr2=0x7f34d81a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34d8077c40 0x7f34d81a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.332+0000 7f34d6a00640 1 -- 172.21.15.137:0/2496861137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d81a5c10 con 0x7f34d8076060 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.333+0000 7f34d6a00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 0x7f34d819dff0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f34c800b4a0 tx=0x7f34c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.333+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8014070 con 0x7f34d8076060 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.333+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f34c8010ac0 con 0x7f34d8076060 2024-03-26T14:18:59.335 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.333+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d81a5f00 con 0x7f34d8076060 2024-03-26T14:18:59.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.333+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8017c30 con 0x7f34d8076060 2024-03-26T14:18:59.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.333+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d81a63c0 con 0x7f34d8076060 2024-03-26T14:18:59.336 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.334+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d8110250 con 0x7f34d8076060 2024-03-26T14:18:59.339 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.338+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f34c8017070 con 0x7f34d8076060 2024-03-26T14:18:59.340 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.339+0000 7f34dca00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f34bc0a2880 0x7f34bc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.340+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f34c80ce5f0 con 0x7f34d8076060 2024-03-26T14:18:59.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.340+0000 7f34d6000640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f34bc0a2880 0x7f34bc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.341+0000 7f34d6000640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f34bc0a2880 0x7f34bc0a4d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f34c4005fd0 tx=0x7f34c4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:59.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.343+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f34c809b2c0 con 0x7f34d8076060 2024-03-26T14:18:59.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:59 smithi179 ceph-mon[35472]: pgmap v956: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:59.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:59 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3044806731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T14:18:59.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:18:59 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3058620099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:59.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:59 smithi159 ceph-mon[35501]: pgmap v956: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-03-26T14:18:59.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:59 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3044806731' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T14:18:59.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:18:59 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3058620099' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:18:59.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.547+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34d802d2c0 con 0x7f34d8076060 2024-03-26T14:18:59.548 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.547+0000 7f34dca00640 1 -- 172.21.15.137:0/2496861137 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34c809abc0 con 0x7f34d8076060 2024-03-26T14:18:59.549 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:18:59.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f34bc0a2880 msgr2=0x7f34bc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.551 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f34bc0a2880 0x7f34bc0a4d40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f34c4005fd0 tx=0x7f34c4005850 comp rx=0 tx=0).stop 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 msgr2=0x7f34d819dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 0x7f34d819dff0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f34c800b4a0 tx=0x7f34c800b960 comp rx=0 tx=0).stop 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 shutdown_connections 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f34bc0a2880 0x7f34bc0a4d40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34d8077c40 0x7f34d81a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34d80772a0 0x7f34d819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 --2- 172.21.15.137:0/2496861137 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f34d8076060 0x7f34d819dff0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 >> 172.21.15.137:0/2496861137 conn(0x7f34d80fee50 msgr2=0x7f34d8100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 shutdown_connections 2024-03-26T14:18:59.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.550+0000 7f34dde00640 1 -- 172.21.15.137:0/2496861137 wait complete. 2024-03-26T14:18:59.570 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-26T14:18:59.785 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 -- 172.21.15.137:0/819343575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 msgr2=0x7fdb8c10bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 --2- 172.21.15.137:0/819343575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c10bd20 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009ca0 tx=0x7fdb8402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 -- 172.21.15.137:0/819343575 shutdown_connections 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 --2- 172.21.15.137:0/819343575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c104ad0 0x7fdb8c104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 --2- 172.21.15.137:0/819343575 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c10bd20 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 --2- 172.21.15.137:0/819343575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c1038e0 0x7fdb8c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 -- 172.21.15.137:0/819343575 >> 172.21.15.137:0/819343575 conn(0x7fdb8c0fee50 msgr2=0x7fdb8c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 -- 172.21.15.137:0/819343575 shutdown_connections 2024-03-26T14:18:59.786 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.784+0000 7fdb90c00640 1 -- 172.21.15.137:0/819343575 wait complete. 2024-03-26T14:18:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.785+0000 7fdb90c00640 1 Processor -- start 2024-03-26T14:18:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.785+0000 7fdb90c00640 1 -- start start 2024-03-26T14:18:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.785+0000 7fdb90c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c1038e0 0x7fdb8c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.788 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.785+0000 7fdb90c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c104ad0 0x7fdb8c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.785+0000 7fdb90c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c1a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.785+0000 7fdb90c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10f410 con 0x7fdb8c105490 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb90c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10f290 con 0x7fdb8c1038e0 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb90c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c10f590 con 0x7fdb8c104ad0 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c1a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c1a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53952/0 (socket says 172.21.15.137:53952) 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 -- 172.21.15.137:0/902484633 learned_addr learned my addr 172.21.15.137:0/902484633 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb89e00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c1038e0 0x7fdb8c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb83e00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c104ad0 0x7fdb8c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 -- 172.21.15.137:0/902484633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c104ad0 msgr2=0x7fdb8c19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c104ad0 0x7fdb8c19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 -- 172.21.15.137:0/902484633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c1038e0 msgr2=0x7fdb8c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c1038e0 0x7fdb8c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.789 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb8a800640 1 -- 172.21.15.137:0/902484633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c1a1700 con 0x7fdb8c105490 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb89e00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c1038e0 0x7fdb8c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.786+0000 7fdb83e00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c104ad0 0x7fdb8c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.787+0000 7fdb8a800640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c1a1090 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402c700 tx=0x7fdb840328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.787+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84041020 con 0x7fdb8c105490 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.787+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb84003e20 con 0x7fdb8c105490 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.787+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c1a1990 con 0x7fdb8c105490 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.787+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84039040 con 0x7fdb8c105490 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.787+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c1a1e80 con 0x7fdb8c105490 2024-03-26T14:18:59.790 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.788+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c10ac50 con 0x7fdb8c105490 2024-03-26T14:18:59.791 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.790+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdb84044020 con 0x7fdb8c105490 2024-03-26T14:18:59.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.792+0000 7fdb82a00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb700a2800 0x7fdb700a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:18:59.793 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.792+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fdb840f1380 con 0x7fdb8c105490 2024-03-26T14:18:59.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.792+0000 7fdb89e00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb700a2800 0x7fdb700a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:18:59.794 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.793+0000 7fdb89e00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb700a2800 0x7fdb700a4cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0058c0 tx=0x7fdb7c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:18:59.798 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.797+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdb8404b020 con 0x7fdb8c105490 2024-03-26T14:18:59.990 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.989+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb8c0622a0 con 0x7fdb8c105490 2024-03-26T14:18:59.991 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.990+0000 7fdb82a00640 1 -- 172.21.15.137:0/902484633 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb8403b2f0 con 0x7fdb8c105490 2024-03-26T14:18:59.991 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 64 2024-03-26T14:18:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.992+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb700a2800 msgr2=0x7fdb700a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb700a2800 0x7fdb700a4cc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0058c0 tx=0x7fdb7c005850 comp rx=0 tx=0).stop 2024-03-26T14:18:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 msgr2=0x7fdb8c1a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:18:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c1a1090 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fdb8402c700 tx=0x7fdb840328a0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.994 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 shutdown_connections 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fdb700a2800 0x7fdb700a4cc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdb8c105490 0x7fdb8c1a1090 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb8c104ad0 0x7fdb8c19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 --2- 172.21.15.137:0/902484633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb8c1038e0 0x7fdb8c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 >> 172.21.15.137:0/902484633 conn(0x7fdb8c0fee50 msgr2=0x7fdb8c107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 shutdown_connections 2024-03-26T14:18:59.995 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:18:59.993+0000 7fdb90c00640 1 -- 172.21.15.137:0/902484633 wait complete. 2024-03-26T14:19:00.013 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.216+0000 7fa384800640 1 -- 172.21.15.137:0/3386029334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c102150 msgr2=0x7fa37c1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.216+0000 7fa384800640 1 --2- 172.21.15.137:0/3386029334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c102150 0x7fa37c1089e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fa378009900 tx=0x7fa37802bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 -- 172.21.15.137:0/3386029334 shutdown_connections 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 --2- 172.21.15.137:0/3386029334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c101790 0x7fa37c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 --2- 172.21.15.137:0/3386029334 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c102150 0x7fa37c1089e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 --2- 172.21.15.137:0/3386029334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37c1005a0 0x7fa37c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.218 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 -- 172.21.15.137:0/3386029334 >> 172.21.15.137:0/3386029334 conn(0x7fa37c0fbb30 msgr2=0x7fa37c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:00.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 -- 172.21.15.137:0/3386029334 shutdown_connections 2024-03-26T14:19:00.219 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.217+0000 7fa384800640 1 -- 172.21.15.137:0/3386029334 wait complete. 2024-03-26T14:19:00.220 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 Processor -- start 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 -- start start 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37c1005a0 0x7fa37c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 0x7fa37c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c102150 0x7fa37c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c10c0e0 con 0x7fa37c101790 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c10bf60 con 0x7fa37c102150 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa384800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c10c260 con 0x7fa37c1005a0 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa381400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 0x7fa37c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa381400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 0x7fa37c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53974/0 (socket says 172.21.15.137:53974) 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.218+0000 7fa381400640 1 -- 172.21.15.137:0/789614989 learned_addr learned my addr 172.21.15.137:0/789614989 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa382800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c102150 0x7fa37c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381400640 1 -- 172.21.15.137:0/789614989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37c1005a0 msgr2=0x7fa37c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381e00640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37c1005a0 0x7fa37c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381400640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37c1005a0 0x7fa37c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.221 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381400640 1 -- 172.21.15.137:0/789614989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c102150 msgr2=0x7fa37c1a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381400640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c102150 0x7fa37c1a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381400640 1 -- 172.21.15.137:0/789614989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c1a1710 con 0x7fa37c101790 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa382800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c102150 0x7fa37c1a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa381400640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 0x7fa37c19a030 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa37000e9f0 tx=0x7fa37000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000cd80 con 0x7fa37c101790 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa3700108e0 con 0x7fa37c101790 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370018920 con 0x7fa37c101790 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.219+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37c1a1a00 con 0x7fa37c101790 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.220+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37c1a1f00 con 0x7fa37c101790 2024-03-26T14:19:00.222 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.221+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37c1a2190 con 0x7fa37c101790 2024-03-26T14:19:00.223 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.223+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa370010a90 con 0x7fa37c101790 2024-03-26T14:19:00.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.224+0000 7fa36ea00640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3680a2880 0x7fa3680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.224+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fa3700cf270 con 0x7fa37c101790 2024-03-26T14:19:00.225 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.224+0000 7fa381e00640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3680a2880 0x7fa3680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.226 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.225+0000 7fa381e00640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3680a2880 0x7fa3680a4d40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa374005c40 tx=0x7fa374005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:00.230 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.229+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa37009bf00 con 0x7fa37c101790 2024-03-26T14:19:00.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:00 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2829847151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:00 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2496861137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:00 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/902484633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:00 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2829847151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:00 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2496861137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:00 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/902484633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.428 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.427+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa37c1a2520 con 0x7fa37c101790 2024-03-26T14:19:00.429 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.428+0000 7fa36ea00640 1 -- 172.21.15.137:0/789614989 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa37009b800 con 0x7fa37c101790 2024-03-26T14:19:00.429 INFO:teuthology.orchestra.run.smithi137.stdout:pg_num: 32 2024-03-26T14:19:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3680a2880 msgr2=0x7fa3680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.431 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3680a2880 0x7fa3680a4d40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa374005c40 tx=0x7fa374005bd0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 msgr2=0x7fa37c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 0x7fa37c19a030 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa37000e9f0 tx=0x7fa37000eeb0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 shutdown_connections 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fa3680a2880 0x7fa3680a4d40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa37c102150 0x7fa37c1a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa37c101790 0x7fa37c19a030 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 --2- 172.21.15.137:0/789614989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa37c1005a0 0x7fa37c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.432 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.430+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 >> 172.21.15.137:0/789614989 conn(0x7fa37c0fbb30 msgr2=0x7fa37c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.431+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 shutdown_connections 2024-03-26T14:19:00.433 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.431+0000 7fa384800640 1 -- 172.21.15.137:0/789614989 wait complete. 2024-03-26T14:19:00.451 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-26T14:19:00.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:00 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2829847151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:00 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2496861137' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:00 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/902484633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 -- 172.21.15.137:0/2214458579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 msgr2=0x7fc404074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 --2- 172.21.15.137:0/2214458579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404074580 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009900 tx=0x7fc3f002bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 -- 172.21.15.137:0/2214458579 shutdown_connections 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 --2- 172.21.15.137:0/2214458579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc404109570 0x7fc40410ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 --2- 172.21.15.137:0/2214458579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404074580 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 --2- 172.21.15.137:0/2214458579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4040756e0 0x7fc404073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 -- 172.21.15.137:0/2214458579 >> 172.21.15.137:0/2214458579 conn(0x7fc4040fcc40 msgr2=0x7fc4040ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:00.643 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 -- 172.21.15.137:0/2214458579 shutdown_connections 2024-03-26T14:19:00.644 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.641+0000 7fc40c400640 1 -- 172.21.15.137:0/2214458579 wait complete. 2024-03-26T14:19:00.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.642+0000 7fc40c400640 1 Processor -- start 2024-03-26T14:19:00.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.642+0000 7fc40c400640 1 -- start start 2024-03-26T14:19:00.645 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc40c400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc40c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4040756e0 0x7fc40419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc40c400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc404109570 0x7fc4041a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc40c400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40410f3f0 con 0x7fc404074100 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc40c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40410f270 con 0x7fc4040756e0 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc40c400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40410f570 con 0x7fc404109570 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:53998/0 (socket says 172.21.15.137:53998) 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 -- 172.21.15.137:0/1634647157 learned_addr learned my addr 172.21.15.137:0/1634647157 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409000640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4040756e0 0x7fc40419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 -- 172.21.15.137:0/1634647157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc404109570 msgr2=0x7fc4041a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc404109570 0x7fc4041a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 -- 172.21.15.137:0/1634647157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4040756e0 msgr2=0x7fc40419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.646 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4040756e0 0x7fc40419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.643+0000 7fc409a00640 1 -- 172.21.15.137:0/1634647157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4041a1740 con 0x7fc404074100 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.644+0000 7fc409a00640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404199b20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400cce0 tx=0x7fc3f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.644+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4004b20 con 0x7fc404074100 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.644+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4041a1a30 con 0x7fc404074100 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.644+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc3f4004cc0 con 0x7fc404074100 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.644+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4041a1f80 con 0x7fc404074100 2024-03-26T14:19:00.647 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.644+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f401d060 con 0x7fc404074100 2024-03-26T14:19:00.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.645+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4041082a0 con 0x7fc404074100 2024-03-26T14:19:00.648 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.647+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc3f4005000 con 0x7fc404074100 2024-03-26T14:19:00.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.648+0000 7fc402000640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3e80a2800 0x7fc3e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:00.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.648+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fc3f40ceef0 con 0x7fc404074100 2024-03-26T14:19:00.649 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.648+0000 7fc409000640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3e80a2800 0x7fc3e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:00.650 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.649+0000 7fc409000640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3e80a2800 0x7fc3e80a4cc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00096c0 tx=0x7fc3f0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:00.653 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.652+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc3f4014030 con 0x7fc404074100 2024-03-26T14:19:00.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.865+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc4040622d0 con 0x7fc404074100 2024-03-26T14:19:00.867 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.866+0000 7fc402000640 1 -- 172.21.15.137:0/1634647157 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+600 (secure 0 0 0) 0x7fc3f409bb80 con 0x7fc404074100 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1621047019 2024-03-27T13:49:52.910355+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/460394311 2024-03-27T13:49:52.910355+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1426358112 2024-03-27T13:49:19.001493+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/2012111099 2024-03-27T13:49:52.910355+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/1602182932 2024-03-27T13:49:19.001493+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6800/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/3592180121 2024-03-27T13:49:19.001493+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:6801/1955125876 2024-03-27T13:49:19.001493+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stdout:172.21.15.137:0/487629871 2024-03-27T13:49:52.910355+0000 2024-03-26T14:19:00.868 INFO:teuthology.orchestra.run.smithi137.stderr:listed 10 entries 2024-03-26T14:19:00.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3e80a2800 msgr2=0x7fc3e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3e80a2800 0x7fc3e80a4cc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00096c0 tx=0x7fc3f0005fd0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 msgr2=0x7fc404199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:00.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404199b20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400cce0 tx=0x7fc3f400a620 comp rx=0 tx=0).stop 2024-03-26T14:19:00.870 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 shutdown_connections 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fc3e80a2800 0x7fc3e80a4cc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4040756e0 0x7fc40419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc404109570 0x7fc4041a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 --2- 172.21.15.137:0/1634647157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc404074100 0x7fc404199b20 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 >> 172.21.15.137:0/1634647157 conn(0x7fc4040fcc40 msgr2=0x7fc404105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 shutdown_connections 2024-03-26T14:19:00.871 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:00.869+0000 7fc40c400640 1 -- 172.21.15.137:0/1634647157 wait complete. 2024-03-26T14:19:00.888 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi137.front.sepia.ceph.com... 2024-03-26T14:19:00.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:19:00.888 DEBUG:teuthology.orchestra.run.smithi137:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-26T14:19:00.979 INFO:teuthology.orchestra.run:waiting for 300 2024-03-26T14:19:00.980 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi137.stderr:ceph-fuse[86381]: fuse finished with error 0 and tester_r 0 2024-03-26T14:19:01.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:01 smithi137 ceph-mon[31539]: pgmap v957: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:19:01.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:01 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/789614989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:19:01.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:01 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1634647157' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T14:19:01.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:01 smithi179 ceph-mon[35472]: pgmap v957: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:19:01.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:01 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/789614989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:19:01.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:01 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1634647157' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T14:19:01.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:01 smithi159 ceph-mon[35501]: pgmap v957: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-03-26T14:19:01.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:01 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/789614989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-03-26T14:19:01.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:01 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1634647157' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-26T14:19:03.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:03 smithi137 ceph-mon[31539]: pgmap v958: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:19:03.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:03 smithi179 ceph-mon[35472]: pgmap v958: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:19:03.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:03 smithi159 ceph-mon[35501]: pgmap v958: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-03-26T14:19:05.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:05 smithi137 ceph-mon[31539]: pgmap v959: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T14:19:05.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:05 smithi179 ceph-mon[35472]: pgmap v959: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T14:19:05.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:05 smithi159 ceph-mon[35501]: pgmap v959: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 5 op/s 2024-03-26T14:19:07.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T14:19:07.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:19:07.081 DEBUG:teuthology.orchestra.run.smithi137:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-26T14:19:07.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:19:07.141 DEBUG:teuthology.orchestra.run.smithi137:> ip netns list 2024-03-26T14:19:07.194 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-26T14:19:07.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:19:07.194 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-26T14:19:07.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:07 smithi137 ceph-mon[31539]: pgmap v960: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:19:07.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-26T14:19:07.292 DEBUG:teuthology.orchestra.run.smithi137:> sudo ip link delete ceph-brx 2024-03-26T14:19:07.421 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-26T14:19:07.433 INFO:tasks.cephadm:Teardown begin 2024-03-26T14:19:07.433 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-26T14:19:07.433 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:19:07.434 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:07 smithi179 ceph-mon[35472]: pgmap v960: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:19:07.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:07 smithi159 ceph-mon[35501]: pgmap v960: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-03-26T14:19:07.626 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 -- 172.21.15.137:0/3611461185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30077c40 msgr2=0x7fcb300784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 --2- 172.21.15.137:0/3611461185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30077c40 0x7fcb300784e0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009900 tx=0x7fcb2802bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 -- 172.21.15.137:0/3611461185 shutdown_connections 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 --2- 172.21.15.137:0/3611461185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb300772a0 0x7fcb30077700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 --2- 172.21.15.137:0/3611461185 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30077c40 0x7fcb300784e0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 --2- 172.21.15.137:0/3611461185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb30076060 0x7fcb30076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 -- 172.21.15.137:0/3611461185 >> 172.21.15.137:0/3611461185 conn(0x7fcb300fee90 msgr2=0x7fcb301012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 -- 172.21.15.137:0/3611461185 shutdown_connections 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.622+0000 7fcb36200640 1 -- 172.21.15.137:0/3611461185 wait complete. 2024-03-26T14:19:07.627 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 Processor -- start 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 -- start start 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 0x7fcb3019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb300772a0 0x7fcb3019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb30077c40 0x7fcb301a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30113860 con 0x7fcb30076060 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301136e0 con 0x7fcb300772a0 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb36200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb301139e0 con 0x7fcb30077c40 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 0x7fcb3019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:07.628 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb2f400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 0x7fcb3019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:54016/0 (socket says 172.21.15.137:54016) 2024-03-26T14:19:07.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.623+0000 7fcb2f400640 1 -- 172.21.15.137:0/3668561326 learned_addr learned my addr 172.21.15.137:0/3668561326 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:07.629 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2ea00640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb300772a0 0x7fcb3019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2fe00640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb30077c40 0x7fcb301a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2f400640 1 -- 172.21.15.137:0/3668561326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb30077c40 msgr2=0x7fcb301a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2f400640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb30077c40 0x7fcb301a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2f400640 1 -- 172.21.15.137:0/3668561326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb300772a0 msgr2=0x7fcb3019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2f400640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb300772a0 0x7fcb3019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2f400640 1 -- 172.21.15.137:0/3668561326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb301a5ab0 con 0x7fcb30076060 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb2f400640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 0x7fcb3019de90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000eab0 tx=0x7fcb2000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:07.630 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.624+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000ce40 con 0x7fcb30076060 2024-03-26T14:19:07.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.625+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcb20011920 con 0x7fcb30076060 2024-03-26T14:19:07.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.625+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20010a00 con 0x7fcb30076060 2024-03-26T14:19:07.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.625+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb301a5da0 con 0x7fcb30076060 2024-03-26T14:19:07.631 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.625+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301a6210 con 0x7fcb30076060 2024-03-26T14:19:07.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.626+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb30110380 con 0x7fcb30076060 2024-03-26T14:19:07.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.628+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcb20011ad0 con 0x7fcb30076060 2024-03-26T14:19:07.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.630+0000 7fcb27e00640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb140a2800 0x7fcb140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:07.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.630+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7fcb200cf4e0 con 0x7fcb30076060 2024-03-26T14:19:07.632 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.630+0000 7fcb2ea00640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb140a2800 0x7fcb140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:07.633 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.631+0000 7fcb2ea00640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb140a2800 0x7fcb140a4cc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb3019f3a0 tx=0x7fcb18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:07.636 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.634+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcb20016030 con 0x7fcb30076060 2024-03-26T14:19:07.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.876+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb3002d070 con 0x7fcb30076060 2024-03-26T14:19:07.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.877+0000 7fcb27e00640 1 -- 172.21.15.137:0/3668561326 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7fcb2009c170 con 0x7fcb30076060 2024-03-26T14:19:07.879 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:19:07.879 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:19:07.880 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:19:07.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.880+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb140a2800 msgr2=0x7fcb140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:07.882 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.880+0000 7fcb36200640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb140a2800 0x7fcb140a4cc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcb3019f3a0 tx=0x7fcb18005f90 comp rx=0 tx=0).stop 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.880+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 msgr2=0x7fcb3019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.880+0000 7fcb36200640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 0x7fcb3019de90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000eab0 tx=0x7fcb2000ef70 comp rx=0 tx=0).stop 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 shutdown_connections 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7fcb140a2800 0x7fcb140a4cc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb300772a0 0x7fcb3019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcb30077c40 0x7fcb301a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 --2- 172.21.15.137:0/3668561326 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb30076060 0x7fcb3019de90 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 >> 172.21.15.137:0/3668561326 conn(0x7fcb300fee90 msgr2=0x7fcb30100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 shutdown_connections 2024-03-26T14:19:07.883 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:07.881+0000 7fcb36200640 1 -- 172.21.15.137:0/3668561326 wait complete. 2024-03-26T14:19:07.900 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-26T14:19:08.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.113+0000 7f1c38c00640 1 -- 172.21.15.137:0/924437553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 msgr2=0x7f1c34109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.113+0000 7f1c38c00640 1 --2- 172.21.15.137:0/924437553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c34109170 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009ca0 tx=0x7f1c2402c100 comp rx=0 tx=0).stop 2024-03-26T14:19:08.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 -- 172.21.15.137:0/924437553 shutdown_connections 2024-03-26T14:19:08.115 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 --2- 172.21.15.137:0/924437553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 0x7f1c3410ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 --2- 172.21.15.137:0/924437553 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c34109170 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 --2- 172.21.15.137:0/924437553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c34107b00 0x7f1c34107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 -- 172.21.15.137:0/924437553 >> 172.21.15.137:0/924437553 conn(0x7f1c34076ab0 msgr2=0x7f1c34078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:08.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 -- 172.21.15.137:0/924437553 shutdown_connections 2024-03-26T14:19:08.116 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.114+0000 7f1c38c00640 1 -- 172.21.15.137:0/924437553 wait complete. 2024-03-26T14:19:08.118 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 Processor -- start 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 -- start start 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c34107b00 0x7f1c3419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c3419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 0x7f1c341a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34113610 con 0x7f1c34108cf0 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34113490 con 0x7f1c34107b00 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c38c00640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34113790 con 0x7f1c341096b0 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.115+0000 7f1c32a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c3419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c3419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41728/0 (socket says 172.21.15.137:41728) 2024-03-26T14:19:08.119 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 -- 172.21.15.137:0/1609911107 learned_addr learned my addr 172.21.15.137:0/1609911107 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c33400640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c34107b00 0x7f1c3419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c33e00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 0x7f1c341a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 -- 172.21.15.137:0/1609911107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 msgr2=0x7f1c341a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 0x7f1c341a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 -- 172.21.15.137:0/1609911107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c34107b00 msgr2=0x7f1c3419dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c34107b00 0x7f1c3419dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 -- 172.21.15.137:0/1609911107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c341a5940 con 0x7f1c34108cf0 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c33e00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 0x7f1c341a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c32a00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c3419e270 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005e30 tx=0x7f1c24035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.116+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24039040 con 0x7f1c34108cf0 2024-03-26T14:19:08.120 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.117+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c24003970 con 0x7f1c34108cf0 2024-03-26T14:19:08.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.117+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c341a5bd0 con 0x7f1c34108cf0 2024-03-26T14:19:08.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.117+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c240044a0 con 0x7f1c34108cf0 2024-03-26T14:19:08.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.117+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c341a60c0 con 0x7f1c34108cf0 2024-03-26T14:19:08.121 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.118+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3410eea0 con 0x7f1c34108cf0 2024-03-26T14:19:08.122 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.119+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1c24046030 con 0x7f1c34108cf0 2024-03-26T14:19:08.123 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.122+0000 7f1c2be00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c180a2800 0x7f1c180a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.122+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f1c240f0e80 con 0x7f1c34108cf0 2024-03-26T14:19:08.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.122+0000 7f1c33400640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c180a2800 0x7f1c180a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.124 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.123+0000 7f1c33400640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c180a2800 0x7f1c180a4cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005fd0 tx=0x7f1c20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:08.128 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.126+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1c2404d060 con 0x7f1c34108cf0 2024-03-26T14:19:08.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:08 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3668561326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:19:08.367 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.366+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c34062340 con 0x7f1c34108cf0 2024-03-26T14:19:08.368 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.367+0000 7f1c2be00640 1 -- 172.21.15.137:0/1609911107 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+11409 (secure 0 0 0) 0x7f1c240bdb10 con 0x7f1c34108cf0 2024-03-26T14:19:08.369 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:19:08.369 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/4090447897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":4090447897},{"type":"v1","addr":"172.21.15.137:6835","nonce":4090447897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4057650912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4057650912},{"type":"v1","addr":"172.21.15.137:6837","nonce":4057650912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14643,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/2542329898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":2542329898},{"type":"v1","addr":"172.21.15.137:6839","nonce":2542329898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6841/2350775776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6840","nonce":2350775776},{"type":"v1","addr":"172.21.15.137:6841","nonce":2350775776}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/2724223163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":2724223163},{"type":"v1","addr":"172.21.15.179:6833","nonce":2724223163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/2045604256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":2045604256},{"type":"v1","addr":"172.21.15.179:6835","nonce":2045604256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-26T13:59:22.728231+0000","modified":"2024-03-26T14:02:21.728835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":34496,"mds_1":34451,"mds_2":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24460":{"gid":24460,"name":"c","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/7146430","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":7146430},{"type":"v1","addr":"172.21.15.179:6837","nonce":7146430}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24481":{"gid":24481,"name":"i","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6839/2441458416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":2441458416},{"type":"v1","addr":"172.21.15.179:6839","nonce":2441458416}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34394":{"gid":34394,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1469472888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1469472888},{"type":"v1","addr":"172.21.15.159:6833","nonce":1469472888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":17,"addr":"172.21.15.159:6835/3125472496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":3125472496},{"type":"v1","addr":"172.21.15.159:6835","nonce":3125472496}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34451":{"gid":34451,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.159:6837/16915360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":16915360},{"type":"v1","addr":"172.21.15.159:6837","nonce":16915360}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34496":{"gid":34496,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.159:6839/2964052982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2964052982},{"type":"v1","addr":"172.21.15.159:6839","nonce":2964052982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34496,"qdb_cluster":[34496,34424,34451]},"id":1}]} 2024-03-26T14:19:08.369 INFO:teuthology.orchestra.run.smithi137.stderr:dumped fsmap epoch 24 2024-03-26T14:19:08.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c180a2800 msgr2=0x7f1c180a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c180a2800 0x7f1c180a4cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1c20005fd0 tx=0x7f1c20005850 comp rx=0 tx=0).stop 2024-03-26T14:19:08.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 msgr2=0x7f1c3419e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c3419e270 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f1c24005e30 tx=0x7f1c24035d60 comp rx=0 tx=0).stop 2024-03-26T14:19:08.372 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 shutdown_connections 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f1c180a2800 0x7f1c180a4cc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1c341096b0 0x7f1c341a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1c34108cf0 0x7f1c3419e270 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 --2- 172.21.15.137:0/1609911107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c34107b00 0x7f1c3419dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 >> 172.21.15.137:0/1609911107 conn(0x7f1c34076ab0 msgr2=0x7f1c3410c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 shutdown_connections 2024-03-26T14:19:08.373 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.370+0000 7f1c38c00640 1 -- 172.21.15.137:0/1609911107 wait complete. 2024-03-26T14:19:08.391 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-26T14:19:08.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:08 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3668561326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:19:08.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:08 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3668561326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:19:08.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 -- 172.21.15.137:0/516746890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701016b0 msgr2=0x7f2370101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 --2- 172.21.15.137:0/516746890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701016b0 0x7f2370101ad0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2364009900 tx=0x7f236402bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:08.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 -- 172.21.15.137:0/516746890 shutdown_connections 2024-03-26T14:19:08.592 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 --2- 172.21.15.137:0/516746890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2370103260 0x7f2370109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 --2- 172.21.15.137:0/516746890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f23701028a0 0x7f2370102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 --2- 172.21.15.137:0/516746890 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701016b0 0x7f2370101ad0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 -- 172.21.15.137:0/516746890 >> 172.21.15.137:0/516746890 conn(0x7f23700fcc80 msgr2=0x7f23700ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:08.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 -- 172.21.15.137:0/516746890 shutdown_connections 2024-03-26T14:19:08.593 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.590+0000 7f2377400640 1 -- 172.21.15.137:0/516746890 wait complete. 2024-03-26T14:19:08.595 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 Processor -- start 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 -- start start 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23701016b0 0x7f2370199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 0x7f237019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2370103260 0x7f23701a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010d220 con 0x7f23701028a0 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010d0a0 con 0x7f23701016b0 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f2377400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237010d3a0 con 0x7f2370103260 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f236fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 0x7f237019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.596 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f236fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 0x7f237019a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41746/0 (socket says 172.21.15.137:41746) 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.592+0000 7f236fe00640 1 -- 172.21.15.137:0/3866588872 learned_addr learned my addr 172.21.15.137:0/3866588872 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f2374a00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23701016b0 0x7f2370199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236fe00640 1 -- 172.21.15.137:0/3866588872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2370103260 msgr2=0x7f23701a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236fe00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2370103260 0x7f23701a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236fe00640 1 -- 172.21.15.137:0/3866588872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23701016b0 msgr2=0x7f2370199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236fe00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23701016b0 0x7f2370199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236fe00640 1 -- 172.21.15.137:0/3866588872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23701a1790 con 0x7f23701028a0 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f2374a00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23701016b0 0x7f2370199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236fe00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 0x7f237019a0b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f235c00d770 tx=0x7f235c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:08.597 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c017070 con 0x7f23701028a0 2024-03-26T14:19:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f235c0054d0 con 0x7f23701028a0 2024-03-26T14:19:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.593+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c004cf0 con 0x7f23701028a0 2024-03-26T14:19:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.594+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23701a1a80 con 0x7f23701028a0 2024-03-26T14:19:08.598 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.594+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23701a1f80 con 0x7f23701028a0 2024-03-26T14:19:08.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.595+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23701a21b0 con 0x7f23701028a0 2024-03-26T14:19:08.599 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.597+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f235c004e90 con 0x7f23701028a0 2024-03-26T14:19:08.600 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.599+0000 7f236d600640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23580a2880 0x7f23580a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.599+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f235c0ce900 con 0x7f23701028a0 2024-03-26T14:19:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.599+0000 7f2374a00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23580a2880 0x7f23580a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:08.601 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.600+0000 7f2374a00640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23580a2880 0x7f23580a4d40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f236402c240 tx=0x7f2364005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:08.604 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.603+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f235c09b590 con 0x7f23701028a0 2024-03-26T14:19:08.804 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.803+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23701a2540 con 0x7f23701028a0 2024-03-26T14:19:08.808 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.807+0000 7f236d600640 1 -- 172.21.15.137:0/3866588872 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24874 (secure 0 0 0) 0x7f235c09ae90 con 0x7f23701028a0 2024-03-26T14:19:08.809 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-03-26T14:19:08.809 INFO:teuthology.orchestra.run.smithi137.stdout:{"epoch":94,"fsid":"66aee2a0-eb77-11ee-95d0-87774f69a715","created":"2024-03-26T13:48:46.854612+0000","modified":"2024-03-26T14:00:55.191525+0000","last_up_change":"2024-03-26T13:58:15.930806+0000","last_in_change":"2024-03-26T13:57:50.812464+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-26T13:53:05.611202+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-26T13:59:20.941382+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-26T13:59:22.182445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-26T13:59:25.824350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f6b26e4-ccfd-4457-91b0-240679242c9e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6803","nonce":4170736921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6805","nonce":4170736921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6809","nonce":4170736921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":4170736921},{"type":"v1","addr":"172.21.15.137:6807","nonce":4170736921}]},"public_addr":"172.21.15.137:6803/4170736921","cluster_addr":"172.21.15.137:6805/4170736921","heartbeat_back_addr":"172.21.15.137:6809/4170736921","heartbeat_front_addr":"172.21.15.137:6807/4170736921","state":["exists","up"]},{"osd":1,"uuid":"40fd05e9-e41a-4c27-9f67-43693dae21d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6801","nonce":3508415474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6803","nonce":3508415474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6807","nonce":3508415474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":3508415474},{"type":"v1","addr":"172.21.15.159:6805","nonce":3508415474}]},"public_addr":"172.21.15.159:6801/3508415474","cluster_addr":"172.21.15.159:6803/3508415474","heartbeat_back_addr":"172.21.15.159:6807/3508415474","heartbeat_front_addr":"172.21.15.159:6805/3508415474","state":["exists","up"]},{"osd":2,"uuid":"76a74368-6c04-4a19-b56d-432b5145e385","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6801","nonce":395733293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6803","nonce":395733293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6807","nonce":395733293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":395733293},{"type":"v1","addr":"172.21.15.179:6805","nonce":395733293}]},"public_addr":"172.21.15.179:6801/395733293","cluster_addr":"172.21.15.179:6803/395733293","heartbeat_back_addr":"172.21.15.179:6807/395733293","heartbeat_front_addr":"172.21.15.179:6805/395733293","state":["exists","up"]},{"osd":3,"uuid":"ba6d4b45-933b-4346-879f-ead75f3faf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6811","nonce":3541368776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6813","nonce":3541368776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6817","nonce":3541368776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":3541368776},{"type":"v1","addr":"172.21.15.137:6815","nonce":3541368776}]},"public_addr":"172.21.15.137:6811/3541368776","cluster_addr":"172.21.15.137:6813/3541368776","heartbeat_back_addr":"172.21.15.137:6817/3541368776","heartbeat_front_addr":"172.21.15.137:6815/3541368776","state":["exists","up"]},{"osd":4,"uuid":"ae280d0f-5c5d-4ff0-a49a-3ddb2e3344e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6809","nonce":2050135218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6811","nonce":2050135218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6815","nonce":2050135218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":2050135218},{"type":"v1","addr":"172.21.15.159:6813","nonce":2050135218}]},"public_addr":"172.21.15.159:6809/2050135218","cluster_addr":"172.21.15.159:6811/2050135218","heartbeat_back_addr":"172.21.15.159:6815/2050135218","heartbeat_front_addr":"172.21.15.159:6813/2050135218","state":["exists","up"]},{"osd":5,"uuid":"bdbb0679-03d9-4423-873d-2c9c968896ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6809","nonce":3906205051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6811","nonce":3906205051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6815","nonce":3906205051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":3906205051},{"type":"v1","addr":"172.21.15.179:6813","nonce":3906205051}]},"public_addr":"172.21.15.179:6809/3906205051","cluster_addr":"172.21.15.179:6811/3906205051","heartbeat_back_addr":"172.21.15.179:6815/3906205051","heartbeat_front_addr":"172.21.15.179:6813/3906205051","state":["exists","up"]},{"osd":6,"uuid":"d6d1d4e9-7548-4ae0-9d1e-141bcedab4c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6819","nonce":102905149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6821","nonce":102905149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6825","nonce":102905149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":102905149},{"type":"v1","addr":"172.21.15.137:6823","nonce":102905149}]},"public_addr":"172.21.15.137:6819/102905149","cluster_addr":"172.21.15.137:6821/102905149","heartbeat_back_addr":"172.21.15.137:6825/102905149","heartbeat_front_addr":"172.21.15.137:6823/102905149","state":["exists","up"]},{"osd":7,"uuid":"54ae5a2e-89d5-4a31-801f-0d04c514f63c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6817","nonce":1788160471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6819","nonce":1788160471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6823","nonce":1788160471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":1788160471},{"type":"v1","addr":"172.21.15.159:6821","nonce":1788160471}]},"public_addr":"172.21.15.159:6817/1788160471","cluster_addr":"172.21.15.159:6819/1788160471","heartbeat_back_addr":"172.21.15.159:6823/1788160471","heartbeat_front_addr":"172.21.15.159:6821/1788160471","state":["exists","up"]},{"osd":8,"uuid":"54d0b6d1-2751-465f-89fb-d866d56a94e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6817","nonce":4118554699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6819","nonce":4118554699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6823","nonce":4118554699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":4118554699},{"type":"v1","addr":"172.21.15.179:6821","nonce":4118554699}]},"public_addr":"172.21.15.179:6817/4118554699","cluster_addr":"172.21.15.179:6819/4118554699","heartbeat_back_addr":"172.21.15.179:6823/4118554699","heartbeat_front_addr":"172.21.15.179:6821/4118554699","state":["exists","up"]},{"osd":9,"uuid":"86070f8d-a3f6-4a0f-9936-8c52710b2e85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6827","nonce":3720717247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6829","nonce":3720717247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6833","nonce":3720717247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3720717247},{"type":"v1","addr":"172.21.15.137:6831","nonce":3720717247}]},"public_addr":"172.21.15.137:6827/3720717247","cluster_addr":"172.21.15.137:6829/3720717247","heartbeat_back_addr":"172.21.15.137:6833/3720717247","heartbeat_front_addr":"172.21.15.137:6831/3720717247","state":["exists","up"]},{"osd":10,"uuid":"95b5ff77-4547-400f-96ef-142faa248b14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6825","nonce":2917781375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6827","nonce":2917781375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6831","nonce":2917781375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":2917781375},{"type":"v1","addr":"172.21.15.159:6829","nonce":2917781375}]},"public_addr":"172.21.15.159:6825/2917781375","cluster_addr":"172.21.15.159:6827/2917781375","heartbeat_back_addr":"172.21.15.159:6831/2917781375","heartbeat_front_addr":"172.21.15.159:6829/2917781375","state":["exists","up"]},{"osd":11,"uuid":"cefee215-43f2-4ed3-adbc-b2784f7cbddc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6825","nonce":1258425145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6827","nonce":1258425145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6831","nonce":1258425145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":1258425145},{"type":"v1","addr":"172.21.15.179:6829","nonce":1258425145}]},"public_addr":"172.21.15.179:6825/1258425145","cluster_addr":"172.21.15.179:6827/1258425145","heartbeat_back_addr":"172.21.15.179:6831/1258425145","heartbeat_front_addr":"172.21.15.179:6829/1258425145","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:51:55.484446+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:52:29.156989+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:01.828224+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:53:36.215990+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:10.707726+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:54:44.578157+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:18.999366+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:55:54.069168+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:56:28.450354+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:03.153698+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:57:38.647737+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-26T13:58:13.581206+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8},{"from":5,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.137:6801/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:6800/1621047019":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/460394311":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1426358112":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/2012111099":"2024-03-27T13:49:52.910355+0000","172.21.15.137:0/1602182932":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6800/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/3592180121":"2024-03-27T13:49:19.001493+0000","172.21.15.137:6801/1955125876":"2024-03-27T13:49:19.001493+0000","172.21.15.137:0/487629871":"2024-03-27T13:49:52.910355+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-26T14:19:08.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23580a2880 msgr2=0x7f23580a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23580a2880 0x7f23580a4d40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f236402c240 tx=0x7f2364005fd0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 msgr2=0x7f237019a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:08.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 0x7f237019a0b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f235c00d770 tx=0x7f235c00dc30 comp rx=0 tx=0).stop 2024-03-26T14:19:08.812 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 shutdown_connections 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f23580a2880 0x7f23580a4d40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2370103260 0x7f23701a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f23701028a0 0x7f237019a0b0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 --2- 172.21.15.137:0/3866588872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23701016b0 0x7f2370199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 >> 172.21.15.137:0/3866588872 conn(0x7f23700fcc80 msgr2=0x7f23700738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 shutdown_connections 2024-03-26T14:19:08.813 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:08.810+0000 7f2377400640 1 -- 172.21.15.137:0/3866588872 wait complete. 2024-03-26T14:19:08.832 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-26T14:19:09.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 -- 172.21.15.137:0/3426375596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64109560 msgr2=0x7f3c6410b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 --2- 172.21.15.137:0/3426375596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64109560 0x7f3c6410b9f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c02bcd0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.044 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 -- 172.21.15.137:0/3426375596 shutdown_connections 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 --2- 172.21.15.137:0/3426375596 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64109560 0x7f3c6410b9f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 --2- 172.21.15.137:0/3426375596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c64074080 0x7f3c640744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 --2- 172.21.15.137:0/3426375596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64075660 0x7f3c64073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 -- 172.21.15.137:0/3426375596 >> 172.21.15.137:0/3426375596 conn(0x7f3c640fcc20 msgr2=0x7f3c640ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 -- 172.21.15.137:0/3426375596 shutdown_connections 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.039+0000 7f3c69600640 1 -- 172.21.15.137:0/3426375596 wait complete. 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 Processor -- start 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 -- start start 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c64074080 0x7f3c64199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.045 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 0x7f3c6419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64109560 0x7f3c641a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6410f440 con 0x7f3c64075660 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6410f2c0 con 0x7f3c64109560 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.040+0000 7f3c69600640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6410f5c0 con 0x7f3c64074080 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 0x7f3c6419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 0x7f3c6419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41766/0 (socket says 172.21.15.137:41766) 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 -- 172.21.15.137:0/4005315697 learned_addr learned my addr 172.21.15.137:0/4005315697 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c63400640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64109560 0x7f3c641a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.046 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62a00640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c64074080 0x7f3c64199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 -- 172.21.15.137:0/4005315697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c64074080 msgr2=0x7f3c64199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c64074080 0x7f3c64199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 -- 172.21.15.137:0/4005315697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64109560 msgr2=0x7f3c641a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64109560 0x7f3c641a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 -- 172.21.15.137:0/4005315697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c641a1720 con 0x7f3c64075660 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c63400640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64109560 0x7f3c641a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.041+0000 7f3c62000640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 0x7f3c6419a040 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400ea80 tx=0x7f3c5400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.042+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400ce40 con 0x7f3c64075660 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.042+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c54005020 con 0x7f3c64075660 2024-03-26T14:19:09.047 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.042+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c641a1a10 con 0x7f3c64075660 2024-03-26T14:19:09.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.042+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540116d0 con 0x7f3c64075660 2024-03-26T14:19:09.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.042+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c641a1f10 con 0x7f3c64075660 2024-03-26T14:19:09.048 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.043+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c64108320 con 0x7f3c64075660 2024-03-26T14:19:09.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.045+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3c540051d0 con 0x7f3c64075660 2024-03-26T14:19:09.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.046+0000 7f3c5b400640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c480a2800 0x7f3c480a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.046+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9216+0+0 (secure 0 0 0) 0x7f3c540cf600 con 0x7f3c64075660 2024-03-26T14:19:09.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.047+0000 7f3c62a00640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c480a2800 0x7f3c480a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.049 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.047+0000 7f3c62a00640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c480a2800 0x7f3c480a4cc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009aa0 tx=0x7f3c4c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:09.050 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.049+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3c54016030 con 0x7f3c64075660 2024-03-26T14:19:09.286 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:09 smithi137 ceph-mon[31539]: pgmap v961: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-26T14:19:09.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/1609911107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:19:09.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:09 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3866588872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:19:09.291 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.290+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3c6402d240 con 0x7f3c64075660 2024-03-26T14:19:09.296 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.295+0000 7f3c5b400640 1 -- 172.21.15.137:0/4005315697 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3c540ce090 con 0x7f3c64075660 2024-03-26T14:19:09.297 INFO:teuthology.orchestra.run.smithi137.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c480a2800 msgr2=0x7f3c480a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c480a2800 0x7f3c480a4cc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009aa0 tx=0x7f3c4c005f90 comp rx=0 tx=0).stop 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 msgr2=0x7f3c6419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 0x7f3c6419a040 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400ea80 tx=0x7f3c5400ef40 comp rx=0 tx=0).stop 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 shutdown_connections 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f3c480a2800 0x7f3c480a4cc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c64109560 0x7f3c641a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.300 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3c64075660 0x7f3c6419a040 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 --2- 172.21.15.137:0/4005315697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c64074080 0x7f3c64199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 >> 172.21.15.137:0/4005315697 conn(0x7f3c640fcc20 msgr2=0x7f3c641054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:09.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.298+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 shutdown_connections 2024-03-26T14:19:09.301 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.299+0000 7f3c69600640 1 -- 172.21.15.137:0/4005315697 wait complete. 2024-03-26T14:19:09.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-26T14:19:09.396 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mon[35501]: pgmap v961: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-26T14:19:09.396 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/1609911107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:19:09.396 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3866588872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:19:09.412 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-mon[35472]: pgmap v961: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-26T14:19:09.412 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/1609911107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-26T14:19:09.412 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3866588872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-26T14:19:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.530+0000 7f411f400640 1 -- 172.21.15.137:0/203383754 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411002cbe0 con 0x7f41200772a0 2024-03-26T14:19:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.530+0000 7f4126400640 1 -- 172.21.15.137:0/203383754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 msgr2=0x7f4120077700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.530+0000 7f4126400640 1 --2- 172.21.15.137:0/203383754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f4120077700 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f4110009900 tx=0x7f411002bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 -- 172.21.15.137:0/203383754 shutdown_connections 2024-03-26T14:19:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 --2- 172.21.15.137:0/203383754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4120077c40 0x7f41200784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.532 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 --2- 172.21.15.137:0/203383754 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f4120077700 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 --2- 172.21.15.137:0/203383754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4120076060 0x7f4120076480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 -- 172.21.15.137:0/203383754 >> 172.21.15.137:0/203383754 conn(0x7f41200fed10 msgr2=0x7f4120101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 -- 172.21.15.137:0/203383754 shutdown_connections 2024-03-26T14:19:09.533 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.531+0000 7f4126400640 1 -- 172.21.15.137:0/203383754 wait complete. 2024-03-26T14:19:09.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 Processor -- start 2024-03-26T14:19:09.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 -- start start 2024-03-26T14:19:09.535 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4120076060 0x7f412019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f412019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4120077c40 0x7f41201a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41201136d0 con 0x7f41200772a0 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4120113550 con 0x7f4120076060 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.532+0000 7f4126400640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4120113850 con 0x7f4120077c40 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f412019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f412019e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41792/0 (socket says 172.21.15.137:41792) 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 -- 172.21.15.137:0/2608763266 learned_addr learned my addr 172.21.15.137:0/2608763266 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f4125000640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4120076060 0x7f412019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f4125a00640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4120077c40 0x7f41201a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 -- 172.21.15.137:0/2608763266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4120077c40 msgr2=0x7f41201a5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.536 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4120077c40 0x7f41201a5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 -- 172.21.15.137:0/2608763266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4120076060 msgr2=0x7f412019ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4120076060 0x7f412019ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.533+0000 7f411fe00640 1 -- 172.21.15.137:0/2608763266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41201a59d0 con 0x7f41200772a0 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.534+0000 7f411fe00640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f412019e300 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4110006010 tx=0x7f41100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.534+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110043020 con 0x7f41200772a0 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.534+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4110004aa0 con 0x7f41200772a0 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.534+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4110003c60 con 0x7f41200772a0 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.534+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41201a5c60 con 0x7f41200772a0 2024-03-26T14:19:09.537 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.534+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41201a6160 con 0x7f41200772a0 2024-03-26T14:19:09.538 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.535+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120110110 con 0x7f41200772a0 2024-03-26T14:19:09.540 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.539+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4110046020 con 0x7f41200772a0 2024-03-26T14:19:09.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.540+0000 7f411d600640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41080a2880 0x7f41080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:09.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.540+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f41100f1920 con 0x7f41200772a0 2024-03-26T14:19:09.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.541+0000 7f4125000640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41080a2880 0x7f41080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:09.542 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.541+0000 7f4125000640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41080a2880 0x7f41080a4d40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f412019f290 tx=0x7f411400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:09.544 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.543+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f411004d020 con 0x7f41200772a0 2024-03-26T14:19:09.676 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i[66013]: ignoring --setuser ceph since I am not root 2024-03-26T14:19:09.676 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i[66013]: ignoring --setgroup ceph since I am not root 2024-03-26T14:19:09.676 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-mds[66017]: -- 172.21.15.179:0/3901766794 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b2e6d224e0 con 0x55b2e6cf8800 2024-03-26T14:19:09.676 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-mds[66017]: -- 172.21.15.179:0/3901766794 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2e6d25a40 con 0x55b2e6cf8800 2024-03-26T14:19:09.676 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i[66013]: starting mds.i at 2024-03-26T14:19:09.676 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c[65709]: ignoring --setuser ceph since I am not root 2024-03-26T14:19:09.677 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c[65709]: ignoring --setgroup ceph since I am not root 2024-03-26T14:19:09.677 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-mds[65713]: -- 172.21.15.179:0/4279523832 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643f4e93a40 con 0x5643f4e66c00 2024-03-26T14:19:09.677 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:19:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c[65709]: starting mds.c at 2024-03-26T14:19:09.704 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: ignoring --setuser ceph since I am not root 2024-03-26T14:19:09.705 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: ignoring --setgroup ceph since I am not root 2024-03-26T14:19:09.705 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mds[65823]: -- 172.21.15.159:0/4279281556 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cba6b69a40 con 0x55cba6b3d000 2024-03-26T14:19:09.705 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: starting mds.b at 2024-03-26T14:19:09.705 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e[65191]: ignoring --setuser ceph since I am not root 2024-03-26T14:19:09.705 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e[65191]: ignoring --setgroup ceph since I am not root 2024-03-26T14:19:09.705 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mds[65195]: -- 172.21.15.159:0/124491522 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56501b383a40 con 0x56501b356c00 2024-03-26T14:19:09.706 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e[65191]: starting mds.e at 2024-03-26T14:19:09.706 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h[65493]: ignoring --setuser ceph since I am not root 2024-03-26T14:19:09.706 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h[65493]: ignoring --setgroup ceph since I am not root 2024-03-26T14:19:09.706 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mds[65497]: -- 172.21.15.159:0/1912340054 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556602e8fa40 con 0x556602e63000 2024-03-26T14:19:09.706 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h[65493]: starting mds.h at 2024-03-26T14:19:09.707 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k[64888]: ignoring --setuser ceph since I am not root 2024-03-26T14:19:09.707 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k[64888]: ignoring --setgroup ceph since I am not root 2024-03-26T14:19:09.707 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-mds[64892]: -- 172.21.15.159:0/3369321367 <== mon.0 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a65a4bba40 con 0x55a65a48e800 2024-03-26T14:19:09.707 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:19:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k[64888]: starting mds.k at 2024-03-26T14:19:09.783 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:09.782+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f412002d300 con 0x7f41200772a0 2024-03-26T14:19:09.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:09 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a[31535]: 2024-03-26T14:19:09.292+0000 7f58bc400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T14:19:10.308 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.306+0000 7f411d600640 1 -- 172.21.15.137:0/2608763266 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f41100be600 con 0x7f41200772a0 2024-03-26T14:19:10.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.310+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41080a2880 msgr2=0x7f41080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:10.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.310+0000 7f4126400640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41080a2880 0x7f41080a4d40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f412019f290 tx=0x7f411400b390 comp rx=0 tx=0).stop 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.310+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 msgr2=0x7f412019e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.310+0000 7f4126400640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f412019e300 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f4110006010 tx=0x7f41100042f0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 shutdown_connections 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f41080a2880 0x7f41080a4d40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4120077c40 0x7f41201a5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41200772a0 0x7f412019e300 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 --2- 172.21.15.137:0/2608763266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4120076060 0x7f412019ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 >> 172.21.15.137:0/2608763266 conn(0x7f41200fed10 msgr2=0x7f4120102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 shutdown_connections 2024-03-26T14:19:10.313 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.311+0000 7f4126400640 1 -- 172.21.15.137:0/2608763266 wait complete. 2024-03-26T14:19:10.331 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4005315697' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T14:19:10.331 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T14:19:10.331 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T14:19:10.331 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/4005315697' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T14:19:10.331 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T14:19:10.331 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-26T14:19:10.332 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:10 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2608763266' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T14:19:10.335 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-26T14:19:10.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4005315697' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T14:19:10.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T14:19:10.425 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T14:19:10.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/4005315697' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T14:19:10.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T14:19:10.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-26T14:19:10.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:10 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2608763266' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T14:19:10.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4005315697' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-26T14:19:10.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-26T14:19:10.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-26T14:19:10.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/4005315697' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-26T14:19:10.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: osdmap e95: 12 total, 12 up, 12 in 2024-03-26T14:19:10.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-03-26T14:19:10.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:10 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2608763266' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 -- 172.21.15.137:0/3668991996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84041005a0 msgr2=0x7f84041009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 --2- 172.21.15.137:0/3668991996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84041005a0 0x7f84041009c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009ca0 tx=0x7f83f402c0f0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 -- 172.21.15.137:0/3668991996 shutdown_connections 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 --2- 172.21.15.137:0/3668991996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404102150 0x7f84041089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 --2- 172.21.15.137:0/3668991996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8404101790 0x7f8404101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 --2- 172.21.15.137:0/3668991996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84041005a0 0x7f84041009c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 -- 172.21.15.137:0/3668991996 >> 172.21.15.137:0/3668991996 conn(0x7f84040fbb30 msgr2=0x7f84040fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 -- 172.21.15.137:0/3668991996 shutdown_connections 2024-03-26T14:19:10.552 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.550+0000 7f840a800640 1 -- 172.21.15.137:0/3668991996 wait complete. 2024-03-26T14:19:10.554 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.551+0000 7f840a800640 1 Processor -- start 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.551+0000 7f840a800640 1 -- start start 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f840a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84041005a0 0x7f8404199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f840a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404101790 0x7f840419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f840a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 0x7f84041a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f840a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840410c0b0 con 0x7f8404102150 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f840a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840410bf30 con 0x7f8404101790 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f840a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840410c230 con 0x7f84041005a0 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f8403e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 0x7f84041a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f8403e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 0x7f84041a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41810/0 (socket says 172.21.15.137:41810) 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f8403e00640 1 -- 172.21.15.137:0/384169594 learned_addr learned my addr 172.21.15.137:0/384169594 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f8403400640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84041005a0 0x7f8404199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.552+0000 7f8402a00640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404101790 0x7f840419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:10.555 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8403e00640 1 -- 172.21.15.137:0/384169594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84041005a0 msgr2=0x7f8404199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8403e00640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84041005a0 0x7f8404199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8403e00640 1 -- 172.21.15.137:0/384169594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404101790 msgr2=0x7f840419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8403e00640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404101790 0x7f840419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8403e00640 1 -- 172.21.15.137:0/384169594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84041a1790 con 0x7f8404102150 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8402a00640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404101790 0x7f840419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f8403e00640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 0x7f84041a1120 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00d750 tx=0x7f83fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc017070 con 0x7f8404102150 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f83fc0054d0 con 0x7f8404102150 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84041a1a80 con 0x7f8404102150 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc004c60 con 0x7f8404102150 2024-03-26T14:19:10.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.553+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84041a1fd0 con 0x7f8404102150 2024-03-26T14:19:10.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.554+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84041a2200 con 0x7f8404102150 2024-03-26T14:19:10.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.557+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f83fc004e00 con 0x7f8404102150 2024-03-26T14:19:10.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.558+0000 7f83fbe00640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83e80a2880 0x7f83e80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:10.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.558+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9474+0+0 (secure 0 0 0) 0x7f83fc0cece0 con 0x7f8404102150 2024-03-26T14:19:10.559 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.558+0000 7f8403400640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83e80a2880 0x7f83e80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:10.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.559+0000 7f8403400640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83e80a2880 0x7f83e80a4d40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f83f402c630 tx=0x7f83f4032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:10.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.561+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f83fc09b8c0 con 0x7f8404102150 2024-03-26T14:19:10.764 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:10.763+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f84041a23d0 con 0x7f8404102150 2024-03-26T14:19:11.312 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.311+0000 7f83fbe00640 1 -- 172.21.15.137:0/384169594 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7f83fc09b1c0 con 0x7f8404102150 2024-03-26T14:19:11.313 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_metadata' removed 2024-03-26T14:19:11.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.317+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83e80a2880 msgr2=0x7f83e80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:11.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.317+0000 7f840a800640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83e80a2880 0x7f83e80a4d40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f83f402c630 tx=0x7f83f4032340 comp rx=0 tx=0).stop 2024-03-26T14:19:11.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 msgr2=0x7f84041a1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:11.319 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 0x7f84041a1120 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00d750 tx=0x7f83fc00dc10 comp rx=0 tx=0).stop 2024-03-26T14:19:11.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 shutdown_connections 2024-03-26T14:19:11.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f83e80a2880 0x7f83e80a4d40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8404101790 0x7f840419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8404102150 0x7f84041a1120 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.320 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 --2- 172.21.15.137:0/384169594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84041005a0 0x7f8404199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.318+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 >> 172.21.15.137:0/384169594 conn(0x7f84040fbb30 msgr2=0x7f84040688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:11.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.319+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 shutdown_connections 2024-03-26T14:19:11.321 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.319+0000 7f840a800640 1 -- 172.21.15.137:0/384169594 wait complete. 2024-03-26T14:19:11.340 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: pgmap v963: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-03-26T14:19:11.340 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T14:19:11.340 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T14:19:11.340 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: Cluster is now healthy 2024-03-26T14:19:11.340 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/2608763266' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T14:19:11.340 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6832/130079239,v1:172.21.15.159:6833/130079239] up:boot 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.179:6836/522774557,v1:172.21.15.179:6837/522774557] up:boot 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6834/1244325371,v1:172.21.15.159:6835/1244325371] up:boot 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6836/3902792239,v1:172.21.15.159:6837/3902792239] up:boot 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.179:6838/1782407342,v1:172.21.15.179:6839/1782407342] up:boot 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: mds.? [v2:172.21.15.159:6838/1788527143,v1:172.21.15.159:6839/1788527143] up:boot 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: fsmap 12 up:standby 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T14:19:11.341 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T14:19:11.342 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T14:19:11.342 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T14:19:11.342 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T14:19:11.342 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:11 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/384169594' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:11.343 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-26T14:19:11.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: pgmap v963: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-03-26T14:19:11.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T14:19:11.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T14:19:11.426 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: Cluster is now healthy 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/2608763266' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6832/130079239,v1:172.21.15.159:6833/130079239] up:boot 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.179:6836/522774557,v1:172.21.15.179:6837/522774557] up:boot 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6834/1244325371,v1:172.21.15.159:6835/1244325371] up:boot 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6836/3902792239,v1:172.21.15.159:6837/3902792239] up:boot 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.179:6838/1782407342,v1:172.21.15.179:6839/1782407342] up:boot 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: mds.? [v2:172.21.15.159:6838/1788527143,v1:172.21.15.159:6839/1788527143] up:boot 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: fsmap 12 up:standby 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T14:19:11.427 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T14:19:11.428 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T14:19:11.428 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T14:19:11.428 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T14:19:11.428 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:11 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/384169594' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: pgmap v963: 129 pgs: 129 active+clean; 13 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: Cluster is now healthy 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/2608763266' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6832/130079239,v1:172.21.15.159:6833/130079239] up:boot 2024-03-26T14:19:11.455 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.179:6836/522774557,v1:172.21.15.179:6837/522774557] up:boot 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6834/1244325371,v1:172.21.15.159:6835/1244325371] up:boot 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6836/3902792239,v1:172.21.15.159:6837/3902792239] up:boot 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.179:6838/1782407342,v1:172.21.15.179:6839/1782407342] up:boot 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: mds.? [v2:172.21.15.159:6838/1788527143,v1:172.21.15.159:6839/1788527143] up:boot 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: fsmap 12 up:standby 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-26T14:19:11.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='mgr.14152 172.21.15.137:0/2909292902' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-26T14:19:11.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:11 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/384169594' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:11.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.554+0000 7f7e2a800640 1 -- 172.21.15.137:0/2422771316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24102120 msgr2=0x7f7e24104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:11.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.554+0000 7f7e2a800640 1 --2- 172.21.15.137:0/2422771316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24102120 0x7f7e24104510 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009900 tx=0x7f7e1c02bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:11.556 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 -- 172.21.15.137:0/2422771316 shutdown_connections 2024-03-26T14:19:11.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 --2- 172.21.15.137:0/2422771316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24104a50 0x7f7e24106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 --2- 172.21.15.137:0/2422771316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24107380 0x7f7e241097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 --2- 172.21.15.137:0/2422771316 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24102120 0x7f7e24104510 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 -- 172.21.15.137:0/2422771316 >> 172.21.15.137:0/2422771316 conn(0x7f7e240fbb10 msgr2=0x7f7e240fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:11.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 -- 172.21.15.137:0/2422771316 shutdown_connections 2024-03-26T14:19:11.557 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.555+0000 7f7e2a800640 1 -- 172.21.15.137:0/2422771316 wait complete. 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 Processor -- start 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 -- start start 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24102120 0x7f7e24199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24104a50 0x7f7e2419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 0x7f7e241a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2410d250 con 0x7f7e24107380 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2410d0d0 con 0x7f7e24102120 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e2a800640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2410d3d0 con 0x7f7e24104a50 2024-03-26T14:19:11.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e23e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 0x7f7e241a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e23e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 0x7f7e241a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41820/0 (socket says 172.21.15.137:41820) 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.556+0000 7f7e23e00640 1 -- 172.21.15.137:0/3391077005 learned_addr learned my addr 172.21.15.137:0/3391077005 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23400640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24102120 0x7f7e24199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e22a00640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24104a50 0x7f7e2419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23e00640 1 -- 172.21.15.137:0/3391077005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24104a50 msgr2=0x7f7e2419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23e00640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24104a50 0x7f7e2419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23e00640 1 -- 172.21.15.137:0/3391077005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24102120 msgr2=0x7f7e24199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23e00640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24102120 0x7f7e24199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23e00640 1 -- 172.21.15.137:0/3391077005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e241a1760 con 0x7f7e24107380 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e22a00640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24104a50 0x7f7e2419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-26T14:19:11.561 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23400640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24102120 0x7f7e24199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-26T14:19:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.557+0000 7f7e23e00640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 0x7f7e241a10f0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b970 tx=0x7f7e1400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.558+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14019070 con 0x7f7e24107380 2024-03-26T14:19:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.558+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e241a1a50 con 0x7f7e24107380 2024-03-26T14:19:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.558+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e14014440 con 0x7f7e24107380 2024-03-26T14:19:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.558+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e140075d0 con 0x7f7e24107380 2024-03-26T14:19:11.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.558+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e241a1fa0 con 0x7f7e24107380 2024-03-26T14:19:11.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.559+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e24108e70 con 0x7f7e24107380 2024-03-26T14:19:11.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.561+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7e1401c070 con 0x7f7e24107380 2024-03-26T14:19:11.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.562+0000 7f7e1be00640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e080a2880 0x7f7e080a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:11.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.562+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8949+0+0 (secure 0 0 0) 0x7f7e140ce360 con 0x7f7e24107380 2024-03-26T14:19:11.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.562+0000 7f7e23400640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e080a2880 0x7f7e080a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:11.564 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.563+0000 7f7e23400640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e080a2880 0x7f7e080a4d40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c000c00 tx=0x7f7e1c0095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:11.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.567+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7e1409b0d0 con 0x7f7e24107380 2024-03-26T14:19:11.771 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:11.769+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7e240622a0 con 0x7f7e24107380 2024-03-26T14:19:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.326+0000 7f7e1be00640 1 -- 172.21.15.137:0/3391077005 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7e1409a9d0 con 0x7f7e24107380 2024-03-26T14:19:12.328 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_data' removed 2024-03-26T14:19:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e080a2880 msgr2=0x7f7e080a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e080a2880 0x7f7e080a4d40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c000c00 tx=0x7f7e1c0095f0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 msgr2=0x7f7e241a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 0x7f7e241a10f0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f7e1400b970 tx=0x7f7e1400be30 comp rx=0 tx=0).stop 2024-03-26T14:19:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 shutdown_connections 2024-03-26T14:19:12.332 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f7e080a2880 0x7f7e080a4d40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7e24107380 0x7f7e241a10f0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7e24104a50 0x7f7e2419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.330+0000 7f7e2a800640 1 --2- 172.21.15.137:0/3391077005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e24102120 0x7f7e24199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.331+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 >> 172.21.15.137:0/3391077005 conn(0x7f7e240fbb10 msgr2=0x7f7e241018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.331+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 shutdown_connections 2024-03-26T14:19:12.333 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.331+0000 7f7e2a800640 1 -- 172.21.15.137:0/3391077005 wait complete. 2024-03-26T14:19:12.358 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-26T14:19:12.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 -- 172.21.15.137:0/2076625990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab8109000 msgr2=0x7f8ab810f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 --2- 172.21.15.137:0/2076625990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab8109000 0x7f8ab810f9e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa802bc30 comp rx=0 tx=0).stop 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 -- 172.21.15.137:0/2076625990 shutdown_connections 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 --2- 172.21.15.137:0/2076625990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ab810ff20 0x7f8ab8112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 --2- 172.21.15.137:0/2076625990 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab8109000 0x7f8ab810f9e0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 --2- 172.21.15.137:0/2076625990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ab81086a0 0x7f8ab8108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 -- 172.21.15.137:0/2076625990 >> 172.21.15.137:0/2076625990 conn(0x7f8ab806e8a0 msgr2=0x7f8ab8070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 -- 172.21.15.137:0/2076625990 shutdown_connections 2024-03-26T14:19:12.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.561+0000 7f8abd200640 1 -- 172.21.15.137:0/2076625990 wait complete. 2024-03-26T14:19:12.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.562+0000 7f8abd200640 1 Processor -- start 2024-03-26T14:19:12.565 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.562+0000 7f8abd200640 1 -- start start 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.562+0000 7f8abd200640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ab81086a0 0x7f8ab81a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8abd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ab8109000 0x7f8ab81a2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8abd200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 0x7f8ab81a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8abd200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8115a00 con 0x7f8ab810ff20 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8abd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8115880 con 0x7f8ab8109000 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8abd200640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8115b80 con 0x7f8ab81086a0 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 0x7f8ab81a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 0x7f8ab81a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:41826/0 (socket says 172.21.15.137:41826) 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 -- 172.21.15.137:0/3691305175 learned_addr learned my addr 172.21.15.137:0/3691305175 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab5e00640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ab81086a0 0x7f8ab81a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8aafe00640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ab8109000 0x7f8ab81a2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:12.566 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 -- 172.21.15.137:0/3691305175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ab81086a0 msgr2=0x7f8ab81a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ab81086a0 0x7f8ab81a22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 -- 172.21.15.137:0/3691305175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ab8109000 msgr2=0x7f8ab81a2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ab8109000 0x7f8ab81a2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.563+0000 7f8ab6800640 1 -- 172.21.15.137:0/3691305175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab81a9f10 con 0x7f8ab810ff20 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.564+0000 7f8ab6800640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 0x7f8ab81a98a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000eaa0 tx=0x7f8ab000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.564+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000cdb0 con 0x7f8ab810ff20 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.564+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8ab0005020 con 0x7f8ab810ff20 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.564+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab81aa200 con 0x7f8ab810ff20 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.564+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00116d0 con 0x7f8ab810ff20 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.564+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab81aa6d0 con 0x7f8ab810ff20 2024-03-26T14:19:12.567 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.565+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab81aa990 con 0x7f8ab810ff20 2024-03-26T14:19:12.568 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.567+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8ab00051d0 con 0x7f8ab810ff20 2024-03-26T14:19:12.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.569+0000 7f8aaea00640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a980a2800 0x7f8a980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-26T14:19:12.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.570+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8324+0+0 (secure 0 0 0) 0x7f8ab00cede0 con 0x7f8ab810ff20 2024-03-26T14:19:12.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.570+0000 7f8ab5e00640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a980a2800 0x7f8a980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-26T14:19:12.571 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.570+0000 7f8ab5e00640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a980a2800 0x7f8a980a4cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005fd0 tx=0x7f8aa0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-26T14:19:12.575 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.574+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8ab0016030 con 0x7f8ab810ff20 2024-03-26T14:19:12.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/384169594' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:12.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:12 smithi179 ceph-mon[35472]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T14:19:12.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:12 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3391077005' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:12.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/384169594' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:12.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:12 smithi159 ceph-mon[35501]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T14:19:12.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:12 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3391077005' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:12.775 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:12.773+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ab81aab60 con 0x7f8ab810ff20 2024-03-26T14:19:12.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/384169594' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:12.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:12 smithi137 ceph-mon[31539]: osdmap e96: 12 total, 12 up, 12 in 2024-03-26T14:19:12.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:12 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3391077005' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.340+0000 7f8aaea00640 1 -- 172.21.15.137:0/3691305175 <== mon.0 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) v1 ==== 178+0+0 (secure 0 0 0) 0x7f8ab009bdc0 con 0x7f8ab810ff20 2024-03-26T14:19:13.341 INFO:teuthology.orchestra.run.smithi137.stderr:pool 'cephfs_data_ec' removed 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a980a2800 msgr2=0x7f8a980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a980a2800 0x7f8a980a4cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005fd0 tx=0x7f8aa0005850 comp rx=0 tx=0).stop 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 msgr2=0x7f8ab81a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 0x7f8ab81a98a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000eaa0 tx=0x7f8ab000ef60 comp rx=0 tx=0).stop 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 shutdown_connections 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:6800/3167395503,v1:172.21.15.137:6801/3167395503] conn(0x7f8a980a2800 0x7f8a980a4cc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ab8109000 0x7f8ab81a2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:13.347 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8ab810ff20 0x7f8ab81a98a0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 --2- 172.21.15.137:0/3691305175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ab81086a0 0x7f8ab81a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-26T14:19:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.345+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 >> 172.21.15.137:0/3691305175 conn(0x7f8ab806e8a0 msgr2=0x7f8ab806ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-26T14:19:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.346+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 shutdown_connections 2024-03-26T14:19:13.348 INFO:teuthology.orchestra.run.smithi137.stderr:2024-03-26T14:19:13.346+0000 7f8abd200640 1 -- 172.21.15.137:0/3691305175 wait complete. 2024-03-26T14:19:13.371 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T14:19:13.449 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T14:19:13.558 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T14:19:13.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:13 smithi179 ceph-mon[35472]: pgmap v965: 97 pgs: 97 active+clean; 577 KiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T14:19:13.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:13 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3391077005' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:13.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:13 smithi179 ceph-mon[35472]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T14:19:13.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:13 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3691305175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:13.660 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-26T14:19:13.660 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T14:19:13.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:13 smithi159 ceph-mon[35501]: pgmap v965: 97 pgs: 97 active+clean; 577 KiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T14:19:13.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:13 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3391077005' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:13.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:13 smithi159 ceph-mon[35501]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T14:19:13.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:13 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3691305175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:13.718 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T14:19:13.774 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-26T14:19:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:13 smithi137 ceph-mon[31539]: pgmap v965: 97 pgs: 97 active+clean; 577 KiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-26T14:19:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:13 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3391077005' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:13 smithi137 ceph-mon[31539]: osdmap e97: 12 total, 12 up, 12 in 2024-03-26T14:19:13.786 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:13 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3691305175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-26T14:19:13.830 INFO:tasks.cephadm:Stopping all daemons... 2024-03-26T14:19:13.831 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-26T14:19:13.831 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a 2024-03-26T14:19:14.287 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:13 smithi137 systemd[1]: Stopping Ceph mon.a for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:14.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:14 smithi179 ceph-mon[35472]: from='client.? 172.21.15.137:0/3691305175' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:14.675 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:14 smithi179 ceph-mon[35472]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T14:19:14.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:14 smithi159 ceph-mon[35501]: from='client.? 172.21.15.137:0/3691305175' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:14.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:14 smithi159 ceph-mon[35501]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T14:19:14.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 ceph-mon[31539]: from='client.? 172.21.15.137:0/3691305175' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-03-26T14:19:14.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 ceph-mon[31539]: osdmap e98: 12 total, 12 up, 12 in 2024-03-26T14:19:14.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a[31535]: 2024-03-26T14:19:14.364+0000 7f58c3200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:14.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a[31535]: 2024-03-26T14:19:14.364+0000 7f58c3200700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-26T14:19:14.787 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 podman[104701]: 2024-03-26 14:19:14.529636525 +0000 UTC m=+0.307809880 container died 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T14:19:15.253 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 podman[104701]: 2024-03-26 14:19:14.991404366 +0000 UTC m=+0.769577701 container cleanup 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:19:15.253 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:14 smithi137 bash[104701]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a 2024-03-26T14:19:15.536 INFO:journalctl@ceph.mon.a.smithi137.stdout:Mar 26 14:19:15 smithi137 podman[104715]: 2024-03-26 14:19:15.25132909 +0000 UTC m=+0.700777888 container remove 15e9559f90d09f4351aae1e274016bdbd68563ced2dc6b85242d5a20c5ac4f4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T14:19:15.773 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.a.service' 2024-03-26T14:19:15.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:15.903 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-26T14:19:15.903 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-26T14:19:15.903 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.b 2024-03-26T14:19:16.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:16 smithi159 systemd[1]: Stopping Ceph mon.b for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:16.954 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b[35497]: 2024-03-26T14:19:16.553+0000 7fcf85400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:16.955 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b[35497]: 2024-03-26T14:19:16.553+0000 7fcf85400700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-26T14:19:16.955 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:16 smithi159 podman[71061]: 2024-03-26 14:19:16.657398378 +0000 UTC m=+0.305079607 container died a88340e27b397d82cf5b106a2419704e567b4a8c2bf15a3bcce42f64c79161ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-26T14:19:17.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:17 smithi159 podman[71061]: 2024-03-26 14:19:17.147934848 +0000 UTC m=+0.795616084 container cleanup a88340e27b397d82cf5b106a2419704e567b4a8c2bf15a3bcce42f64c79161ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T14:19:17.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:17 smithi159 bash[71061]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b 2024-03-26T14:19:17.704 INFO:journalctl@ceph.mon.b.smithi159.stdout:Mar 26 14:19:17 smithi159 podman[71075]: 2024-03-26 14:19:17.449135396 +0000 UTC m=+0.776418898 container remove a88340e27b397d82cf5b106a2419704e567b4a8c2bf15a3bcce42f64c79161ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T14:19:17.964 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.b.service' 2024-03-26T14:19:18.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:18.127 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-26T14:19:18.128 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-26T14:19:18.128 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.c 2024-03-26T14:19:18.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:18 smithi179 systemd[1]: Stopping Ceph mon.c for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:18.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-c[35463]: 2024-03-26T14:19:18.739+0000 7f1dff600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:18.925 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-c[35463]: 2024-03-26T14:19:18.739+0000 7f1dff600700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-26T14:19:19.354 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:19 smithi179 podman[71213]: 2024-03-26 14:19:19.036478382 +0000 UTC m=+0.472342338 container died df3ce41e900ce8d2eb2b7d8b0b5ff2736e4c70a0c328b3f5b551725ba7ec6cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-26T14:19:19.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:19 smithi179 podman[71213]: 2024-03-26 14:19:19.498840362 +0000 UTC m=+0.934704312 container cleanup df3ce41e900ce8d2eb2b7d8b0b5ff2736e4c70a0c328b3f5b551725ba7ec6cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:19:19.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:19 smithi179 bash[71213]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-c 2024-03-26T14:19:20.175 INFO:journalctl@ceph.mon.c.smithi179.stdout:Mar 26 14:19:19 smithi179 podman[71235]: 2024-03-26 14:19:19.767356239 +0000 UTC m=+0.720818853 container remove df3ce41e900ce8d2eb2b7d8b0b5ff2736e4c70a0c328b3f5b551725ba7ec6cf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-03-26T14:19:20.269 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mon.c.service' 2024-03-26T14:19:20.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:20.405 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-26T14:19:20.405 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-26T14:19:20.405 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x 2024-03-26T14:19:20.786 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:20 smithi137 systemd[1]: Stopping Ceph mgr.x for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:21.287 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:19:20.968+0000 7f8c37200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:21.287 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x[31769]: 2024-03-26T14:19:20.968+0000 7f8c37200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-26T14:19:21.287 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:21 smithi137 podman[104821]: 2024-03-26 14:19:21.085498863 +0000 UTC m=+0.245353930 container died fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:19:21.786 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:21 smithi137 podman[104821]: 2024-03-26 14:19:21.663977436 +0000 UTC m=+0.823832526 container cleanup fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-26T14:19:21.787 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:21 smithi137 bash[104821]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x 2024-03-26T14:19:22.286 INFO:journalctl@ceph.mgr.x.smithi137.stdout:Mar 26 14:19:21 smithi137 podman[104835]: 2024-03-26 14:19:21.932310116 +0000 UTC m=+0.834920830 container remove fc3bed4ebd1054b9bbeb074b25b44df4e6140b203d691505756761ff83f32d20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3be9a5c9bc793e11e2800a8c0c696e8b46742033, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-x, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-26T14:19:22.471 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.x.service' 2024-03-26T14:19:22.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:22.590 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-26T14:19:22.590 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-26T14:19:22.591 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.y 2024-03-26T14:19:22.954 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 14:19:22 smithi159 systemd[1]: Stopping Ceph mgr.y for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:23.562 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 14:19:23 smithi159 podman[71183]: 2024-03-26 14:19:23.240388237 +0000 UTC m=+0.254291840 container died 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T14:19:23.954 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 14:19:23 smithi159 podman[71183]: 2024-03-26 14:19:23.728106558 +0000 UTC m=+0.742010175 container cleanup 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-03-26T14:19:23.954 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 14:19:23 smithi159 bash[71183]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y 2024-03-26T14:19:24.454 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 14:19:23 smithi159 podman[71197]: 2024-03-26 14:19:23.996278782 +0000 UTC m=+0.746367858 container remove 14dc75baac4f26447e53ec9148e9e72567b0999c6cfd5f27efcf12acaa89f222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:19:24.454 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Mar 26 14:19:24 smithi159 systemd[1]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-26T14:19:24.513 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.y.service' 2024-03-26T14:19:24.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:24.635 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-26T14:19:24.635 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-26T14:19:24.635 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.z 2024-03-26T14:19:25.175 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 14:19:24 smithi179 systemd[1]: Stopping Ceph mgr.z for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:25.596 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 14:19:25 smithi179 podman[71349]: 2024-03-26 14:19:25.283833093 +0000 UTC m=+0.230460029 container died c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-26T14:19:25.925 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 14:19:25 smithi179 podman[71349]: 2024-03-26 14:19:25.771383558 +0000 UTC m=+0.718010464 container cleanup c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T14:19:25.925 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 14:19:25 smithi179 bash[71349]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z 2024-03-26T14:19:26.425 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 14:19:25 smithi179 podman[71363]: 2024-03-26 14:19:25.998208457 +0000 UTC m=+0.698542925 container remove c08f9860921cf4c56f145e315c44ec1b97e306176f5cfb681f635aa46146bfc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-26T14:19:26.425 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Mar 26 14:19:26 smithi179 systemd[1]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-26T14:19:26.468 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mgr.z.service' 2024-03-26T14:19:26.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:26.584 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-26T14:19:26.585 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-26T14:19:26.585 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.0 2024-03-26T14:19:27.036 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:26 smithi137 systemd[1]: Stopping Ceph osd.0 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:27.537 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0[46370]: 2024-03-26T14:19:27.165+0000 7ff413400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:27.537 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0[46370]: 2024-03-26T14:19:27.165+0000 7ff413400700 -1 osd.0 98 *** Got signal Terminated *** 2024-03-26T14:19:27.537 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0[46370]: 2024-03-26T14:19:27.165+0000 7ff413400700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:19:32.958 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:32 smithi137 podman[104941]: 2024-03-26 14:19:32.595225819 +0000 UTC m=+5.556365185 container died 1ff2c4b90707939856008f641ce50a858cdbcef19ee0cb82553f3e299d99a7b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-26T14:19:33.286 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:33 smithi137 podman[104941]: 2024-03-26 14:19:33.085580206 +0000 UTC m=+6.046719584 container cleanup 1ff2c4b90707939856008f641ce50a858cdbcef19ee0cb82553f3e299d99a7b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T14:19:33.286 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:33 smithi137 bash[104941]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0 2024-03-26T14:19:33.786 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:33 smithi137 podman[104961]: 2024-03-26 14:19:33.312022014 +0000 UTC m=+0.698259685 container remove 1ff2c4b90707939856008f641ce50a858cdbcef19ee0cb82553f3e299d99a7b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T14:19:34.409 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:34 smithi137 podman[105059]: 2024-03-26 14:19:33.973458053 +0000 UTC m=+0.035192241 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:19:34.409 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:34 smithi137 podman[105059]: 2024-03-26 14:19:34.129944239 +0000 UTC m=+0.191678464 container create cfbdd59756ec73a5ac574a85fee615e937c0f5b7c07a2311b6d417cc1793a13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-26T14:19:34.787 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:34 smithi137 podman[105059]: 2024-03-26 14:19:34.408376564 +0000 UTC m=+0.470110802 container init cfbdd59756ec73a5ac574a85fee615e937c0f5b7c07a2311b6d417cc1793a13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-03-26T14:19:34.787 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:34 smithi137 podman[105059]: 2024-03-26 14:19:34.419333466 +0000 UTC m=+0.481067661 container start cfbdd59756ec73a5ac574a85fee615e937c0f5b7c07a2311b6d417cc1793a13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0-deactivate, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T14:19:34.787 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:34 smithi137 podman[105059]: 2024-03-26 14:19:34.524309682 +0000 UTC m=+0.586043954 container attach cfbdd59756ec73a5ac574a85fee615e937c0f5b7c07a2311b6d417cc1793a13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-26T14:19:35.258 INFO:journalctl@ceph.osd.0.smithi137.stdout:Mar 26 14:19:34 smithi137 podman[105059]: 2024-03-26 14:19:34.838967097 +0000 UTC m=+0.900701352 container died cfbdd59756ec73a5ac574a85fee615e937c0f5b7c07a2311b6d417cc1793a13f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-0-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:19:35.681 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.0.service' 2024-03-26T14:19:35.806 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:35.807 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-26T14:19:35.807 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-26T14:19:35.807 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.1 2024-03-26T14:19:36.204 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:35 smithi159 systemd[1]: Stopping Ceph osd.1 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:36.705 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1[40085]: 2024-03-26T14:19:36.421+0000 7fcb97800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:36.705 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1[40085]: 2024-03-26T14:19:36.421+0000 7fcb97800700 -1 osd.1 98 *** Got signal Terminated *** 2024-03-26T14:19:36.705 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1[40085]: 2024-03-26T14:19:36.421+0000 7fcb97800700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:19:42.204 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:41 smithi159 podman[71303]: 2024-03-26 14:19:41.866991521 +0000 UTC m=+5.605262124 container stop 8b7036724f6fbc4d3443ce49853af7b1b9d37827e47fc2db2f38b374aa6fac10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-03-26T14:19:42.204 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:41 smithi159 podman[71303]: 2024-03-26 14:19:41.893029378 +0000 UTC m=+5.631299978 container died 8b7036724f6fbc4d3443ce49853af7b1b9d37827e47fc2db2f38b374aa6fac10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-26T14:19:42.635 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:42 smithi159 podman[71303]: 2024-03-26 14:19:42.374155023 +0000 UTC m=+6.112425665 container cleanup 8b7036724f6fbc4d3443ce49853af7b1b9d37827e47fc2db2f38b374aa6fac10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T14:19:42.635 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:42 smithi159 bash[71303]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1 2024-03-26T14:19:42.954 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:42 smithi159 podman[71321]: 2024-03-26 14:19:42.63407478 +0000 UTC m=+0.749518090 container remove 8b7036724f6fbc4d3443ce49853af7b1b9d37827e47fc2db2f38b374aa6fac10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:19:43.823 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:43 smithi159 podman[71409]: 2024-03-26 14:19:43.399733581 +0000 UTC m=+0.033496529 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:19:43.823 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:43 smithi159 podman[71409]: 2024-03-26 14:19:43.518344073 +0000 UTC m=+0.152107055 container create b5adb36b19650c6fce7aaafc6a37e0de7ef96221377da560c4b77cb9620a70a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-26T14:19:44.204 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:43 smithi159 podman[71409]: 2024-03-26 14:19:43.822024907 +0000 UTC m=+0.455787896 container init b5adb36b19650c6fce7aaafc6a37e0de7ef96221377da560c4b77cb9620a70a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:19:44.204 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:43 smithi159 podman[71409]: 2024-03-26 14:19:43.832976989 +0000 UTC m=+0.466739955 container start b5adb36b19650c6fce7aaafc6a37e0de7ef96221377da560c4b77cb9620a70a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T14:19:44.204 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:43 smithi159 podman[71409]: 2024-03-26 14:19:43.929424934 +0000 UTC m=+0.563187976 container attach b5adb36b19650c6fce7aaafc6a37e0de7ef96221377da560c4b77cb9620a70a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-26T14:19:44.689 INFO:journalctl@ceph.osd.1.smithi159.stdout:Mar 26 14:19:44 smithi159 podman[71409]: 2024-03-26 14:19:44.269991281 +0000 UTC m=+0.903754277 container died b5adb36b19650c6fce7aaafc6a37e0de7ef96221377da560c4b77cb9620a70a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:19:45.162 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.1.service' 2024-03-26T14:19:45.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:45.284 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-26T14:19:45.284 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-26T14:19:45.284 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.2 2024-03-26T14:19:45.676 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:45 smithi179 systemd[1]: Stopping Ceph osd.2 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:46.175 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2[40490]: 2024-03-26T14:19:45.899+0000 7f63cf400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:46.176 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2[40490]: 2024-03-26T14:19:45.899+0000 7f63cf400700 -1 osd.2 98 *** Got signal Terminated *** 2024-03-26T14:19:46.176 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2[40490]: 2024-03-26T14:19:45.899+0000 7f63cf400700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:19:51.825 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:51 smithi179 podman[71471]: 2024-03-26 14:19:51.50946673 +0000 UTC m=+5.764478337 container stop 98a0f8a9c54d719af2d7338f3814e3ab1ff7f2e34ee3093362f3b8f66c504015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T14:19:51.825 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:51 smithi179 podman[71471]: 2024-03-26 14:19:51.540236135 +0000 UTC m=+5.795247735 container died 98a0f8a9c54d719af2d7338f3814e3ab1ff7f2e34ee3093362f3b8f66c504015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:19:52.175 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:51 smithi179 podman[71471]: 2024-03-26 14:19:51.959084301 +0000 UTC m=+6.214095971 container cleanup 98a0f8a9c54d719af2d7338f3814e3ab1ff7f2e34ee3093362f3b8f66c504015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:19:52.175 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:51 smithi179 bash[71471]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2 2024-03-26T14:19:52.675 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:52 smithi179 podman[71490]: 2024-03-26 14:19:52.260523252 +0000 UTC m=+0.732660212 container remove 98a0f8a9c54d719af2d7338f3814e3ab1ff7f2e34ee3093362f3b8f66c504015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:19:53.369 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:53 smithi179 podman[71586]: 2024-03-26 14:19:52.944177293 +0000 UTC m=+0.034273624 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:19:53.369 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:53 smithi179 podman[71586]: 2024-03-26 14:19:53.06374274 +0000 UTC m=+0.153839101 container create cce9c831f047c522a8605c396570cd88513c6448cdc21ac48459a2818931dd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:19:53.369 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:53 smithi179 podman[71586]: 2024-03-26 14:19:53.367544962 +0000 UTC m=+0.457641329 container init cce9c831f047c522a8605c396570cd88513c6448cdc21ac48459a2818931dd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-26T14:19:53.675 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:53 smithi179 podman[71586]: 2024-03-26 14:19:53.379011373 +0000 UTC m=+0.469107784 container start cce9c831f047c522a8605c396570cd88513c6448cdc21ac48459a2818931dd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-26T14:19:53.675 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:53 smithi179 podman[71586]: 2024-03-26 14:19:53.466469048 +0000 UTC m=+0.556565419 container attach cce9c831f047c522a8605c396570cd88513c6448cdc21ac48459a2818931dd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:19:54.175 INFO:journalctl@ceph.osd.2.smithi179.stdout:Mar 26 14:19:53 smithi179 podman[71613]: 2024-03-26 14:19:53.871380675 +0000 UTC m=+0.033847752 container died cce9c831f047c522a8605c396570cd88513c6448cdc21ac48459a2818931dd60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-26T14:19:54.614 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.2.service' 2024-03-26T14:19:54.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:19:54.738 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-26T14:19:54.738 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-26T14:19:54.738 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.3 2024-03-26T14:19:55.263 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:54 smithi137 systemd[1]: Stopping Ceph osd.3 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:19:55.537 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:55.260+0000 7f4435200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:19:55.537 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:55.260+0000 7f4435200700 -1 osd.3 98 *** Got signal Terminated *** 2024-03-26T14:19:55.537 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:55.260+0000 7f4435200700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:19:55.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:19:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:19:55.522+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:19:56.536 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:56 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:56.136+0000 7f4430a00700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.235922+0000 front 2024-03-26T14:19:30.235291+0000 (oldest deadline 2024-03-26T14:19:56.135102+0000) 2024-03-26T14:19:56.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:19:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:19:56.434+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:19:56.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:19:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:19:56.557+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:19:57.536 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:57 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:57.177+0000 7f4430a00700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.235922+0000 front 2024-03-26T14:19:30.235291+0000 (oldest deadline 2024-03-26T14:19:56.135102+0000) 2024-03-26T14:19:57.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:19:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:19:57.407+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:19:57.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:19:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:19:57.604+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:19:57.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:19:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:19:57.777+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:19:58.536 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:58 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:58.190+0000 7f4430a00700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.235922+0000 front 2024-03-26T14:19:30.235291+0000 (oldest deadline 2024-03-26T14:19:56.135102+0000) 2024-03-26T14:19:58.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:19:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:19:58.383+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:19:58.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:19:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:19:58.589+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:19:58.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:19:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:19:58.733+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:19:59.536 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:19:59 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:19:59.234+0000 7f4430a00700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.235922+0000 front 2024-03-26T14:19:30.235291+0000 (oldest deadline 2024-03-26T14:19:56.135102+0000) 2024-03-26T14:19:59.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:19:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:19:59.341+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:19:59.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:19:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:19:59.540+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:19:59.925 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:19:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:19:59.692+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:00.536 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:00 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3[52904]: 2024-03-26T14:20:00.229+0000 7f4430a00700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.235922+0000 front 2024-03-26T14:19:30.235291+0000 (oldest deadline 2024-03-26T14:19:56.135102+0000) 2024-03-26T14:20:00.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:00.354+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:00.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:00.491+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:00.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:00.686+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:01.036 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:00 smithi137 podman[105166]: 2024-03-26 14:20:00.741356826 +0000 UTC m=+5.612349974 container died c6c07313ead3f7f1c7e68a64d1c3899f0008640a438bcd89cdca2c57de1f7dbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-26T14:20:01.439 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:01 smithi137 podman[105166]: 2024-03-26 14:20:01.188846582 +0000 UTC m=+6.059839730 container cleanup c6c07313ead3f7f1c7e68a64d1c3899f0008640a438bcd89cdca2c57de1f7dbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.vendor=CentOS) 2024-03-26T14:20:01.439 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:01 smithi137 bash[105166]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3 2024-03-26T14:20:01.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:01.334+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:01.786 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:01 smithi137 podman[105185]: 2024-03-26 14:20:01.435327167 +0000 UTC m=+0.676065026 container remove c6c07313ead3f7f1c7e68a64d1c3899f0008640a438bcd89cdca2c57de1f7dbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:20:01.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:01 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:01.489+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:01.925 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:01 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:01.646+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:02.404 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:02 smithi137 podman[105272]: 2024-03-26 14:20:02.133757951 +0000 UTC m=+0.035605472 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:02.404 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:02 smithi137 podman[105272]: 2024-03-26 14:20:02.249352849 +0000 UTC m=+0.151200394 container create e7fed47da6107b1a9c0cd638a1fdcdf1752f3ffa8bff4557a8e5d57f04a3a5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-26T14:20:02.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:02.354+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:02.720 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:02 smithi137 podman[105272]: 2024-03-26 14:20:02.494644255 +0000 UTC m=+0.396491816 container init e7fed47da6107b1a9c0cd638a1fdcdf1752f3ffa8bff4557a8e5d57f04a3a5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux ) 2024-03-26T14:20:02.720 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:02 smithi137 podman[105272]: 2024-03-26 14:20:02.505564637 +0000 UTC m=+0.407412168 container start e7fed47da6107b1a9c0cd638a1fdcdf1752f3ffa8bff4557a8e5d57f04a3a5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T14:20:02.720 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:02 smithi137 podman[105272]: 2024-03-26 14:20:02.59643707 +0000 UTC m=+0.498284664 container attach e7fed47da6107b1a9c0cd638a1fdcdf1752f3ffa8bff4557a8e5d57f04a3a5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-26T14:20:02.828 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:02.529+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:02.829 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:02.658+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:03.036 INFO:journalctl@ceph.osd.3.smithi137.stdout:Mar 26 14:20:02 smithi137 podman[105272]: 2024-03-26 14:20:02.932959254 +0000 UTC m=+0.834806826 container died e7fed47da6107b1a9c0cd638a1fdcdf1752f3ffa8bff4557a8e5d57f04a3a5c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:20:03.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:02 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:02.757+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:03.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:02.825+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:03.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:03.316+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:03.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:03.324+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:03.742 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.3.service' 2024-03-26T14:20:03.852 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:03.574+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:03.852 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:03.672+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:03.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:03.859 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-26T14:20:03.860 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-26T14:20:03.860 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.4 2024-03-26T14:20:04.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:03 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:03.798+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:04.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:03.850+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:04.205 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:03.935+0000 7f7357c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.336875+0000 front 2024-03-26T14:19:40.337063+0000 (oldest deadline 2024-03-26T14:20:03.837029+0000) 2024-03-26T14:20:04.205 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:03 smithi159 systemd[1]: Stopping Ceph osd.4 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:04.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:04.337+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:04.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:04.292+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:04.705 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:04.459+0000 7f735c400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:04.705 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:04.459+0000 7f735c400700 -1 osd.4 98 *** Got signal Terminated *** 2024-03-26T14:20:04.705 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:04.459+0000 7f735c400700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:20:04.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:04.595+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:04.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:04.833+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:04.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:04.659+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:05.204 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:04.923+0000 7f7357c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.336875+0000 front 2024-03-26T14:19:40.337063+0000 (oldest deadline 2024-03-26T14:20:03.837029+0000) 2024-03-26T14:20:05.286 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:04 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:04.822+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:05.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:05 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:05.271+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:05.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:05 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:05.343+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:05.824 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:05.572+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:05.824 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:05.672+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:06.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:05 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:05.786+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:06.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:05 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:05.618+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:06.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:05.822+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:06.300 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:05 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:05.962+0000 7f7357c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.336875+0000 front 2024-03-26T14:19:40.337063+0000 (oldest deadline 2024-03-26T14:20:03.837029+0000) 2024-03-26T14:20:06.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:06 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:06.324+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:06.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:06 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:06.298+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:06.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:06 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:06.298+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:06.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:06.776+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:06.925 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:06.715+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:06.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:06.559+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:06.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:06.559+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:07.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:06 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:06.770+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:07.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:06 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:06.603+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:07.288 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:07 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:07.006+0000 7f7357c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.336875+0000 front 2024-03-26T14:19:40.337063+0000 (oldest deadline 2024-03-26T14:20:03.837029+0000) 2024-03-26T14:20:07.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:07 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:07.358+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:07.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:07 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:07.285+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:07.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:07 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:07.285+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:07.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:07.774+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:07.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:07.744+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:07.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:07.525+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:07.927 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:07.525+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:08.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:07 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:07.723+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:08.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:07 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:07.611+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:08.307 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:07 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:07.990+0000 7f7357c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.336875+0000 front 2024-03-26T14:19:40.337063+0000 (oldest deadline 2024-03-26T14:20:03.837029+0000) 2024-03-26T14:20:08.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:08 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:08.318+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:08.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:08 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:08.305+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:08.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:08 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:08.305+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:08.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:08.726+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:08.925 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:08.708+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:08.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:08.558+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:08.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:08.558+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:09.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:08 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:08.767+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:09.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:08 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:08.623+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:09.317 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:08 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4[46610]: 2024-03-26T14:20:08.986+0000 7f7357c00700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.336875+0000 front 2024-03-26T14:19:40.337063+0000 (oldest deadline 2024-03-26T14:20:03.837029+0000) 2024-03-26T14:20:09.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:09.343+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:09.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:09.314+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:09.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:09 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:09.314+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:09.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:09.749+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:09.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:09.589+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:09.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:09.589+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:09.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:09.692+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:10.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:09 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:09.786+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:10.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:09 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:09.669+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:10.204 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:09 smithi159 podman[71529]: 2024-03-26 14:20:09.944761061 +0000 UTC m=+5.634880127 container died 964ea10d4c88ff459b9dc8f9f0332e3d3c1530352d544b2394a891eb77de30fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-26T14:20:10.638 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:10 smithi159 podman[71529]: 2024-03-26 14:20:10.38603398 +0000 UTC m=+6.076153055 container cleanup 964ea10d4c88ff459b9dc8f9f0332e3d3c1530352d544b2394a891eb77de30fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-26T14:20:10.639 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:10 smithi159 bash[71529]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4 2024-03-26T14:20:10.639 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:10 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:10.346+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:10.639 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:10 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:10.346+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:10.639 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:10 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:10.336+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:10.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:10.701+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:10.925 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:10.691+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:10.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:10.574+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:10.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:10.574+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:10.954 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:10 smithi159 podman[71549]: 2024-03-26 14:20:10.637021513 +0000 UTC m=+0.678129857 container remove 964ea10d4c88ff459b9dc8f9f0332e3d3c1530352d544b2394a891eb77de30fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T14:20:11.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:10 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:10.777+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:11.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:10 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:10.696+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:11.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:11 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:11.330+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:11.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:11 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:11.330+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:11.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:11 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:11.310+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:11.705 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:11 smithi159 podman[71636]: 2024-03-26 14:20:11.36709355 +0000 UTC m=+0.037764302 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:11.705 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:11 smithi159 podman[71636]: 2024-03-26 14:20:11.554890377 +0000 UTC m=+0.225561147 container create 2262fb345d2c83636cf9e797728c6c485d6a283f9a72661543f0a1fdfaa61dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-26T14:20:11.925 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:11.740+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:11.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:11.682+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:11.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:11.613+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:11.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:11.613+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:11.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:11.613+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:11.975 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:11 smithi159 podman[71636]: 2024-03-26 14:20:11.854047485 +0000 UTC m=+0.524718280 container init 2262fb345d2c83636cf9e797728c6c485d6a283f9a72661543f0a1fdfaa61dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-26T14:20:11.975 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:11 smithi159 podman[71636]: 2024-03-26 14:20:11.865906793 +0000 UTC m=+0.536577588 container start 2262fb345d2c83636cf9e797728c6c485d6a283f9a72661543f0a1fdfaa61dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:20:12.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:11 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:11.752+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:12.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:11 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:11.739+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:12.296 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:11 smithi159 podman[71636]: 2024-03-26 14:20:11.974235956 +0000 UTC m=+0.644906752 container attach 2262fb345d2c83636cf9e797728c6c485d6a283f9a72661543f0a1fdfaa61dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-26T14:20:12.673 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:12 smithi159 podman[71636]: 2024-03-26 14:20:12.295691365 +0000 UTC m=+0.966362174 container died 2262fb345d2c83636cf9e797728c6c485d6a283f9a72661543f0a1fdfaa61dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:20:12.674 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:12 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:12.357+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:12.674 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:12 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:12.357+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:12.674 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:12 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:12.344+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:12.674 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:12 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:12.344+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:12.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:12.599+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:12.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:12.599+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:12.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:12.599+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:12.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:12.647+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:12.927 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:12.647+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:12.927 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:12.710+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:12.927 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:12.710+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:13.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:12.763+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:13.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:12.727+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:13.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:12 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:12.727+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:13.176 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.4.service' 2024-03-26T14:20:13.249 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:13 smithi159 podman[71662]: 2024-03-26 14:20:13.128989617 +0000 UTC m=+0.816628049 container remove 2262fb345d2c83636cf9e797728c6c485d6a283f9a72661543f0a1fdfaa61dae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T14:20:13.249 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:13 smithi159 systemd[1]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-26T14:20:13.249 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:13 smithi159 systemd[1]: Stopped Ceph osd.4 for 66aee2a0-eb77-11ee-95d0-87774f69a715. 2024-03-26T14:20:13.249 INFO:journalctl@ceph.osd.4.smithi159.stdout:Mar 26 14:20:13 smithi159 systemd[1]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.4.service: Consumed 39.338s CPU time. 2024-03-26T14:20:13.282 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:13.283 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-26T14:20:13.283 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-26T14:20:13.283 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.5 2024-03-26T14:20:13.625 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 systemd[1]: Stopping Ceph osd.5 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:13.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:13 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:13.327+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:13.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:13 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:13.327+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:13.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:13 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:13.321+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:13.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:13 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:13.321+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:13.925 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:13.625+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:13.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:13.625+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:13.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:13.719+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:13.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:13.719+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:13.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:13.623+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:13.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:13.623+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:13.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:13.623+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:13.927 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:13.855+0000 7f8168600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:13.927 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:13.855+0000 7f8168600700 -1 osd.5 98 *** Got signal Terminated *** 2024-03-26T14:20:13.927 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:13.855+0000 7f8168600700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:20:14.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:13 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:13.800+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:14.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:13 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:13.715+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:14.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:13 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:13.715+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:14.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:14 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:14.357+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:14.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:14 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:14.357+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:14.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:14 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:14.342+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:14.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:14 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:14.342+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:14.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:14.620+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:14.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:14.620+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:14.926 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:14.620+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:14.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:14.576+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:14.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:14.576+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:14.927 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:14.764+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:14.927 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:14 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:14.764+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:15.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:14.838+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:15.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:14.673+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:15.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:14 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:14.673+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:15.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:15 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:15.353+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:15.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:15 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:15.353+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:15.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:15 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:15.298+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:15.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:15 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:15.298+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:15.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:15.583+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:15.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:15.583+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:15.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:15.583+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:15.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:15.588+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:15.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:15.588+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:15.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:15.808+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:15.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:15 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:15.808+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:16.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:15.827+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:16.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:15.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:16.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:15 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:15.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:16.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:16.333+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:16.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:16.333+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:16.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:16.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:16.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:16 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:16.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:16.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:16.590+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:16.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:16.590+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:16.925 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:16.590+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:16.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:16.608+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:16.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:16.608+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:16.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:16.829+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:16.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:16 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:16.829+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:17.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:16.800+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:17.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:16.800+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:17.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:16.672+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:17.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:16 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:16.672+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:17.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:17 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:17.336+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:17.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:17 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:17.336+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:17.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:17 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:17.373+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:17.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:17 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:17.373+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:17.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:17 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:17.373+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:17.880 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:17.612+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:17.880 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:17.612+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:17.881 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:17.549+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:17.881 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:17.549+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:17.881 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:17.549+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:18.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:17.832+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:18.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:17.832+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:18.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:17.663+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:18.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:17.663+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:18.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:17.879+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:18.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:17 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:17.879+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:18.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:18 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:18.356+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:18.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:18 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:18.356+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:18.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:18 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:18.360+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:18.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:18 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:18.360+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:18.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:18 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:18.360+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:18.891 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:18.604+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:18.891 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:18.604+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:18.891 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:18.563+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:28.674055+0000 front 2024-03-26T14:19:28.673721+0000 (oldest deadline 2024-03-26T14:19:54.574007+0000) 2024-03-26T14:20:18.891 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:18.563+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.377285+0000 front 2024-03-26T14:19:40.377507+0000 (oldest deadline 2024-03-26T14:20:05.677535+0000) 2024-03-26T14:20:18.891 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5[47565]: 2024-03-26T14:20:18.563+0000 7f815ee00700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:45.678897+0000 front 2024-03-26T14:19:45.678976+0000 (oldest deadline 2024-03-26T14:20:11.579187+0000) 2024-03-26T14:20:19.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:18.822+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:19.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:18.822+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:19.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:18.669+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:19.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:18 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:18.669+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:19.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:18.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:19.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:18 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:18.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:19.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:19 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:19.360+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:19.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:19 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:19.360+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:19.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:19 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:19.360+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:19.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:19 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:19.360+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:19.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:19 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:19.360+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:19.723 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:19 smithi179 podman[71694]: 2024-03-26 14:20:19.430233021 +0000 UTC m=+5.729781387 container died 6764f31fd2b3414584596cc376d66ed9f37b2d2580b33f5a8c940f7efa28b491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-26T14:20:19.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:19 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:19.607+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:19.723 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:19 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:19.607+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:20.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:19 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:19.775+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:20.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:19 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:19.775+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:20.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:19 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:19.670+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:20.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:19 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:19.670+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:20.143 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:19 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:19.845+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:20.143 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:19 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:19.845+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:20.143 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:19 smithi179 podman[71694]: 2024-03-26 14:20:19.856909884 +0000 UTC m=+6.156458272 container cleanup 6764f31fd2b3414584596cc376d66ed9f37b2d2580b33f5a8c940f7efa28b491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T14:20:20.143 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:19 smithi179 bash[71694]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5 2024-03-26T14:20:20.425 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:20 smithi179 podman[71714]: 2024-03-26 14:20:20.141631117 +0000 UTC m=+0.705047385 container remove 6764f31fd2b3414584596cc376d66ed9f37b2d2580b33f5a8c940f7efa28b491 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T14:20:20.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:20 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:20.334+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:20.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:20 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:20.334+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:20.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:20 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:20.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:20.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:20 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:20.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:20.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:20 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:20.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:20.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:20 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:20.642+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:20.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:20 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:20.642+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:20.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:20 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:20.850+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:20.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:20 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:20.850+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:21.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:20.815+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:21.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:20.815+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:21.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:20.640+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:21.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:20 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:20.640+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:21.307 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:20 smithi179 podman[71803]: 2024-03-26 14:20:20.866747258 +0000 UTC m=+0.039096308 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:21.307 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:21 smithi179 podman[71803]: 2024-03-26 14:20:21.009888802 +0000 UTC m=+0.182237870 container create 87ddbc335cff13b02fd455cb63bca52e939c9ce14a883d89eb4d5c4a81463086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:20:21.307 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:21 smithi179 podman[71803]: 2024-03-26 14:20:21.305119813 +0000 UTC m=+0.477468918 container init 87ddbc335cff13b02fd455cb63bca52e939c9ce14a883d89eb4d5c4a81463086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:20:21.608 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:21 smithi179 podman[71803]: 2024-03-26 14:20:21.316015491 +0000 UTC m=+0.488364526 container start 87ddbc335cff13b02fd455cb63bca52e939c9ce14a883d89eb4d5c4a81463086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T14:20:21.608 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:21 smithi179 podman[71803]: 2024-03-26 14:20:21.421061532 +0000 UTC m=+0.593410637 container attach 87ddbc335cff13b02fd455cb63bca52e939c9ce14a883d89eb4d5c4a81463086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T14:20:21.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:21 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:21.361+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:21.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:21 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:21.361+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:21.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:21 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:21.429+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:21.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:21 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:21.429+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:21.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:21 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:21.429+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:21.884 INFO:journalctl@ceph.osd.5.smithi179.stdout:Mar 26 14:20:21 smithi179 podman[71803]: 2024-03-26 14:20:21.747123755 +0000 UTC m=+0.919472789 container died 87ddbc335cff13b02fd455cb63bca52e939c9ce14a883d89eb4d5c4a81463086 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-26T14:20:21.885 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:21 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:21.605+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:21.885 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:21 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:21.605+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:22.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:21 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:21.808+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:22.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:21 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:21.808+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:22.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:21 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:21.673+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:22.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:21 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:21.673+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:22.171 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:21 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:21.881+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:22.171 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:21 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:21.881+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:22.171 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:21 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:21.881+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:22.553 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.5.service' 2024-03-26T14:20:22.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:22.649 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-26T14:20:22.649 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-26T14:20:22.649 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.6 2024-03-26T14:20:22.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:22 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:22.406+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:22.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:22 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:22.406+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:22.704 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:22 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:22.439+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:22.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:22 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:22.439+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:22.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:22 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:22.439+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:22.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:22 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:22.439+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:22.887 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:22 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:22.580+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:22.887 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:22 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:22.580+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:23.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:22.653+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:23.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:22.653+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:23.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:22 smithi137 systemd[1]: Stopping Ceph osd.6 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:23.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:22.842+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:23.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:22.842+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:23.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:22.842+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.573187+0000 front 2024-03-26T14:19:56.573698+0000 (oldest deadline 2024-03-26T14:20:22.473110+0000) 2024-03-26T14:20:23.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:22 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:22.884+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:23.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:22 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:22.884+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:23.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:22 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:22.884+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:23.536 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:23.239+0000 7f153bc00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:23.537 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:23.239+0000 7f153bc00700 -1 osd.6 98 *** Got signal Terminated *** 2024-03-26T14:20:23.537 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:23.239+0000 7f153bc00700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:20:23.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:23 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:23.371+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:23.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:23 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:23.371+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:23.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:23 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:23.419+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:23.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:23 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:23.419+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:23.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:23 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:23.419+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:23.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:23 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:23.419+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:23.915 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:23 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:23.566+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:23.915 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:23 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:23.566+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:23.915 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:23 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:23.566+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:24.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:23.811+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:24.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:23.811+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:24.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:23.811+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.573187+0000 front 2024-03-26T14:19:56.573698+0000 (oldest deadline 2024-03-26T14:20:22.473110+0000) 2024-03-26T14:20:24.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:23.680+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:24.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:23.680+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:24.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:23 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:23.680+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:24.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:23 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:23.911+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:24.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:23 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:23.911+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:24.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:23 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:23.911+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:24.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:24.377+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:24.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:24.377+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:24.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:24.378+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:24.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:24.376+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:24.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:24.376+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:24.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:24.376+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:24.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:24 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:24.376+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:24.904 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:24 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:24.534+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:24.904 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:24 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:24.534+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:24.904 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:24 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:24.534+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:25.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:24.858+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:25.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:24.858+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:25.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:24.858+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.573187+0000 front 2024-03-26T14:19:56.573698+0000 (oldest deadline 2024-03-26T14:20:22.473110+0000) 2024-03-26T14:20:25.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:24.688+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:25.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:24.688+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:25.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:24 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:24.688+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:25.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:24 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:24.901+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:25.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:24 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:24.901+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:25.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:24 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:24.901+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:25.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:25.347+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:25.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:25.347+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:25.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:25.347+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:25.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:25.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:25.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:25.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:25.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:25.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:25.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:25 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:25.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:25.875 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:25 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:25.572+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:25.875 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:25 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:25.572+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:25.875 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:25 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:25.572+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:26.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:25 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:25.836+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:26.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:25 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:25.836+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:26.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:25 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:25.836+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.573187+0000 front 2024-03-26T14:19:56.573698+0000 (oldest deadline 2024-03-26T14:20:22.473110+0000) 2024-03-26T14:20:26.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:25 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:25.660+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:26.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:25 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:25.660+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:26.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:25 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:25.660+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:26.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:25 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:25.873+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:26.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:25 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:25.873+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:26.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:25 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:25.873+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:26.606 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:26.328+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:26.607 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:26.328+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:26.607 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:26.328+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:26.607 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:26.328+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:26.607 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:26.314+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:26.607 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:26.314+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:26.607 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:26 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:26.314+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:26.892 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:26.590+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:26.892 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:26.590+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:26.892 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:26.590+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:27.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:26.858+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:27.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:26.858+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:27.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:26.858+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.573187+0000 front 2024-03-26T14:19:56.573698+0000 (oldest deadline 2024-03-26T14:20:22.473110+0000) 2024-03-26T14:20:27.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:26.651+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:27.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:26.651+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:27.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:26.651+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:27.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:26.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:27.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:26.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:27.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:26 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:26.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:27.631 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:27.298+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:27.631 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:27.298+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:27.631 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:27.298+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:27.631 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:27.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:27.631 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:27.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:27.631 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:27.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:27.632 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:27 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:27.335+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:27.849 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:27.557+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:27.850 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:27.557+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:27.850 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:27.557+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:28.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:27.895+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:38.368569+0000 front 2024-03-26T14:19:38.368659+0000 (oldest deadline 2024-03-26T14:20:01.868765+0000) 2024-03-26T14:20:28.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:27.895+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.672137+0000 front 2024-03-26T14:19:50.672164+0000 (oldest deadline 2024-03-26T14:20:16.572283+0000) 2024-03-26T14:20:28.037 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6[60007]: 2024-03-26T14:20:27.895+0000 7f1532400700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.573187+0000 front 2024-03-26T14:19:56.573698+0000 (oldest deadline 2024-03-26T14:20:22.473110+0000) 2024-03-26T14:20:28.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:27.669+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:28.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:27.669+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:28.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:27 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:27.669+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:28.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:27.847+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:28.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:27.847+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:28.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:27 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:27.847+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:28.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:28.338+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:28.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:28.338+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:28.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:28.338+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:28.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:28.356+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:28.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:28.356+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:28.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:28.356+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:28.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:28 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:28.356+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:28.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:28.593+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:28.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:28.593+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:28.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:28.593+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:28.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:28.810+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:28.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:28.810+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:28.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:28 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:28.810+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:29.036 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:28 smithi137 podman[105386]: 2024-03-26 14:20:28.751556333 +0000 UTC m=+5.650904304 container died c049927a6c530c021f5bb598ead6896a78f8b7af5d5d3b3e80d9556bb8e5eaf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:20:29.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:28 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:28.698+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:29.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:28 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:28.698+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:29.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:28 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:28.698+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:29.385 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:29 smithi137 podman[105386]: 2024-03-26 14:20:29.19084818 +0000 UTC m=+6.090196181 container cleanup c049927a6c530c021f5bb598ead6896a78f8b7af5d5d3b3e80d9556bb8e5eaf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-26T14:20:29.385 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:29 smithi137 bash[105386]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6 2024-03-26T14:20:29.454 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:29.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:29.455 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:29.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:29.455 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:29.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:29.455 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:29.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:29.455 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:29.305+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:29.455 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:29.305+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:29.455 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:29 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:29.305+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:29.651 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:29 smithi137 podman[105407]: 2024-03-26 14:20:29.383849356 +0000 UTC m=+0.615964422 container remove c049927a6c530c021f5bb598ead6896a78f8b7af5d5d3b3e80d9556bb8e5eaf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-26T14:20:29.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:29.623+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:29.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:29.623+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:29.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:29.623+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:29.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:29.766+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:29.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:29.766+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:29.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:29 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:29.766+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:30.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:29 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:29.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:30.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:29 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:29.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:30.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:29 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:29.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:30.500 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:30 smithi137 podman[105493]: 2024-03-26 14:20:30.095052192 +0000 UTC m=+0.034552055 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:30.501 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:30 smithi137 podman[105493]: 2024-03-26 14:20:30.239445374 +0000 UTC m=+0.178945274 container create 686402bb52587c3e8a07abfa1e739e69c7ad7664053594ea346c8514f7f70793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T14:20:30.501 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:30 smithi137 podman[105493]: 2024-03-26 14:20:30.499303856 +0000 UTC m=+0.438803759 container init 686402bb52587c3e8a07abfa1e739e69c7ad7664053594ea346c8514f7f70793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:20:30.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:30.355+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:30.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:30.355+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:30.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:30.355+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:30.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:30.331+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:30.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:30.331+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:30.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:30.331+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:30.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:30 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:30.331+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:30.787 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:30 smithi137 podman[105493]: 2024-03-26 14:20:30.510812207 +0000 UTC m=+0.450312123 container start 686402bb52587c3e8a07abfa1e739e69c7ad7664053594ea346c8514f7f70793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5) 2024-03-26T14:20:30.787 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:30 smithi137 podman[105493]: 2024-03-26 14:20:30.595355223 +0000 UTC m=+0.534855152 container attach 686402bb52587c3e8a07abfa1e739e69c7ad7664053594ea346c8514f7f70793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-26T14:20:30.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:30 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:30.677+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:30.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:30 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:30.677+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:30.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:30 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:30.677+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:30.591+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:30.591+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:30.591+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:30.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:30.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:30.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:30.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:30 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:30.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:31.287 INFO:journalctl@ceph.osd.6.smithi137.stdout:Mar 26 14:20:30 smithi137 podman[105493]: 2024-03-26 14:20:30.942763229 +0000 UTC m=+0.882263105 container died 686402bb52587c3e8a07abfa1e739e69c7ad7664053594ea346c8514f7f70793 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T14:20:31.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:31.378+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:31.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:31.378+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:31.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:31.378+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:31.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:31.314+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:31.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:31.314+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:31.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:31.314+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:31.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:31 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:31.314+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:31.739 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.6.service' 2024-03-26T14:20:31.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:31.855 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-26T14:20:31.856 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-26T14:20:31.856 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.7 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:31.639+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:31.639+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:31.639+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:31.812+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:31.812+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:31.812+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:31.926 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:31 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:31.812+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:32.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:31 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:31.645+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:32.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:31 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:31.645+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:32.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:31 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:31.645+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:32.204 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:31 smithi159 systemd[1]: Stopping Ceph osd.7 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:32.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:32.310+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:32.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:32.310+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:32.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:32.310+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:32.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:32.310+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:32.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:32.408+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:32.706 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:32.408+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:32.706 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:32.408+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:32.706 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:32.495+0000 7f0149600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:32.706 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:32.495+0000 7f0149600700 -1 osd.7 98 *** Got signal Terminated *** 2024-03-26T14:20:32.706 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:32 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:32.495+0000 7f0149600700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:20:33.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:32 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:32.641+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:33.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:32 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:32.641+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:33.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:32 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:32.641+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:33.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:32 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:32.641+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:32.689+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:32.689+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:32.689+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:32.823+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:32.823+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:32.823+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:33.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:32 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:32.823+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:33.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:33.421+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:33.421+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:33.421+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:33.421+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.737389+0000 front 2024-03-26T14:20:08.737449+0000 (oldest deadline 2024-03-26T14:20:32.836936+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:33.350+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:33.350+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:33.350+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:33.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:33 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:33.350+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:33.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:33 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:33.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:33.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:33 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:33.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:33.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:33 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:33.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:33.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:33 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:33.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:34.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:33.682+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:34.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:33.682+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:34.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:33.682+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:34.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:33.783+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:34.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:33.783+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:34.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:33.783+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:34.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:33 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:33.783+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:34.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:34.443+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:34.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:34.443+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:34.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:34.443+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:34.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:34.443+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.737389+0000 front 2024-03-26T14:20:08.737449+0000 (oldest deadline 2024-03-26T14:20:32.836936+0000) 2024-03-26T14:20:34.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:34.343+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:34.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:34.343+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:34.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:34.343+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:34.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:34.343+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:34.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:34 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:34.614+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:34.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:34 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:34.614+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:34.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:34 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:34.614+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:34.787 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:34 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:34.614+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:34.726+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:34.726+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:34.726+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:34.739+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:34.739+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:34.739+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:35.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:34 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:34.739+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:35.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:35.470+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:35.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:35.470+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:35.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:35.470+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:35.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:35.470+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.737389+0000 front 2024-03-26T14:20:08.737449+0000 (oldest deadline 2024-03-26T14:20:32.836936+0000) 2024-03-26T14:20:35.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:35.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:35.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:35.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:35.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:35.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:35.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:35.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:35.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:35 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:35.375+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:36.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:35 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:35.632+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:36.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:35 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:35.632+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:36.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:35 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:35.632+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:36.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:35 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:35.632+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:36.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:35.720+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:35.720+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:35.720+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:35.720+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:35.779+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:35.779+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:35.779+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:36.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:35 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:35.779+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:36.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:36.464+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:36.464+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:36.464+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:36.464+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.737389+0000 front 2024-03-26T14:20:08.737449+0000 (oldest deadline 2024-03-26T14:20:32.836936+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:36.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:36.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:36.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:36.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:36.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:36.706 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:36 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:36.389+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:37.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:36 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:36.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:37.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:36 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:36.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:37.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:36 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:36.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:37.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:36 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:36.648+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:37.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:36.733+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:37.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:36.733+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:37.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:36.733+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:37.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:36.733+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:37.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:36.770+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:37.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:36.770+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:37.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:36.770+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:37.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:36 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:36.770+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:37.704 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:37.482+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.325554+0000 front 2024-03-26T14:19:40.325374+0000 (oldest deadline 2024-03-26T14:20:02.625351+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:37.482+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.830995+0000 front 2024-03-26T14:19:48.830917+0000 (oldest deadline 2024-03-26T14:20:12.330926+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:37.482+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.134433+0000 front 2024-03-26T14:19:58.134539+0000 (oldest deadline 2024-03-26T14:20:23.434635+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7[53545]: 2024-03-26T14:20:37.482+0000 7f013fe00700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.737389+0000 front 2024-03-26T14:20:08.737449+0000 (oldest deadline 2024-03-26T14:20:32.836936+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:37.436+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:37.436+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:37.436+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:37.436+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:37.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:37 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:37.436+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:38.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:37 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:37.646+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:38.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:37 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:37.646+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:38.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:37 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:37.646+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:38.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:37 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:37.646+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:38.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:37.774+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:37.774+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:37.774+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:37.774+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:37.730+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:37.730+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:37.730+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:38.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:37 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:37.730+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:38.204 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:37 smithi159 podman[71768]: 2024-03-26 14:20:37.924987109 +0000 UTC m=+5.603327432 container died 714b40031f41f43f14f173b2a7c39921c4338d89671f67e1d9a956e8bb976371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-26T14:20:38.644 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:38 smithi159 podman[71768]: 2024-03-26 14:20:38.416059513 +0000 UTC m=+6.094399873 container cleanup 714b40031f41f43f14f173b2a7c39921c4338d89671f67e1d9a956e8bb976371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-26T14:20:38.644 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:38 smithi159 bash[71768]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7 2024-03-26T14:20:38.645 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:38.426+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:38.645 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:38.426+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:38.645 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:38.426+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:38.645 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:38.426+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:38.645 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:38 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:38.427+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:38.954 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:38 smithi159 podman[71787]: 2024-03-26 14:20:38.701749944 +0000 UTC m=+0.761124919 container remove 714b40031f41f43f14f173b2a7c39921c4338d89671f67e1d9a956e8bb976371 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:20:39.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:38 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:38.668+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:39.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:38 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:38.668+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:39.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:38 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:38.668+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:39.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:38 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:38.668+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:39.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:38.782+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:39.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:38.782+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:39.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:38.782+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:39.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:38.782+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:39.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:38.759+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:39.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:38.760+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:39.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:38.760+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:39.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:38 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:38.760+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:39.570 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:39 smithi159 podman[71872]: 2024-03-26 14:20:39.449204725 +0000 UTC m=+0.036426709 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:39.570 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:39.420+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:39.570 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:39.420+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:39.570 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:39.420+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:39.570 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:39.420+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:39.570 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:39.420+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:39.833 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:39 smithi159 podman[71872]: 2024-03-26 14:20:39.569149235 +0000 UTC m=+0.156371222 container create 19fc9d54c7a17586b8609132820d890e39c41a792fca38935e3e0d14edff4fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-03-26T14:20:39.833 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:39 smithi159 podman[71872]: 2024-03-26 14:20:39.831438426 +0000 UTC m=+0.418660450 container init 19fc9d54c7a17586b8609132820d890e39c41a792fca38935e3e0d14edff4fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-26T14:20:40.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:39 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:39.683+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:40.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:39 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:39.683+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:40.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:39 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:39.683+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:40.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:39 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:39.683+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:40.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:39.827+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:39.827+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:39.827+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:39.827+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:39.717+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:39.717+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:39.717+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:40.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:39 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:39.717+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:40.204 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:39 smithi159 podman[71872]: 2024-03-26 14:20:39.842473273 +0000 UTC m=+0.429695275 container start 19fc9d54c7a17586b8609132820d890e39c41a792fca38935e3e0d14edff4fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-26T14:20:40.204 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:39 smithi159 podman[71872]: 2024-03-26 14:20:39.930091223 +0000 UTC m=+0.517313216 container attach 19fc9d54c7a17586b8609132820d890e39c41a792fca38935e3e0d14edff4fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-26T14:20:40.599 INFO:journalctl@ceph.osd.7.smithi159.stdout:Mar 26 14:20:40 smithi159 podman[71872]: 2024-03-26 14:20:40.27219156 +0000 UTC m=+0.859413584 container died 19fc9d54c7a17586b8609132820d890e39c41a792fca38935e3e0d14edff4fff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-26T14:20:40.599 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:40 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:40.434+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:40.599 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:40 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:40.434+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:40.599 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:40 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:40.434+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:40.599 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:40 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:40.434+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:40.599 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:40 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:40.434+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:41.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:40 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:40.657+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:41.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:40 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:40.657+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:41.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:40 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:40.657+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:41.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:40 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:40.657+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:41.044 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.7.service' 2024-03-26T14:20:41.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:41.167 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-26T14:20:41.167 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-26T14:20:41.167 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.8 2024-03-26T14:20:41.178 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:40.848+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:41.178 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:40.848+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:41.178 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:40.848+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:41.178 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:40.848+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:41.178 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:40.848+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:15.364813+0000 front 2024-03-26T14:20:15.365038+0000 (oldest deadline 2024-03-26T14:20:40.064719+0000) 2024-03-26T14:20:41.178 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:40.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:41.179 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:40.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:41.179 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:40.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:41.179 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:40 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:40.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:41.675 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 systemd[1]: Stopping Ceph osd.8 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:41.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:41 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:41.469+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:41.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:41 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:41.469+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:41.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:41 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:41.469+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:41.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:41 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:41.469+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:41.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:41 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:41.469+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:41.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:41 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:41.469+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:42.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:41 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:41.653+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:42.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:41 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:41.653+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:42.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:41 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:41.653+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:42.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:41 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:41.653+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:42.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:41.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:42.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:41.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:42.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:41.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:42.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:41.735+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:42.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.742+0000 7f46d2000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.742+0000 7f46d2000700 -1 osd.8 98 *** Got signal Terminated *** 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.742+0000 7f46d2000700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.855+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.855+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.855+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.855+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:42.177 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:41 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:41.855+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:15.364813+0000 front 2024-03-26T14:20:15.365038+0000 (oldest deadline 2024-03-26T14:20:40.064719+0000) 2024-03-26T14:20:42.847 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:42 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:42.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:42.847 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:42 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:42.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:42.847 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:42 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:42.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:42.848 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:42 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:42.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:42.848 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:42 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:42.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:42.848 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:42 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:42.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:43.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:42 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:42.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:43.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:42 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:42.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:43.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:42 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:42.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:43.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:42 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:42.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:43.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:42 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:42.634+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:43.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:42.808+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:43.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:42.808+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:43.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:42.808+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:43.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:42.808+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:43.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:42.808+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:15.364813+0000 front 2024-03-26T14:20:15.365038+0000 (oldest deadline 2024-03-26T14:20:40.064719+0000) 2024-03-26T14:20:43.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:42.706+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:43.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:42.706+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:43.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:42.706+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:43.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:42.706+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:43.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:42 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:42.706+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:43.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:43.441+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:43.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:43.441+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:43.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:43.441+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:43.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:43.441+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:43.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:43.441+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:43.705 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:43.441+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:43.906 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:43 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:43.618+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:43.906 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:43 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:43.618+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:43.906 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:43 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:43.618+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:43.906 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:43 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:43.618+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:43.906 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:43 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:43.618+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:44.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:43.786+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:44.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:43.786+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:44.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:43.786+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:44.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:43.786+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:44.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:43.786+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:15.364813+0000 front 2024-03-26T14:20:15.365038+0000 (oldest deadline 2024-03-26T14:20:40.064719+0000) 2024-03-26T14:20:44.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:43.705+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:44.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:43.705+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:44.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:43.705+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:44.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:43.705+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:44.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:43 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:43.705+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:44.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:44 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:44.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:44.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:44 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:44.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:44.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:44 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:44.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:44.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:44 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:44.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:44.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:44 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:44.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:44.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:44 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:44.455+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:45.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:44.616+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:45.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:44.616+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:45.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:44.616+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:45.038 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:44.616+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:45.038 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:44 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:44.616+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:44.772+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:44.772+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:44.772+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:44.772+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:44.772+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:15.364813+0000 front 2024-03-26T14:20:15.365038+0000 (oldest deadline 2024-03-26T14:20:40.064719+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:44.716+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:44.716+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:45.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:44.716+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:45.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:44.716+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:45.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:44 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:44.716+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:45.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:45 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:45.478+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:45.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:45 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:45.478+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:45.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:45 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:45.478+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:45.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:45 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:45.478+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:45.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:45 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:45.478+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:45.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:45 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:45.478+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:46.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:45.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:46.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:45.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:46.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:45.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:46.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:45.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:46.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:45 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:45.625+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:45.775+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:36.253012+0000 front 2024-03-26T14:19:36.253078+0000 (oldest deadline 2024-03-26T14:20:02.153259+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:45.775+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.355973+0000 front 2024-03-26T14:19:47.356281+0000 (oldest deadline 2024-03-26T14:20:11.957315+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:45.775+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.360888+0000 front 2024-03-26T14:19:58.360211+0000 (oldest deadline 2024-03-26T14:20:23.561393+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:45.775+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:03.562511+0000 front 2024-03-26T14:20:03.562456+0000 (oldest deadline 2024-03-26T14:20:35.363909+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8[54084]: 2024-03-26T14:20:45.775+0000 7f46c8800700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:15.364813+0000 front 2024-03-26T14:20:15.365038+0000 (oldest deadline 2024-03-26T14:20:40.064719+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:45.749+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:45.749+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:45.749+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:46.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:45.749+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:46.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:45 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:45.749+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:46.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:46 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:46.514+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:46.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:46 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:46.514+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:46.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:46 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:46.514+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:46.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:46 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:46.514+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:46.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:46 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:46.514+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:46.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:46 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:46.514+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:47.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:46.659+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:47.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:46.659+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:47.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:46.659+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:47.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:46.659+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:47.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:46 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:46.659+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:47.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:46 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:46.780+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:47.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:46 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:46.780+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:47.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:46 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:46.780+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:47.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:46 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:46.780+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:47.115 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:46 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:46.780+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:47.383 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:47 smithi179 podman[71920]: 2024-03-26 14:20:47.114030272 +0000 UTC m=+5.522552018 container died fd27a7ee26369fb7f9fc68a760b69c2264a75bb3078eebdfa4b8f971c10832cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-26T14:20:47.675 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:47 smithi179 podman[71920]: 2024-03-26 14:20:47.485544824 +0000 UTC m=+5.894066601 container cleanup fd27a7ee26369fb7f9fc68a760b69c2264a75bb3078eebdfa4b8f971c10832cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-26T14:20:47.675 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:47 smithi179 bash[71920]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8 2024-03-26T14:20:47.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:47.534+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:47.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:47.534+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:47.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:47.534+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:47.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:47.534+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:47.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:47.534+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:47.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:47.534+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:48.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:47 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:47.694+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:48.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:47 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:47.694+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:48.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:47 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:47.694+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:48.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:47 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:47.694+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:48.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:47 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:47.694+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:48.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:47 smithi179 podman[71939]: 2024-03-26 14:20:47.728430153 +0000 UTC m=+0.599155406 container remove fd27a7ee26369fb7f9fc68a760b69c2264a75bb3078eebdfa4b8f971c10832cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-26T14:20:48.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:47 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:47.745+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:48.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:47 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:47.745+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:48.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:47 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:47.745+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:48.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:47 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:47.745+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:48.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:47 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:47.745+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:48.795 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:48 smithi179 podman[72029]: 2024-03-26 14:20:48.450233821 +0000 UTC m=+0.033590858 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:48.795 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:48 smithi179 podman[72029]: 2024-03-26 14:20:48.579157804 +0000 UTC m=+0.162514877 container create c6cf7a43297debd7f592539ede0a69453a52e0c7c26635dd81ce3cb1731667cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:20:48.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:48.496+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:48.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:48.496+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:48.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:48.496+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:48.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:48.496+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:48.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:48.496+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:48.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:48 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:48.496+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:49.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:48.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:49.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:48.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:49.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:48.737+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:49.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:48.737+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:49.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:48 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:48.737+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:49.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:48 smithi179 podman[72029]: 2024-03-26 14:20:48.883292774 +0000 UTC m=+0.466649875 container init c6cf7a43297debd7f592539ede0a69453a52e0c7c26635dd81ce3cb1731667cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True) 2024-03-26T14:20:49.175 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:48 smithi179 podman[72029]: 2024-03-26 14:20:48.89411318 +0000 UTC m=+0.477470244 container start c6cf7a43297debd7f592539ede0a69453a52e0c7c26635dd81ce3cb1731667cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-26T14:20:49.176 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:48 smithi179 podman[72029]: 2024-03-26 14:20:48.965443055 +0000 UTC m=+0.548800146 container attach c6cf7a43297debd7f592539ede0a69453a52e0c7c26635dd81ce3cb1731667cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:20:49.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:48 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:48.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:49.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:48 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:48.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:49.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:48 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:48.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:49.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:48 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:48.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:49.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:48 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:48.793+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:49.675 INFO:journalctl@ceph.osd.8.smithi179.stdout:Mar 26 14:20:49 smithi179 podman[72029]: 2024-03-26 14:20:49.329366325 +0000 UTC m=+0.912723416 container died c6cf7a43297debd7f592539ede0a69453a52e0c7c26635dd81ce3cb1731667cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-8-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T14:20:49.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:49 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:49.539+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:49.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:49 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:49.539+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:49.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:49 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:49.539+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:49.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:49 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:49.539+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:49.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:49 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:49.539+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:49.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:49 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:49.539+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:50.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:49.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:50.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:49.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:50.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:49.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:50.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:49.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:50.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:49 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:49.736+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:50.054 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:49 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:49.804+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:50.054 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:49 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:49.804+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:50.054 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:49 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:49.804+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:50.054 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:49 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:49.804+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:50.054 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:49 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:49.804+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:50.101 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.8.service' 2024-03-26T14:20:50.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:50.222 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-26T14:20:50.222 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-26T14:20:50.222 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.9 2024-03-26T14:20:50.695 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 systemd[1]: Stopping Ceph osd.9 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:50.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:50 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:50.585+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:50.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:50 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:50.585+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:50.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:50 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:50.585+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:50.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:50 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:50.585+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:50.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:50 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:50.585+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:50.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:50 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:50.585+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:51.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.693+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:51.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.693+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:51.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.693+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:51.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.693+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:51.038 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.693+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:51.038 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.808+0000 7f841d000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:51.038 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.808+0000 7f841d000700 -1 osd.9 98 *** Got signal Terminated *** 2024-03-26T14:20:51.038 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:50 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:50.808+0000 7f841d000700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:20:51.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:50 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:50.849+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:51.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:50 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:50.849+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:51.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:50 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:50.849+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:51.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:50 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:50.849+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:51.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:50 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:50.849+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:51.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:51 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:51.605+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:51.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:51 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:51.605+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:51.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:51 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:51.605+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:51.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:51 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:51.605+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:51.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:51 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:51.605+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:51.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:51 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:51.605+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:52.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:51.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:52.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:51.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:52.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:51.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:52.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:51.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:52.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:51 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:51.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:52.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:51.883+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:52.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:51.883+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:52.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:51.883+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:52.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:51.883+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:52.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:51.883+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:52.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:51.883+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:52.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:52.596+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:52.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:52.596+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:52.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:52.596+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:52.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:52.596+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:52.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:52.596+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:52.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:52 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:52.596+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:53.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:52.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:53.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:52.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:53.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:52.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:53.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:52.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:53.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:52.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:53.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:52 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:52.728+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.562622+0000 front 2024-03-26T14:20:27.562715+0000 (oldest deadline 2024-03-26T14:20:52.262481+0000) 2024-03-26T14:20:53.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:52.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:53.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:52.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:53.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:52.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:53.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:52.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:53.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:52.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:53.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:52 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:52.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:53.840 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:53 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:53.547+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:53.840 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:53 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:53.547+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:53.840 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:53 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:53.547+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:53.840 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:53 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:53.547+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:53.840 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:53 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:53.547+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:53.840 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:53 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:53.547+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:54.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:53 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:53.697+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:54.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:53 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:53.697+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:54.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:53 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:53.697+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:54.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:53 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:53.697+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:54.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:53 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:53.697+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:54.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:53 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:53.697+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.562622+0000 front 2024-03-26T14:20:27.562715+0000 (oldest deadline 2024-03-26T14:20:52.262481+0000) 2024-03-26T14:20:54.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:53 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:53.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:54.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:53 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:53.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:54.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:53 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:53.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:54.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:53 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:53.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:54.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:53 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:53.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:54.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:53 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:53.889+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:54.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:54.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:54.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:54.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:54.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:54.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:54.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:54 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:54.523+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:20:55.036 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:54 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:54.722+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:55.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:54 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:54.722+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:55.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:54 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:54.722+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:55.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:54 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:54.722+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:55.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:54 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:54.722+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:55.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:54 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:54.722+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.562622+0000 front 2024-03-26T14:20:27.562715+0000 (oldest deadline 2024-03-26T14:20:52.262481+0000) 2024-03-26T14:20:55.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:54 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:54.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:55.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:54 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:54.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:55.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:54 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:54.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:55.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:54 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:54.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:55.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:54 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:54.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:55.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:54 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:54.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:55.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:55.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:55.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:55.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:55.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:55.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:55.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:55 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:55.567+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:20:56.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:55.685+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:40.047689+0000 front 2024-03-26T14:19:40.047497+0000 (oldest deadline 2024-03-26T14:20:05.347449+0000) 2024-03-26T14:20:56.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:55.685+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:50.049895+0000 front 2024-03-26T14:19:50.049441+0000 (oldest deadline 2024-03-26T14:20:11.749296+0000) 2024-03-26T14:20:56.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:55.685+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:58.652583+0000 front 2024-03-26T14:19:58.652628+0000 (oldest deadline 2024-03-26T14:20:23.352461+0000) 2024-03-26T14:20:56.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:55.685+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:09.056721+0000 front 2024-03-26T14:20:09.056654+0000 (oldest deadline 2024-03-26T14:20:32.556491+0000) 2024-03-26T14:20:56.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:55.685+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:17.561042+0000 front 2024-03-26T14:20:17.561139+0000 (oldest deadline 2024-03-26T14:20:41.660719+0000) 2024-03-26T14:20:56.037 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:55 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9[66733]: 2024-03-26T14:20:55.685+0000 7f8418800700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.562622+0000 front 2024-03-26T14:20:27.562715+0000 (oldest deadline 2024-03-26T14:20:52.262481+0000) 2024-03-26T14:20:56.175 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:55.919+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:56.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:55.919+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:56.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:55.919+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:56.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:55.919+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:56.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:55.919+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:56.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:55.919+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:56.537 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:56 smithi137 podman[105609]: 2024-03-26 14:20:56.282555988 +0000 UTC m=+5.634186765 container died 649c0c7e61155350f6edbadb475552c5d0c8cb7d7953b9eedfa9006bb0122e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T14:20:56.903 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:56 smithi137 podman[105609]: 2024-03-26 14:20:56.667428536 +0000 UTC m=+6.019059341 container cleanup 649c0c7e61155350f6edbadb475552c5d0c8cb7d7953b9eedfa9006bb0122e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2) 2024-03-26T14:20:56.903 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:56 smithi137 bash[105609]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9 2024-03-26T14:20:56.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:56.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:56.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:56.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:56.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:56.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:56.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:56 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:56.617+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:20:57.286 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:56 smithi137 podman[105628]: 2024-03-26 14:20:56.901656356 +0000 UTC m=+0.604895911 container remove 649c0c7e61155350f6edbadb475552c5d0c8cb7d7953b9eedfa9006bb0122e50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:20:57.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:56.942+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:57.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:56.942+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:57.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:56.942+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:57.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:56.942+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:57.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:56.942+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:57.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:56 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:56.942+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:57.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:57.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:57.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:57.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:57.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:57.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:57.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:57 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:57.657+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:20:57.966 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:57 smithi137 podman[105716]: 2024-03-26 14:20:57.589437456 +0000 UTC m=+0.033806555 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:20:57.966 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:57 smithi137 podman[105716]: 2024-03-26 14:20:57.702569415 +0000 UTC m=+0.146938552 container create 84ebc9b063791fac81dbe2a2a3f4a781188f1b9f15922ad9759d603466015a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-26T14:20:57.966 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:57 smithi137 podman[105716]: 2024-03-26 14:20:57.964382844 +0000 UTC m=+0.408751974 container init 84ebc9b063791fac81dbe2a2a3f4a781188f1b9f15922ad9759d603466015a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T14:20:58.287 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:57 smithi137 podman[105716]: 2024-03-26 14:20:57.975799402 +0000 UTC m=+0.420168526 container start 84ebc9b063791fac81dbe2a2a3f4a781188f1b9f15922ad9759d603466015a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-26T14:20:58.288 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:58 smithi137 podman[105716]: 2024-03-26 14:20:58.063281537 +0000 UTC m=+0.507650634 container attach 84ebc9b063791fac81dbe2a2a3f4a781188f1b9f15922ad9759d603466015a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:20:58.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:57.975+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:58.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:57.975+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:58.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:57.975+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:58.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:57.975+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:58.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:57.975+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:58.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:57 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:57.975+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:58.746 INFO:journalctl@ceph.osd.9.smithi137.stdout:Mar 26 14:20:58 smithi137 podman[105716]: 2024-03-26 14:20:58.403241501 +0000 UTC m=+0.847610625 container died 84ebc9b063791fac81dbe2a2a3f4a781188f1b9f15922ad9759d603466015a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-9-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:20:58.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:58.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:58.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:58 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:58.642+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:20:59.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.9.service' 2024-03-26T14:20:59.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:20:59.263 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-26T14:20:59.263 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-26T14:20:59.263 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.10 2024-03-26T14:20:59.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:58.929+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:20:59.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:58.929+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:20:59.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:58.929+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:20:59.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:58.929+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:20:59.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:58.929+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:20:59.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:58.929+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:20:59.665 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 systemd[1]: Stopping Ceph osd.10 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.816+0000 7fd3c3800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:20:59.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.816+0000 7fd3c3800700 -1 osd.10 98 *** Got signal Terminated *** 2024-03-26T14:20:59.956 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:20:59 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:20:59.816+0000 7fd3c3800700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:21:00.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:59.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:00.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:59.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:00.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:59.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:00.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:59.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:00.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:59.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:00.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:20:59 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:20:59.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:00.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:21:00.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:21:00.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:21:00.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:21:00.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:21:00.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:21:00.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:00 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:00.648+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:21:01.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:00.984+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:01.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:00.984+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:01.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:00.984+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:01.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:00.984+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:01.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:00.984+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:01.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:00 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:00.984+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:01.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:21:01.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:21:01.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:21:01.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:21:01.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:21:01.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:21:01.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:01 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:01.663+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:02.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.032+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:02.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:21:02.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:21:02.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:21:02.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:21:02.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:21:02.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:21:02.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:02 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:02.676+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:03.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:02 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:02.988+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:03.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:21:03.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:21:03.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:21:03.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:21:03.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:21:03.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:21:03.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:03 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:03.633+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:21:04.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:04.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:04.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:04.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:04.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:04.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:04.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:03.964+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:30.274893+0000 front 2024-03-26T14:19:30.275024+0000 (oldest deadline 2024-03-26T14:19:55.574832+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-03-26T14:19:39.578007+0000 front 2024-03-26T14:19:39.578150+0000 (oldest deadline 2024-03-26T14:20:05.477721+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:48.880877+0000 front 2024-03-26T14:19:48.880905+0000 (oldest deadline 2024-03-26T14:20:17.082028+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:57.083026+0000 front 2024-03-26T14:19:57.083260+0000 (oldest deadline 2024-03-26T14:20:21.782737+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:06.985511+0000 front 2024-03-26T14:20:06.985876+0000 (oldest deadline 2024-03-26T14:20:34.586927+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.889010+0000 front 2024-03-26T14:20:16.888707+0000 (oldest deadline 2024-03-26T14:20:40.889389+0000) 2024-03-26T14:21:04.955 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:04 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10[60459]: 2024-03-26T14:21:04.631+0000 7fd3bf000700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:27.892074+0000 front 2024-03-26T14:20:27.891938+0000 (oldest deadline 2024-03-26T14:20:53.791892+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:05.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:04 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:04.921+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:05.700 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:05 smithi159 podman[71987]: 2024-03-26 14:21:05.3484988 +0000 UTC m=+5.659346109 container died e2b3fe1a1f69b7adfbbcca01ecd1dea9282ff272487686e5b31036ea9adedcbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-26T14:21:05.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:05 smithi159 podman[71987]: 2024-03-26 14:21:05.802494212 +0000 UTC m=+6.113341533 container cleanup e2b3fe1a1f69b7adfbbcca01ecd1dea9282ff272487686e5b31036ea9adedcbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:21:05.954 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:05 smithi159 bash[71987]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:06.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:05 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:05.898+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:06.454 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:05 smithi159 podman[72007]: 2024-03-26 14:21:05.998556031 +0000 UTC m=+0.632611374 container remove e2b3fe1a1f69b7adfbbcca01ecd1dea9282ff272487686e5b31036ea9adedcbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, ceph=True) 2024-03-26T14:21:07.030 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:06 smithi159 podman[72098]: 2024-03-26 14:21:06.684735315 +0000 UTC m=+0.036279460 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:21:07.030 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:06 smithi159 podman[72098]: 2024-03-26 14:21:06.847241606 +0000 UTC m=+0.198785771 container create bc124b97e1e639d77831ddde083be52efee65a3b085a8f5fb92301bd342c2bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:07.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:06 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:06.939+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:07.454 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:07 smithi159 podman[72098]: 2024-03-26 14:21:07.142260185 +0000 UTC m=+0.493804357 container init bc124b97e1e639d77831ddde083be52efee65a3b085a8f5fb92301bd342c2bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-26T14:21:07.454 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:07 smithi159 podman[72098]: 2024-03-26 14:21:07.153793891 +0000 UTC m=+0.505338027 container start bc124b97e1e639d77831ddde083be52efee65a3b085a8f5fb92301bd342c2bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-26T14:21:07.455 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:07 smithi159 podman[72098]: 2024-03-26 14:21:07.282678716 +0000 UTC m=+0.634222859 container attach bc124b97e1e639d77831ddde083be52efee65a3b085a8f5fb92301bd342c2bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-26T14:21:07.918 INFO:journalctl@ceph.osd.10.smithi159.stdout:Mar 26 14:21:07 smithi159 podman[72124]: 2024-03-26 14:21:07.634733389 +0000 UTC m=+0.031628378 container died bc124b97e1e639d77831ddde083be52efee65a3b085a8f5fb92301bd342c2bb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-26T14:21:08.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:08.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:08.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:08.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:08.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:08.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:08.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:08.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:07 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:07.906+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:08.337 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.10.service' 2024-03-26T14:21:08.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:08.457 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-26T14:21:08.457 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-26T14:21:08.458 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.11 2024-03-26T14:21:08.867 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 systemd[1]: Stopping Ceph osd.11 for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:09.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:09.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:08 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:08.865+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.033+0000 7f3386c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.033+0000 7f3386c00700 -1 osd.11 98 *** Got signal Terminated *** 2024-03-26T14:21:09.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.033+0000 7f3386c00700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-26T14:21:10.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:10.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:10.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:10.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:10.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:10.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:10.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:10.177 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:09 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:09.902+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:11.176 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:10 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:10.912+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:12.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:12.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:12.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:12.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:12.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:12.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:12.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:12.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:11 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:11.930+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:13.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:13.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:13.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:13.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:13.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:13.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:13.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:13.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:12 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:12.943+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6806 osd.0 since back 2024-03-26T14:19:31.547880+0000 front 2024-03-26T14:19:31.547270+0000 (oldest deadline 2024-03-26T14:19:57.446842+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-03-26T14:19:47.950522+0000 front 2024-03-26T14:19:47.950651+0000 (oldest deadline 2024-03-26T14:20:12.050305+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6814 osd.3 since back 2024-03-26T14:19:56.752230+0000 front 2024-03-26T14:19:56.752301+0000 (oldest deadline 2024-03-26T14:20:20.852311+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-03-26T14:20:08.455797+0000 front 2024-03-26T14:20:08.455560+0000 (oldest deadline 2024-03-26T14:20:30.755512+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-03-26T14:20:16.558477+0000 front 2024-03-26T14:20:16.558828+0000 (oldest deadline 2024-03-26T14:20:42.458669+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.137:6822 osd.6 since back 2024-03-26T14:20:26.560440+0000 front 2024-03-26T14:20:26.560288+0000 (oldest deadline 2024-03-26T14:20:51.860164+0000) 2024-03-26T14:21:14.426 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-03-26T14:20:35.862709+0000 front 2024-03-26T14:20:35.862768+0000 (oldest deadline 2024-03-26T14:21:01.762377+0000) 2024-03-26T14:21:14.427 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:13 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11[60796]: 2024-03-26T14:21:13.927+0000 7f337d400700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.179:6820 osd.8 since back 2024-03-26T14:20:42.264357+0000 front 2024-03-26T14:20:42.264541+0000 (oldest deadline 2024-03-26T14:21:07.564379+0000) 2024-03-26T14:21:14.729 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:14 smithi179 podman[72148]: 2024-03-26 14:21:14.437391356 +0000 UTC m=+5.548426356 container died 259b8bc17bca67f0a54f1967fbb69fd5de148683e2d47411edfa6fdbc73fe390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS) 2024-03-26T14:21:15.074 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:14 smithi179 podman[72148]: 2024-03-26 14:21:14.868649924 +0000 UTC m=+5.979684964 container cleanup 259b8bc17bca67f0a54f1967fbb69fd5de148683e2d47411edfa6fdbc73fe390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319) 2024-03-26T14:21:15.074 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:14 smithi179 bash[72148]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11 2024-03-26T14:21:15.425 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:15 smithi179 podman[72166]: 2024-03-26 14:21:15.071826972 +0000 UTC m=+0.628160607 container remove 259b8bc17bca67f0a54f1967fbb69fd5de148683e2d47411edfa6fdbc73fe390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-26T14:21:16.110 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:15 smithi179 podman[72258]: 2024-03-26 14:21:15.73749822 +0000 UTC m=+0.034897831 image pull c664afe56bb01db4f9afa9532c20598e201e664f2e9ba2be50db0d52674a26d2 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc 2024-03-26T14:21:16.110 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:15 smithi179 podman[72258]: 2024-03-26 14:21:15.839329266 +0000 UTC m=+0.136728940 container create 7ba1163b75633e5973047f047b31ad7cd20a7fd09777c7c641aa4f7d98bc4353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-26T14:21:16.110 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:16 smithi179 podman[72258]: 2024-03-26 14:21:16.10900537 +0000 UTC m=+0.406404999 container init 7ba1163b75633e5973047f047b31ad7cd20a7fd09777c7c641aa4f7d98bc4353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:21:16.368 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:16 smithi179 podman[72258]: 2024-03-26 14:21:16.119382451 +0000 UTC m=+0.416782077 container start 7ba1163b75633e5973047f047b31ad7cd20a7fd09777c7c641aa4f7d98bc4353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:21:16.368 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:16 smithi179 podman[72258]: 2024-03-26 14:21:16.199706288 +0000 UTC m=+0.497105917 container attach 7ba1163b75633e5973047f047b31ad7cd20a7fd09777c7c641aa4f7d98bc4353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5) 2024-03-26T14:21:16.675 INFO:journalctl@ceph.osd.11.smithi179.stdout:Mar 26 14:21:16 smithi179 podman[72258]: 2024-03-26 14:21:16.573750562 +0000 UTC m=+0.871150216 container died 7ba1163b75633e5973047f047b31ad7cd20a7fd09777c7c641aa4f7d98bc4353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-osd-11-deactivate, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-26T14:21:17.324 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@osd.11.service' 2024-03-26T14:21:17.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:17.427 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-26T14:21:17.427 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-26T14:21:17.427 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.a 2024-03-26T14:21:17.786 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:17 smithi137 systemd[1]: Stopping Ceph mds.a for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:18.286 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a[73357]: 2024-03-26T14:21:17.965+0000 7f1af5e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:18.287 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:17 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a[73357]: 2024-03-26T14:21:17.965+0000 7f1af5e00700 -1 mds.a *** got signal Terminated *** 2024-03-26T14:21:21.148 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:20 smithi137 podman[105820]: 2024-03-26 14:21:20.803527967 +0000 UTC m=+2.973682721 container died cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-26T14:21:21.494 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:21 smithi137 podman[105820]: 2024-03-26 14:21:21.266473242 +0000 UTC m=+3.436628012 container cleanup cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319) 2024-03-26T14:21:21.494 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:21 smithi137 bash[105820]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a 2024-03-26T14:21:21.786 INFO:journalctl@ceph.mds.a.smithi137.stdout:Mar 26 14:21:21 smithi137 podman[105846]: 2024-03-26 14:21:21.492519463 +0000 UTC m=+0.675930502 container remove cfbcad657b2a637886da499986beace9f0b070885982eccf0e57056b04b726f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-26T14:21:21.956 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.a.service' 2024-03-26T14:21:22.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:22.067 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-26T14:21:22.067 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-26T14:21:22.067 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.d 2024-03-26T14:21:22.536 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:22 smithi137 systemd[1]: Stopping Ceph mds.d for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:23.037 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-d[72279]: 2024-03-26T14:21:22.683+0000 7fa0ff200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:23.037 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:22 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-d[72279]: 2024-03-26T14:21:22.683+0000 7fa0ff200700 -1 mds.d *** got signal Terminated *** 2024-03-26T14:21:24.761 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:24 smithi137 podman[105952]: 2024-03-26 14:21:24.481924778 +0000 UTC m=+1.930582988 container died 4b9571b3711905edbd1459bcda6066cd0b7c6d971f5eb231b5b3df9b6eab94a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-d, org.label-schema.build-date=20240319, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-26T14:21:25.037 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:24 smithi137 podman[105952]: 2024-03-26 14:21:24.870029431 +0000 UTC m=+2.318687634 container cleanup 4b9571b3711905edbd1459bcda6066cd0b7c6d971f5eb231b5b3df9b6eab94a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-26T14:21:25.037 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:24 smithi137 bash[105952]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-d 2024-03-26T14:21:25.416 INFO:journalctl@ceph.mds.d.smithi137.stdout:Mar 26 14:21:25 smithi137 podman[105966]: 2024-03-26 14:21:25.054207837 +0000 UTC m=+0.557576834 container remove 4b9571b3711905edbd1459bcda6066cd0b7c6d971f5eb231b5b3df9b6eab94a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-d, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-26T14:21:25.451 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.d.service' 2024-03-26T14:21:25.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:25.567 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-26T14:21:25.567 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-26T14:21:25.567 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.g 2024-03-26T14:21:26.036 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:25 smithi137 systemd[1]: Stopping Ceph mds.g for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:26.537 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-g[73000]: 2024-03-26T14:21:26.254+0000 7f5c55c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:26.537 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:26 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-g[73000]: 2024-03-26T14:21:26.254+0000 7f5c55c00700 -1 mds.g *** got signal Terminated *** 2024-03-26T14:21:29.287 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:28 smithi137 podman[106072]: 2024-03-26 14:21:28.853710628 +0000 UTC m=+2.837125326 container died 17d9460e4d861f8dab5e189e4708a9759280ef0bc94865f72a18099bec0dfc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:21:29.786 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:29 smithi137 podman[106072]: 2024-03-26 14:21:29.597853394 +0000 UTC m=+3.581268042 container cleanup 17d9460e4d861f8dab5e189e4708a9759280ef0bc94865f72a18099bec0dfc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-g, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-26T14:21:29.786 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:29 smithi137 bash[106072]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-g 2024-03-26T14:21:30.235 INFO:journalctl@ceph.mds.g.smithi137.stdout:Mar 26 14:21:29 smithi137 podman[106085]: 2024-03-26 14:21:29.840635834 +0000 UTC m=+0.965727553 container remove 17d9460e4d861f8dab5e189e4708a9759280ef0bc94865f72a18099bec0dfc8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-g, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:21:30.351 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.g.service' 2024-03-26T14:21:30.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:30.464 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-26T14:21:30.464 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-26T14:21:30.464 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.j 2024-03-26T14:21:30.786 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:30 smithi137 systemd[1]: Stopping Ceph mds.j for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:31.287 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:31 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j[71894]: 2024-03-26T14:21:31.031+0000 7f1b62600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:31.287 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:31 smithi137 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j[71894]: 2024-03-26T14:21:31.031+0000 7f1b62600700 -1 mds.j *** got signal Terminated *** 2024-03-26T14:21:32.590 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:32 smithi137 podman[106190]: 2024-03-26 14:21:32.331922039 +0000 UTC m=+1.424088199 container died 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:21:32.879 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:32 smithi137 podman[106190]: 2024-03-26 14:21:32.710197675 +0000 UTC m=+1.802363833 container cleanup 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-26T14:21:32.879 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:32 smithi137 bash[106190]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j 2024-03-26T14:21:33.287 INFO:journalctl@ceph.mds.j.smithi137.stdout:Mar 26 14:21:32 smithi137 podman[106204]: 2024-03-26 14:21:32.877582874 +0000 UTC m=+0.533957965 container remove 667c102acc8c414aaa9d9ec9a4eed109e1ff0a1e8c4c332ccab9e2a9bb87fc73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:21:33.402 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.j.service' 2024-03-26T14:21:33.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:33.513 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-26T14:21:33.513 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-26T14:21:33.514 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.b 2024-03-26T14:21:33.933 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:33 smithi159 systemd[1]: Stopping Ceph mds.b for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:34.205 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: 2024-03-26T14:21:34.085+0000 7ff468000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:34.205 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:34 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b[65814]: 2024-03-26T14:21:34.085+0000 7ff468000700 -1 mds.b *** got signal Terminated *** 2024-03-26T14:21:37.704 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:37 smithi159 podman[72201]: 2024-03-26 14:21:37.450387289 +0000 UTC m=+3.504945217 container died 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-26T14:21:38.165 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:37 smithi159 podman[72201]: 2024-03-26 14:21:37.912143276 +0000 UTC m=+3.966701229 container cleanup 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-26T14:21:38.165 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:37 smithi159 bash[72201]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b 2024-03-26T14:21:38.454 INFO:journalctl@ceph.mds.b.smithi159.stdout:Mar 26 14:21:38 smithi159 podman[72227]: 2024-03-26 14:21:38.163800825 +0000 UTC m=+0.697291015 container remove 12dac58e7cafb5d13c022d2f5d044492db3b69d2f7a6407cc4cd742e110c748c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_CLEAN=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-26T14:21:38.726 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.b.service' 2024-03-26T14:21:38.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:38.844 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-26T14:21:38.845 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-26T14:21:38.845 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.e 2024-03-26T14:21:39.454 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:39 smithi159 systemd[1]: Stopping Ceph mds.e for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:39.954 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e[65191]: 2024-03-26T14:21:39.462+0000 7f8221c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:39.955 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:39 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e[65191]: 2024-03-26T14:21:39.462+0000 7f8221c00700 -1 mds.e *** got signal Terminated *** 2024-03-26T14:21:41.704 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:41 smithi159 podman[72333]: 2024-03-26 14:21:41.432037419 +0000 UTC m=+2.087217597 container died d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:21:42.126 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:41 smithi159 podman[72333]: 2024-03-26 14:21:41.882298462 +0000 UTC m=+2.537478669 container cleanup d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:21:42.127 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:41 smithi159 bash[72333]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e 2024-03-26T14:21:42.454 INFO:journalctl@ceph.mds.e.smithi159.stdout:Mar 26 14:21:42 smithi159 podman[72347]: 2024-03-26 14:21:42.125133612 +0000 UTC m=+0.681948509 container remove d84026a869aa95b12d1a2013a659100cad51ba228226326a66d943c44487db94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-e, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-26T14:21:42.657 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.e.service' 2024-03-26T14:21:42.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:42.770 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-26T14:21:42.770 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-26T14:21:42.770 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.h 2024-03-26T14:21:43.204 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:42 smithi159 systemd[1]: Stopping Ceph mds.h for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:43.704 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h[65493]: 2024-03-26T14:21:43.341+0000 7fc868800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:43.704 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:43 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h[65493]: 2024-03-26T14:21:43.341+0000 7fc868800700 -1 mds.h *** got signal Terminated *** 2024-03-26T14:21:45.816 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:45 smithi159 podman[72452]: 2024-03-26 14:21:45.464523004 +0000 UTC m=+2.243999067 container died 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:21:46.180 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:45 smithi159 podman[72452]: 2024-03-26 14:21:45.977349271 +0000 UTC m=+2.756825365 container cleanup 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-26T14:21:46.180 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:45 smithi159 bash[72452]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h 2024-03-26T14:21:46.454 INFO:journalctl@ceph.mds.h.smithi159.stdout:Mar 26 14:21:46 smithi159 podman[72466]: 2024-03-26 14:21:46.178412882 +0000 UTC m=+0.696675526 container remove 66af434dd95e2f9cc05a68a9814b9f1113a8385f952488756824eafecef9f09b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:21:46.734 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.h.service' 2024-03-26T14:21:46.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:46.845 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-26T14:21:46.845 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-26T14:21:46.845 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.k 2024-03-26T14:21:47.204 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:46 smithi159 systemd[1]: Stopping Ceph mds.k for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:47.704 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k[64888]: 2024-03-26T14:21:47.386+0000 7f7bc6000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:47.705 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:47 smithi159 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k[64888]: 2024-03-26T14:21:47.386+0000 7f7bc6000700 -1 mds.k *** got signal Terminated *** 2024-03-26T14:21:49.704 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:49 smithi159 podman[72575]: 2024-03-26 14:21:49.448454054 +0000 UTC m=+2.182628911 container died 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, GIT_BRANCH=HEAD) 2024-03-26T14:21:50.047 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:49 smithi159 podman[72575]: 2024-03-26 14:21:49.855851687 +0000 UTC m=+2.590026538 container cleanup 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:21:50.047 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:49 smithi159 bash[72575]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k 2024-03-26T14:21:50.454 INFO:journalctl@ceph.mds.k.smithi159.stdout:Mar 26 14:21:50 smithi159 podman[72587]: 2024-03-26 14:21:50.047984709 +0000 UTC m=+0.584348661 container remove 69ca7f16293f0e1abdcb355f69d9e2f385ddb682d90dceed54d66d71ce2fb7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:21:50.490 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.k.service' 2024-03-26T14:21:50.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:50.604 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-26T14:21:50.605 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-26T14:21:50.605 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.c 2024-03-26T14:21:51.136 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:50 smithi179 systemd[1]: Stopping Ceph mds.c for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:51.425 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c[65709]: 2024-03-26T14:21:51.132+0000 7f2912800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:51.425 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:51 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c[65709]: 2024-03-26T14:21:51.132+0000 7f2912800700 -1 mds.c *** got signal Terminated *** 2024-03-26T14:21:53.745 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:53 smithi179 podman[72388]: 2024-03-26 14:21:53.46634156 +0000 UTC m=+2.435592085 container died 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-26T14:21:54.067 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:53 smithi179 podman[72388]: 2024-03-26 14:21:53.853061615 +0000 UTC m=+2.822312140 container cleanup 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-26T14:21:54.067 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:53 smithi179 bash[72388]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c 2024-03-26T14:21:54.425 INFO:journalctl@ceph.mds.c.smithi179.stdout:Mar 26 14:21:54 smithi179 podman[72402]: 2024-03-26 14:21:54.064811103 +0000 UTC m=+0.587750583 container remove 2a298516b5dc087bed331671b7a9a1fc97a2913391236af7563db599675c8747 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-26T14:21:54.461 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.c.service' 2024-03-26T14:21:54.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:54.571 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-26T14:21:54.571 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-26T14:21:54.571 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.f 2024-03-26T14:21:55.145 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:54 smithi179 systemd[1]: Stopping Ceph mds.f for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:55.425 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f[65409]: 2024-03-26T14:21:55.143+0000 7fbf0f600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:55.426 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:55 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f[65409]: 2024-03-26T14:21:55.143+0000 7fbf0f600700 -1 mds.f *** got signal Terminated *** 2024-03-26T14:21:57.117 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:56 smithi179 podman[72511]: 2024-03-26 14:21:56.859875764 +0000 UTC m=+1.874488731 container died a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-26T14:21:57.426 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:57 smithi179 podman[72511]: 2024-03-26 14:21:57.223404124 +0000 UTC m=+2.238017085 container cleanup a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-26T14:21:57.426 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:57 smithi179 bash[72511]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f 2024-03-26T14:21:57.925 INFO:journalctl@ceph.mds.f.smithi179.stdout:Mar 26 14:21:57 smithi179 podman[72525]: 2024-03-26 14:21:57.491242527 +0000 UTC m=+0.612860864 container remove a78f758c342b660b379bb5ebd0a7ab817b6145bcd6f079a534938f7e0883d29e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-f, CEPH_POINT_RELEASE=, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-26T14:21:58.021 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.f.service' 2024-03-26T14:21:58.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:21:58.139 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-26T14:21:58.140 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-26T14:21:58.140 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.i 2024-03-26T14:21:58.676 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:21:58 smithi179 systemd[1]: Stopping Ceph mds.i for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:21:59.175 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:21:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i[66013]: 2024-03-26T14:21:58.716+0000 7f1820e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:21:59.176 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:21:58 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i[66013]: 2024-03-26T14:21:58.716+0000 7f1820e00700 -1 mds.i *** got signal Terminated *** 2024-03-26T14:22:01.776 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:22:01 smithi179 podman[72630]: 2024-03-26 14:22:01.442484814 +0000 UTC m=+2.847389389 container died 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-26T14:22:02.130 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:22:01 smithi179 podman[72630]: 2024-03-26 14:22:01.885307667 +0000 UTC m=+3.290212212 container cleanup 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-26T14:22:02.130 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:22:01 smithi179 bash[72630]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i 2024-03-26T14:22:02.425 INFO:journalctl@ceph.mds.i.smithi179.stdout:Mar 26 14:22:02 smithi179 podman[72644]: 2024-03-26 14:22:02.127767466 +0000 UTC m=+0.664135210 container remove 9e79f1a555a61fd2f636fe60b2eccc4439907c755b68958dfcb56a872c5fa9d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-26T14:22:02.650 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.i.service' 2024-03-26T14:22:02.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:22:02.763 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-26T14:22:02.763 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-26T14:22:02.763 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.l 2024-03-26T14:22:02.925 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:02 smithi179 systemd[1]: Stopping Ceph mds.l for 66aee2a0-eb77-11ee-95d0-87774f69a715... 2024-03-26T14:22:03.676 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l[65090]: 2024-03-26T14:22:03.310+0000 7f7694c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-26T14:22:03.676 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:03 smithi179 ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l[65090]: 2024-03-26T14:22:03.310+0000 7f7694c00700 -1 mds.l *** got signal Terminated *** 2024-03-26T14:22:04.676 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:04 smithi179 podman[72748]: 2024-03-26 14:22:04.37436141 +0000 UTC m=+1.185719013 container died c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-26T14:22:05.149 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:04 smithi179 podman[72748]: 2024-03-26 14:22:04.911283728 +0000 UTC m=+1.722641325 container cleanup c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c, org.label-schema.schema-version=1.0) 2024-03-26T14:22:05.149 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:04 smithi179 bash[72748]: ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l 2024-03-26T14:22:05.425 INFO:journalctl@ceph.mds.l.smithi179.stdout:Mar 26 14:22:05 smithi179 podman[72762]: 2024-03-26 14:22:05.147619442 +0000 UTC m=+0.759322014 container remove c586187f6c74d37334cf761f36f0dac51ca5aad9d9d8f7aac4996be582d8491b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b48cbed4f912cd766666ab034815156a50a4f9b95481e98a662f737d54a2abc, name=ceph-66aee2a0-eb77-11ee-95d0-87774f69a715-mds-l, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-mchangir-testing-main-20240318.032620-3be9a5c) 2024-03-26T14:22:05.654 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-66aee2a0-eb77-11ee-95d0-87774f69a715@mds.l.service' 2024-03-26T14:22:05.772 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-26T14:22:05.772 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-26T14:22:05.772 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T14:22:06.092 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:22:10.674 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T14:22:10.998 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:22:15.937 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --force --keep-logs 2024-03-26T14:22:16.257 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:22:21.083 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T14:22:21.187 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T14:22:21.289 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-26T14:22:21.383 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-26T14:22:21.385 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi137/crash 2024-03-26T14:22:21.386 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash -- . 2024-03-26T14:22:21.493 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T14:22:21.493 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-03-26T14:22:21.499 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi159/crash 2024-03-26T14:22:21.500 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash -- . 2024-03-26T14:22:21.638 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T14:22:21.639 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-03-26T14:22:21.645 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi179/crash 2024-03-26T14:22:21.646 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash -- . 2024-03-26T14:22:21.759 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/crash: Cannot open: No such file or directory 2024-03-26T14:22:21.760 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-03-26T14:22:21.765 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-26T14:22:21.765 DEBUG:teuthology.orchestra.run.smithi137:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T14:22:21.874 INFO:teuthology.orchestra.run.smithi137.stdout:2024-03-26T13:55:59.837757+0000 mon.a (mon.0) 632 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T14:22:21.876 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-26T14:22:21.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo egrep '\[SEC\]' /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T14:22:21.967 DEBUG:teuthology.orchestra.run.smithi137:> sudo egrep '\[ERR\]' /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T14:22:22.054 DEBUG:teuthology.orchestra.run.smithi137:> sudo egrep '\[WRN\]' /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-03-26T14:22:22.138 INFO:teuthology.orchestra.run.smithi137.stdout:2024-03-26T13:55:59.837757+0000 mon.a (mon.0) 632 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-03-26T14:22:22.139 INFO:tasks.cephadm:Compressing logs... 2024-03-26T14:22:22.139 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T14:22:22.143 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T14:22:22.150 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-26T14:22:22.207 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T14:22:22.207 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T14:22:22.207 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-26T14:22:22.208 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-03-26T14:22:22.209 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.6.log 2024-03-26T14:22:22.210 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.a.log 2024-03-26T14:22:22.212 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T14:22:22.212 INFO:teuthology.orchestra.run.smithi137.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-03-26T14:22:22.213 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.g.log 2024-03-26T14:22:22.215 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.j.log 2024-03-26T14:22:22.215 INFO:teuthology.orchestra.run.smithi137.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T14:22:22.215 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.3.log 2024-03-26T14:22:22.218 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.x.log 2024-03-26T14:22:22.218 INFO:teuthology.orchestra.run.smithi137.stderr: 90.2% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log.gz/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.3.log: 2024-03-26T14:22:22.218 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.0.log 2024-03-26T14:22:22.221 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.x.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T14:22:22.223 INFO:teuthology.orchestra.run.smithi137.stderr: 94.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.g.log.gz 2024-03-26T14:22:22.224 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log 2024-03-26T14:22:22.224 INFO:teuthology.orchestra.run.smithi137.stderr: 94.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.j.log.gz 2024-03-26T14:22:22.225 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T14:22:22.228 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log: 83.5% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T14:22:22.229 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.9.log 2024-03-26T14:22:22.230 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.d.log 2024-03-26T14:22:22.231 INFO:teuthology.orchestra.run.smithi137.stderr: 88.1% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T14:22:22.231 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.a.log 2024-03-26T14:22:22.232 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-26T14:22:22.233 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-26T14:22:22.234 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.10.log 2024-03-26T14:22:22.235 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T14:22:22.236 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.b.log 2024-03-26T14:22:22.236 INFO:teuthology.orchestra.run.smithi159.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T14:22:22.237 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.4.log 2024-03-26T14:22:22.238 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log: No such file or directory 2024-03-26T14:22:22.238 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-03-26T14:22:22.238 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.1.log 2024-03-26T14:22:22.239 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.l.log 2024-03-26T14:22:22.240 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log 2024-03-26T14:22:22.240 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.k.log 2024-03-26T14:22:22.240 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T14:22:22.241 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.h.log 2024-03-26T14:22:22.241 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.a.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.9.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.a.log.gz 2024-03-26T14:22:22.242 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.i.log 2024-03-26T14:22:22.242 INFO:teuthology.orchestra.run.smithi179.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-26T14:22:22.244 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.7.log 2024-03-26T14:22:22.244 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.11.log 2024-03-26T14:22:22.244 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.z.log 2024-03-26T14:22:22.245 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.e.log 2024-03-26T14:22:22.245 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.8.log 2024-03-26T14:22:22.246 INFO:teuthology.orchestra.run.smithi179.stderr: 90.4% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.audit.log.gz 2024-03-26T14:22:22.247 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.5.log 2024-03-26T14:22:22.248 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.7.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.y.log 2024-03-26T14:22:22.249 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.f.log 2024-03-26T14:22:22.249 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.8.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.5.log: 94.4% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.l.log.gz 2024-03-26T14:22:22.250 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T14:22:22.250 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log 2024-03-26T14:22:22.252 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log 2024-03-26T14:22:22.254 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log: 83.3% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T14:22:22.255 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.2.log 2024-03-26T14:22:22.256 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.y.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log 2024-03-26T14:22:22.256 INFO:teuthology.orchestra.run.smithi159.stderr: 83.3% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.cephadm.log.gz 2024-03-26T14:22:22.259 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T14:22:22.259 INFO:teuthology.orchestra.run.smithi137.stderr: 94.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.d.log.gz 2024-03-26T14:22:22.259 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.c.log 2024-03-26T14:22:22.260 INFO:teuthology.orchestra.run.smithi179.stderr: 93.3% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.z.log.gz 2024-03-26T14:22:22.260 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log 2024-03-26T14:22:22.260 INFO:teuthology.orchestra.run.smithi179.stderr: 94.4% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.f.log.gz 2024-03-26T14:22:22.260 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.c.log 2024-03-26T14:22:22.260 INFO:teuthology.orchestra.run.smithi179.stderr: 88.2%/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.c.log: -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T14:22:22.261 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.b.log 2024-03-26T14:22:22.266 INFO:teuthology.orchestra.run.smithi137.stderr: 90.8% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T14:22:22.272 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.b.log: 88.2% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph.log.gz 2024-03-26T14:22:22.278 INFO:teuthology.orchestra.run.smithi159.stderr: 93.2% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.y.log.gz 2024-03-26T14:22:22.278 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log: /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T14:22:22.294 INFO:teuthology.orchestra.run.smithi159.stderr: 90.8% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-volume.log.gz 2024-03-26T14:22:22.319 INFO:teuthology.orchestra.run.smithi179.stderr: 94.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.i.log.gz 2024-03-26T14:22:22.360 INFO:teuthology.orchestra.run.smithi179.stderr: 94.5% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.c.log.gz 2024-03-26T14:22:22.406 INFO:teuthology.orchestra.run.smithi159.stderr: 94.7% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.h.log.gz 2024-03-26T14:22:22.421 INFO:teuthology.orchestra.run.smithi159.stderr: 94.7% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.e.log.gz 2024-03-26T14:22:22.591 INFO:teuthology.orchestra.run.smithi137.stderr: 90.1% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mgr.x.log.gz 2024-03-26T14:22:22.693 INFO:teuthology.orchestra.run.smithi179.stderr: 91.9% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.c.log.gz 2024-03-26T14:22:22.811 INFO:teuthology.orchestra.run.smithi159.stderr: 95.5% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.k.log.gz 2024-03-26T14:22:22.846 INFO:teuthology.orchestra.run.smithi159.stderr: 92.1% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.b.log.gz 2024-03-26T14:22:23.934 INFO:teuthology.orchestra.run.smithi137.stderr: 90.5% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mon.a.log.gz 2024-03-26T14:22:24.166 INFO:teuthology.orchestra.run.smithi137.stderr: 92.0% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.3.log.gz 2024-03-26T14:22:24.243 INFO:teuthology.orchestra.run.smithi179.stderr: 92.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.11.log.gz 2024-03-26T14:22:24.509 INFO:teuthology.orchestra.run.smithi159.stderr: 91.8% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.10.log.gz 2024-03-26T14:22:24.520 INFO:teuthology.orchestra.run.smithi137.stderr: 92.3% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.9.log.gz 2024-03-26T14:22:24.665 INFO:teuthology.orchestra.run.smithi137.stderr: 92.1% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.0.log.gz 2024-03-26T14:22:24.838 INFO:teuthology.orchestra.run.smithi159.stderr: 92.2% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.1.log.gz 2024-03-26T14:22:24.868 INFO:teuthology.orchestra.run.smithi179.stderr: 92.1% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.8.log.gz 2024-03-26T14:22:25.134 INFO:teuthology.orchestra.run.smithi159.stderr: 92.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.7.log.gz 2024-03-26T14:22:25.332 INFO:teuthology.orchestra.run.smithi137.stderr: 92.3% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.6.log.gz 2024-03-26T14:22:25.379 INFO:teuthology.orchestra.run.smithi159.stderr: 92.2% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.4.log.gz 2024-03-26T14:22:25.441 INFO:teuthology.orchestra.run.smithi179.stderr: 92.6% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.2.log.gz 2024-03-26T14:22:26.293 INFO:teuthology.orchestra.run.smithi179.stderr: 93.0% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-osd.5.log.gz 2024-03-26T14:22:26.298 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-03-26T14:22:26.298 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m4.097s 2024-03-26T14:22:26.298 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m11.601s 2024-03-26T14:22:26.299 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m1.024s 2024-03-26T14:22:26.825 INFO:teuthology.orchestra.run.smithi159.stderr: 93.4% -- replaced with /var/log/ceph/66aee2a0-eb77-11ee-95d0-87774f69a715/ceph-mds.b.log.gz 2024-03-26T14:22:26.829 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-03-26T14:22:26.830 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m4.635s 2024-03-26T14:22:26.830 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m15.642s 2024-03-26T14:22:26.830 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m1.295s 2024-03-26T14:22:58.055 INFO:teuthology.orchestra.run.smithi137.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-26T14:22:58.060 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-03-26T14:22:58.060 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m35.886s 2024-03-26T14:22:58.060 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m44.091s 2024-03-26T14:22:58.060 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m3.786s 2024-03-26T14:22:58.061 INFO:tasks.cephadm:Archiving logs... 2024-03-26T14:22:58.062 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi137/log 2024-03-26T14:22:58.063 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T14:23:02.990 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi159/log 2024-03-26T14:23:02.991 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T14:23:05.319 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi179/log 2024-03-26T14:23:05.320 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-26T14:23:07.120 INFO:tasks.cephadm:Removing cluster... 2024-03-26T14:23:07.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --force 2024-03-26T14:23:07.444 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:23:08.057 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --force 2024-03-26T14:23:08.371 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:23:08.966 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 66aee2a0-eb77-11ee-95d0-87774f69a715 --force 2024-03-26T14:23:09.290 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: 66aee2a0-eb77-11ee-95d0-87774f69a715 2024-03-26T14:23:09.855 INFO:tasks.cephadm:Removing cephadm ... 2024-03-26T14:23:09.855 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T14:23:09.911 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T14:23:09.969 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-26T14:23:10.028 INFO:tasks.cephadm:Teardown complete 2024-03-26T14:23:10.028 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-26T14:23:10.039 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-26T14:23:10.039 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T14:23:10.046 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T14:23:10.054 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-26T14:23:10.220 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-26T14:23:10.230 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-26T14:23:10.230 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T14:23:10.237 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T14:23:10.244 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-26T14:23:10.283 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-03-26T14:23:10.290 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-03-26T14:23:10.292 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T14:23:10.292 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-03-26T14:23:10.292 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 7 +529us[ +529us] +/- 86ms 2024-03-26T14:23:10.292 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv02.front.sepia.ceph.com 3 7 377 75 -638us[ -611us] +/- 81ms 2024-03-26T14:23:10.292 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 7 +170us[ +170us] +/- 85ms 2024-03-26T14:23:10.292 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T14:23:10.295 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-03-26T14:23:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T14:23:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-03-26T14:23:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 +590us[ +590us] +/- 86ms 2024-03-26T14:23:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 7 377 11 -526us[ -530us] +/- 54ms 2024-03-26T14:23:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +228us[ +228us] +/- 85ms 2024-03-26T14:23:10.300 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T14:23:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-26T14:23:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-03-26T14:23:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 10 +557us[ +557us] +/- 86ms 2024-03-26T14:23:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 7 377 74 -575us[ -594us] +/- 81ms 2024-03-26T14:23:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +191us[ +191us] +/- 85ms 2024-03-26T14:23:10.304 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-26T14:23:10.305 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-26T14:23:10.314 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-26T14:23:10.315 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-26T14:23:10.325 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T14:23:10.553 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T14:23:10.777 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-26T14:23:11.003 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T14:23:11.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T14:23:11.107 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T14:23:11.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T14:23:11.213 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-26T14:23:11.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T14:23:11.318 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-26T14:23:11.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-26T14:23:11.342 INFO:teuthology.task.internal:Duration was 2659.735018 seconds 2024-03-26T14:23:11.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-26T14:23:11.355 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-26T14:23:11.355 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T14:23:11.362 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T14:23:11.370 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-26T14:23:11.519 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T14:23:11.528 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T14:23:11.528 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-26T14:23:11.885 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-26T14:23:11.886 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-03-26T14:23:11.886 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T14:23:11.953 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-03-26T14:23:11.953 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T14:23:12.052 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-03-26T14:23:12.053 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-26T14:23:12.116 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-26T14:23:12.116 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T14:23:12.124 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T14:23:12.131 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-26T14:23:12.224 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-26T14:23:12.225 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T14:23:12.232 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T14:23:12.236 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-26T14:23:12.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-26T14:23:12.753 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-26T14:23:12.753 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T14:23:12.844 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T14:23:12.945 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-26T14:23:13.033 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-26T14:23:13.046 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T14:23:13.054 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T14:23:13.061 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-26T14:23:13.150 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-03-26T14:23:13.161 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-03-26T14:23:13.168 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-03-26T14:23:13.259 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T14:23:13.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T14:23:13.313 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T14:23:13.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T14:23:13.367 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-26T14:23:13.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-26T14:23:13.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-26T14:23:13.433 INFO:teuthology.task.internal:Transferring archived files... 2024-03-26T14:23:13.434 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi137 2024-03-26T14:23:13.434 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T14:23:13.548 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi159 2024-03-26T14:23:13.548 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T14:23:13.656 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940/remote/smithi179 2024-03-26T14:23:13.656 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-26T14:23:13.764 INFO:teuthology.task.internal:Removing archive directory... 2024-03-26T14:23:13.764 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T14:23:13.772 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T14:23:13.780 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-26T14:23:13.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-26T14:23:13.850 INFO:teuthology.task.internal:Not uploading archives. 2024-03-26T14:23:13.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-26T14:23:13.924 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-26T14:23:13.924 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T14:23:13.931 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T14:23:13.938 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-26T14:23:13.980 INFO:teuthology.orchestra.run.smithi137.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 14:23 /home/ubuntu/cephtest 2024-03-26T14:23:13.989 INFO:teuthology.orchestra.run.smithi159.stdout: 393776 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 14:23 /home/ubuntu/cephtest 2024-03-26T14:23:13.994 INFO:teuthology.orchestra.run.smithi179.stdout: 395173 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 26 14:23 /home/ubuntu/cephtest 2024-03-26T14:23:13.997 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-26T14:23:14.010 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-26T14:23:14.042 INFO:teuthology.nuke:Checking targets against current locks 2024-03-26T14:23:14.066 DEBUG:teuthology.nuke:shortname: smithi137 2024-03-26T14:23:14.067 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T14:23:14.083 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 13:28:50.995636', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxphBMmXNF5997hhTfJvfFX87xNP8nNzUJue/G8xxLyEA5aHmZvyq4GdEN1upO8Ac7fqmYN6PW7ZYSGnxHpQc0='} 2024-03-26T14:23:14.087 DEBUG:teuthology.nuke:shortname: smithi159 2024-03-26T14:23:14.087 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T14:23:14.106 DEBUG:teuthology.nuke:shortname: smithi179 2024-03-26T14:23:14.106 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-26T14:23:14.109 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 13:28:50.998389', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFaz4TwhsPhkEN5rr9iXf6Tvy5HEfimoIVWxx+bM9NLts2Ga8D3Nqf2LP8UVejyAo9PK4I4d/Bv5+mHUl2TFmEc='} 2024-03-26T14:23:14.125 INFO:teuthology.orchestra.console.smithi137:Power off 2024-03-26T14:23:14.125 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T14:23:14.138 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-26_07:59:35-fs-wip-mchangir-testing-main-20240318.032620-testing-default-smithi/7622940', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-26 13:28:51.000042', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMXfKgTlsjA+aIMd4i8oHJ7WfQSVYhz5rZzq7S7A2jCksrbOmDQa07XUBN0GQea2sd8zC5OC7KYWPMsxDGzdo5k='} 2024-03-26T14:23:14.148 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-03-26T14:23:14.149 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:14.168 INFO:teuthology.orchestra.console.smithi159:Power off 2024-03-26T14:23:14.169 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T14:23:14.178 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-26T14:23:14.191 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-03-26T14:23:14.191 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:14.206 INFO:teuthology.orchestra.console.smithi179:Power off 2024-03-26T14:23:14.206 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-26T14:23:14.216 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-26T14:23:14.249 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2024-03-26T14:23:14.249 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:14.273 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-03-26T14:23:18.179 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:18.217 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:18.274 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:18.289 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-26T14:23:18.326 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-26T14:23:18.383 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-03-26T14:23:22.295 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:22.326 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:22.385 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:22.405 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-03-26T14:23:22.435 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-03-26T14:23:22.493 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2024-03-26T14:23:22.594 INFO:teuthology.orchestra.console.smithi179:Power off completed 2024-03-26T14:23:26.410 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:26.436 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-26T14:23:26.521 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-03-26T14:23:26.545 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-03-26T14:23:26.622 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-03-26T14:23:26.646 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-03-26T14:23:26.751 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/direct_io}} duration: 2659.7350182533264 failure_reason: '"2024-03-26T13:55:59.837757+0000 mon.a (mon.0) 632 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.c-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: mchangir success: false 2024-03-26T14:23:26.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-26T14:23:26.828 INFO:teuthology.run:FAIL